Deep-dive discussions with the smartest developers we know, explaining what they're working on, how they're trying to move the industry forward, and what we can learn from them.
You might find the solution to your next architectural headache, pick up a new programming language, or just hear some good war stories from the frontline of technology.
Join your host Kris Jenkins as we try to figure out what tomorrow's computing will look like the best way we know how - by listening directly to the developers' voices.
All content for Developer Voices is the property of Kris Jenkins and is served directly from their servers
with no modification, redirects, or rehosting. The podcast is not affiliated with or endorsed by Podjoint in any way.
Deep-dive discussions with the smartest developers we know, explaining what they're working on, how they're trying to move the industry forward, and what we can learn from them.
You might find the solution to your next architectural headache, pick up a new programming language, or just hear some good war stories from the frontline of technology.
Join your host Kris Jenkins as we try to figure out what tomorrow's computing will look like the best way we know how - by listening directly to the developers' voices.
DataFusion - The Database Building Toolkit (with Andrew Lamb)
Developer Voices
1 hour 32 minutes
6 months ago
DataFusion - The Database Building Toolkit (with Andrew Lamb)
Building a database is a serious undertaking. There are just so many parts that you have to implement before you even get to a decent prototype, and so many hours of work before you could begin working on the ideas that would make your database unique. Apache DataFusion is a project that hopes to change all that, but building an extensible, composable toolkit of database pieces, which could let you build a viable database extremely quickly, and then innovate from that starting point. And even if you're not building a database, it's a fascinating project to explain how databases are built.
Joining me to explain it all is Andrew Lamb, one of DataFusion's core contributors, and he's going to take us through the whole stack, how it's built and how you could use it. Along the way we cover everything from who's building interesting new databases and how you manage a large, open-source Rust project.
–
DataFusion Homepage: https://datafusion.apache.org/
DataFusion on Github: https://github.com/apache/datafusion
DataFusion Architecture (with diagrams!): https://youtu.be/NVKujPxwSBA?si=tw9ACxlbdpBuVsnv&t=1045
Datalog: https://docs.racket-lang.org/datalog/
Tokio: https://tokio.rs/
Andrew's Homepage: http://andrew.nerdnetworks.org/
Andrew's Blog Post about Tokio: https://thenewstack.io/using-rustlangs-async-tokio-runtime-for-cpu-bound-tasks/
Velox: https://velox-lib.io/
Arroyo: https://www.arroyo.dev/
Synnada: https://www.synnada.ai/
LanceDB: https://lancedb.com/
SDF+DBT: https://docs.sdf.com/integrations/dbt/integrating
Support Developer Voices on Patreon: https://patreon.com/DeveloperVoices
Support Developer Voices on YouTube: https://www.youtube.com/@developervoices/join
Kris on Bluesky: https://bsky.app/profile/krisajenkins.bsky.social
Kris on Mastodon: http://mastodon.social/@krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Developer Voices
Deep-dive discussions with the smartest developers we know, explaining what they're working on, how they're trying to move the industry forward, and what we can learn from them.
You might find the solution to your next architectural headache, pick up a new programming language, or just hear some good war stories from the frontline of technology.
Join your host Kris Jenkins as we try to figure out what tomorrow's computing will look like the best way we know how - by listening directly to the developers' voices.
