The GeekNarrator podcast is a show hosted by Kaivalya Apte who is a Software Engineer and loves to talk about Technology, Technical Interviews, Self Improvement...
The GeekNarrator memberships can be joined here: https://www.youtube.com/channel/UC_mGuY4g0mggeUGM6V1osdA/join
Membership will get you access to member only videos, exclusive notes and monthly 1:1 with me.
Here you can see all the member only videos: https://www.youtube.com/playlist?list=UUMO_mGuY4g0mggeUGM6V1osdA
------------------------------------------------------------------------------------------------------------------------------------------------------------------
About this episode:
------------------------------------------------------------------------------------------------------------------------------------------------------------------
In this episode we are talking to Peter and Qian, co-founders of DBOS. The conversation covers the challenges of creating fault-tolerant applications, the architecture of DBOS, and how it addresses reliability at multiple layers.
Chapters:
00:00 Introduction to the Geeknerder Podcast
00:29 Meet the Co-Founders of DBOSS
01:25 The Core Problem: Building Reliable Systems
02:05 How DBOSS Solves Reliability Issues
04:29 Understanding DBOSS Architecture
06:09 Deep Dive into DBOSS Library
08:36 Postgres and State Management
18:31 Handling Parallel Steps and Performance Concerns
26:00 Observability and Version Control
30:18 Running Multiple Code Versions
30:58 Managing Workflow Versions
32:03 Surgery on Workflow States
33:15 Library Annotations and Durable Execution
34:24 Migrating to the Cloud Version
37:23 Handling Email Workflows
42:41 Transactional Guarantees with Postgres
48:44 Technical Challenges and Multi-Tenancy
54:12 Real-World Use Cases and Benefits
59:45 Conclusion and Final Thoughts
Some important links:
- Main website: https://www.dbos.dev/
- DBOS docs: https://docs.dbos.dev/
- Open-source DBOS Transact libraries:
- Python: https://github.com/dbos-inc/dbos-transact-py
- TypeScript: https://github.com/dbos-inc/dbos-transact-ts
------------------------------------------------------------------------------------------------------------------------------------------------------------------
Like building real stuff?
------------------------------------------------------------------------------------------------------------------------------------------------------------------
Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.
https://app.codecrafters.io/join?via=geeknarrator
------------------------------------------------------------------------------------------------------------------------------------------------------------------
Link to other playlists. LIKE, SHARE and SUBSCRIBE
------------------------------------------------------------------------------------------------------------------------------------------------------------------
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.
Database internals series: https://youtu.be/yV_Zp0Mi3xs
Popular playlists:
Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
Stay Curios! Keep Learning!
--------
1:01:58
Database Trends and More with Peter Zaitsev
Deep Dive into Databases with Peter Zaitsev | The GeekNarrator Podcast
Join host Kaivalya Apte and special guest Peter Zaitsev from Percona on this episode of the Geeknerder Podcast. They discuss Peter's fascinating journey into the world of databases, founding Percona, and the evolution of open source database solutions. Topics include the rise of PostgreSQL, the comparison between MySQL and PostgreSQL, database observability, the impact of cloud and Kubernetes on database management, licensing changes in popular databases like Redis, and career advice for database administrators and developers. Stay tuned for insights on the future of databases, observability strategies, and the role of AI in database management.
00:00 Introduction and Guest Welcome
00:14 Peter's Journey into Databases
04:15 The Rise of PostgreSQL vs MySQL
18:17 Challenges in Managing Database Clusters
24:36 Common Developer Mistakes with Databases
30:59 MongoDB's Success and Future
34:53 Redis and Licensing Changes
37:07 Elastic's License Change and Its Impact
38:25 Redis Fork and Industry Collaboration
40:27 Kubernetes and Cloud-Native Databases
47:47 Challenges in Database Upgrades and Migrations
54:58 Load Testing and Observability
01:09:02 Future of Database Administration and Development
01:15:13 Conclusion and Final Thoughts
Become a member of The GeekNarrator to get access to member only videos, notes and monthly 1:1 with me.
Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.
https://app.codecrafters.io/join?via=geeknarrator
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.
Database internals series: https://youtu.be/yV_Zp0Mi3xs
Popular playlists:
Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
Stay Curios! Keep Learning!
--------
1:04:13
How would you design a database on Object Storage?
Join Kaivalya Apte and Simon Hørup Eskildsen from Turbopuffer as they talk about the complexities of building a database on top of object storage. Discover the key challenges, the nuances of various storage formats, and the critical trade-offs involved.
Learn from Simon's rich experience, from his time at Shopify to creating Turbopuffer. This episode covers everything—from approaches to write-ahead logs to multi-tenancy and object storage advancements. Perfect for database enthusiasts and those keen on first-principles thinking!
00:00 Introduction
00:17 Simon's Background and Journey to TurboBuffer
02:42 Challenges in Database Scalability
04:21 Experimenting with Vector Databases
05:02 Cost Implications of Vector Databases
05:52 Architectural Considerations for Search Workloads
07:39 Building a Database on Object Storage
16:14 Designing a Simple Database on Object Storage
26:01 Handling Multiple Writers and Consistency
31:26 Trade-offs in Write Operations
32:36 Optimizing MySQL Write Performance
34:03 Batching Writes in Object Storage
35:08 Time-Based vs Size-Based Batching
36:32 Understanding Amplification in Databases
42:26 Challenges with Cold Queries
44:02 Building and Persisting B-Trees
50:53 Separating Workloads in Databases
56:07 Multi-Tenancy Challenges
01:00:39 Choosing Storage Formats
01:06:10 Key Innovations in Object Storage Databases
Important links:
- https://github.com/sirupsen/napkin-math (numbers)
- https://turbopuffer.com/
- https://turbopuffer.com/architecture
- https://sirupsen.com/napkin/problem-10-mysql-transactions-per-second
- https://sirupsen.com (my blog, napkin math)
- https://sirupsen.com/subscribe (napkin math newsletter)
- https://github.com/rkyv/rkyv rkyv rust
Become a member of The GeekNarrator to get access to member only videos, notes and monthly 1:1 with me.
Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.
https://app.codecrafters.io/join?via=geeknarrator
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.
Database internals series: https://youtu.be/yV_Zp0Mi3xs
Popular playlists:
Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
Stay Curios! Keep Learning!
--------
1:08:26
Practical Systems Learning & Verification with Jack Vanlightly
Welcome to The GeekNarrator podcast! In this episode, host Kaivalya Apte goes deeper into the practical applications of formal methods with Jack Vanlightly, a principal technologist at Confluent. With years of experience in distributed systems, Jack discusses his journey and how formal methods have been instrumental in system design verification and bug detection. The conversation covers Jack's background, his process of using formal methods, the significance of modelling, verification, documentation, and systems learning, as well as the future evolution of tooling and its applications. Tune in to understand the intricacies of how formal methods can transform your approach to distributed systems!
Chapters:
00:00 Introduction to the episode
00:37 Meet Jack VanLightly: Principal Technologist at Confluent
02:17 Jack's Journey into Distributed Systems
04:29 Discovering the Power of Formal Methods
08:11 Modeling and Simulation in Formal Methods
13:43 Verification and Safety Properties
19:02 Documentation and Communication Challenges
20:43 Formal Methods as a Systems Learning Tool
24:26 Practical Applications and Case Studies
56:38 Future of Formal Verification and Closing Thoughts
Jack's Blog: https://jack-vanlightly.com/
Become a member of The GeekNarrator to get access to member only videos, notes and monthly 1:1 with me.
Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.
https://app.codecrafters.io/join?via=geeknarrator
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.
Database internals series: https://youtu.be/yV_Zp0Mi3xs
Popular playlists:
Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
Stay Curios! Keep Learning!
--------
1:01:07
Database Internals - NileDB Postgres re-engineered for multitenant apps
Database Internals - NileDB: Postgres Re-engineered for Multitenant Apps with Gwen Shapira
Join us in this episode as we dive deep into the intricacies of NileDB, a groundbreaking database re-engineered for multi-tenant applications. Our special guest, Gwen Shapira, co-founder of NileDB and a notable figure in the database field, shares her insights and technical know-how on solving the common challenges faced by multitenant SaaS applications. From the benefits of using Postgres as the underlying database to the unique tenant isolation features of NileDB, we cover it all. Don't miss out on learning about AI native capabilities, handling schema migrations, and ensuring zero downtime data migrations.
Chapters:
00:00 Introduction
07:19 Challenges in Multi-Tenant Databases
11:09 Tenant Isolation and NILDB's Approach
34:16 Necessary Modifications for Tenant Data
37:47 Zero Downtime Data Migrations
44:32 Handling Schema Migrations
59:11 AI Use Cases and Vector Embedding Storage
59:51 Technical and Non-Technical Learnings from Building Nile
01:05:03 Future Plans and Upcoming Features
NileDB: https://www.thenile.dev/
Blog: https://www.thenile.dev/blog
Gwen's Linkedin: https://www.linkedin.com/in/gwenshapira
Gwen's Twitter: https://twitter.com/gwenshap
#postgres #sql #ai
Become a member of The GeekNarrator to get access to member only videos, notes and monthly 1:1 with me.
Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.
https://app.codecrafters.io/join?via=geeknarrator
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.
Database internals series: https://youtu.be/yV_Zp0Mi3xs
Popular playlists:
Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
Stay Curios! Keep Learning!
The GeekNarrator podcast is a show hosted by Kaivalya Apte who is a Software Engineer and loves to talk about Technology, Technical Interviews, Self Improvement, Best Practices and Hustle.
Connect with Kaivalya Apte https://www.linkedin.com/in/kaivalya-apte-2217221a
Tech blogs: https://kaivalya-apte.medium.com/
Wanna talk? Book a slot here: https://calendly.com/speakwithkv/hey
Enjoy the show and please follow to get more updates. Also please don’t forget to rate and review the show.
Cheers