Episodes
How Chained Hashing Reshaped Database Indexing
In episode 166 of The Technical Co-Founder Podcast, Lucas and Luna dive into chained hashing—the classic collison-handling technique that quietly powers database indexing. They explore how it works, why it beats linear probing in real-world workloads, and how modern systems like PostgreSQL and Redis still rely on it today. With a concrete example of a simple key-value store and a look at how hash tables made search faster, this episode connects a foundational data structure to the performance…
How Delta Lake Handles ACID on a Data Lake
In this episode, Lucas and Luna take a close look at Delta Lake, the open-source storage layer that brings transactional integrity to data lakes. They start with the problem: a data lake where a routine job can leave files half-written and consumers reading inconsistent snapshots. Then they walk through Delta Lake's solution — a transaction log that records each change as an atomic commit, letting readers see a consistent view while writers work in parallel. They discuss time travel, schema…
How Probabilistic Data Structures Shrink Memory Footprints
In this episode of The Technical Co-Founder Podcast, Lucas and Luna explore how probabilistic data structures like HyperLogLog and Count-Min Sketch are transforming the way startups handle massive datasets. They break down how a real-time analytics company cut its memory footprint by 95 percent using HyperLogLog for unique user counts, and how Count-Min Sketch powers frequency estimation in distributed systems. The hosts explain the trade-offs of approximate answers, the math behind error…
How Causal Profiling Reveals Hidden Latency in Production
In this episode, Lucas and Luna explore causal profiling, a technique that traces performance bottlenecks back to their true root cause in production systems. Using the real-world example of a fintech startup that reduced API latency by 40 percent, they explain how causal profiles go beyond CPU time to capture wait times, lock contention, and I/O stalls. They contrast it with traditional sampling profilers, discuss the challenges of overhead and distributed tracing, and highlight the rise of…
How Secret Sharing Secures Multi-Party Computation
Lucas and Luna explore how Shamir's secret sharing underpins secure multi-party computation, using the example of a fictional consortium of three hospitals that want to jointly train a medical model without exposing patient data. They break down the math of polynomial interpolation, the threshold scheme, and how it enables privacy-preserving analytics. They also discuss real-world applications like secure voting and private set intersection, and contrast it with homomorphic encryption. The…
How Zstandard Replaced Snappy in Big Data Pipelines
Episode 161 of The Technical Co-Founder Podcast digs into the quiet but massive shift from Snappy to Zstandard (zstd) in big data pipelines. Lucas walks through the compression ratios and speed trade-offs that made Facebook engineers build zstd in 2016, and why it's now a default in Hadoop, Kafka, and cloud data warehouses by 2026. Luna pushes on practical adoption: when does zstd actually save you money, and when is Snappy still the right call? The conversation lands on a concrete example from…
How gRPC Multiplexing Cuts Latency in Microservices
In this episode, Lucas and Luna dive into the world of gRPC multiplexing and how it's transforming microservice communication. They explore the mechanics of HTTP/2 multiplexing, the practical benefits for startups, and the real-world impact on latency and throughput. With a focus on a concrete example—how a fintech startup reduced their inter-service latency by 35 percent—they break down the trade-offs between gRPC and REST, the role of protobufs, and the operational considerations like…
How Merkle Prefix Trees Shrink Blockchain State
In this episode, Lucas and Luna dig into the engineering behind blockchain state storage, focusing on a technique that keeps full nodes lean: Merkle prefix trees. They trace how a simple idea—combining cryptographic hashing with radix-style path compression—lets Ethereum-style networks verify state without re-downloading everything. Along the way, they look at real numbers, like how a full Ethereum node can store hundreds of gigabytes of state, and why that matters for decentralization. They…
How Bloom Filters Cut Database Lookups to Near Zero
In episode 158 of The Technical Co-Founder Podcast, Lucas and Luna dive into Bloom filters: the probabilistic data structure that lets databases like PostgreSQL and Cassandra check 'definitely not in this set' without touching disk. They walk through the classic case of PostgreSQL's index-only scans, where a Bloom filter over visibility maps cuts unnecessary heap fetches. They explain how the trade-off between false positives and memory works, why the optimal number of hash functions matters…
How Read Replicas Scale PostgreSQL Without the Headache
In this episode of The Technical Co-Founder Podcast, Lucas and Luna dig into the often-overlooked workhorse of modern databases: read replicas. Using a concrete example of a startup that scaled its PostgreSQL database from a single instance to handle 40x read traffic, they explain how read replicas work under the hood — from physical replication to lag and consistency trade-offs. They also tackle the tricky part: what happens when a replica falls behind, and how to handle failover without…
How S2 Cells Make Google Maps Blazing Fast
Have you ever wondered how Google Maps instantly finds nearby restaurants or draws perfect boundaries around neighborhoods? The secret isn't just GPS—it's an open-source geometry library called S2, used by Google, Uber, Foursquare, and many others. In this episode, Lucas and Luna dive into how S2 cells work: they project the Earth onto a cube, then recursively subdivide into a hierarchy of cells, each with a unique 64-bit ID. You'll learn how these IDs enable lightning-fast spatial indexing…
How Functional Indexes Speed Up PostgreSQL Queries
This episode of The Technical Co-Founder Podcast dives into a deceptively simple PostgreSQL feature that can transform query performance: functional indexes. Most developers know that indexes speed up lookups on columns, but few realize you can index the result of a function applied to that column. Lucas and Luna walk through a real-world case: a startup's analytics dashboard that was crawling because it filtered on date_trunc('month', created_at). By creating a functional index on that…
Why Consistent Hashing Is the Secret to Your Sorted Set
In this episode, Lucas and Luna dive into the mechanics behind one of the most quietly powerful data structures in modern systems: the skip list. You'll learn how skip lists power Redis sorted sets and in-memory indexes, why they beat balanced trees in practice, and how they enable the lightning-fast range queries your favorite apps rely on. We break down the probabilistic nature of skip lists, walk through a real-world example of how a startup uses them for real-time leaderboards, and explore…
How ZSTD Compression Speeds Up Modern Data Pipelines
Lucas and Luna dig into Zstandard, the compression algorithm that's quietly replacing gzip and zlib across modern data infrastructure. They trace its origins at Facebook, where a team led by Yann Collet set out to close the gap between compression ratio and speed. The episode walks through Zstd's key innovations: entropy coding with Finite State Entropy, dictionary compression for small records, and the ability to tune compression levels without recompressing. Lucas and Luna discuss why Zstd's…
How CRDTs Tame Offline Collaboration
When two engineers edit the same file offline, who wins? In this episode, Lucas and Luna dig into conflict-free replicated data types (CRDTs) — the algorithms behind collaborative editing in tools like Figma, Notion, and Google Docs. They walk through why traditional locking breaks down at scale, what makes CRDTs tick, and how one startup used them to cut sync conflicts by 90 percent. Expect concrete examples, a real-world case study, and a look at the hard trade-offs — from merge semantics to…
How Rebuildable Snapshots Make Git History Cheap and Safe
Lucas and Luna explore how Git's snapshot model, combined with rebuildable snapshots and shallow clones, keeps repositories fast and history cheap. They discuss the trade-offs of depth, the role of tags as durable snapshots, and how tools like git replace can reshape history without rewriting it. The episode centers on a real case: a monorepo that cut clone time by 70 percent by switching to shallow clones with filters, while preserving integrity through signed tags. They also touch on how…
How Orphaned Blocks Make Blockchain Storage Cheaper
In this episode, Lucas and Luna explore how blockchain networks handle the growing problem of data storage. They focus on a clever mechanism used by Ethereum: pruning orphaned blocks. These are valid but discarded blocks that still consume storage. The episode explains how pruning works, why it matters for node operators, and how it reduces storage costs by up to 80 percent in some cases. Lucas breaks down the technical details, while Luna asks about the trade-offs, like security risks and the…
How Btrfs Snapshots Power Instant Backup
In this episode, we explore how Btrfs snapshots enable instant, low-cost backups for modern data pipelines. We break down the copy-on-write mechanics, the difference between snapshots and full copies, and why incremental snapshot chains make nightly backups nearly free. We also look at real-world use cases, from container filesystems to database backups, and discuss the trade-offs—like performance overhead and the need for careful snapshot management. If you're a developer or systems admin…
How Raft Consensus Powers CockroachDB's Global Consistency
In this episode, Lucas and Luna dive into the Raft consensus algorithm and how CockroachDB uses it to achieve global consistency across distributed databases. They break down the mechanics of leader election, log replication, and safety guarantees, and explain why Raft is simpler to implement than Paxos. With real-world examples and a look at the trade-offs, you'll understand how modern distributed systems stay reliable. Perfect for engineers and business leaders alike, this episode connects…
How One Startup Uses Sparse Merkle Trees for Secure Key-Value Stores
In this episode, Lucas and Luna explore how a growing data infrastructure startup uses sparse Merkle trees to secure its distributed key-value store. They break down the clever trick of using a fixed-depth tree with almost all leaves empty, and how that structure lets the company prove data integrity without shipping gigabytes of hashes. Lucas walks through a concrete example: how a single key update only touches a handful of nodes along the path to the root, keeping the proof size small and…
How LFU Caching Powers Content Delivery Networks
In this episode, Lucas and Luna dive into the engineering behind content delivery networks, focusing on the least-frequently-used caching strategy. They explore how LFU differs from LRU, why it matters for serving billions of requests, and the real-world tradeoffs. Using Cloudflare's approach as a case study, they break down the 'cache thundering herd' problem and how LFU's frequency tracking handles it. They also discuss the hybrid approaches that modern CDNs use, and why LFU isn't a silver…
How One Startup Uses Delta Encoding for 90 Percent Smaller Backups
Lucas and Luna dig into delta encoding, the unsung hero behind incremental backups and versioned storage. They break down how block-level deltas and content-defined chunking let companies like Dropbox and Git store only what changed, slashing storage costs and network traffic. Learn the difference between delta and deduplication, why rolling hashes matter, and how one startup cut backup size by 90 percent with rolling checksums. If you're building anything that moves or stores data, this…
How LZ4 Compression Speeds Up Real-Time Data Pipelines
Episode 144 of The Technical Co-Founder Podcast dives into LZ4, the compression algorithm that's become a default for high-throughput data systems. Lucas and Luna break down why LZ4 trades a smaller ratio for blazing speed, how it powers everything from Kafka to RocksDB to real-time log pipelines, and why it's often a better fit than zstd when latency matters more than disk space. They walk through a concrete example from a fintech startup that cut pipeline latency by 60% just by switching…
How One Startup Uses Apache Arrow for Zero-Copy Analytics
In this episode, Lucas and Luna explore how a startup called Synthara uses Apache Arrow to achieve zero-copy analytics, cutting query latency by 75 percent and memory overhead by half. They break down the columnar format, the concept of zero-copy, and how Synthara leveraged Arrow's ecosystem to build a real-time analytics engine that handles billions of rows without breaking a sweat. With concrete numbers and practical insights, this conversation is a must-listen for anyone building…
How Btrfs Snapshots Power Instant Backup
Lucas and Luna dive into how Btrfs snapshots enable instant, space-efficient backups for modern infrastructure. They unpack the copy-on-write mechanism, compare it with ZFS, and walk through a real-world example where a startup cut backup time from hours to seconds. Along the way, they discuss the trade-offs of Btrfs in production, from data integrity to performance overhead, and why understanding the file system layer matters for tech founders. If you've ever wondered how snapshots work under…
How Bitmap Indexes Power 100x Faster Analytics
Lucas and Luna explore how a startup called Pulse Analytics uses bitmap indexes — a decades-old data structure — to run complex filter queries over 10 billion rows in under 200 milliseconds. They break down how compressed bitmaps (Roaring bitmaps in particular) turn multi-dimensional count queries into bitwise operations, what trade-offs bitmap indexes require versus B-trees, and why this approach is reshaping real-time analytics for event streams, ad-tech, and IoT. Along the way, they touch on…
How One Startup Uses Merkle Trees for Tamper-Evident Audit Logs
Audit logs are supposed to be the unassailable record of what happened inside a system. But if logs can be modified after the fact, they're worthless. In this episode, we break down how startup LogChain built a tamper-evident audit log using Merkle trees. We explore the specific data structure choices—why they chose a binary Merkle tree over a hash chain, how they batch leaves to keep performance under 5 microseconds per append, and how their scheme makes it computationally infeasible to alter…
How CockroachDB Uses Raft for Global Consistency
Distributed databases face a fundamental challenge: keeping data consistent across multiple data centers without sacrificing performance. In this episode, Lucas and Luna explore how CockroachDB leverages the Raft consensus algorithm to achieve linearizable transactions across continents. They break down Raft's core mechanisms—leader election, log replication, and safety guarantees—and explain how CockroachDB implements a 'multi-Raft' architecture, where each range of data has its own Raft…
How Skip Lists Power Redis Sorted Sets and In-Memory Indexes
In this episode, we dive into skip lists, the probabilistic data structure that underlies Redis sorted sets, LevelDB's memtable, and countless in-memory indexes. Lucas and Luna explore how skip lists achieve O(log n) operations with a simple randomized balancing mechanism, making them easier to implement and more concurrent-friendly than balanced trees like red-black trees. We walk through a concrete example: how Redis uses skip lists to power leaderboard operations like ZADD and ZRANK at…
How T-Digest Powers Real-Time Percentile Estimation
Lucas and Luna dive into the T-Digest algorithm, a probabilistic data structure that enables accurate percentile estimation (P50, P95, P99) from streaming data using a fraction of the memory. They explore how a major observability platform uses T-Digest to monitor millions of requests per second without storing every data point, achieving sub-1% relative error while keeping memory under 100KB per metric. The conversation covers how T-Digest works—clustering centroids, adjusting buffer…
How Perfect Hashing Powers Sub-100 Nanosecond Lookups
DNS startup FastDomain uses minimal perfect hashing to resolve domain names against a 100-million-record dataset in under 100 nanoseconds. Lucas and Luna break down how the algorithm works – from the concept of a perfect hash function to the Hash, Displace, and Compress construction method. They explore the trade-offs: static datasets require rebuilding the function on updates, but the payoff is zero collisions, minimal memory overhead, and dramatic latency wins. The episode dives into why…
How HNSW Graphs Power Real-Time Vector Search
Vector similarity search is behind every recommendation engine, image lookup, and semantic search you use. But scaling it to billions of vectors with millisecond latency requires clever data structures. In this episode, Lucas and Luna explore Hierarchical Navigable Small World (HNSW) graphs—the algorithm that one startup, Vectara, uses to deliver sub-10ms search on 10 billion vectors. They break down how HNSW works, why it beats brute-force and tree-based methods, and the memory trade-offs…
How Consistent Hashing Cut Cache Misses by 80%
Episode 134 of The Technical Co-Founder Podcast explores how a real-world startup slashed cache miss rates from 30% to 6% by adopting consistent hashing. Lucas and Luna break down the algorithm's mechanics—the ring, virtual nodes, and minimal reshuffling—and discuss why it's a go-to for distributed systems engineers. Packed with specific numbers and trade-offs, this episode is a masterclass in algorithmic thinking for production systems. #ConsistentHashing #DistributedSystems #Caching…
How Fermyon Uses WebAssembly for Sub-5ms Edge Computing
Episode 133 dives into how startup Fermyon leverages WebAssembly (WASM) to run untrusted multi-tenant code at the edge with cold starts under 5 milliseconds—compared to hundreds of milliseconds for containers. We explore their Spin framework, capability-based security model, and why WASM's minimal runtime enables dense packing of thousands of instances per server. Plus, a look at the trade-offs: limited system calls, debugging challenges, and why edge workloads like API gateways are the sweet…
How HyperLogLog Powers Real-Time Analytics
In this episode, Lucas and Luna explore how a fictional startup called Streamlytics uses the HyperLogLog probabilistic data structure to count unique events in real-time – like distinct users or IPs – with minimal memory. They break down why exact counts fail at scale, how HyperLogLog achieves 97% accuracy using only 1.5 KB of memory for billions of events, and the tradeoffs around mergeability and error bounds. Along the way, they discuss practical deployment in streaming pipelines with Apache…
How One Startup Uses Count-Min Sketch for Real-Time Frequency Estimation
In this episode, Lucas and Luna dive into the Count-Min Sketch, a probabilistic data structure that lets startups track frequencies of millions of items using a fraction of the memory. They break down how one social media analytics company, TrendSpot, uses a 2D array of counters and just three hash functions to handle 500,000 events per second while keeping memory under 100MB. Learn how the sketch works, the tradeoffs between accuracy and memory, and why the minimum-of-hashes trick gives you a…
How One Startup Uses Multi-Version Concurrency Control for Zero-Downtime Migrations
In this episode, Lucas and Luna explore how a fast-growing fintech startup uses multi-version concurrency control (MVCC) to perform schema migrations without taking the database offline. They walk through the problem of locking tables during ALTER statements, how MVCC keeps multiple versions of rows active, and the specific approach the startup took: progressive column additions, lazy backfills, and a version-based query router. The case study is a company processing 2 million transactions…
How One Startup Uses Differential Privacy for Analytics Without Leaking User Data
Episode 129 dives into differential privacy — the statistical technique that lets companies extract useful insights from datasets while making it nearly impossible to re-identify any single individual. Lucas and Luna break down how one startup, a health-data analytics platform called Synthos, applies differential privacy to its customer-facing dashboards. They walk through the math behind adding 'calibrated noise,' the trade-off between accuracy and privacy, and how Synthos uses a privacy…
How One Startup Uses Zero-Knowledge Proofs for Verifiable Outsourced Computation
In episode 128 of The Technical Co-Founder Podcast, Lucas and Luna explore how a startup called Veridise uses zero-knowledge proofs to let customers verify that a cloud provider actually ran the computation they paid for — without re-executing it or exposing private data. Lucas explains why zk-SNARKs matter for trust in outsourced compute, how Veridise handles the performance trade-off between proof generation and verification, and why a financial-services client cut audit latency from 24 hours…
How One Startup Uses CRDTs for Peer-to-Peer Databases
In episode 127 of The Technical Co-Founder Podcast, Lucas and Luna explore how a startup called Ditto uses Conflict-Free Replicated Data Types (CRDTs) to build peer-to-peer databases that work offline. Lucas explains the core idea behind CRDTs—data structures that automatically resolve conflicts without a central server—and how Ditto applies them to sync data across devices in retail and logistics environments. Luna asks about the trade-offs, including consistency guarantees and the complexity…
How One Startup Uses Bloom Filters to Speed Up Database Joins by 40x
In this episode, Lucas and Luna explore how a small data infrastructure startup, Raft, deployed Bloom filters to accelerate join operations in their distributed SQL engine. Bloom filters are a space-efficient probabilistic data structure that can quickly test whether an element is a member of a set, with a small chance of false positives. Raft integrated Bloom filters into their query optimizer to skip entire partitions during hash joins, reducing I/O and network transfer. The result: a 40x…
How One Startup Uses LSM Trees for High-Throughput Time Series
Episode 125 of The Technical Co-Founder Podcast dives into Log-Structured Merge Trees (LSM Trees) and how one startup, TimescaleDB, uses them to handle millions of data points per second for time-series workloads. Lucas and Luna break down the write amplification problem, the role of SSTables and compaction strategies, and why LSM Trees outperform B-Trees for sequential writes in IoT and observability. They also discuss real-world trade-offs: read latency, space amplification, and how tiered…
How One Startup Uses Lock-Free Data Structures for Real-Time Analytics
Episode 124 of The Technical Co-Founder Podcast dives into lock-free data structures and how one startup, a real-time analytics platform called OmniScope, replaced traditional mutex-based queues with lock-free ring buffers to handle over 10 million events per second per node. Lucas and Luna break down the mechanics of compare-and-swap (CAS) operations, memory ordering, and the ABA problem, using concrete examples from OmniScope's journey. They discuss why lock-free solutions matter for…
How One Startup Uses Formal Verification for Smart Contracts
Episode 123 of The Technical Co-Founder Podcast dives into formal verification for smart contracts. Lucas and Luna explore how one early-stage startup, Quantify Labs, mathematically proves the correctness of DeFi smart contracts before deployment. They break down the difference between fuzzing, auditing, and formal verification, and walk through a concrete example where formal verification caught a critical reentrancy bug that static analysis missed. The hosts discuss the trade-offs: the steep…
How One Startup Uses Property-Based Testing for Data Pipelines
In this episode of The Technical Co-Founder Podcast, Lucas and Luna dive into property-based testing—a technique that upends traditional unit testing by verifying invariants across random inputs. They explore a real case: a fintech startup called FinFlow that rebuilt its data pipeline after catching a rounding error that had silently corrupted six months of transaction reports. Lucas explains how tools like Hypothesis and QuickCheck let engineers describe 'properties' the system should always…
How One Startup Uses Conflict-Free Replicated Data Types for Real-Time Sync
In this episode, Lucas and Luna explore how a small collaborative design startup called Blueprint uses Conflict-Free Replicated Data Types (CRDTs) to enable real-time, offline-capable syncing across thousands of users. They dive into the specific CRDT algorithm — a last-writer-wins register combined with a grow-only set — that Blueprint chose for its vector-based drawing tool. Lucas explains how CRDTs differ from operational transforms (OT), the trade-offs in convergence guarantees, and why…
How One Startup Uses Trusted Execution Environments for Confidential AI Inference
Episode 120 dives into a specific technical approach for running AI inference on sensitive data without exposing it to the cloud provider. Lucas and Luna break down how one startup, Anjuna Security, uses Intel SGX enclaves to process healthcare and financial models while keeping data encrypted in memory. They walk through the real overhead numbers—roughly 5-15 percent performance hit—and the deployment trade-offs versus homomorphic encryption. The conversation covers attestation, the memory…
How One Startup Uses eBPF to Replace Service Meshes
Episode 119 of The Technical Co-Founder Podcast dives into eBPF — the Linux kernel technology that lets you run sandboxed programs without changing kernel source code. Lucas and Luna examine how a startup called Isovalent built Cilium, an eBPF-based networking and security layer that replaces heavyweight sidecar proxies like Envoy. They unpack the architecture: how eBPF hooks into the kernel at XDP and TC layers, why it reduces latency by 40-60% compared to traditional service meshes, and what…
How One Startup Uses Data Tombs for Historical Analytics
In Episode 118 of The Technical Co-Founder Podcast, Lucas and Luna dive into the architectural pattern known as 'data tombs' — a scalable, low-cost approach to storing and querying historical data that rarely changes. Using the example of a logistics startup that processes 50 million tracking events daily, they explore how the company separates hot and cold data into different storage tiers, using Apache Iceberg and S3 for the tomb layer, and how they achieve sub-second query times on…
How One Startup Uses Scheduler-Aware Networking for Sub-100 Microsecond Latency
In Episode 117 of The Technical Co-Founder Podcast, Lucas and Luna explore how a startup called InferSched uses scheduler-aware networking to achieve sub-100 microsecond latency for distributed inference. They break down the problem: how traditional kernel networking introduces multi-microsecond jitter when scheduling threads and interrupts compete for CPU cores. The startup's solution: a user-space network stack pinned to isolated cores, combined with a custom cooperative scheduler that passes…
Showing the latest 50 episodes. The full archive of 166 is on Apple Podcasts, Spotify and every major podcast app — or via the RSS feed above.