Skip to content
Radio Logo

EDGE AI POD

EDGE AI FOUNDATION
Latest episode

122 episodes

  • EDGE AI POD

    Distributed SLM-Based Agentic AI for the Edge

    2026/11/19 | 24 mins.
    What happens when AI stops chatting and starts doing? We walk through the leap from single-shot prompts to agentic loops that plan, choose tools, and act in fast-changing physical environments. The twist: small, specialized language models often beat their giant cousins on the edge, delivering higher accuracy for focused tasks while slashing latency, bandwidth, and cost.

    We break down the surge of small language models and why the 1–10B parameter range hits a sweet spot. With advances in Mixture-of-Experts routing, KV-cache efficiency, attention, quantization, and distillation, SLMs run lean and still reason well. Even better, specialization pays: models tuned for math, vision, or reasoning routinely outscore generalists with far fewer parameters. That efficiency multiplies in real-world loops where cameras, robots, and drones stream constant updates and decisions must keep up with reality.

    To make this practical, we share a layered architecture for edge agents. The application control layer handles goals, plans, constraints, guardrails, and tool selection. The model service layer hosts multiple fit-for-purpose models and converts them to each hardware target, tracked by a metastore for performance insights. A data management layer shapes sensor streams into clean inputs, while orchestration coordinates capabilities across heterogeneous devices, sharing distributed memory and balancing load. We also compare tool choices: lightweight, single-frame vision keeps network traffic low, while multi-frame, high-resolution pipelines drive logarithmic growth in transfers and inference calls.

    Heterogeneous hardware is a feature, not a bug, if the scheduler is smart. Random placement underutilizes strong nodes and drags out pipelines; capability- and load-aware scheduling evens utilization and improves completion time. We close with deployment avenues across warehouses, farms, drones, and clinical monitors, and a candid view on readiness: the building blocks exist, but robust guardrails, predictable iteration limits, and resource-aware orchestration are the difference between a demo and a dependable system.

    If you enjoyed this deep dive, follow the show, share it with a colleague who builds on the edge, and leave a quick review to help others discover it. Your feedback guides what we explore next.
    Send us Fan Mail
    Support the show
    Learn more about the EDGE AI FOUNDATION - edgeaifoundation.org
  • EDGE AI POD

    Designing the Next-GenerationFoundation Model Architecture for Edge AI

    2026/11/12 | 18 mins.
    What if your phone could outpace bigger cloud models on the tasks you care about—and still keep quality high? We walk through how we built LFM2, our family of “Liquid Foundation Models,” by starting from the hardware up. Phones face tight memory and bandwidth, laptops have more RAM but similar bottlenecks, and cloud GPUs invite scale but at a cost. So instead of forcing one transformer to fit all, we engineered inference‑aware architectures that adapt to each device class.

    We explain how our STAR evolutionary search explores a modular design space that mixes attention with state space models, linear attention, and convolutional components. Every candidate is judged in the real world: hardware‑in‑the‑loop measurements capture KV cache, activation memory, prefill throughput, and decode latency on a Galaxy S24, then models are trained and checked for perplexity and benchmark quality. Over generations, weak designs are pruned and strong ones recombine, moving toward an architecture that is not only smaller and faster but also competitive on MMLU, multilingual MMLU, and math.

    The results are tangible. LFM2 outperforms larger transformer baselines like Llama variants on edge decode speed, and the gains transfer to laptops with AMD Ryzen CPUs. We’ve released open weights at multiple sizes (350M to 1.2B, plus 2.6B and an MoE variant) with integrations in llama.cpp and NPU paths for Qualcomm, with AMD support coming soon. Beyond text, we extend the same efficient backbone to vision‑language models for on‑device video understanding and to a native audio stack that interleaves text and audio tokens—reducing latency and simplifying the classic STT→LLM→TTS chain.

    If you care about low latency AI, local privacy, and real‑world deployment on phones and PCs, this conversation is for you. Try the open‑weight models, fine‑tune on your data, and let us know how they run on your hardware. Subscribe, share with a teammate who ships on-device, and leave a review to help more builders find the show.
    Send us Fan Mail
    Support the show
    Learn more about the EDGE AI FOUNDATION - edgeaifoundation.org
  • EDGE AI POD

    From Cloud Costs To Tiny Chips: Building Reliable Voice AI On The Edge - with femto.AI

    2026/11/05 | 13 mins.
    Voice should feel instant, private, and reliable—and today’s cloud-only pipelines rarely deliver all three. We walk through a builder’s playbook for voice AI that actually works on the edge, showing how to keep accuracy high while slashing latency, power, and cost. The conversation starts with the real trade-offs of cloud speech systems—network round trips, privacy exposure, and compute bills—then moves into a pragmatic hybrid model where the device handles fast, predictable steps and the cloud adds breadth when needed.

    We break down the techniques that make this possible. Sparsity lets us prune large speech models to a fraction of their size while preserving the capabilities products rely on, backed by production thresholds for true positive rates and minimal false activations. Data augmentation and confusable-phrase training harden models against room acoustics, accents, and near-sound commands like play, plain, and pay, cutting false wakes and improving user trust. We also show how synthetic data from big cloud models can teach compact on-device models to be robust without endless data collection.

    The other unlock comes from merging AI with classic DSP on the same edge processor. By integrating steps like noise suppression, beamforming, and VAD, we feed cleaner signals into recognizers and even boost cloud performance when offloading, halving character error rates for models like Whisper in noisy scenes. Finally, we share how to scale this approach using sparsity-driven Pareto curves, selecting the smallest viable model for each product’s power and memory budget. If you care about instant responses, privacy-first design, and shipping voice interfaces that hold up under real-world noise, this deep dive offers a roadmap you can use today. Subscribe, share with a teammate who builds audio products, and leave a review with the toughest edge constraint you want solved next.
    Send us Fan Mail
    Support the show
    Learn more about the EDGE AI FOUNDATION - edgeaifoundation.org
  • EDGE AI POD

    AI solutions for embedded systems and challenges for GenAI implementations

    2026/10/29 | 9 mins.
    What does it really take to ship AR glasses you can wear all day without a tether to the cloud? We take you behind the scenes of edge AI design, where every gram on the frame and every milliwatt in the battery shapes what’s possible. From comfort thresholds to compute budgets, we map the constraints that make or break a usable product.

    We start with the human limits: sub‑40‑gram weight, eight to ten hours of battery life, high‑resolution displays, and a hands‑free UI that leans on eye‑gaze and a few reliable voice actions. Then we dig into the technical spine of modern wearables—an Arm‑based MCU paired with a next‑gen NPU that runs transformer operators and small language models locally. You’ll hear how native generative support on an MCU changes the experience, cutting latency and boosting privacy while keeping assistants, translation, and visual guidance on‑device. We also explain why an integrated ISP matters for clean, low‑light imaging and how quantization, operator fusion, and memory layout turn peak TOPS into real‑world performance.

    Next, we break down the memory story. Tiny models aren’t tiny enough without faster external memory, so we upgraded from Octal SPI to Hex SPI to move data at much higher rates. That bandwidth feeds vision pipelines and token streams without burning through battery budgets. Along the way, we make the case for open ecosystems: transparent kernels, flexible toolchains, and community momentum beat proprietary walls when model architectures evolve monthly. Our roadmap from early cores to the U85 shows steady gains in perf‑per‑watt, transformer coverage, and concurrent workloads across voice and vision.

    If you’re curious about building wearables that feel instant, private, and truly useful, this conversation lays out the path—from silicon choices and NPU design to UX trade‑offs and developer workflows. Come see the live generative AI demo running on our MCU platform, and let’s talk about what you want to build next. Subscribe, share with a teammate, and leave a review with the one feature you think AR glasses must nail.
    Send us Fan Mail
    Support the show
    Learn more about the EDGE AI FOUNDATION - edgeaifoundation.org
  • EDGE AI POD

    Edge AI From Atoms To Apps with Dr. Mohamed Sabry of Nanoveu

    2026/10/22 | 15 mins.
    What if real AI at the edge didn’t drain your battery, overheat your device, or bloat your bill of materials? We pull back the curtain on EMASS, our edge AI system-on-chip built from the ground up to squeeze maximum intelligence out of milliwatts, not watts. The journey starts “from atoms to apps,” where device physics, circuits, architecture, and algorithms are tuned together—so every layer contributes to performance instead of fighting it.

    We walk through the architecture: a RISC-V core that manages dataflow and OS tasks, plus two specialized AI accelerators that the compiler selects automatically based on workload. With SRAM and non-volatile memory on board, the chip preserves system state, then powers down aggressively and wakes in microseconds. Add model quantization, pruning, knowledge distillation, and coding, and you can shrink networks by up to 100x; a hardware weight decompressor restores precision just in time, preserving accuracy while slashing memory and bandwidth.

    Performance isn’t just on paper. Fabricated at 22 nm and moving to 16 nm, XDOT hits up to 12 TOPS/W and around 30 GOPS at roughly 2 mW. Against public edge benchmarks, we match or beat latency while cutting energy by 20x to 200x. Developers can compile from TensorFlow or PyTorch and even remote-run on our boards—hosted in Singapore—so you see real hardware results from anywhere. Partnerships with Literal Labs yield dramatic speed and energy gains, while our Arrow Electronics modular boards let you stack sensors for predictive maintenance, biometrics, cold-chain tracking, storage, and wireless in minutes.

    The standout use case: drones. Instead of piling on compute-heavy features, we target the battery subsystem with AI-driven control, extending flight endurance by up to 60% and, in some trials, even higher. That means longer missions on the same charge or more payload without sacrificing airtime. It’s a taste of what becomes possible when the entire stack is co-designed—wearables that last, IoT nodes that think locally, AR devices that stay cool, and fleets that fly farther.

    If you’re building edge AI and want real results on real silicon, subscribe, share this episode with a teammate, and leave a review. Ready to test your model on hardware today? Reach out and try the remote run.
    Send us Fan Mail
    Support the show
    Learn more about the EDGE AI FOUNDATION - edgeaifoundation.org
More Technology podcasts
About EDGE AI POD
Discover the cutting-edge world of energy-efficient machine learning, edge AI, hardware accelerators, software algorithms, and real-world use cases with this podcast feed from all things in the world's largest EDGE AI community. These are shows like EDGE AI Talks, EDGE AI Blueprints as well as EDGE AI FOUNDATION event talks on a range of research, product and business topics. Join us to stay informed and inspired!
Podcast website

Listen to EDGE AI POD, All-In with Chamath, Jason, Sacks & Friedberg and many other podcasts from around the world with the radio.net app

Get the free radio.net app

  • Stations and podcasts to bookmark
  • Stream via Wi-Fi or Bluetooth
  • Supports Carplay & Android Auto
  • Many other app features