Risky Business Features

In this solo episode of Risky Business Features James Wilson explores how distillation techniques are both a legitimate way to train smaller models, as well as a way to steal model capabilities. It’s not just a problem for frontier labs! Any LLM-based product could have its competitive advantage stolen through these attacks.

James covers:

High-level concept of distillation

Why it matters including close/open-weight/open-source explanation

Types of distillation and the prompts used

The distillation pipeline end to end

Distillation at scale and mitigation techniques

Hardware resource constraints for distillation



Show notes



Self-Instruct: Aligning Language Models with Self-Generated Instructions


Alpaca: A Strong, Replicable Instruction-Following Model


Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality


Orca: Progressive Learning from Complex Explanation Traces of GPT-4


Zephyr: Direct Distillation of LM Alignment


Stealing Part of a Production Language Model


Microsoft probes if DeepSeek-linked group improperly obtained OpenAI data, Bloomberg News reports


Detecting and preventing distillation attacks

In this episode, Anthropic’s Nicholas Carlini joins Patrick Gray and James Wilson to talk about advancements in AI-driven vulnerability research and exploit development.

Nicholas’ talk at the recent [un]prompted conference demonstrated how Anthropic’s Opus 4.6 could find and exploit vulnerabilities in popular open source projects. In the short few weeks since then, Anthropic announced a new model that’s already identifying hundreds of bug fixes across critical software. Nicholas talks us through the work he does at Anthropic, what’s possible and the limitations with current frontier models, and where this goes from here.

This episode is also available on YouTube



Show notes

In this episode, James Wilson is joined by entrepreneur and investor Yaniv Bernstein to discuss Anthropic’s Mythos through the lens of startups and growing businesses. Yaniv is Google’s former VP of Engineering, and is former VP Eng and COO at Airtasker. He’s now an investor and advisor to startups and he co-hosts The Startup Podcast.



Show notes

In this episode, James Wilson is joined by Brad Arkin who provides a CISO’s perspective on Anthropic’s Mythos. As former CISO at Adobe, Cisco and Salesforce, Brad’s perspective challenges the notion that finding and fixing exploits makes us safer.



Show notes

In this episode of Risky Business Features, James Wilson chats to professional hacker Jamieson O’Reilly about Anthropic’s Mythos and the impact it could have on offensive security. Jamieson is CEO of DVULN and co-founder of Aether AI. He’s been hacking into organisations for more than a decade, and knows a thing or two about combining AI and offensive security.



Show notes