AI Engineer

@aidotengineer

节目详情

Talks, workshops, events, and training for AI Engineers.

单集更新

Don't Build Agents You Can't Answer For — Addy Osmani

18:26

For his closing keynote, Addy Osmani explores the evolving role of software engineers in the age of AI agents. He argues that as coding tasks become increasingly automated, the true value of an engineer shifts from mere code production to accountability, judgment, and system ownership. https://addyosmani.com/ https://x.com/addyosmani/status/2074927530482835916 Timestamps 0:00 Introduction and the human side of engineering 1:46 Rebundling roles and ownership of systems 2:34 Harnesses, loop engineering, and software factories 3:34 The shift to answerability as an engineering requirement 4:26 Reviewing AI-assisted code and organizational bottlenecks 5:55 Redefining leverage through human judgment 6:15 Alpha, decay, and the role of "taste" 8:49 Defining the modern software engineer 9:50 Risks to avoid: cognitive debt and surrender 11:51 Orchestration tax and system design 12:39 Accountability as the foundation for scaling 13:16 Career math: credibility vs. capability 14:13 High agency and the decision-making ladder 15:13 Defining the boundary between agents and humans 16:13 Operational rule: explain it or don't ship it 17:20 Future outlook: unlocking latent demand

Don't Ship Skills Without Evals — Philipp Schmid, Google DeepMind

00:00

There are thousands of agent skills. Almost none of them are tested. They get vibe-checked with two manual runs, maybe a thumbs-up from a colleague, then shipped. You wouldn't merge code without tests — so why are we shipping skills without evals? This talk covers the full lifecycle of building reliable agent skills: what a skill actually is (and isn't), how to write one that triggers correctly, and how to build a lightweight eval harness that catches failures before your users do. ### Philipp Schmid Staff Engineer · Google DeepMind [X/Twitter](https://x.com/_philschmid) · [LinkedIn](https://www.linkedin.com/in/philipp-schmid-a6a2bb196/) · [Website](https://www.philschmid.de/) · [Blog](https://www.philschmid.de) Philipp Schmid is a Staff Engineer at Google DeepMind working on Gemini and Gemma. His work focuses on helping developers build and benefit from AI responsibly. ## About This Session — [View on the schedule](https://www.ai.engineer/worldsfair/schedule?session=asn_slot_2026_07_01_breakout_track_05_1545_2026_06_26t08_57_00_975z)

Forward Deployed Engineering at Cursor — Pauline Brunet

00:00

# How Forward Deployed Engineering is done at Cursor **Location:** Forward Deployed Engineering / Room 2020 **When:** Day 2 - June 30, 2026 · 11:10am-11:30am ## Speakers ### Pauline Brunet VP, Forward Deployed Engineering · Cursor [LinkedIn](https://www.linkedin.com/in/pauline-brunet/) VP of Forward Deployed Engineering at Cursor. Building the motion and team to help customers adopt Cursor and drive meaningful returns. We configure and co-build alongside customer software and transformation teams. Spent 10 years in AI deployments across enterprises. — [View on the schedule](https://www.ai.engineer/worldsfair/schedule?session=asn_slot_2026_06_30_breakout_track_08_1110_2026_06_20t00_13_36_943z)

WTF Is the Context Layer? The Missing Infrastructure for Production Agents — Prukalpa Sankar

00:00

In the last two years, models have gotten exponentially smarter. Two years ago they couldn't pass the bar. Today, top 1% of test scorers. And yet most agents still can't answer a simple business question correctly. You ship a demo that works. You deploy it. The business abandons it in a month. The missing variable is context: the business definitions, procedural knowledge, and operational norms that make a human expert valuable. Drawing on hundreds of production deployments, Prukalpa Sankar will break down what it actually takes to give agents contextual intelligence — and get them past the demo stage. She'll walk through the architecture of a context layer: how context repos work (versioned, testable, portable), how simulation environments catch failures before deployment, how agent traces compound back into shared context, and why context engineering scales where fine-tuning and prompting don't. She'll also cover why your context needs to be open (MCP, Iceberg, deploy to any framework) — and what happens when it isn't. ### Prukalpa Sankar Founder & Co-CEO · Atlan [X/Twitter](https://x.com/prukalpa) · [LinkedIn](https://www.linkedin.com/in/prukalpa) Prukalpa Sankar is the Founder & Co-CEO of Atlan, the context layer for AI. She's been early to a defining idea of the AI era: context is king. AI systems are only as good as the business context behind the data they rely on. Under her leadership, Atlan has become a Leader in the Gartner Magic Quadrants for both Data & Analytics and Metadata Management, serves 300+ enterprises including Mastercard, GM, JPMorgan Chase, and Nasdaq, and has raised $200M+ from Sequoia, GIC, and Salesforce Ventures. Before Atlan, Prukalpa co-founded SocialCops, the world's largest government data lake powering the UN's SDG monitoring — recognized by the New York Times and the World Economic Forum. She's been featured in Forbes 30 Under 30 and Fortune 40 Under 40.

The Great Loops Debate — Dex Horthy, Geoff Huntley, Ian Livingstone, Greg Pstrucha, @insecure-agents

00:00

Oxford Style Debate: There is, or is not, a delta between the hype behind loops and what actually works in practice. Team No Delta (pro the way we do loops today) The hype around loops is valid and loops work well today in practice. Loops today can be a silver bullet and result in outsize productivity gains, and marks an important step up the autonomy curve towards real software factories. Ian Livingstone Geoff Huntley Team Delta (anti the way we do loops today) There is a delta between the hype behind loops and what actually works in practice. The way we are doing loops today is wrong. Loops are not a silver bullet and there is no magic. The hype is outrunning the discipline "Stop writing loops, start writing control loops." A bare repeat-the-agent loop isn't magic. The leverage comes from the Kubernetes-style reconciliation around it: read current state → read desired state → one incremental change → repeat. Dex's tell when shown a fake loop: "where's the recur condition?" (Jun 21) A software factory can run the mechanical, spec-gated, test-covered slices unattended; it cannot autonomously decide whether it built the right thing. Dex Horthy Greg Pstrucha Main Debate Loop History - Why now as the inflection point and not some of the earlier ones? Loop Anatomy - What makes a good loop? Loop Future - Given what we’re seeing with loop usage now, are we well positioned for software factories? If we can’t use loops well today how do we expect to operate software factories? Appendix Research https://x.com/AnatoliKopadze/status/2068328135611822149?s=20 https://x.com/ericzakariasson/status/2070493377267646797?s=20 https://x.com/MilksandMatcha/status/2069838072515281386?s=20 https://x.com/AnatoliKopadze/status/2070156017262793008?s=20 https://ghuntley.com/loop/ https://ghuntley.com/ralph/ https://www.anthropic.com/institute/recursive-self-improvement Anthropic's Absorption of the Ralph Loop Verifying Agents in GitHub

In Code They Act, In Proof We Trust — Erik Meijer, Leibniz Labs

21:13

AI agents today execute on blind trust, and the failure modes are already in the headlines: a dealership chatbot agreeing to sell a $76,000 Chevy Tahoe for $1, a coding agent wiping a production database during a code freeze, an "agent skill" quietly installing a keylogger on a developer's machine. These are not edge cases. They are the predictable consequence of allowing agents to act without any mechanical guarantee of correctness or safety. Execution is irreversible. You cannot unsend a message, unwire a payment, or un-delete a database. In that regime, permitting an unsafe action costs far more than withholding a safe one, and thus the economically rational choice is to refuse to let agents act on unchecked intent alone. Automind is an agent harness that enforces this discipline by construction. Before any action runs, the agent must submit its execution plan together with a machine-checkable proof of safety and correctness, written in Universalis, a literate logic programming language designed to be read by humans and verified by machines. A small, auditable checker decides whether the plan is allowed to execute. By left-shifting the trust boundary, we no longer have to trust the agent's proposal, or even its proof; only the checker. Policy compliance becomes a static property, established before the first side effect. We can finally demand formal proofs, not vibes, from the agents we deploy. More about Erik: https://x.com/headinthebox and automind: https://spawn-queue.acm.org/doi/pdf/10.1145/3676287 Erik Meijer has spent more than three decades designing programming languages and developer tools that help humans express intent more clearly to machines. His work has influenced languages and technologies including Haskell, Mondrian, Cω, C#, Visual Basic, Dart, Hack, LINQ, and Rx. Today, he is building Universalis, the world's first programming language for AI agents. By combining formal verification with large language models, Universalis aims to make agentic systems safe, transparent, and trustworthy enough for real-world knowledge work. Timestamps 0:00 Introduction and purpose of the talk 1:54 The inherent dangers of AI and accidental file deletion 3:39 The history and impact of LLMs (the "Pandora's box") 5:36 The problem of prompt injection and model safety 7:03 Formal verification and using Lean for safety proofs 10:45 The introduction of tool calls and the leap into chaos 13:59 The "lethal trifecta" of AI risks 14:13 The proposed solution: "air-gapping" the agentic loop 16:36 Refying plans into programs and using Free Monads 19:17 The concept of proof-carrying code and summary

Stop Evaluating Models Like It's the 50s - Alejandro Vidal, Mindmakers

23:35

Psychologists spent the last century learning how to measure something invisible and uncooperative: a human mind. AI evaluation, meanwhile, still scores like it is 1950. Count the right answers, treat every question as equal, trust the percentage (this is Classical Test Theory). We are sitting on decades of measurement theory built for exactly this problem, and we forgot to use it. Borrow it and the picture changes. Item Response Theory (or IRT, the math behind the SAT and the GRE) models every item on top of a shared scale with real error bars. That tells you which of your test items are pure noise, which are optimal, and where the knowledge gaps and unexpected behaviours are. Adaptive testing then measures the same ability with a fraction of the questions, which means private, rotating benchmarks that resist contamination instead of saturating in a month (tinyBenchmarks already hinted you can shrink a benchmark with IRT). It goes further than scoring. The statistical properties of how a model fits the test reveal something a single number never could: data leakage, the moment an agent has quietly seen the answers before. The same machinery that catches a cheating student catches a contaminated benchmark. And instead of one flat score, you get a shape: where the jagged frontier actually is, which abilities are solid and which are luck, so you know which direction to push next. You will leave this talk with a way to build evals that are cheaper, harder to game, and that tell you what your model actually learned instead of how lucky it got. This is not about handing human tests to a model. It is about borrowing a century of how to measure a mind that does not want to be measured. Speakers: - Alejandro Vidal (Mindmakers): Alex Vidal is the founder of Mindmakers, a psychologist and computer scientist who teaches humans to use AI and teaches AIs to teach humans, building adaptive learning technology and the agents, evals and boring infrastructure that keep it from falling over. X/Twitter: https://x.com/dobleio

From fork() to Fleet: Designing an Agent Sandbox Cloud — Abhishek Bhardwaj, OpenAI

44:34

Sandboxes unleash agents by giving them secure, fully functional computers where they can tackle diverse tasks with minimal setup. This talk explores the architectural challenges of building an agent sandbox cloud. We compare runtime isolation technologies and their trade-offs, examine persistence and storage as the next major unlock for agent capabilities, and discuss the key decisions involved in orchestrating and scaling sandboxes. Abhishek Bhardwaj works on Agent and Reinforcement Learning Infrastructure at OpenAI. He builds systems that enable large-scale model training in RL environments, as well as secure and scalable cloud sandboxes for OpenAI’s agents. Before joining OpenAI, he created Arrakis, an open-source sandbox for AI agents. Previously, he worked at Google on ChromeOS and foundational microVM technologies, and at Replit on core infrastructure and early versions of Replit Agent. Timestamps 0:00 Introduction and motivation for AI agent sandboxes 1:31 Why AI models need tools and execution environments 3:51 Product-side challenges: Security and the need for sandboxing 6:44 Comparing research vs. product sandbox requirements 8:24 Overview of the three pillars: Runtime, Persistence, and Orchestration 9:05 First principles of Linux execution: System calls and security vectors 11:15 Evaluating fork() and exec models 12:06 Understanding containers: Namespaces and cgroups 16:26 GVisor as an application kernel alternative 18:29 Hardware-level virtualization (Virtual Machines) 20:34 How VMMs (Virtual Machine Monitors) work with KVM 23:16 Evolution of modern VMMs and Rust-based safety 24:32 What defines a "microVM"? 25:43 Orchestrating microVMs via APIs 27:16 Trade-offs of microVMs (performance vs. security) 30:05 The need for persistent storage in agent sandboxes 31:40 Use cases for persistence: Reliability, long-running tasks, and research 34:36 Design choices for disk snapshotting 36:03 First principles of Linux block storage and file systems 37:25 Implementing always-on vs. explicit persistence 41:20 Scaling and orchestrating sandboxes at fleet level

The Prime Intellect Stack — Will Brown, Prime Intellect

46:52

Deep dive into Prime Intellect's open-source ecosystem of post-training tools, including the verifiers and prime-rl libraries, as well as the Lab platform for self-serve training and inference. Speaker: Will Brown — Research Lead, Prime Intellect Will Brown leads Applied Research at Prime Intellect and builds open research infrastructure to enable every company to train, deploy, and self-improve their own frontier agentic models. He holds a PhD in Computer Science from Columbia University. X: https://x.com/willccbb LinkedIn: https://www.linkedin.com/in/willcb/ GitHub: https://github.com/willccbb Website: https://willcb.com TImestamps 0:00 Introduction and Overview of Prime Intellect 4:20 Defining the Environment in Post-Training 9:33 Decomposing Environments: Tasks, Harnesses, and Runtimes 12:46 Verifiers V1: The New Modular Pattern 17:46 Rewards, Metrics, and Group-Level Rewards 20:25 Tooling, User Simulators, and MCP Integration 22:00 The Interception Server Pattern 24:13 Trace Graphs and Handling Tokenization 25:35 The Renderers Library for Chat Templates 29:20 Primaril: Asynchronous Reinforcement Learning 38:02 Customizing Training Algorithms and Losses 42:35 The Lab Platform and Hosted Training

RLM: Recursive Language Models for Large Codebases - Shashi, Superagentic AI

17:27

Large codebases break coding agents: they lose the architecture and drown in tool output as context grows. This talk introduces Recursive Language Models (RLM) from a MIT paper a pattern that loads the repo into a programmable REPL where the model writes code to inspect it and recursively delegates focused sub-questions via llm_query. With a live demo on RLM Code (independent, unofficial), you'll see the loop run end to end on local and cloud models, with a fully inspectable trajectory. Speakers: - Shashi (Superagentic AI): Building tools and frameworks for AI Agents X/Twitter: https://x.com/Shashikant86 LinkedIn: https://www.linkedin.com/in/shashikantjagtap/ GitHub: https://github.com/Shashikant86

The AI bugpocalypse is here. Now what? - Jack Cable, Corridor

19:44

Something shifted in the past year that most security teams haven't fully reckoned with yet: AI models can now find serious vulnerabilities in production code, at scale, with minimal human skill required. Not in toy examples. In libraries that have been reviewed hundreds of times by the best researchers in the world. Jack Cable, Co-Founder and CEO of Corridor, will walk through what this means for the 80% of organizations that have never had to defend against adversaries doing in-house vuln discovery: where the real exposure is, what the available playbooks actually get right, and what concrete steps security teams can take right now to reduce their blast radius before open-weight models make this everybody's problem. Speakers: - Jack Cable (Corridor): Jack Cable is a hacker who serves as the Co-Founder and CEO at Corridor, the security platform for AI coding. X/Twitter: https://x.com/jackhcable LinkedIn: https://www.linkedin.com/in/jackcable/

Semantic Blindness: 500,000 Sensors Confused an LLM - Raahul Singh & Vanč Levstik, Phaidra

16:25

You cannot solve a combinatorial engineering problem with a next token prediction engine. We learned this the hard way. Modern LLMs can write code, summarize research papers, and reason across massive datasets. But what happens when you connect them to mission-critical physical infrastructure with 50,000 live sensors, deterministic dependencies, and real-world thermodynamic constraints? We deployed state-of-the-art LLMs to manage real time operations within industrial and AI factory environments to tackle root cause analysis, alarm triage, and operational decision support. What we discovered was a fundamental architectural mismatch between probabilistic language models and deterministic engineering systems. In this talk, we introduce a failure mode we call Semantic Blindness: the inability of general-purpose LLMs to maintain structural awareness of physical systems, even when provided enormous amounts of context. This talk dissects three specific failure modes we encountered — and why each one exposes a gap in how the industry thinks about scaling LLMs to complex systems: 1) The Topology Trap. Vector embeddings don't understand pipes, wires, or physical causality. Sensor_445_Temp is just a string. But in reality, it's attached to Valve B, which controls coolant to Generator 3. 2) The Illusion of Scale. At a small scale, dumping 100 sensors into the context window works surprisingly well. It’s a reasonable solution and it holds up. At 500,000 sensors, the same approach collapses. It creates new problems: attention degrades, critical anomalies get buried in the middle, and latency spikes to unusable levels for real-time response. 3) The Repetition Kill Switch. Industrial tag naming conventions are nearly identical at scale. Feeding the same naming conventions across hundreds of variants, you’ll trip the model’s repetition penalty. It thinks it’s stuck in a degenerate loop and it will literally stop. The data is correct. The model just can’t handle it. Rather than focusing on prompt engineering tricks, this session explores the architectural patterns required to make AI reliable in real-world engineering environments. We’ll present a practical hybrid design approach that combines: - semantic ontologies, - deterministic query systems, - structured synthesis layers, - and LLM orchestration architectures purpose-built for operational infrastructure. Attendees will leave with a clear understanding of: - why naive RAG architectures fail in industrial environments, - how to design AI systems that respect physical reality, - how to make LLMs work reliably against massive scale of data - and what the next generation of “AI-enabled intent resolution” actually looks like beyond semantic search. This session is designed for senior AI engineers, infrastructure architects, CTOs, and technical leaders building AI systems that must operate reliably under real-world constraints — not just benchmark well in demos. Speakers: - Raahul Singh (Phaidra): Raahul Singh is a Staff AI Research Engineer at Phaidra and the lead architect behind the company's agentic AI platform for data center infrastructure. LinkedIn: https://www.linkedin.com/in/raahulsingh42 GitHub: https://github.com/raahul-singh - Vanč Levstik (Phaidra): Vanč Levstik is a Senior Engineering Manager at Phaidra and leading the teams developing Phaidra Prism

The Agentic Web and the Bazaar Era of AI - Ramesh Raskar, MIT Media Lab

12:11

The AI agent industry is currently focused on memory, orchestration, enterprise deployment, and tooling. But these are the first steps toward a larger transformation: the emergence of the Agentic Web. Today’s ecosystem resembles the early days of AOL: closed platforms, proprietary agent stores, and siloed orchestration layers. The next era of AI agents will require open infrastructure that allows agents to discover, transact, and co-learn across organizational boundaries. This talk explores three layers of the Agentic Web. First, the Discovery Layer: agents will require discovery infrastructure analogous to AltaVista or Google—but for agents instead of webpages. The challenge is no longer PageRank, but “AgentRank”: how agents are discovered, trusted, verified, and coordinated across the open web. This creates the need for ICANN- and W3C-like governance and standards for agents. Second, the Commerce Layer: what is the dollar value of intelligence? Agents will pay for reasoning, inference, memory, capabilities, and context through emerging “knowledge pricing” markets. Intelligence itself will be discovered, priced before use, coordinated among untrusted entities, and delivered in new ways. Third, the Bazaar Layer: the last 14 years were about machine learning. The next decade will be about machine co-learning. Speakers: - Ramesh Raskar (MIT Media Lab): Ramesh Raskar is an Associate Professor at the MIT Media Lab and founding architect of NANDA whose pioneering work spans distributed AI agent architectures, health technology, and computational imaging, holding 100+ US patents and earning honors including the National Academy of Inventors award (2024), the Lemelson Award (2016), and the ACM SIGGRAPH Achievement Award (2017), alongside research roles at Google [X], Apple, and Facebook and the co-founding or advising of several companies. LinkedIn: https://www.linkedin.com/in/raskar

A Song of Types and Agents - Roberto Stagi, Ratel

14:16

Python ruled unchallenged for a decade, sitting comfortably on the AIron Throne. But a quiet rebellion is brewing: the entire stack that actually deploys AI agents in production runs on npm, not pip. This lightning talk is an opinionated, slightly unhinged tour of how TypeScript is taking over the AI throne, why this happened and how you can prepare for it. Speakers: - Roberto Stagi (Ratel): Roberto is the CTO & Co-Founder of Ratel, context layer for AI Agents, EU-Ambassador at AI Socratic, and deep into the mission of making context engineering simple for everyone. X/Twitter: https://x.com/rstagi_ LinkedIn: https://linkedin.com/in/rstagi GitHub: https://github.com/rstagi

ReviewDebt: a practical framework for scoring every pull request — Sachin Gupta, Ebay

25:00

Coding agents ship PRs faster than humans can trust them. The gap is filling up with a debt nobody is measuring — and it's about to swallow your engineering velocity. Every team in 2026 measures coding agents the same way: PR count, lines of code, cycle time, developer NPS. None of those see the real cost — bloated diffs, weak tests, ambiguous rationale, ownership sprawl, and human reviewers spending more time verifying AI code than they used to spend writing their own. This talk introduces ReviewDebt: a practical framework for scoring every pull request on the hidden review burden it creates. The scoring is deterministic — diff size, test-coverage delta, ownership spread, generated-code smells, evidence and rationale gaps — so the number is defensible in a real engineering review. We'll walk three real PRs side-by-side (clean human PR, high-debt AI PR, refactored AI PR), watch the scoring play out signal by signal, and look at a 90-day dashboard from a production backend org where review debt climbs in lockstep with AI-PR share. Speakers: - Sachin Gupta: Sachin Gupta is a Staff Software Engineer with 15+ years building backend platforms at internet scale, currently focused on the runtime trust boundaries that LLM coding agents blur and the creator of HeapLens, a Java heap analyzer extension used in 50+ countries. LinkedIn: https://www.linkedin.com/in/guptasachin1/ GitHub: https://github.com/sachinkg12

Stop Evaluating Models Like It's the 50s - Alejandro Vidal, Mindmakers

23:00

Psychologists spent the last century learning how to measure something invisible and uncooperative: a human mind. AI evaluation, meanwhile, still scores like it is 1950. Count the right answers, treat every question as equal, trust the percentage (this is Classical Test Theory). We are sitting on decades of measurement theory built for exactly this problem, and we forgot to use it. Borrow it and the picture changes. Item Response Theory (or IRT, the math behind the SAT and the GRE) models every item on top of a shared scale with real error bars. That tells you which of your test items are pure noise, which are optimal, and where the knowledge gaps and unexpected behaviours are. Adaptive testing then measures the same ability with a fraction of the questions, which means private, rotating benchmarks that resist contamination instead of saturating in a month (tinyBenchmarks already hinted you can shrink a benchmark with IRT). It goes further than scoring. The statistical properties of how a model fits the test reveal something a single number never could: data leakage, the moment an agent has quietly seen the answers before. The same machinery that catches a cheating student catches a contaminated benchmark. And instead of one flat score, you get a shape: where the jagged frontier actually is, which abilities are solid and which are luck, so you know which direction to push next. You will leave this talk with a way to build evals that are cheaper, harder to game, and that tell you what your model actually learned instead of how lucky it got. This is not about handing human tests to a model. It is about borrowing a century of how to measure a mind that does not want to be measured. Speakers: - Alejandro Vidal (Mindmakers): Alex Vidal is the founder of Mindmakers, a psychologist and computer scientist who teaches humans to use AI and teaches AIs to teach humans, building adaptive learning technology and the agents, evals and boring infrastructure that keep it from falling over. X/Twitter: https://x.com/dobleio

What Does Done Even Mean? Agents and Paperclip's Liveness Model - Dotta, Paperclip

7:14

What does “done” mean when agents can produce more work than humans can possibly review? This talk argues that the future of agentic work is not just faster output, but a stronger trust protocol: systems where “done” means an artifact has met a stated standard, carries evidence, has been checked by the right verifier, assigns ownership of remaining risk, and clearly authorizes the next action. Drawing from Paperclip’s liveness model, it shows how teams can avoid approval theater, keep work moving, route review by risk, and turn agent completion from a vague confidence signal into something others can safely build on. Speakers: - Dotta (Paperclip): Dotta is the creator of Paperclip, the Open-source app for zero human companies X/Twitter: https://x.com/dotta GitHub: https://github.com/cryppadotta

Claws Out: Securing and Building with OpenClaw - Nick Taylor, Pomerium

17:12

Running OpenClaw without hardening access to it is a bad idea. We'll cover how I secured my OpenClaw, McClaw, contributed trusted-proxy auth mode to the OpenClaw project, and how I use it to build tools. We're going to build something live during the talk using OpenClaw, the same way I built Clawspace, a browser-based file explorer/editor for your OpenClaw workspace. feat(gateway): add trusted-proxy auth modegiithub.com/nickytonline/clawspace, a browser-based file explorer/editor for an OpenClaw workspace.github.com/pomerium/pomerium, an open core Identity-Aware Proxy

Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes, AWS

55:19

AI agents that book 15 guests in a 10-person room. Agents that fabricate statistics when data doesn't exist. Agents that pick wrong tools from 29 options, wasting $47 in tokens. These aren't prompt engineering failures, they're architectural limitations that need structural solutions. This hands-on workshop covers 5 research-backed techniques to prevent agent hallucinations: 1. Graph-RAG (Neo4j) - Replace vector similarity guessing with precise entity relationships. Result: 73% fewer fabricated statistics. 2. Semantic Tool Selection - Filter 29 tools to the relevant 5 using embeddings. Result: 89% token reduction, accurate tool selection. 3. Multi-Agent Validation - Executor-Validator-Critic swarms catch fabrications through cross-checking. Result: 92% detection rate. 4. Neurosymbolic Guardrails - Framework-enforced rules (lifecycle hooks) that agents cannot bypass. Result: Zero business rule violations. 5. Agent Steering - Guide agents to self-correct instead of blocking them. Result: Task completion without hard failures. Each demo includes live code, before/after metrics, and failure case analysis. Final module shows production deployment. You'll walk away with working Python implementations, a decision framework for when to apply each technique, and an open-source repository adaptable to your domain. code: https://github.com/elizabethfuentes12/why-agents-fail-sample-for-amazon-agentcore Speakers: - Elizabeth Fuentes (AWS): Elizabeth Fuentes is a developer advocate and AI engineer focused on what makes agents fast, cheap, and correct in production. She turns failure modes (hallucination, token blowups, context overflow, lost memory) into named, measurable fixes, each backed by a runnable demo and before/after numbers. Her work covers the architectural decisions behind reliable agents: context offloading, the split between conversation and data memory, semantic versus exact-reference retrieval, guardrails, and agent evaluation. With 107+ published technical articles and a Master's in Data Science, she shares production agent patterns across English and Spanish developer communities, and likes turning complex concepts into something anyone can learn. X/Twitter: https://x.com/ElizabethFue12 LinkedIn: https://www.linkedin.com/in/lizfue/ GitHub: https://github.com/elizabethfuentes12

The Factory That Dreams: 39 AI Agents, No Framework - Rushabh Doshi, Machinecraft

9:58

Most AI demos are built around a toy workflow. Ira was built around a factory. This talk is the story of how a third-generation Indian machinery company built a multi-agent operating system that helps run sales, business development, recruitment, quoting, marketing, production context, email workflows, and organizational memory. Ira is not a chatbot and not a wrapper around a single framework. It is a company brain: 39 bounded specialist agents, Athena as orchestrator, a 17-stage request pipeline, Qdrant for document memory, Neo4j for relationships, Mem0 for long-term semantic memory, Postgres for CRM and recruiting data, Redis for coordination, and Cursor as the operating cockpit. The deeper lesson is architectural: companies do not need generic AI assistants. They need digital brains grounded in their own documents, relationships, processes, and values. I will show how Ira ingests company files through a "digestive system", routes work through a pantheon of agents, verifies claims through immune-system style guardrails, learns through memory and corrections, and "dreams" through a nightly consolidation cycle. I will also explain why we gave Ira a SOUL.md: a philosophical constitution based on Anekantavada, Syadvada, Svadharma, and operational truthfulness. The talk ends with the Fork My Brain thesis: the right way to build company AI is not to sell another SaaS dashboard. It is to send a special-ops AI team inside a company for a week, map the business from the inside out, ingest the right files into Qdrant and Neo4j, wire the operational databases, and leave behind a forkable digital brain that employees can run through Cursor and LLMs. Speakers: - Rushabh Doshi (Machinecraft / Fork My Brain): Rushabh Doshi builds and operates Ira, a multi-agent AI operating system for Machinecraft, an Indian thermoforming machinery manufacturer, combining Cursor, LLMs, retrieval, memory, and business operations into one living company brain. LinkedIn: https://www.linkedin.com/in/rdd0101/ GitHub: https://github.com/doshirush1901

Chat and citations won't save your vertical AI - Atul Ramachandran, Filed Inc

15:12

Most vertical SaaS teams are doing the same things: chasing higher accuracy, building better model harnesses, shipping more features. And their customers are saying the same things: the AI got this wrong, it hallucinated, the accuracy is not good enough. So teams go back and push the numbers higher. We did the same at Filed. We built AI data entry for tax firms and hit 80%+ accuracy against an industry baseline of 50-60%. Many users still complained. Same model, same stack, different outcomes. So we dug in. The unhappy customers were not experiencing worse AI. They were reverse-engineering everything we produced. We had not removed work from their day. We had just changed its shape. Chat interfaces and citation trails feel like the fix. They are not. They hand the verification burden back to the user with extra steps. Accuracy %s are the score you get after the game is already over. The complaints, the hallucination reports, all of it: symptoms of the same underlying problem. Users are still holding the bag, and when they are, every error is catastrophic. When we started building the real fix, we realised the coding world had already been here. Early coding AI dumped a full function and asked engineers to review 200 lines. Same problem. The fix was not a better model. It was Copilot in the editor, not a separate tab. The planner pattern instead of dumping full outputs. Skills and memory that compound with every use. We reached the same conclusion independently, from taxes. This talk is those three patterns and what they look like in a vertical SaaS product. Go where the work is. Most users will try a new feature. Almost none will adopt a new platform. AI has to live inside existing workflows, not alongside them. 1000 feet first. The right unit of work matters more than accuracy on any given unit. Start at the macro level, let users orient, then drill down. Each level is small enough to verify fast. Users stop auditing and start deciding. Skills over models. Every edge case is a skill waiting to be encoded, not a model failure. Turn real usage into institutional knowledge that makes every future user better off. The specific lessons are from taxes. The pattern is universal. Speakers: - Atul Ramachandran (Filed Inc): Atul has cofounded multiple startups and is currently CTO of Filed, which has raised over $17M to build AI infrastructure for tax firms. He is an active open source contributor in the JavaScript ecosystem, with projects like NodeGui. He is currently based out of Stockholm, Sweden. X/Twitter: https://x.com/a7ulr LinkedIn: https://www.linkedin.com/in/atulanand94/ GitHub: https://github.com/a7ul

State of the Union: Why Local, Why Now — NVIDIA, Osmantic, Roboflow, EXO Labs, @matthew_berman

44:29

Local AI has crossed from interesting to useful, driven by stronger open models, better hardware, and a maturing ecosystem for running intelligence outside the cloud. This panel explores what that shift unlocks for sovereignty, defense, regulated industries, privacy, cost, and resilience, and why open-source AI may be central to who benefits from the next wave of intelligence. Moderator: Nader Khalil (NVIDIA). Panelists: Joseph Nelson (Roboflow), Alex Cheema (Exo Labs), Ahmad Osman (r/LocalLLaMA). Timestamps * **0:00:00** - Introduction to the Local AI Summit * **0:02:34** - Panelist Introductions * **0:04:36** - Defining the Inflection Point in Local AI * **0:11:18** - Lessons from Vision AI for Language Models * **0:13:42** - The Shift to a Multi-Model World * **0:16:34** - Sovereignty and Control in Enterprise AI * **0:19:30** - Optimizing Performance on Specialized Hardware * **0:22:18** - The Culture of "Swarming" and Collaborative Innovation * **0:26:03** - Infrastructure Needs for Future Growth * **0:27:07** - Closing the Gap for Mainstream Users * **0:30:52** - The Difficulty of Specializing Models * **0:34:36** - Distillation and Real-World Deployment Examples * **0:39:51** - Q&A: Addressing the Big Open Problems in Local AI * **0:41:59** - The Role of Open Source Advocacy

Every Solo Agent Builder Eventually Reinvents a Worse Version of CI/CD - Sumaiya Shrabony

10:51

If you build agents alone long enough, you will independently reinvent five things software engineering solved decades ago. A way to test whether your agent's output is still correct after you changed something. A way to run it on a schedule and know if it failed. A way to prevent one skill's schema change from silently breaking three downstream skills. A way to roll back when today's run produces garbage. A way to validate outputs before they hit production. You just reinvented regression testing, cron monitoring, contract testing, version control, and staging. Badly. Without realizing it. The dangerous failure in an agent system is not bad output. Bad output is easy to catch. The dangerous failure is a polished artifact that looks ready but violates a production contract: it uses the wrong voice patterns, makes an unverified claim, repeats an old angle, and gets labeled "READY TO PUBLISH" anyway. That is the agent equivalent of shipping because the code compiled, even though the tests never ran. This talk uses a real, open-source 19-skill Claude Code agent system (github.com/safrin96/agentic-content-system) as the case study. Through an interactive live demo, I show three ways an agent system silently lies to you and what a boundary looks like that catches it. The takeaway is simple: the infrastructure gap in the agent ecosystem is not another framework. It is the equivalent of what CI/CD gave software teams in 2015, a standard, boring, reliable way to test, deploy, and roll back agent behavior. Before you add another agent, add one boundary. Speakers: - Sumaiya Shrabony: Sumaiya Shrabony is a Technical Program Manager, enterprise AI practitioner, and content creator across LinkedIn, Instagram (@thedata_ai.girl), and Substack (Ground Truth) building toward thought leadership at the intersection of enterprise data infrastructure, AI adoption, and the immigrant-in-tech experience. LinkedIn: https://www.linkedin.com/in/sumaiya-shrabony GitHub: https://github.com/safrin96

Develop at Idea Velocity - Jeffrey Lee-Chan, Snapchat

15:28

The biggest gap in production AI agent systems is not the model—it's the harness. After 1,000 hours of orchestrating autonomous fleets under human direction, the pattern is unmistakable: agents that finish complex tasks on the first run routinely fail on subsequent iterations because the surrounding loop lacks persistent memory and contextual guardrails. In this talk, I dissects the key multi-agent primitives required to turn raw models into deterministic teammates. Moving beyond simple API wrappers, we explore how separating your stack into distinct "Agent Orchestrator Managers" and specialized workers prevents low-level implementation bias. Using concrete examples from production systems, we will walk through real-time terminal routing via CMUX, analyze the token-burn tradeoffs between leading models, and look under the hood of high-context consumer applications like WorldAI and Consensus ML. You will walk away with a practical architectural checklist you can drop directly into your own agent infrastructure on Monday morning. Once you're setup you can truly develop at idea velocity ie. natural language to code to automated iteration to evidence produced to human review where human interaction in the middle is pushed to the beginning or end allowing improved parallelization. Speakers: - Jeffrey Lee-Chan (Snapchat): Most teams use AI tools wrong — humans still on the critical path. I build parallel multi-agent harnesses so one engineer directs 10–20 coding agents instead of becoming the rate limiter. X/Twitter: https://x.com/jleechan2015 LinkedIn: https://www.linkedin.com/in/jeffrey-lee-chan/ GitHub: https://github.com/jleechanorg

From Writing Code to Designing Systems: How the Developer Role is Changing — Chris Noring, Microsoft

23:05

For decades, developers have been valued primarily for how much code they could write and how quickly they could write it. That model no longer scales. As AI becomes a first-class collaborator, the bottleneck is no longer syntax or implementation speed—it’s clarity of intent, architectural thinking, and the ability to coordinate work across many autonomous contributors. Today’s challenge is not "How do I write this code?" but "How do I ensure this system is built correctly, consistently, and to company standards—across dozens of moving parts?" Without structure, AI-assisted development risks fragmentation: inconsistent patterns, duplicated logic, and solutions that technically work but fail architectural, security, or organizational expectations. This talk introduces a new mental model for modern development: the developer as planner, system designer, and orchestrator of agents. Using GitHub Copilot, GitHub Copilot CLI, and custom Copilot agents driven by agents.md, we’ll explore how developers can decompose large problems, delegate implementation to specialized AI agents, and encode standards, constraints, and intent directly into the workflow. Instead of prompting ad-hoc, we define explicit instructions per layer—frontend, backend, infrastructure, testing—so every agent builds the right thing in the right way. The result is not less control, but more leverage: faster delivery, higher consistency, and systems that reflect deliberate design rather than accidental outcomes. What you’ll learn: How the developer role is shifting from code producer to system designer, planner, and agent orchestrator How to structure projects for agent-driven development, using GitHub Copilot CLI, Copilot Chat, and agents.md to encode standards and intent How to ensure architectural consistency and quality at scale by giving agents clear responsibilities, constraints, and ownership boundaries

Special Topics in Kernels, RL, Reward Hacking in Agents — Daniel Han, Unsloth

00:00

An advanced seminar (good prerequisites: Daniel's 2024 and 2025 hit AIE workshops, but all are welcome!) PLS WATCH: https://www.youtube.com/@aiDotEngineer/search?query=daniel%20han

Design Patterns for AI Trust: Juries, Libraries, and Agent Tiers — Alex Bauer, Upside.tech

17:09

A couple of years ago, everyone worried about AI hallucinating. We rarely hear that word anymore, but it’s just because the problem grew up. Today, your AI still doesn’t know how to say “I’m not sure.” Instead, it hands you a revenue number that’s wrong in ways that look exactly like being right. The good news is we already solved this once, for people: you onboard a new hire so they understand your business; you put subjective, high-stakes calls in front of more than one set of eyes. This talk walks through patterns we run at Upside, including a librarian every agent consults before it acts, a jury-and-judge model for the questions a single pass can’t be trusted to answer, and knowing when the model itself is just too dumb for the job. Live demos and real failures included. Speaker: Alex Bauer - (https://Upside.tech) Alex Bauer is co-founder of Upside, the data layer for GTM engineers. He spent 2016–2024 at Branch as the public voice of mobile attribution and deep-linking. He now builds the clean, normalized GTM data that revenue teams point Claude and Cursor at to answer "what actually happened, and did it work?" X: https://x.com/alexdbauer LinkedIn: https://www.linkedin.com/in/alexdbauer/ GitHub: https://github.com/aeromusek Website: https://alexbauer.net/

On AI and Knowledge — Pablo Castro, Distinguished Engineer & CVP for AI Knowledge, Microsoft

00:00

Pablo Castro explores AI and knowledge systems for building better applications and agents. Speaker: Pablo Castro — Distinguished Engineer and CVP, Microsoft Pablo leads the AI Knowledge team in Microsoft's CoreAI division, where he focuses on state-of-the-art information understanding and retrieval systems for AI applications and agents, including Foundry IQ, Azure AI Search, and Azure Content Understanding. LinkedIn: https://www.linkedin.com/in/pabloc

Understanding is the new bottleneck — Geoffrey Litt, Notion

19:33

Autonomous loops are hot, but the reality is that most agentic tasks still require human judgement. And to guide your agents well, it's not enough to just verify correctness -- you actually need to understand the work they're doing. In this talk, I'll share some techniques for staying in the loop and efficiently developing understanding, combining old ideas from education and cognitive science with modern agent capabilities. You'll walk away with some practical tips for moving faster with agents by understanding more, not less.