Today we're talking with one of our favorite engineers, Rafal Wilinski. Rafal has been on the cutting edge of AI development in the last few years as he has led AI teams at Zapier and Vendr.
Rafal walks us through the hard-won lessons about actually integrating AI tools into the applications you're building. One of the hardest things in integrating these AI tools is how to ensure you're getting better and not regressing as you improve your prompts and upgrade your models. He shows how using evals is one part of the story along with deeply investigating customer signals to see how they are or aren't succeeding with AI.
Along the way, we also talk about RAG, his favorite models, his AI development toolset, and why Poland has been killing it lately. Check it out and be sure to follow Rafal if you want to learn more on building with AI.
*Timestamps*
01:10 Start
08:05 Zapier Agents
09:41 How does it differ from previous workflows?
13:10 Does text transform into a workflow?
14:31 Using Zapier's existing tools to power Agents?
15:48 Agents are given a only set of actions from which they can choose
17:45 Browser related stuff?
21:16 Choosing a model
25:28 Different prompts for different models?
40:36 Most often used models
41:45 TypeScript or Python
44:19 Cost
45:35 Less talk about non AI stuff
47:32 On regular devs building with AI
52:18 Protecting against user behaviour
54:25 Future of RAG
56:26 Is AI progress slowing down?
01:00:43 Personal AI tools
01:02:41 Losing understanding of repos?
01:04:33 Would you need Software Background in a few years?
01:06:08 GitHub Actions
01:07:31 Poland
