Training Data By cover art

Training Data

By: Sequoia Capital
Listen for free

  • Summary

  • Join us as we train our neural nets on the theme of the century: AI. Sonya Huang, Pat Grady and more Sequoia Capital partners host conversations with leading AI builders and researchers to ask critical questions and develop a deeper understanding of the evolving technologies—and their implications for technology, business and society. The content of this podcast does not constitute investment advice, an offer to provide investment advisory services, or an offer to sell or solicitation of an offer to buy an interest in any investment fund.
    Show more Show less
activate_primeday_promo_in_buybox_DT
Episodes
  • Klarna CEO Sebastian Siemiatkowski on Getting AI to Do the Work of 700 Customer Service Reps

    Klarna CEO Sebastian Siemiatkowski on Getting AI to Do the Work of 700 Customer Service Reps
    Jul 23 2024
    In February, Sebastian Siemiatkowski boldly announced that Klarna’s new OpenAI-powered assistant handled two thirds of the Swedish fintech’s customer service chats in its first month. Not only were customer satisfaction metrics better, but by replacing 700 full-time contractors the bottom line impact is projected to be $40M. Since then, every company we talk to wants to know, “How do we get the Klarna customer support thing?” Co-founder and CEO Sebastian Siemiatkowski tells us how the Klarna team shipped this new product in record time—and how embracing AI internally with an experimental mindset is transforming the company. He discusses how AI development is proliferating inside the company, from customer support to marketing to internal knowledge to customer-facing experiences. Sebastian also reflects on the impacts of AI on employment, society, and the arts while encouraging lawmakers to be open minded about the benefits. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital Mentioned in this episode: DeepL: Language translation app that Sebastian says makes 10,000 translators in Brussels redundant The Klarna brand: The offbeat optimism that the company is now augmenting with AI Neo4j: The graph database management system that Klarna is using to build Kiki, their internal knowledge base 00:00 Introduction 01:57 Klarna’s business 03:00 Pitching OpenAI 08:51 How we built this 10:46 Will Klara ever completely replace its CS team with AI? 14:22 The benefits 17:25 If you had a policy magic wand… 21:12 What jobs will be most affected by AI? 23:58 How about marketing? 27:55 How creative are LLMs? 30:11 Klarna’s knowledge graph, Kiki 33:10 Reducing the number of enterprise systems 35:24 Build vs buy? 39:59 What’s next for Klarna with AI? 48:48 Lightning round
    Show more Show less
    52 mins
  • Reflection AI’s Misha Laskin on the AlphaGo Moment for LLMs

    Reflection AI’s Misha Laskin on the AlphaGo Moment for LLMs
    Jul 16 2024
    LLMs are democratizing digital intelligence, but we’re all waiting for AI agents to take this to the next level by planning tasks and executing actions to actually transform the way we work and live our lives. Yet despite incredible hype around AI agents, we’re still far from that “tipping point” with best in class models today. As one measure: coding agents are now scoring in the high-teens % on the SWE-bench benchmark for resolving GitHub issues, which far exceeds the previous unassisted baseline of 2% and the assisted baseline of 5%, but we’ve still got a long way to go. Why is that? What do we need to truly unlock agentic capability for LLMs? What can we learn from researchers who have built both the most powerful agents in the world, like AlphaGo, and the most powerful LLMs in the world? To find out, we’re talking to Misha Laskin, former research scientist at DeepMind. Misha is embarking on his vision to build the best agent models by bringing the search capabilities of RL together with LLMs at his new company, Reflection AI. He and his cofounder Ioannis Antonoglou, co-creator of AlphaGo and AlphaZero and RLHF lead for Gemini, are leveraging their unique insights to train the most reliable models for developers building agentic workflows. Hosted by: Stephanie Zhan and Sonya Huang, Sequoia Capital 00:00 Introduction 01:11 Leaving Russia, discovering science 10:01 Getting into AI with Ioannis Antonoglou 15:54 Reflection AI and agents 25:41 The current state of Ai agents 29:17 AlphaGo, AlphaZero and Gemini 32:58 LLMs don’t have a ground truth reward 37:53 The importance of post-training 44:12 Task categories for agents 45:54 Attracting talent 50:52 How far away are capable agents? 56:01 Lightning round Mentioned: The Feynman Lectures on Physics: The classic text that got Misha interested in science. Mastering the game of Go with deep neural networks and tree search: The original 2016 AlphaGo paper. Mastering the game of Go without human knowledge: 2017 AlphaGo Zero paper Scaling Laws for Reward Model Overoptimization: OpenAI paper on how reward models can be gamed at all scales for all algorithms. Mapping the Mind of a Large Language Model: Article about Anthropic mechanistic interpretability paper that identifies how millions of concepts are represented inside Claude Sonnet Pieter Abeel: Berkeley professor and founder of Covariant who Misha studied with A2C and A3C: Advantage Actor Critic and Asynchronous Advantage Actor Critic, the two algorithms developed by Misha’s manager at DeepMind, Volodymyr Mnih, that defined reinforcement learning and deep reinforcement learning
    Show more Show less
    1 hr and 7 mins
  • Microsoft CTO Kevin Scott on How Far Scaling Laws Will Extend

    Microsoft CTO Kevin Scott on How Far Scaling Laws Will Extend
    Jul 9 2024
    The current LLM era is the result of scaling the size of models in successive waves (and the compute to train them). It is also the result of better-than-Moore’s-Law price vs performance ratios in each new generation of Nvidia GPUs. The largest platform companies are continuing to invest in scaling as the prime driver of AI innovation. Are they right, or will marginal returns level off soon, leaving hyperscalers with too much hardware and too few customer use cases? To find out, we talk to Microsoft CTO Kevin Scott who has led their AI strategy for the past seven years. Scott describes himself as a “short-term pessimist, long-term optimist” and he sees the scaling trend as durable for the industry and critical for the establishment of Microsoft’s AI platform. Scott believes there will be a shift across the compute ecosystem from training to inference as the frontier models continue to improve, serving wider and more reliable use cases. He also discusses the coming business models for training data, and even what ad units might look like for autonomous agents. Hosted by: Pat Grady and Bill Coughran, Sequoia Capital Mentioned: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, the 2018 Google paper that convinced Kevin that Microsoft wasn’t moving fast enough on AI. Dennard scaling: The scaling law that describes the proportional relationship between transistor size and power use; has not held since 2012 and is often confused with Moore’s Law. Textbooks Are All You Need: Microsoft paper that introduces a new large language model for code, phi-1, that achieves smaller size by using higher quality “textbook” data. GPQA and MMLU: Benchmarks for reasoning Copilot: Microsoft product line of GPT consumer assistants from general productivity to design, vacation planning, cooking and fitness. Devin: Autonomous AI code agent from Cognition Labs that Microsoft recently announced a partnership with. Ray Solomonoff: Participant in the 1956 Dartmouth Summer Research Project on Artificial Intelligence that named the field; Kevin admires his prescience about the importance of probabilistic methods decades before anyone else. 00:00 - Introduction 01:20 - Kevin’s backstory 06:56 - The role of PhDs in AI engineering 09:56 - Microsoft’s AI strategy 12:40 - Highlights and lowlights 16:28 - Accelerating investments 18:38 - The OpenAI partnership 22:46 - Soon inference will dwarf training 27:56 - Will the demand/supply balance change? 30:51 - Business models for data 36:54 - The value function 39:58 - Copilots 44:47 - The 98/2 rule 49:34 - Solving zero-sum games 57:13 - Lightning round
    Show more Show less
    1 hr
Economics
Show more Show less

What listeners say about Training Data

Average customer ratings

Reviews - Please select the tabs below to change the source of reviews.

Audible.com reviews

Amazon reviews
No Reviews are Available