• 674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation)

  • Apr 28 2023
  • Length: 5 mins
  • Podcast

674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation)

Listen for free

View show details

  • Summary

  • Models like Alpaca, Vicuña, GPT4All-J and Dolly 2.0 have relatively small model architectures, but they're prohibitively expensive to train even on a small amount of your own data. The standard model-training protocol can also lead to catastrophic forgetting. In this week's episode, Jon explores a solution to these problems, introducing listeners to Parameter-Efficient Fine-Tuning (PEFT) and the leading approach: Low-Rank Adaptation (LoRA).Additional materials: www.superdatascience.com/674Interested in sponsoring a SuperDataScience Podcast episode? Visit JonKrohn.com/podcast for sponsorship information.
    Show more Show less
Science
Show more Show less

What listeners say about 674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation)

Average customer ratings

Reviews - Please select the tabs below to change the source of reviews.

Audible.com reviews

Amazon reviews
No Reviews are Available