New technical deep dive course: Generative AI Foundations on AWS

Generative AI Foundations on AWS is a brand new technical deep dive course that offers you the conceptual fundamentals, sensible recommendation, and hands-on steerage to pre-train, fine-tune, and deploy state-of-the-art basis fashions on AWS and past. Developed by AWS generative AI worldwide foundations lead Emily Webber, this free hands-on course and the supporting GitHub supply code launched through AWS Youtube. In case you are in search of a curated playlist of the highest assets, ideas, and steerage to rise up to hurry on basis fashions, and particularly those who unlock generative capabilities in your information science and machine studying tasks, then look no additional.

Throughout this 8-hour deep dive, you’ll be launched to the important thing methods, providers, and traits that may make it easier to perceive basis fashions from the bottom up. This implies breaking down principle, arithmetic, and summary ideas mixed with hands-on workout routines to realize useful instinct for sensible utility. All through the course, we deal with a large spectrum of progressively complicated generative AI methods, supplying you with a robust base to grasp, design, and apply your individual fashions for the perfect efficiency. We’ll begin with recapping basis fashions, understanding the place they arrive from, how they work, how they relate to generative AI, and what you possibly can to do customise them. You’ll then find out about choosing the right basis mannequin to fit your use case.

When you’ve developed a robust contextual understanding of basis fashions and how you can use them, you’ll be launched to the core topic of this course: pre-training new basis fashions. You’ll study why you’d need to do that in addition to how and the place it’s aggressive. You’ll even discover ways to use the scaling legal guidelines to select the fitting mannequin, dataset, and compute sizes. We’ll cowl making ready coaching datasets at scale on AWS, together with choosing the right cases and storage methods. We’ll cowl fine-tuning your basis fashions, evaluating current methods, and understanding how you can run these along with your scripts and fashions. We’ll dive into reinforcement studying with human suggestions, exploring how you can use it skillfully and at scale to really maximize your basis mannequin efficiency.

Lastly, you’ll discover ways to apply principle to manufacturing by deploying your new basis mannequin on Amazon SageMaker, together with throughout a number of GPUs and utilizing prime design patterns like retrieval augmented technology and chained dialogue. As an added bonus, we’ll stroll you thru a Steady Diffusion deep dive, immediate engineering greatest practices, standing up LangChain, and extra.

Extra of a reader than a video client? You possibly can take a look at my 15-chapter guide “Pretrain Imaginative and prescient and Massive Language Fashions in Python: Finish-to-end methods for constructing and deploying basis fashions on AWS,” which launched Could 31, 2023, with Packt publishing and is accessible now on Amazon. Wish to soar proper into the code? I’m with you—each video begins with a 45-minute overview of the important thing ideas and visuals. Then I’ll provide you with a 15-minute walkthrough of the hands-on portion. All the instance notebooks and supporting code will ship in a public repository, which you should use to step by way of by yourself. Be happy to achieve out to me on Medium, LinkedIn, GitHub, or by way of your AWS groups. Study extra about generative AI on AWS.

Joyful trails!

Course define

1. Introduction to Basis Fashions

What are giant language fashions and the way do they work? The place do they arrive from? What are different forms of generative AI? How do you customise a basis mannequin? How do you consider a Generative mannequin? Fingers-on stroll by way of: Basis Fashions on SageMaker

Lesson 1 slides Lesson 1 hands-on demo resources

2. Choosing the right basis mannequin

Why beginning with the fitting basis mannequin issues Contemplating dimension Contemplating accuracy Contemplating licensing Contemplating earlier examples of this mannequin working properly in your trade Contemplating exterior benchmarks

Lesson 2 slides Lesson 2 hands-on demo resources

3. Utilizing pretrained basis fashions: immediate engineering and fine-tuning

The advantages of beginning with a pre-trained basis mannequin Immediate engineering: Zero-shot Single-shot Few-shot Summarization Translation Effective-tuning Traditional fine-tuning Parameter environment friendly fine-tuning Hugging Face’s new library Fingers-on stroll by way of: immediate engineering and fine-tuning on SageMaker

Lesson 3 slides Lesson 3 hands-on demo resources

4. Pretraining a brand new basis mannequin

Why would you need or have to create a brand new basis mannequin? Evaluating pretraining to fine-tuning Getting ready your dataset for pretraining Distributed coaching on SageMaker: libraries, scripts, jobs, assets Why and how you can adapt a brand new script to SageMaker distributed coaching

Lesson 4 slides Lesson 4 hands-on demo resources

5. Getting ready information and coaching at scale

Choices for prepping information at scale on AWS Clarify SageMaker job parallelism on CPU cases Clarify modes of sending information to SageMaker Coaching Introduction to FSx for Lustre Utilizing FSx for Lustre at scale for SageMaker Coaching Fingers-on stroll by way of: configuring Lustre for SageMaker Coaching

Lesson 5 slides Lesson 5 hands-on demo resources

6. Reinforcement studying with human suggestions

What’s this system and why will we care about it The way it will get round issues with subjectivity and objectivity by way of rating human preferences at scale How does it work? How to do that with SageMaker Floor Fact Up to date reward modeling Fingers-on stroll by way of: RLFH on SageMaker

Lesson 6 slides Lesson 6 hands-on demo resources

7. Deploying a basis mannequin

Why will we need to deploy fashions? Completely different choices for deploying FM’s on AWS optimize your mannequin for deployment Massive mannequin deployment container deep dive High configuration suggestions for deploying FM’s on SageMaker Immediate engineering suggestions for invoking basis fashions Utilizing retrieval augmented technology to mitigate hallucinations Fingers-on stroll by way of: Deploying an FM on SageMaker

Lesson 7 slides Lesson 7 hands-on demo resources

---

Concerning the writer

Emily Webber joined AWS simply after SageMaker launched, and has been making an attempt to inform the world about it ever since! Outdoors of constructing new ML experiences for purchasers, Emily enjoys meditating and finding out Tibetan Buddhism.