So you have a pretty good grasp on web development, but you want to learn more about building with generative AI? You’re not alone. In this live-coding course, follow along as we build a full-stack generative AI app using LangChain.js, Replicate, OpenAI, and Convex.
Step-by-step GitHub repo:
https://github.com/get-convex/ai-storybook
Topics:
* Turning your React app into a full-stack app
* Running AI-centric background tasks on the backend, including subtleties about throttling and data consistency
* Using text-to-image models to create generated images in your apps
* Prompt design and iteration
* Utilizing model chaining to further refine your prompts and images
Technologies:
* Convex (https://convex.dev) — Backend application platform
* LangChain.js (https://github.com/hwchase17/langchainjs) – Model chaining and composition
* OpenAI (https://platform.openai.com) – Chat model service (GPT-3.5)
* Replicate (https://replicate.com) – Text-to-image model service (Kandinsky-2)
* React (https://react.dev/) – UI framework
* Tailwind (https://tailwindcss.com/) – CSS framework
* DaisyUI (https://daisyui.com/) – Tailwind-based component library
* Vite (https://vitejs.dev/) – Frontend development environment
Table of Contents:
0:00 What we’re making
2:01 What you need to know
2:25 Step 0: React web app
4:36 Step 1: Full stack app
5:36 – 1.1 Why we need a backend?
6:29 – 1.2 Provisioning our backend
8:00 – 1.3 Creating our first schema
9:37 – 1.4 Adding an index for common queries
11:01 – 1.5 Write a mutation to store pages
14:21 – …Hilarity as the author slowly realizes he’s working in the wrong directory
15:48 – 1.6 Testing out functions with the Convex dashboard
16:40 – 1.7 Write a query to fetch the book state
20:45 – 1.8 Connecting our app components to the backend query
24:25 – 1.9 Testing our our full-stack app
25:29 – 1.10 Hooking up our app’s page save to our mutation
26:57 – …TypeScript confusion that gets resolved later in the project
28:19 – …Wherein we discover a small bug around insertions vs. updates
29:07 – 1.11 Bug fix the mutation to update if the page exists
32:16 – 1.12 Final full-stack app testing
33:49 Step 1 Review
35:00 Step 2: Simple AI image generation
36:35 – 2.1 Play with models and prompts to determine your target
39:55 – 2.2 Introducing LangChain.js
40:58 – 2.3 Debouncing and consistency with long-running AI jobs
47:22 – 2.4 Introduce content versioning
54:44 – 2.5 Explore Convex actions for long-running AI
56:10 – 2.6 “Hello world” of actions
57:39 – 2.7 Using the Node.js runtime within Convex
59:53 – 2.8 Scheduling actions from mutations
1:07:04 – 2.9 Monitoring background actions with the dashboard
1:08:28 – 2.10 Fetching the book state from our AI action
1:11:37 – 2.11 Ensure the action only runs on the current content version
1:13:50 – 2.12 More scaffolding for calling the AI service
1:16:29 – 2.13 Basic prompt engineering and persistence
1:18:54 – 2.14 LangChain basics
1:21:37 – 2.15 Injecting secrets into your backend environment
1:22:38 – 2.16 Finishing the call to Replicate
1:24:45 – 2.17 Our first generated images (and wow they’re bleak!)
1:27:02 – 2.18 Saving the results of AI
1:34:21 – 2.19 Basic AI app testing
1:35:32 – Step 2 bonus: Regenerate button
1:37:25 Step 2 Review and project assessment
1:43:02 Step 3: Prompt engineering and model chaining
1:44:08 – 3.1 Deeper on chaining and using LangChain.js composably
1:45:47 – 3.2 Prompt templates for a consistent visual style
1:51:37 – 3.3 Survey PromptTemplate style improvements
1:53:49 – 3.4 Bug fix: clear all obsolete images
1:56:08 – 3.5 Let’s make a chain of chains
1:59:01 – 3.6 Refactor for nested chaining
2:01:11 – 3.7 ChatGPT chaining
2:02:36 – 3.8 Prompt engineering our GPT-provided illustration instructions
2:08:58 – 3.9 Implement ChatGPT model and chain
2:11:22 – 3.10 Composing the LLMChains with SequentialChain
2:28:22 – 3.11 Trying out the chained models in our app
2:23:56 – 3.12 Tryhard ChatGPT is causing problems…
2:25:47 – 3.13 Chill out ChatGPT with a “system” prompt
2:29:54 – That’s a wrap! Project summary and ideas for further improvements
About the author:
Jamie Turner has been creating web apps and frameworks for more than twenty years. Formerly he was the Head of Engineering at Bump Technologies, a Principal Engineer at Dropbox, he’s now the co-founder of Convex.
