10. Voice Cloning, Eleven Labs API, OpenAI Embeddings, ChatGPT API, Whisper API

I will be starting a spinoff channel on AI in music, art, and gaming in 2023. Subscribe at:

In this video, we build a question answering voice assistant that responds with a more realistic voice based on samples. To accomplish this, we combine multiple tools and techniques from the previous videos in the seires: OpenAI Embeddings, ChatGPT API, Whisper API, Eleven Labs API, Gradio UI, and Midjourney images.

0:00 Project Description: Q&A + Voice Cloning
1:45 The movie “Her” and the Idea of Smarter Assistants
2:27 Voice Sampling
3:15 Demo Voice #1 (Samantha Voice)
4:00 Demo Voice #2 (Jay-Z Voice), Rhyming Responses
5:20 Hip Hop Music and Sampling Analogy
6:26 Hip Hop Production, Rick Rubin, Taste and Technical Ability Clip
8:10 Recap of OpenAI For Finance Series So Far, Prerequisites
9:16 Building a Q&A Corpus, Vector Embeddings, Cosine Similarity Review
17:52 Building a User Interface with Gradio, Starter Code from Video #9
20:05 Voice Cloning with Eleven Labs API
24:57 Python Code Walkthrough – constants, voice ID, custom prompts
25:52 Eleven Labs API – Example Request and Response Payloads
26:28 Avatars and AI Art Generation with Midjourney, Nvidia Stock Win
30:37 Python Code Walkthrough –, requirements.txt
31:44 Gradio User Interface Development, Microphone Input, Avatar Display
35:15 UI Launch, Debugging Mode, Sharing Your App, Mobile Devices
36:32 Transcribe Function, OpenAI Whisper API
38:40 Incorporating Word Embeddings, Question Vector, Cosine Similarity, Answers
40:25 ChatGPT API, Conversation History, Stuffing the Prompt with Context
42:57 Eleven Labs API Request with Python, Text to Speech, Voice Synthesis Settings
44:25 Outputting Binary Response / MP3 to Audio Output
46:00 Final Words of Advice from Jay-Z

Her Movie Clip:
Knowledge Base / Transcriptions From:
Buy Me a Coffee:

Video #5 Word Embeddings:
Video #5 Notebook:
Video #6 Financial Q&A
Video #9 ChatGPT API + Whisper API and Gradio
Video #9 Code:
Video #10 Code (This Video)

Since I am starting to do more content featuring AI + art and sound, I will be starting a spinoff channel on AI in music, art, and gaming in 2023. Subscribe at:

OpenAI Gym – Lunar Lander Training Demo


Tongue-Controlled MouthPad Enables Computer Interaction for Paralyzed Users