On this video, I’m going over the best way to obtain and run the open-source implementation of GPT3, known as GPT Neo. This mannequin is 2.7 billion parameters, which is identical measurement as GPT3 Ada. The outcomes are excellent and are a big enchancment over GPT-2. I’m excited to mess around with this mannequin extra and for the way forward for even bigger NLP fashions.
Pocket book https://github.com/mallorbc/GPTNeo_notebook
GPT Neo github https://github.com/EleutherAI/gpt-neo (use the primary launch tag)
GPT Neo HuggingFace docs https://huggingface.co/transformers/model_doc/gpt_neo.html
A helpful article about transformer parameters https://huggingface.co/blog/how-to-generate
00:00 – GPT3 Background
01:07 – GPT3 Interview
02:06 – GPT Neo Github
03:14 – GPT Neo HuggingFace
03:52 – Establishing Anaconda and Jupyter
05:05 – Beginning the Jupyter pocket book
06:14 – Putting in dependencies within the pocket book
06:58 – Importing wanted dependencies
07:33 – Deciding on what GPT mannequin to make use of
08:45 – Checking our pc {hardware}
09:36 – Loading the tokenizer
09:55 – Giving our inputs
10:39 – Producing the tokens with the mannequin
11:42 – Decoding and studying the consequence
13:22 – Reflections on Transformers
14:26 – Outro questions future work
