Tune, Distill, and Evaluate Meta Llama 3 on Vertex AI
Tuning a general LLM like Llama 3 with your own data can transform it into a powerful model tailored to your specific business and use cases. When developers access Llama 3 through Vertex AI, they will soon have access to multiple state of the art tuning options made available through Colab Enterprise. These include preconfigured notebooks for supervised tuning (LoRA), reinforcement learning through human feedback (RLHF), and distillation.
Vertex AI also makes it simple for developers to evaluate their tuned Llama models, either through preconfigured notebooks directly in Model Garden or with Auto SxS, Vertex AI’s pairwise model-based evaluation tool. These easy-to-use interfaces mean that developers can spend less time on operational details and start optimizing and deploying Llama 3 for their use case immediately.
State of the Art Hardware & Software for Efficient Tuning and Serving
Vertex AI offers the most flexibility and choice with accelerators, with both TPU and GPU offerings. Last week at Next ‘24, we announced that Cloud TPU v5e is now generally available for online prediction on Vertex AI, meaning developers can now serve their tuned Llama 3 models from Google’s state of the art, latest generation TPUs. PyTorch users can now also use the Optimum-TPU package to train and serve Llama 3 on TPUs.
And with robust features like Model Registry, Vertex AI makes it easy to manage and monitor model variants and endpoints and scale them appropriately for your needs.
A thriving, open ecosystem for enterprise model builders
With over 130 first-party, third-party, and open models, Vertex AI Model Garden is a one-stop destination for enterprise developers to discover, tune, and manage models. We are thrilled to bring developers not only the latest state of the art models like Llama 3 but the best infrastructure and tooling to build real generative AI agents on these models. Join us at I/O on May 14th for more exciting updates on Vertex Model Garden.
