Amazon SageMaker Model Cards allow you to standardize how fashions are documented, thereby attaining visibility into the lifecycle of a mannequin, from designing, constructing, coaching, and analysis. Mannequin playing cards are supposed to be a single supply of fact for enterprise and technical metadata in regards to the mannequin that may reliably be used for auditing and documentation functions. They supply a factsheet of the mannequin that’s necessary for mannequin governance.
Till now, mannequin playing cards have been logically related to a mannequin within the Amazon SageMaker Model Registry utilizing mannequin identify match. Nonetheless, when fixing a enterprise downside by a machine studying (ML) mannequin, as prospects iterate on the issue, they create a number of variations of the mannequin and they should operationalize and govern a number of mannequin variations. Subsequently, they want the power to affiliate a mannequin card to a selected mannequin model.
On this publish, we talk about a brand new function that helps integrating mannequin playing cards with the mannequin registry on the deployed mannequin model stage. We talk about the answer structure and finest practices for managing mannequin card variations, and stroll by the right way to arrange, operationalize, and govern the mannequin card integration with the mannequin model within the mannequin registry.
Resolution overview
SageMaker mannequin playing cards make it easier to standardize documenting your fashions from a governance perspective, and the SageMaker mannequin registry helps you deploy and operationalize ML fashions. The mannequin registry helps a hierarchical construction for organizing and storing ML fashions with mannequin metadata info.
When a corporation solves a enterprise downside utilizing ML, similar to a buyer churn prediction, we suggest the next steps:
- Create a mannequin card for the enterprise downside to be solved.
- Create a mannequin bundle group for the enterprise downside to be solved.
- Construct, practice, consider, and register the primary model of the mannequin bundle model (for instance, Buyer Churn V1).
- Replace the mannequin card linking the mannequin bundle model to the mannequin card.
- As you iterate on new mannequin bundle model, clone the mannequin card from the earlier model and hyperlink to the brand new mannequin bundle model (for instance, Buyer Churn V2).
The next determine illustrates how a SageMaker mannequin card integrates with the mannequin registry.
As illustrated within the previous diagram, the mixing of SageMaker mannequin playing cards and the mannequin registry means that you can affiliate a mannequin card with a particular mannequin model within the mannequin registry. This allows you to set up a single supply of fact to your registered mannequin variations, with complete and standardized documentation throughout all levels of the mannequin’s journey on SageMaker, facilitating discoverability and selling governance, compliance, and accountability all through the mannequin lifecycle.
Greatest practices for managing mannequin playing cards
Working in machine studying with governance is a crucial requirement for a lot of enterprise organizations right now, notably in extremely regulated industries. As a part of these necessities, AWS offers a number of companies that allow dependable operation of the ML surroundings.
SageMaker mannequin playing cards doc crucial particulars about your ML fashions in a single place for streamlined governance and reporting. Mannequin playing cards make it easier to seize particulars such because the supposed use and threat score of a mannequin, coaching particulars and metrics, analysis outcomes and observations, and extra call-outs similar to issues, suggestions, and customized info.
Mannequin playing cards must be managed and up to date as a part of your growth course of, all through the ML lifecycle. They’re an necessary a part of steady supply and pipelines in ML. In the identical method {that a} Properly-Architected ML challenge implements steady integration and steady supply (CI/CD) underneath the umbrella of MLOps, a steady ML documentation course of is a crucial functionality in lots of regulated industries or for larger threat use instances. Mannequin playing cards are a part of the very best practices for accountable and clear ML growth.
The next diagram reveals how mannequin playing cards needs to be a part of a growth lifecycle.
Think about the next finest practices:
- We suggest creating mannequin playing cards early in your challenge lifecycle. Within the first part of the challenge, if you find yourself engaged on identifying the business goal and framing the ML problem, you need to provoke the creation of the mannequin card. As you’re employed by the completely different steps of enterprise necessities and necessary efficiency metrics, you may create the mannequin card in a draft standing and decide the enterprise particulars and supposed makes use of.
- As a part of your model development lifecycle phase, you need to use the mannequin registry to catalog fashions for manufacturing, handle mannequin variations, and affiliate metadata with a mannequin. The mannequin registry permits lineage monitoring.
- After you’ve got iterated efficiently and are able to deploy your mannequin to manufacturing, it’s time to replace the mannequin card. Within the deployment lifecycle phase, you may replace the mannequin particulars of the mannequin card. You also needs to replace coaching particulars, analysis particulars, moral issues, and caveats and proposals.
Mannequin playing cards have variations related to them. A given mannequin model is immutable throughout all attributes aside from the mannequin card standing. In the event you make some other modifications to the mannequin card, similar to analysis metrics, description, or supposed makes use of, SageMaker creates a brand new model of the mannequin card to replicate the up to date info. That is to make sure that a mannequin card, as soon as created, can’t be tampered with. Moreover, every distinctive mannequin identify can have just one related mannequin card and it will possibly’t be modified after you create the mannequin card.
ML fashions are dynamic and workflow automation parts allow you to simply scale your means to construct, practice, check, and deploy lots of of fashions in manufacturing, iterate quicker, scale back errors resulting from guide orchestration, and construct repeatable mechanisms.
Subsequently, the lifecycle of your mannequin playing cards will look as described within the following diagram. Each time you replace your mannequin card by the mannequin lifecycle, you routinely create a brand new model of the mannequin card. Each time you iterate on a brand new mannequin model, you create a brand new mannequin card that may inherit some mannequin card info of the earlier mannequin variations and observe the identical lifecycle.
Pre-requisites
This publish assumes that you have already got fashions in your mannequin registry. If you wish to observe alongside, you should utilize the next SageMaker instance on GitHub to populate your mannequin registry: SageMaker Pipelines integration with Model Monitor and Clarify.
Combine a mannequin card with the mannequin model within the mannequin registry
On this instance, now we have the model-monitor-clarify-group bundle in our mannequin registry.
On this bundle, two mannequin variations can be found.
For this instance, we hyperlink Model 1 of the mannequin to a brand new mannequin card. Within the mannequin registry, you may see the small print for Model 1.
We will now use the brand new function within the SageMaker Python SDK. From the sagemaker.model_card ModelPackage module, you may choose a particular mannequin model from the mannequin registry that you simply wish to hyperlink the mannequin card to.
Now you can create a brand new mannequin card for the mannequin model and specify the model_package_details parameter with the earlier mannequin bundle retrieved. You should populate the mannequin card with all the extra particulars crucial. For this publish, we create a easy mannequin card for instance.
You may then use that definition to create a mannequin card utilizing the SageMaker Python SDK.
When loading the mannequin card once more, you may see the related mannequin underneath "__model_package_details".
You even have the choice to replace an current mannequin card with the model_package as proven within the instance code snippet under:
Lastly, when creating or updating a brand new mannequin bundle model in an current mannequin bundle, if a mannequin card already exists in that mannequin bundle group, some info such because the enterprise particulars and supposed makes use of could be carried over to the brand new mannequin card.
Clear up
Customers are answerable for cleansing up sources if created utilizing the pocket book talked about within the pre-requisites part. Please observe the directions within the pocket book to wash up sources.
Conclusion
On this publish, we mentioned the right way to combine a SageMaker mannequin card with a mannequin model within the mannequin registry. We shared the answer structure with finest practices for implementing a mannequin card and confirmed the right way to arrange and operationalize a mannequin card to enhance your mannequin governance posture. We encourage you to check out this resolution and share your suggestions within the feedback part.
Concerning the Authors
Ram Vittal is a Principal ML Options Architect at AWS. He has over 20 years of expertise architecting and constructing distributed, hybrid, and cloud purposes. He’s captivated with constructing safe and scalable AI/ML and massive information options to assist enterprise prospects with their cloud adoption and optimization journey to enhance their enterprise outcomes. In his spare time, he rides his motorbike and walks together with his 2-year-old sheep-a-doodle!
Natacha Fort is the Authorities Information Science Lead for Public Sector Australia and New Zealand, Principal SA at AWS. She helps organizations navigate their machine studying journey, supporting them from framing the machine studying downside to deploying into manufacturing, all of the whereas ensuring the very best structure practices are in place to make sure their success. Natacha focuses with organizations on MLOps and accountable AI.
