Use Amazon SageMaker Mannequin Card sharing to enhance mannequin governance

As Synthetic Intelligence (AI) and Machine Studying (ML) applied sciences have change into mainstream, many enterprises have been profitable in constructing crucial enterprise purposes powered by ML fashions at scale in manufacturing. Nonetheless, since these ML fashions are making crucial enterprise choices for the enterprise, it’s vital for enterprises so as to add correct guardrails all through their ML lifecycle. Guardrails be certain that safety, privateness, and high quality of the code, configuration, and information and mannequin configuration utilized in mannequin lifecycle are versioned and preserved.

Implementing these guardrails is getting tougher for enterprises as a result of the ML processes and actions inside enterprises have gotten extra advanced as a result of inclusion of deeply concerned processes that require contributions from a number of stakeholders and personas. Along with information engineers and information scientists, there have been inclusions of operational processes to automate & streamline the ML lifecycle. Moreover, the surge of enterprise stakeholders and in some circumstances authorized and compliance opinions want capabilities so as to add transparency for managing entry management, exercise monitoring, and reporting throughout the ML lifecycle.

The framework that provides systematic visibility into ML mannequin improvement, validation, and utilization known as ML governance. Throughout AWS re:Invent 2022, AWS introduced new ML governance tools for Amazon SageMaker which simplifies entry management and enhances transparency over your ML initiatives. One of many instruments out there as a part of the ML governance is Amazon SageMaker Model Cards, which has the aptitude to create a single supply of fact for mannequin data by centralizing and standardizing documentation all through the mannequin lifecycle.

SageMaker mannequin playing cards allow you to standardize how fashions are documented, thereby reaching visibility into the lifecycle of a mannequin, from designing, constructing, coaching, and analysis. Mannequin playing cards are meant to be a single supply of fact for enterprise and technical metadata concerning the mannequin that may reliably be used for auditing and documentation functions. They supply a reality sheet of the mannequin that’s vital for mannequin governance.

As you scale your fashions, initiatives, and groups, as a finest apply we advocate that you just undertake a multi-account technique that gives mission and crew isolation for ML mannequin improvement and deployment. For extra details about bettering governance of your ML fashions, confer with Improve governance of your machine learning models with Amazon SageMaker.

Structure overview

The structure is applied as follows:

• Knowledge Science Account – Knowledge Scientists conduct their experiments in SageMaker Studio and construct an MLOps setup to deploy fashions to staging/manufacturing environments utilizing SageMaker Projects.
• ML Shared Providers Account – The MLOps arrange from the Knowledge Science account will set off steady integration and steady supply (CI/CD) pipelines utilizing AWS CodeCommit and AWS CodePipeline.
• Dev Account – The CI/CD pipelines will additional set off ML pipelines on this account overlaying information pre-processing, mannequin coaching and publish processing like mannequin analysis and registration. Output of those pipelines will deploy the mannequin in SageMaker endpoints to be consumed for inference functions. Relying in your governance necessities, Knowledge Science & Dev accounts will be merged right into a single AWS account.
• Knowledge Account – The ML pipelines operating within the Dev Account will pull the info from this account.
• Take a look at and Prod Accounts – The CI/CD pipelines will proceed the deployment after the Dev Account to arrange SageMaker endpoint configuration in these accounts.
• Safety and Governance – Providers like AWS Identification and Entry Administration (IAM), AWS IAM Identification Heart, AWS CloudTrail, AWS Key Administration Service (AWS KMS), Amazon CloudWatch, and AWS Safety Hub can be used throughout these accounts as a part of safety and governance.

The next diagram illustrates this structure.

For extra details about setting scalable multi account ML structure, confer with MLOps foundation for enterprises with Amazon SageMaker.

Our clients want the aptitude to share mannequin playing cards throughout accounts to enhance visibility and governance of their fashions by means of data shared within the mannequin card. Now, with cross-account mannequin playing cards sharing, clients can get pleasure from the advantages of multi-account technique whereas having accessibility into the out there mannequin playing cards of their group, to allow them to speed up collaboration and guarantee governance.

On this publish, we present the best way to arrange and entry mannequin playing cards throughout Mannequin Growth Lifecycle (MDLC) accounts utilizing the brand new cross-account sharing function of the mannequin card. First, we are going to describe a situation and structure for organising the cross-account sharing function of the mannequin card, after which dive deep into every element of the best way to arrange and entry shared mannequin playing cards throughout accounts to enhance visibility and mannequin governance.

Resolution overview

When constructing ML fashions, we advocate organising a multi-account structure to offer workload isolation bettering safety, reliability, and scalability. For this publish, we are going to assume constructing and deploying a mannequin for Buyer Churn use case. The structure diagram that follows exhibits one of many really helpful approaches – centralized mannequin card – for managing a mannequin card in a multi-account Machine Studying Mannequin-Growth Lifecycle (MDLC) structure. Nonetheless, you too can undertake one other strategy, a hub-and-spoke mannequin card. On this publish, we are going to focus solely on a centralized mannequin card strategy, however the identical rules will be prolonged to a hub-and-spoke strategy. The principle distinction is that every spoke account will keep their very own model of mannequin card and it’ll have processes to mixture and replica to a centralized account.

The next diagram illustrates this structure.

The structure is applied as follows:

1. Lead Knowledge Scientist is notified to resolve the Buyer Churn use case utilizing ML, they usually begin the ML mission by means of creation of a mannequin card for Buyer Churn V1 mannequin in Draft standing within the ML Shared Providers Account
2. By means of automation, that mannequin card is shared with ML Dev Account
3. Knowledge Scientist builds the mannequin and begins to populate data by way of APIs into the mannequin card primarily based on their experimentation outcomes and the mannequin card standing is about to Pending Overview
4. By means of automation, that mannequin card is shared with the ML check account
5. ML Engineer (MLE) runs integration and validation exams in ML Take a look at account and the mannequin within the central registry is marked Pending Approval
6. Mannequin Approver opinions the mannequin outcomes with the supporting documentation offered within the central mannequin card and approves the mannequin card for manufacturing deployment.
7. By means of automation, that mannequin card is shared with ML Prod account in read-only mode.

Conditions

Earlier than you get began, be sure to have the next stipulations:

• Two AWS accounts.
• In each AWS accounts, an IAM federation position with administrator entry to do the next:
  • Create, edit, view, and delete mannequin playing cards inside Amazon SageMaker.
  • Create, edit, view, and delete useful resource share inside AWS RAM.

For extra data, confer with Example IAM policies for AWS RAM.

Organising mannequin card sharing

The account the place the mannequin playing cards are created is the mannequin card account. Customers within the mannequin card account share them with the shared accounts the place they are often up to date. Customers within the mannequin card account can share their mannequin playing cards by means of AWS Resource Access Manager (AWS RAM). AWS RAM helps you share sources throughout AWS accounts.

Within the following part, we present the best way to share mannequin playing cards.

First, create a mannequin card for a Buyer Churn use case as beforehand described. On the Amazon SageMaker console, broaden the Governance part and select Mannequin playing cards.

We create the mannequin card in Draft standing with the title Buyer-Churn-Mannequin-Card. For extra data, confer with Create a model card. On this demonstration, you’ll be able to depart the rest of the fields clean and create the mannequin card.

Alternatively, you need to use the next AWS CLI command to create the mannequin card:

aws sagemaker create-model-card --model-card-name Buyer-Churn-Mannequin-Card --content "{"model_overview": {"model_owner": "model-owner","problem_type": "Buyer Churn Mannequin"}}" --model-card-status Draft

Now, create the cross-account share utilizing AWS RAM. Within the AWS RAM console, choose Create a useful resource share.

Enter a reputation for the useful resource share, for instance “Buyer-Churn-Mannequin-Card-Share”. Within the Sources – non-compulsory part, choose the useful resource sort as SageMaker Mannequin Playing cards. The mannequin card we created within the earlier step will seem within the itemizing.

Choose that mannequin and it’ll seem within the Chosen sources part. Choose that useful resource once more as proven within the following steps and select Subsequent.

On the subsequent web page, you’ll be able to choose the Managed permissions. You may create customized permissions or use the default choice “AWSRAMPermissionSageMakerModelCards” and choose Subsequent. For extra data, confer with Managing permissions in AWS RAM.

On the subsequent web page, you’ll be able to choose Principals. Underneath Choose principal sort, select AWS Account and enter the ID of the account of the share the mannequin card. Choose Add and proceed to the subsequent web page.

On the final web page, evaluate the data and choose “Create useful resource share”. Alternatively, you need to use the next AWS CLI command to create a useful resource share:

aws ram create-resource-share --name <Title of the Mannequin Card>

aws ram associate-resource-share --resource-share-arn <ARN of useful resource share create from the earlier command> --resource-arns <ARN of the Mannequin Card>

On the AWS RAM console, you see the attributes of the useful resource share. Ensure that Shared sources, Managed permissions, and Shared principals are within the “Related” standing.

After you employ AWS RAM to create a useful resource share, the principals specified within the useful resource share will be granted entry to the share’s sources.

• Should you activate AWS RAM sharing with AWS Organizations, and your principals that you just share with are in the identical group because the sharing account, these principals can obtain entry as quickly as their account administrator grants them permissions.
• Should you don’t activate AWS RAM sharing with Organizations, you’ll be able to nonetheless share sources with particular person AWS accounts which might be in your group. The administrator within the consuming account receives an invite to hitch the useful resource share, they usually should settle for the invitation earlier than the principals specified within the useful resource share can entry the shared sources.
• You can even share with accounts exterior of your group if the useful resource sort helps it. The administrator within the consuming account receives an invite to hitch the useful resource share, they usually should settle for the invitation earlier than the principals specified within the useful resource share can entry the shared sources.

For extra details about AWS RAM, confer with Terms and concepts for AWS RAM.

Accessing shared mannequin playing cards

Now we will log in to the shared AWS account to entry the mannequin card. Just remember to are accessing the AWS console utilizing IAM permissions (IAM position) which permit entry to AWS RAM.

With AWS RAM, you’ll be able to view the useful resource shares to which you could have been added, the shared sources that you could entry, and the AWS accounts which have shared sources with you. You can even depart a useful resource share whenever you not require entry to its shared sources.

To view the mannequin card within the shared AWS account:

1. Navigate to the Shared with me: Shared resources web page within the AWS RAM console.
2. Just remember to are working in the identical AWS area the place the share was created.
3. The mannequin shared from the mannequin account can be out there within the itemizing. If there’s a lengthy listing of sources, then you’ll be able to apply a filter to seek out particular shared sources. You may apply a number of filters to slim your search.
4. The next data is obtainable:
   1. Useful resource ID – The ID of the useful resource. That is the title of the mannequin card that we created earlier within the mannequin card account.
   2. Useful resource sort – The kind of useful resource.
   3. Final share date – The date on which the useful resource was shared with you.
   4. Useful resource shares – The variety of useful resource shares during which the useful resource is included. Select the worth to view the useful resource shares.
   5. Proprietor ID – The ID of the principal who owns the useful resource.

You can even entry the mannequin card utilizing the AWS CLI choice. For the AWS IAM coverage configured with the proper credentials, just remember to have permissions to create, edit, and delete mannequin playing cards inside Amazon SageMaker. For extra data, confer with Configure the AWS CLI.

You should use the next AWS IAM permissions coverage as template:

{
     "Model": "2012-10-17",
     "Assertion": [
        {
             "Effect": "Allow",
             "Action": [
                 "sagemaker:DescribeModelCard",
                 "sagemaker:UpdateModelCard",
                 "sagemaker:CreateModelCardExportJob",
                 "sagemaker:ListModelCardVersions",
                 "sagemaker:DescribeModelCardExportJob"
             ],
             "Useful resource": [
                 "arn:aws:sagemaker:AWS-Region:AWS-model-card-account-id:model-card/example-model-card-name-0",
                 "arn:aws:sagemaker:AWS-Region:AWS-model-card-account-id:model-card/example-model-card-name-1/*"
             ]
        },
        { 
             "Impact": "Enable", 
             "Motion": "s3:PutObject",
             "Useful resource": "arn:aws:s3:::Amazon-S3-bucket-storing-the-pdf-of-the-model-card/model-card-name/*"
        }
    ]
}

You may run the next AWS CLI command to entry the small print of the shared mannequin card.

aws sagemaker describe-model-card --model-card-name <ARN of the mannequin card>

Now you may make adjustments to this mannequin card from this account.

aws sagemaker update-model-card --model-card-name <ARN of the Mannequin Card> --content "{"model_overview": {"model_owner": "model-owner","problem_type": "Buyer Churn Mannequin"}}"

After you make adjustments, return to the mannequin card account to see the adjustments that we made on this shared account.

The issue sort has been up to date to “Buyer Churn Mannequin” which we had offered as a part of the AWS CLI command enter.

Clear up

Now you can delete the mannequin card you created. Just remember to delete the AWS RAM useful resource share that you just created to share the mannequin card.

Conclusion

On this publish, we offered an outline of multi-account structure for scaling and governing your ML workloads securely and reliably. We mentioned the structure patterns for organising mannequin card sharing and illustrated how centralized mannequin card sharing patterns work. Lastly, we arrange mannequin card sharing throughout a number of accounts for bettering visibility and governance in your mannequin improvement lifecycle. We encourage you check out the brand new mannequin card sharing function and tell us your suggestions.

Concerning the authors

Vishal Naik is a Sr. Options Architect at Amazon Internet Providers (AWS). He’s a builder who enjoys serving to clients accomplish their enterprise wants and clear up advanced challenges with AWS options and finest practices. His core space of focus consists of Machine Studying, DevOps, and Containers. In his spare time, Vishal loves making brief movies on time journey and alternate universe themes.

Ram Vittal is a Principal ML Options Architect at AWS. He has over 20 years of expertise architecting and constructing distributed, hybrid, and cloud purposes. He’s captivated with constructing safe and scalable AI/ML and massive information options to assist enterprise clients with their cloud adoption and optimization journey to enhance their enterprise outcomes. In his spare time, he rides his motorbike and walks together with his 2-year-old sheep-a-doodle!