In order to succesfully run Deeploy on AWS a simple set of cloud resources is required. Below we walk through their setup in AWS. We advise to install the aws cli and eksctl in order to complete the steps listed.
Typical Customer Deployment
Typical customer deployments consist of a separate development and production environment. The high level architecture overview with the relevant AWS resources is shown below:
- Deeploy integrates with version control systems in or model framework for ML code deployments. For the following system we maintain support:
- Gitlab
- GitHub
- Bitbucket
- Azure DevOps - Deeploy advises to use a managed Kubernetes cluster (Stateless): AWS EKS. For normal usage Deeploy requires about 3 medium nodes; minimal requirements: 3 (v)CPU and 6 GB RAM). We advise to use auto scaling nodegroups.
- Deeploy advises to use a managed PostgreSQL database server: AWS RDS for PostgreSQL. Every microservice has his own database. In the RDS database all data shared with or created with Deeploy will be stored. We advise to enable AWS RDS Storage Autoscaling.
- Deeploy advises to use S3 storage to store ML model artefacts.
- Deeploy advises to isolate single Deeploy environment installation within a separate AWS VPC (see Networking)
- Finally, Deeploy needs an AWS KMS Symmetric key. This key is used to encrypt/decrypt Deeploy tokens in the database.
- The easiest way to install Deeploy on the cloud resources is to use the AWS ECR images as provided by the AWS Marketplace.
- Deeploy will authenticate with AWS resources using an IAM user or IAM role. In order to use roles with Deeploy uses AWS EKS Pod Identity Webhooks.
Data is stored in the following AWS resources:
Set up the cloud resources
In a production environment, we advise to manage cloud resources as infrastructure as code, in for example, AWS Cloudformation or Terraform.
AWS EKS
Follow the steps to set up AWS EKS using the AWS EKS guide. Specific considerations regarding your Deeploy installation:
- Kubernetes version: we advise to use version 1.23+ This is currently the latest supported version of Kubernetes.
- Deeploy can receive traffic on the Istio ingress gateway via a AWS loadbalancer
- Enable autoscaling for your managed node group: we advise to enable Cluster autoscaling.
- Deeploy does check the health of the cluster using readiness probes
The fastest way to setup a EKS cluster is to use eksctl:
eksctl create cluster -f <your-config>.yamlCheck the documentation for specific configuration instructions. Pay attention to the single or multi availability zone configuration as described here.
AWS RDS for PostgreSQL
Follow the steps to set up the AWS RDS using the AWS RDS for PostgreSQL guide. Make sure to implement the best practices to backup and restore data at any point in time as described here. Specific considerations regarding your Deeploy installation:
- Since the amount of data in the RDS will increase with usage over time, make sure to enable AWS RDS Storage Autoscaling to prevent manual interventions.
- We advise to align the network configuration of the RDS database with the EKS cluster (same VPC and subnets). This will allow for data transfers over the internal AWS network.
- Make sure to create a separate user called
deeploywith read and write rights and save the credentials to use in the Helmvalues.yamlfile that we will create later on.
AWS S3
Follow the steps below to set up a AWS S3 bucket using the AWS S3 guide. Specific considerations regarding your Deeploy installation:
- Deeploy advises to use just a single IAM user/role to access the S3 bucket from the EKS cluster using the following minimal required IAM access policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::<YOUR-BUCKET-NAME>"
]
},
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::<YOUR-BUCKET-NAME>/*"
]
}
]
}
- Create an access key and a secret key and save the credentials to use in the Helm
values.yamlfile that we will create later on. - Create an S3 Gateway endpoint for your VPC. This will allow for data transfers over the internal AWS network.
AWS KMS
Follow the steps to set up a AWS KMS Symmetric Encryption key using the AWS KMS Guide. Specific considerations regarding your Deeploy installation:
- Deeploy advises to use just a single IAM user/ to access the KMS from the EKS cluster using the following minimal required IAM access policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<YOUR-ACCOUNT-ID>:user/<YOUR-DEEPLOY-IAM-USERNAME>"
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::<YOUR-ACCOUNT-ID>:root"
]
},
"Action": [
"kms:*
],
"Resource": "*"
}
]
}
- Create an access key and a secret key and save the credentials to use in the Helm
values.yamlfile that we will create later on.
Networking
We advise to get familiar with AWS specific networking as described in this guide. Follow the best practices for networking for the EKS and RDS deployments. Below an example AWS architecture diagram including networking configuration is shown below.
Specific considerations regarding your Deeploy installation:
- Deploy the RDS and EKS cluster within the same VPC but use different security groups to specifically whitelist the EKS nodegroup within the RDS database.
AWS Resource Health
Amazon CloudWatch
We advise to set up CloudWatch for monitoring and alerting resource health and credit balance.
Service Health Dashboard
If you assume there is a problem with your AWS Cloud Resources, make sure to check this service health dashboard.
Comments
0 comments
Please sign in to leave a comment.