Run interactive IDEs on Amazon EKS with SageMaker AI to power up your AI workflows

To power up AI workflows on Amazon Elastic Kubernetes Service (Amazon EKS), data scientists need interactive IDEs like JupyterLab and Code Editor. Yet running those IDEs usually means leaving the cluster that hosts their pipelines, moving to a standalone JupyterHub deployment or a local laptop.…

To power up AI workflows on Amazon Elastic Kubernetes Service (Amazon EKS), data scientists need interactive IDEs like JupyterLab and Code Editor. Yet running those IDEs usually means leaving the cluster that hosts their pipelines, moving to a standalone JupyterHub deployment or a local laptop. That switch leaves them without the GPU nodes, shared storage, and AWS Identity and Access Management (IAM) roles their pipelines depend on. The Amazon SageMaker AI Spaces add-on for Amazon EKS closes that gap. It runs managed JupyterLab and Code Editor environments on the cluster that you already operate. Standing up a standalone JupyterHub environment with GPU access, storage, and authentication typically takes a platform team 3–5 days. With the add-on, a data scientist launches a fully configured Space in about 5 minutes. In this post, you install the SageMaker AI Spaces add-on on an Amazon EKS cluster. You set up the supporting add-ons and IAM roles, deploy the AWS Load Balancer Controller, request a TLS certificate, and create an AWS Key Management System (AWS KMS) encryption key. You then create your first Space and reach it through a presigned URL in the browser and from VS Code over SSH-over-SSM. Finally, you review how to move your team to OpenID Connect (OIDC) sign-in with Amazon Cognito. Solution overview The solution runs on a single EKS cluster in three layers: Network and access. Amazon Route 53 resolves a wildcard domain to an internet-facing Application Load Balancer (ALB) with TLS from AWS Certificate Manager (ACM). For VS Code, AWS Systems Manager tunnels directly to the Space pod. Cluster routing. The AWS Load Balancer Controller provisions the ALB. Traefik routes by hostname. Auth middleware validates tokens using AWS Key Management Service (AWS KMS) for JSON Web Token (JWT) encryption. Compute and storage. Space pods run on private-subnet workers. The Amazon Elastic Block Store (Amazon EBS) CSI driver provides persistent volumes, and Amazon Elastic File System (Amazon EFS) or Amazon FSx handle shared or high-throughput storage. EKS Pod Identity grants pods scoped IAM roles. Consolidating interactive and training workloads on one cluster keeps GPU nodes busy between jobs. This can lift GPU utilization by up to 30 percent compared with a dedicated notebook fleet. It also avoids the cost of an always-on GPU environment, which can run into thousands of dollars a month. Figure 1: Solution architecture Prerequisites To follow along, you need an AWS account with the AWS Command Line Interface (AWS CLI) 2.x or later configured for your target AWS Region, plus kubectl 1.30 or later and Helm v3. You also need a Route 53 public hosted zone for a domain you own, referenced as throughout this post, and IAM permissions to create roles, policies, EKS add-ons, access entries, Pod Identity associations, ACM certificates, and KMS keys. The Spaces add-on must be version 0.1.4 or later, because earlier versions supported Amazon SageMaker HyperPod only. Figure 2: Route 53 hosted zone with validation records Set these variables once. The rest of the post reuses them. export CLUSTER_NAME= export REGION= export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) Every IAM role in this post is assumed by a Kubernetes service account through EKS Pod Identity, so they all share one trust policy. Save it once and reuse it: cat > pod-identity-trust.json

Source: AWS Machine Learning — Published — Category: Tools

🔗 Read full article on AWS Machine Learning →