Distributed training with Ray Train DLCs and EFA on EKS
Distributed training with Ray Train DLCs and EFA on EKS
This AWS post shows how to fine-tune Qwen2.5-1.5B with PyTorch FSDP across eight NVIDIA L4 GPUs on two EFA-enabled g6.12xlarge instances on Amazon EKS, using the published Ray Train DLC (public.ecr.aws/deep-learning-containers/ray:train-ml-cuda-v1.1) without building any image. KubeRay creates the Ray head and worker pods from one RayCluster manifest, while hardware, transport and per-job Python packages are configured from outside the image via resource requests, environment variables and Ray runtime_env. The sample repository includes idempotent deploy scripts that locate g6.12xlarge capacity, provision the cluster and node group, install the GPU and EFA device plugins and KubeRay, and submit the job from inside the head pod, with the script failing if NCCL did not use EFA.
Source: AWS Machine Learning Blog · aws.amazon.com