[Epic][Feature] KubeRay Cache Pod Pool

[lw309637554]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

KubeRay Pod Pool Virtual Kubelet is an optional and standalone component of the KubeRay Ecosystem. It provides warmed-up pod pools by registering itself as a virtual kubelet to a Kubernetes cluster. When KubeRay requests pods from the virtual kubelet, the actual pods are taken from one of the pod pools specified by the users and effectively skip:

1. resource scheduling time
2. image pulling time
3. volume preparation time
   because those pods are already active and waiting in pools.

Use case

1. Improve the elasticity and efficiency of job-based operations, such as efficient analysis of Ray data.

2. Improve the efficiency of autoscaling in cluster-based operations to meet peak business load demands. For example, elastic scaling of inference and agent scaling.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!