Kueue v0.17: What’s new

Kueue v0.17 brings some solid architectural updates. I’ve put together a look at my top highlights:

Priority Boost: More flexible priority management

Kueue now offers the ability to adjust a workload’s importance dynamically. Introduction of the new **kueue.k8s-x.io/priority-boost** annotation allows external controllers to change the effective priority of a workload on top of existing priority class. This change allows for several scenarios that were previously difficult to orchestrate:

• Checkpoint-Aware Scheduling: You can now apply a boost to workloads that have reached a significant milestone, preventing Kueue from preempting them and wasting hours of expensive progress.
• Starvation Prevention: Low-priority jobs could be continuously preempted. Now, external controllers can increase their priority to ensure they cross the finish line.
• Phase-based Prioritization: Priority can be adjusted based on the job’s lifecycle — for example the running workload can get a higher boost than the one that is still initializing.

Ecosystem integration

Kueue v0.17 broadens its support for distributed framework controllers:

• Native RayService Support: Kueue now natively manages RayService workloads alongside RayCluster and RayJob. As the Ray framework remains a primary driver for elastic workloads, it is important to note that these now benefit from topology-aware scheduling to further optimize resource placement.
• **Kubeflow Trainer v2.2 Support:** The newly released Trainer v2.2 is now fully integrated into the Kueue ecosystem, expanding support for standardized ML training workflows.
• Kubeflow SparkApplication Support: Kueue now officially welcomes the Apache Spark community, providing native management for Spark-based workloads to streamline big data processing within the ecosystem.
• LeaderWorkerSet (LWS) in MultiKueue: MultiKueue now supports LWS workloads, enabling the seamless dispatch of complex, multi-node AI models across distributed clusters.

Scheduler Improvements for high-pressure scenarios

Historically, the Kueue scheduler could encounter “stuck” states when queues were congested by large-scale jobs. v0.17.0 addresses these bottlenecks with critical architectural enhancements:

• SchedulerLongRequeueInterval Feature Gate: It extends the requeue interval for inadmissible workloads from 1s to 10s. In high-pressure environments with massive pending queues, this prevents the scheduler from getting trapped in a loop of re-evaluating the same top-of-queue workloads, allowing it to process deeper, schedulable tasks.
• SchedulingEquivalenceHashing Feature Gate: By grouping identical workloads through equivalence hashing, Kueue moves away from individual job evaluation. If a representative workload is flagged as “NoFit,” the scheduler can now bulk-bypass all redundant workloads, significantly reducing redundant compute cycles.

For a comprehensive breakdown of these scheduler optimizations, explore my detailed technical deep dive.

Advanced Orchestration

Kueue v0.17 significantly matures the way Kueue handles complex infrastructure and specialized hardware by unifying several core resource management pillars:

• **DRA Support for Extended Resources: **To provide a smooth transition from the legacy device plugin model to Dynamic Resource Allocation (DRA), Kueue now supports extended resource syntax. This ensures that as vendors update their drivers, existing workloads maintain consistent quota management within Kueue. This integration is vital to prevent tracking errors, such as double-counting or missed resource counts, when using DRA-backed DeviceClasses.
• Multi-Layer Topology: Topology Aware Scheduling (TAS) logic now supports up to 3 hierarchical slice layers. This is a critical update for large-scale clusters, enabling much more complex data-center topology mapping than the single-layer approach.
• **Orchestrated Preemption in MultiKueue:** A centralized layer to prevent simultaneous workload evictions caused by worker cluster isolation. This is managed via the new PreemptionGate API, which regulates which workloads are permitted to trigger preemption.

Better Observability

In previous versions, Kueue metrics were limited to built-in dimensions, often leaving a visibility gap when trying to map infrastructure usage back to business units.

• **Custom Metric Labels**: With the new metrics configuration, you can now propagate metadata (like department, project, or cost-center) from your ClusterQueue or Cohort objects directly into Prometheus. This allows for far more granular reporting.
• **Common Grafana Queries:** A set of curated PromQL queries is now available in the official documentation to help batch administrators gain deeper insights into their cluster performance and resource utilization.
• **New Cohort-level Metrics:** Administrators can now monitor the resource health of entire organizational branches within cohorts.

For a comprehensive list of all updates and changes, you can explore the official changelog.

If you found this article helpful, **follow me** for more deep dives into Kueue and the evolving landscape of Kubernetes workload scheduling.