Skip to content

Ephemeral-Storage in MaxNodePoolResources Uses Raw Bytes (×1000 Off) in Error Message #400

@JosefNagelschmidt

Description

@JosefNagelschmidt

What happened?

In KAI Scheduler’s MaxNodePoolResources predicate, ephemeral-storage requests and limits are printed in the unschedulable message using raw integer bytes, without unit conversion, and are also off by a factor of 1000 compared to other resources.

MaxNodePoolResources: The pod <mypod> requires GPU: 0, CPU: 2.5 (cores), memory: 3.758 (GB), ephemeral-storage: 32212254720000

Cause:

  • CPU and memory values are converted to cores/GB before formatting.
  • Scalar resources like ephemeral-storage skip conversion and use the stored int64 value directly.
  • The stored value is in bytes, so it shows huge numbers and doesn’t match the GB format used elsewhere.

Impact:

Error messages are misleading, showing unreadable byte counts that are ~1000× larger than the intended GB values.

I hope it does not affect actual scheduling logic and is purely a display/formatting issue.

Reproduce via adding the following test case to the Test_podToMaxNodeResourcesFiltering:

{
	"not enough ephemeral storage",
	args{
		nodesMap: map[string]*node_info.NodeInfo{
			"n1": {
				Allocatable: resource_info.ResourceFromResourceList(v1.ResourceList{
					v1.ResourceCPU:                resource.MustParse("100m"),
					v1.ResourceMemory:             resource.MustParse("200Mi"),
					resource_info.GPUResourceName: resource.MustParse("1"),
					"kai.scheduler/r1":            resource.MustParse("2"),
					v1.ResourceEphemeralStorage:   resource.MustParse("10Gi"),
				}),
			},
			"n2": {
				Allocatable: resource_info.ResourceFromResourceList(v1.ResourceList{
					v1.ResourceCPU:                resource.MustParse("500m"),
					v1.ResourceMemory:             resource.MustParse("200Mi"),
					resource_info.GPUResourceName: resource.MustParse("1"),
					"kai.scheduler/r1":            resource.MustParse("2"),
					v1.ResourceEphemeralStorage:   resource.MustParse("20Gi"),
				}),
			},
		},
		pod: &v1.Pod{
			ObjectMeta: metav1.ObjectMeta{
				Name:      "name1",
				Namespace: "n1",
			},
			Spec: v1.PodSpec{
				Containers: []v1.Container{
					{
						Name: "c1",
						Resources: v1.ResourceRequirements{
							Requests: map[v1.ResourceName]resource.Quantity{
								v1.ResourceEphemeralStorage: resource.MustParse("25G"),
							},
						},
					},
				},
			},
		},
	},
	expected{
		k8sframework.NewStatus(k8sframework.Unschedulable,
			"The pod n1/name1 requires GPU: 0, CPU: 0 (cores), memory: 0 (GB), ephemeral-storage: 25 (GB). "+
				"Max ephemeral-storage resources available in a single node in the default node-pool is topped at 10.737 GB"),
	},
},

What did you expect to happen?

Expected that ephemeral-storage is shown in human readable format.

Environment

  • Kubernetes version: v1.33.3
  • KAI Scheduler version: v0.7.8
  • Cloud provider or hardware configuration
  • Tools that you are using KAI together with
  • Anything else that is relevant

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions