perf(endpoint): optimize ProviderSpecific to use map for O(1) access

[u-kai]

What does it do ?

This PR optimizes the ProviderSpecific field in the Endpoint struct by changing it from a slice-based implementation ([]ProviderSpecificProperty) to a map-based implementation (map[string]string).
This change improves property access performance from O(n) linear search to O(1) constant-time lookups while maintaining full backward compatibility for CRD users.

Motivation

The current slice-based ProviderSpecific implementation creates significant performance bottlenecks in large-scale environments.
This optimization was identified as a prerequisite for addressing the performance concerns raised in PR #5799.
Before implementing provider-specific property warnings, the underlying data structure needed to be optimized to prevent the warnings themselves from becoming a performance bottleneck.

More

• Yes, this PR title follows Conventional Commits
• Yes, I added unit tests
• Yes, I updated end user documentation accordingly

[k8s-ci-robot]

Hi @u-kai. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ivankatliarchuk]

As you can see, the path is apis/v1alpha1. Moving Endpoints under this folder, means we are doing versioning for Endpoint object. Not against, but not sure if this is the right approach. As now we most likely going to have v1alpha.Endpoint....

I have no solution, not an easy one though

[u-kai]

Thanks for raising this — you’re right that moving Endpoint under apis/v1alpha1 makes it part of the public API surface and therefore versioned.

A few clarifications on intent and impact:

Intentional separation:
We’re explicitly decoupling the CRD type (apis/v1alpha1.Endpoint) from the internal type (endpoint.Endpoint).
This lets us keep the CRD schema stable while giving us freedom to optimize the internal representation (e.g., map-based ProviderSpecific) without leaking those changes into the API.

No user-facing change:
The CRD schema shape for Endpoints remains the same from a user point of view; we only replaced the reference to the internal type with the versioned API type.

On future versions (v1, etc.):
If v1’s Endpoint is identical to v1alpha1, we can avoid extra helpers by using a type alias or a shared converter.
If it diverges, we’ll add a thin conversion layer and still keep the internals map cleanly.
Typically the controller consumes one served version; the apiserver handles storage-version normalization.

Maintenance cost:
We’re aware this assigns versioning responsibility to Endpoint, but it also prevents tight coupling to internals and reduces the risk for future internal refactors — or at least, we believe it can reduce such risks.

[ivankatliarchuk]

Have you generated CRDs, are they still the same or there is diff?

[u-kai]

Sorry, I didn't handle this properly initially.
I found that the Endpoints field should have the omitempty tag.
After adding omitempty to the endpoints field and regenerating the CRDs, there are no diffs.
Everything is now in the proper state.

[ivankatliarchuk]

/ok-to-test

[vflaux]

I'm not sure if we want to maintain code that mimic the default output from fmt?
I would prefer to keep the ProviderSpecificProperty struct (but package-private), build a slice of it from ProviderSpecific and use fmt.Sprintf().
Performance wise your code is faster though.
@ivankatliarchuk wdyt ?

edit:
Something like that:

func (ps ProviderSpecific) String() string {
	data := make([]providerSpecificProperty, 0, len(ps))
	for k, v := range ps {
		data = append(data, providerSpecificProperty{Name: k, Value: v})
	}
	slices.SortFunc(data, func(a, b providerSpecificProperty) int {
		return strings.Compare(a.Name, b.Name)
	})
	return fmt.Sprint(data)
}

[ivankatliarchuk]

Not too shure what this method is for.

[vflaux]

It's to keep the log output identical. But that's mostly for debug logs.

external-dns/endpoint/endpoint.go

Line 383 in 7792e78

log.Debugf(`Skipping endpoint %v because of missing owner label (required: "%s")`, ep, ownerID)

But there is some info logs too:

external-dns/registry/dynamodb.go

Line 305 in abdf8bb

log.Infof("Skipping endpoint %v because owner does not match", ep)

external-dns/source/wrappers/targetfiltersource.go

Line 65 in b31afb4

log.WithField("endpoint", ep).Debugf("Skipping endpoint because all targets were filtered out")

[u-kai]

@ivankatliarchuk
As @vflaux mentioned, this logic is to maintain compatibility of the log output.

[ivankatliarchuk]

it's a map, so the output should be consistent. I think I'm questioning the actual need for this method. Is formatting with %q or %v is not enough?

[ivankatliarchuk]

Hi @mloiseleur. Wdyt, do we want to keep same output in logging or Map : ProviderSpecific map[key1:value1 key2:value2 key3:value3] is just enough?

[mloiseleur]

🤔 Changing logs output is not the goal of this PR.
Nonetheless, it changes the struct, so it can be somehow expected that it impacts log output.

The default output provided by Go on map is fine and well known in go software.
My recommendation is to use it as a start, for this PR.

We'll see with time if it's not enough and if we need to update it with specific code and/or struct.

@vflaux @ivankatliarchuk @u-kai Wdyt ? Does it make sense to you ?

[u-kai]

Thanks for the feedback.
My original motivation for adding the custom String() was to keep backward compatibility in log output — since the struct changed, I wanted to avoid surprising changes in log messages.

That said, I’m also fine with simplifying the code and just returning the default map output. It makes the code cleaner, though it will slightly change the log format. If we go this route, I think it’s worth mentioning in release notes so users are aware of the change.

If you prefer that approach, I can drop the custom String() method and just let it print the map directly.

[mloiseleur]

Yes, I prefer the simpler approach and drop String() method.
No problem to add a line on this in the next release notes.

[u-kai]

I've removed the String() method and related tests as suggested.
@mloiseleur @ivankatliarchuk
Please review when you have a chance 🙇

[vflaux]

Maybe add a test case for this as this is not covered?

[u-kai]

I'll address the test coverage for the if ep.ProviderSpecific !=nil branch in a separate PR.
The current tests use makeZone helper function which doesn't set ProviderSpecific, and modifying it would require updating all existing tests that depend on it.
A dedicated PR focused on test infrastructure improvements would be more appropriate to maintain consistency across the codebase.

[ivankatliarchuk]

I dunno. When I merge with master I just do git merge master, but hard to say what exact configs applied in the background

[ivankatliarchuk]

Was it tested on real environment? The potential risk here - it may keep try to recreate/update same endpoints over and over again, due some of the changes.

[u-kai]

@ivankatliarchuk

I dunno. When I merge with master I just do git merge master, but hard to say what exact configs applied in the background

Alright, then I’ll resolve conflicts with --no-rebase for now.

Was it tested on real environment? The potential risk here - it may keep try to recreate/update same endpoints over and over again, due some of the changes.

I tested the following scenario and didn’t see any issues: Ran with v0.19.0 using DNSEndpoint and Ingress as sources.

Then switched to the version with this change applied and confirmed it kept working fine. After that, modified the record contents and verified the DNS records were updated accordingly.

Of course, it’s not realistic to cover every possible case. That said, since this change doesn’t alter any external
interfaces and all unit tests are still passing, I think the risk of problems is quite low.
If you have any specific scenarios in mind where this could be risky, I’d be happy to check them.

[mloiseleur]

The code LGTM.
🤔 The last thing we should double check / confirm: does it have impact on webhook providers ?

[u-kai]

@mloiseleur
Thanks for flagging this — you’re absolutely right.
This does affect webhook providers: we need to align the types that webhooks receive and providers send with the CRD-defined types, and ensure backward compatibility (likely via an adapter/translation layer).
I’m working on this now and will follow up with an update. If you have a specific webhook provider in mind, I can test against it as well.

[u-kai]

@mloiseleur
Verified webhook provider compatibility by adding an adapter layer between internal and API endpoint types.

• Implemented conversion functions
• Updated webhook to use the adapter
• Added comprehensive tests

All tests pass and backward compatibility is maintained. Ready for review!

[ivankatliarchuk]

Overall lgtm,. few questions left to resolve

[ivankatliarchuk]

Is deepcopy no longer works here? Not sure if we should be using endpoint.Endpoint in refactorings, dedicated methods more preferred, so we could incapsulate things in the future.

Why this code is required, with null checks and map copy?

if ep.Labels != nil {
			v4EP.Labels = make(endpoint.Labels, len(ep.Labels))
			maps.Copy(v4EP.Labels, ep.Labels)
		}

		if ep.ProviderSpecific != nil {
			v4EP.ProviderSpecific = make(endpoint.ProviderSpecific, len(ep.ProviderSpecific))
			maps.Copy(v4EP.ProviderSpecific, ep.ProviderSpecific)
		}

My understanding v4EP created without labels or provider specific, is this not just?

v4ep.Labels = ep.Labels

[u-kai]

On DeepCopy:
This code now uses an internal Endpoint type instead of the CRD code-generated one, so we don’t have generated DeepCopy methods anymore.
I don’t think it’s worth re-implementing DeepCopy just for this use case—being explicit in the adapter keeps the conversion type-safe and makes future encapsulation easier.

On the nil checks and map copy:
A direct assignment would share the same map (Go maps are reference types), so changes to the converted value could leak back to the source.
Copying avoids aliasing and preserves nil vs empty.

[ivankatliarchuk]

[ivankatliarchuk]

[ivankatliarchuk]

name crdEp to specific and method description is missing for exported method

[ivankatliarchuk]

Suggested change

	Copyright 2017 The Kubernetes Authors.
	Copyright 2025 The Kubernetes Authors.

[ivankatliarchuk]

same as other method

[ivankatliarchuk]

is the map copy even required, why not a direct assignment?

[u-kai]

On the map copy: direct assignment would just alias the underlying map, since maps in Go are reference types.
That would mean v4EP and ep share the same backing map, and any mutation on one would affect the other.
To avoid this aliasing we explicitly copy into a new map, so the converted object is fully independent.

This struct is used in a variety of ways in tests, and we wanted to minimize any chance of shared state leaking between instances.
Making explicit copies ensures data isolation and makes test behavior more predictable.

[ivankatliarchuk]

[u-kai]

Since we are only encoding valid types into an in-memory buffer, this call cannot realistically fail.
Because the failure path is not reproducible in this context, we are intentionally not covering it with tests.

[ivankatliarchuk]

It was not covered before, which probably explains why tests were green when there was an issue with webhook

[ivankatliarchuk]

The name ToInternalEndpoint not necessary is clear engough. external-dns Endpoint is not internal, as is exported. I do get the meaning probably slighly different here.

Maybe ToAPI and FromApi or ToVersionedApi and FromVersionedApi

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from ivankatliarchuk. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[u-kai]

@ivankatliarchuk
Thank you very much for all the reviews!
I’ve made the necessary fixes and replied to the comments where a response was needed.
I’d appreciate it if you could take another look.

[ivankatliarchuk]

Overall lgtm. Need to find some time to execute few smoke tests on my side.

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.