Use `DNSEndpoint` for all records

[lucasfcnunes]

What would you like to be added:

Use the kind DNSEndpoint for all generated DNS records.

• When a service, ingress, etc needs to have their records created, it should always trigger the creation of a respective DNSEndpoint with a metadata.ownerReferences[0] self-reference in it.
• The real resource referenced in metadata.ownerReferences[0] shouldn't be used for anything but to generate and refresh their respective DNSEndpoint content.
• Only the DNSEndpoint should be used as the source of truth for the syncing of all records.

Example (merely illustrative):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: airflow-ingress
  namespace: airflow
  annotations:
    external-dns.alpha.kubernetes.io/cloudflare-proxied: "false"
spec:
  ingressClassName: nginx
  rules:
    - host: airflow.example.com
      http:
        paths:
          - backend:
              service:
                name: airflow-webserver
                port:
                  name: airflow-ui
            path: /
            pathType: ImplementationSpecific
    - # ...
status:
  loadBalancer:
    ingress:
      - ip: 123.4.5.6
---
apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
  name: airflow-ingress
  namespace: airflow
  ownerReferences:
    - apiVersion: networking.k8s.io/v1
      blockOwnerDeletion: true
      controller: true
      kind: Ingress
      name: airflow-ingress
spec:
  endpoints:
    - dnsName: airflow.example.com
      recordType: A
      targets:
        - 123.4.5.6
      providerSpecific:
        - name: external-dns.alpha.kubernetes.io/cloudflare-proxied
          value: "false"
    - # ...
status:
  conditions:
    - lastTransitionTime: "2024-10-14T10:24:24Z"
      message: DNSEndpoint is up to date
      observedGeneration: 1
      reason: Ready
      status: "True"
      type: Ready
  notAfter: "2025-01-12T09:25:51Z"
  notBefore: "2024-10-14T09:25:52Z"
  renewalTime: "2024-12-13T09:25:51Z"
  revision: 1

Why is this needed:

• Transparency on what external-dns is doing or trying to do with the domains
• Keep track of all DNS records
• Possibility of a detailed status for all records being synced
  • e.g.
    • [Feature Request] Domain Expiration Info #4609
• ownerReferences is used in multiple systems e.g. Argo CD to link related k8s resources
• Possibility to kubectl get DNSEndpoint -A -o yaml and have the actual state of all managed records
• Make external-dns flow more like what we see in cert-manager (DNSEnpoint <=> Certificate)

PS

There are architectural difficulties to be considered. Some are:

• DNSEnpoint name collision.
  • solutions:
    • prefix/suffix it with the kind of the owner)
• Same record (dnsName, recordType, target) in distinct DNSEnpoint resources.
  • solutions:
    • same as always has been when it happens
• txtOwnerId
• txtPrefix, txtSuffix
• ?

[laserpedro]

In my case we need to create records from DNSEndpoints resources coming from the AW Gateway API Controller and therefore we need to pass extra arguments to the controller which creates two ways of managing records in the cluster, via the ingress and via the dns endpoints, I agree with the above it would be easier to manage records in a unified way. Also in my case, and it is not mentioned by the doc, we do not know the controller behavior once we add as ExtraArgs:
--source crd --crd-source-apiversion externaldns.k8s.io/v1alpha1 --crd-source-kind DNSEndpoint
regarding the records that are already managed using Ingresses ...

[ivankatliarchuk]

/help

[k8s-ci-robot]

@ivankatliarchuk:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ivankatliarchuk]

link #5079

[hjoshi123]

@ivankatliarchuk I can work on this and will comment if I have any queries regarding this

[hjoshi123]

@ivankatliarchuk @lucasfcnunes do we handle the deletion of DNSEndpoints too or are they just one to one of the owner reference?

Also, I wanted to know if adding a flag helps in this scenario, where if the flag is turned on we only use the DNSEndpoint source as the truth else keep the normal behavior of using svc/ingress/node etc as source

[hjoshi123]

/assign

[lucasfcnunes]

do we handle the deletion of DNSEndpoints too or are they just one to one of the owner reference?

We could do as cert-manager does it - I think they

• delete everything normally (as if it didn't have an owner)
• then (re)create in the next loop

Also, I wanted to know if adding a flag helps in this scenario, where if the flag is turned on we only use the DNSEndpoint source as the truth else keep the normal behavior of using svc/ingress/node etc as source

A flag is a good way to enforce a feature gate, backward compatibility and alpha > beta > GA flows.

[hjoshi123]

yup that tracks with my approach. thank you @lucasfcnunes

[hjoshi123]

@lucasfcnunes / @ivankatliarchuk quick question.. so how would we know which source to track? In the sense if there is service and an ingress what do we create DNSEndpoint for?

[lucasfcnunes]

@lucasfcnunes / @ivankatliarchuk quick question.. so how would we know which source to track? In the sense if there is service and an ingress what do we create DNSEndpoint for?

Both. #4859 (comment) (on DNSEnpoint name collision.)