Add a new version of the communication protocol between DocC and external link resolver executables #1292

d-ronnqvist · 2025-09-22T09:32:43Z

Bug/issue #, if applicable: rdar://159610082 #802 rdar://76656957&77906558&86334416&92254059&120424365 #341

Summary

This adds a new "version 2" of the communication protocol between DocC and external link resolver executables.

Some of the benefits of the new version:

It uses the same structure for successfully resolved pages as DocC creates itself, allowing for more detailed information and less code duplication (Pass "link summaries" instead of "resolved information" in external documentation source responses #802)
It can send more information about why an external link failed to resolve allowing external warnings to display suggestions.
It isn't frozen and can in the future grow to add new requests and responses tied to new capabilities.

One thing to note about the new version; because it encodes the abstract with the full fidelity and includes any references from the abstract, an externally curated page can display both transient links and images. Because of this, the warning about links or images in abstract is no longer needed (and is deprecated, to be removed (#341)).

Dependencies

None.

Testing

The easiest way to test this is by configuring DocC to call bin/test-data-external-resolver (updated to use the new version of the protocol) for external links and then add some external links with a "com.test.bundle" identifier in some minimal content.

Note: the test-data-external-resolver need to be updated to reference an image in the abstract to verify that functionality.

Note: external links on the page won't be clickable unless the archive is merged with an archive with that bundle identifier. To simulate this you can either create a separate archive and merge the two or you can add "com.test.bundle" in the only archive's includedArchiveIdentifiers list in its /index/index.json file.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

Added tests
Ran the ./bin/test script and it succeeded
Updated documentation if necessary

d-ronnqvist · 2025-09-22T09:32:54Z

@swift-ci please test

… capabilities rdar://159610082 swiftlang#802

rdar://76656957 rdar://77906558 rdar://86334416 rdar://92254059 rdar://120424365 swiftlang#341

d-ronnqvist · 2025-09-22T09:49:47Z

@swift-ci please test

d-ronnqvist · 2025-09-22T13:22:52Z

This test failure happens on other PRs as well. I opened #1293 to fix it.

d-ronnqvist · 2025-09-22T13:39:13Z

@swift-ci please test

d-ronnqvist · 2025-09-22T13:45:33Z

@swift-ci please test

d-ronnqvist · 2025-09-22T13:49:36Z

@swift-ci please test

anferbui

The code changes look correct to me, and I didn't spot any issues or bugs in the implementation. I left some comments on areas where it was hard for me to understand why some of the changes were made.

The way the commits were organised with multiple changes within one commit made it hard for me to review and ensure there were no regressions in behaviour being introduced; I would have found it helpful to have longer commit descriptions and/or smaller commits.

Thanks!

...ocC/Infrastructure/External Data/OutOfProcessReferenceResolver+DeprecatedCommunication.swift

Sources/SwiftDocC/Infrastructure/External Data/OutOfProcessReferenceResolver.swift

Tests/SwiftDocCTests/Rendering/RenderMetadataTests.swift

Sources/SwiftDocC/LinkTargets/LinkDestinationSummary.swift

Sources/SwiftDocC/Infrastructure/Link Resolution/LinkResolver+NavigatorIndex.swift

Sources/SwiftDocC/Infrastructure/External Data/OutOfProcessReferenceResolver.swift

Sources/SwiftDocC/Infrastructure/Link Resolution/LinkResolver.swift

anferbui · 2025-09-22T16:22:28Z

Sources/SwiftDocC/Model/Rendering/RenderContext.swift

+            for case let dependency as TopicRenderReference in (entity.references ?? []) {
+                guard let url = URL(string: dependency.identifier.identifier), let rawBundleID = url.host else {
+                    // This dependency doesn't have a valid topic reference, skip adding it to the render context.
+                    continue
+                }
+
+                let dependencyReference = ResolvedTopicReference(
+                    bundleID: .init(rawValue: rawBundleID),
+                    path: url.path,
+                    fragment: url.fragment,
+                    // TopicRenderReference doesn't have language information. Also, the reference's languages _doesn't_ specify the languages of the linked entity.
+                    sourceLanguages: reference.sourceLanguages
+                )
+                topics[dependencyReference] = .init(renderReference: dependency, canonicalPath: nil, taskGroups: nil, source: nil, isDocumentationExtensionContent: false)
+            }


I'm not sure what this is fixing, chould we add this in a separate PR to reduce the amount of things this PR is changing?

No. This change is necessary for this to work. It ensures that any topic references in the external entity's abstract is included in the linking page's references section.

If you want to see what this is for you can try commenting it out and see which tests fail.

Adding a comment here with "This logic ensures that any topic references in the external entity's abstract are included in the linking page's references section" would be helpful for future maintainability

Tests/SwiftDocCTests/Infrastructure/TestExternalReferenceResolvers.swift

Tests/SwiftDocCTests/LinkTargets/LinkDestinationSummaryTests.swift

Sources/SwiftDocC/Infrastructure/External Data/OutOfProcessReferenceResolver.swift

d-ronnqvist · 2025-09-23T17:24:12Z

@swift-ci please test

d-ronnqvist · 2025-09-23T17:28:53Z

@swift-ci please test

d-ronnqvist · 2025-09-23T17:35:30Z

@swift-ci please test

anferbui

LGTM, thank you for addressing my comments

d-ronnqvist · 2025-09-24T12:15:03Z

@swift-ci please test

…and external link resolver executables (swiftlang#1292)" This reverts commit 9413c59.

…and external link resolver executables (#1292)" (#1299) This reverts commit 9413c59.

…en DocC and external link resolver executables (#1292)" (#1299)" 257c62d

* Revert "Revert "Add a new version of the communication protocol between DocC and external link resolver executables (#1292)" (#1299)" 257c62d * Update missed Windows API in #if conditional

d-ronnqvist added 2 commits September 19, 2025 10:22

Move old request and response to a separate file

16c379d

Introduce new request and response versions

1c8fd95

d-ronnqvist requested a review from anferbui September 22, 2025 09:32

d-ronnqvist added 3 commits September 22, 2025 11:49

Allow external link executables to more information based on declared…

046c2f0

… capabilities rdar://159610082 swiftlang#802

Support transient references in external content

0c87cd0

Stop emitting no longer necessary warning about external references

dfc61c4

rdar://76656957 rdar://77906558 rdar://86334416 rdar://92254059 rdar://120424365 swiftlang#341

d-ronnqvist force-pushed the external-resolver-protocol-v2 branch from 14a4f88 to dfc61c4 Compare September 22, 2025 09:49

Merge branch 'main' into external-resolver-protocol-v2

6a25aa9

Remove trailing comma in parameters for Swift 5.9 compatibility

7f5678c

This was linked to issues Sep 22, 2025

Pass "link summaries" instead of "resolved information" in external documentation source responses #802

Closed

Warning: "Link in document summary will not be displayed." is inaccurate. #341

Closed

d-ronnqvist mentioned this pull request Sep 22, 2025

Warning: "Link in document summary will not be displayed." is inaccurate. #341

Closed

2 tasks

anferbui reviewed Sep 22, 2025

View reviewed changes

d-ronnqvist commented Sep 22, 2025

View reviewed changes

Tests/SwiftDocCTests/LinkTargets/LinkDestinationSummaryTests.swift Outdated Show resolved Hide resolved

d-ronnqvist commented Sep 22, 2025

View reviewed changes

Tests/SwiftDocCTests/LinkTargets/LinkDestinationSummaryTests.swift Outdated Show resolved Hide resolved

d-ronnqvist commented Sep 22, 2025

View reviewed changes

Sources/SwiftDocC/Infrastructure/External Data/OutOfProcessReferenceResolver.swift Outdated Show resolved Hide resolved

d-ronnqvist commented Sep 22, 2025

View reviewed changes

Sources/SwiftDocC/Infrastructure/External Data/OutOfProcessReferenceResolver.swift Outdated Show resolved Hide resolved

d-ronnqvist force-pushed the external-resolver-protocol-v2 branch from 005d1ca to cf1dfb4 Compare September 23, 2025 17:28

d-ronnqvist force-pushed the external-resolver-protocol-v2 branch from cf1dfb4 to 2c6beba Compare September 23, 2025 17:35

anferbui approved these changes Sep 24, 2025

View reviewed changes

Address code review feedback

1b7c3f9

Add missing public initializers to new response diagnostic info type

cced91e

d-ronnqvist force-pushed the external-resolver-protocol-v2 branch from 2c6beba to cced91e Compare September 24, 2025 12:14

Merge branch 'main' into external-resolver-protocol-v2

1663b20

d-ronnqvist merged commit 9413c59 into swiftlang:main Sep 24, 2025
2 checks passed

d-ronnqvist deleted the external-resolver-protocol-v2 branch September 24, 2025 12:19

Steelskin mentioned this pull request Sep 24, 2025

Fix Windows build after 9413c59 #1298

Closed

kavon added a commit to kavon/swift-docc that referenced this pull request Sep 25, 2025

Revert "Add a new version of the communication protocol between DocC …

df5d553

…and external link resolver executables (swiftlang#1292)" This reverts commit 9413c59.

kavon mentioned this pull request Sep 25, 2025

Unblock Swift CI #1299

Merged

eeckstein pushed a commit that referenced this pull request Sep 25, 2025

Revert "Add a new version of the communication protocol between DocC …

257c62d

…and external link resolver executables (#1292)" (#1299) This reverts commit 9413c59.

d-ronnqvist added a commit that referenced this pull request Sep 25, 2025

Revert "Revert "Add a new version of the communication protocol betwe…

1dccb57

…en DocC and external link resolver executables (#1292)" (#1299)" 257c62d

Add a new version of the communication protocol between DocC and external link resolver executables #1292

Add a new version of the communication protocol between DocC and external link resolver executables #1292

Uh oh!

Conversation

d-ronnqvist commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Dependencies

Testing

Checklist

Uh oh!

d-ronnqvist commented Sep 22, 2025

Uh oh!

d-ronnqvist commented Sep 22, 2025

Uh oh!

d-ronnqvist commented Sep 22, 2025

Uh oh!

d-ronnqvist commented Sep 22, 2025

Uh oh!

d-ronnqvist commented Sep 22, 2025

Uh oh!

d-ronnqvist commented Sep 22, 2025

Uh oh!

anferbui left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

anferbui Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

d-ronnqvist Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

anferbui Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

d-ronnqvist commented Sep 23, 2025

Uh oh!

d-ronnqvist commented Sep 23, 2025

Uh oh!

d-ronnqvist commented Sep 23, 2025

Uh oh!

anferbui left a comment

Choose a reason for hiding this comment

Uh oh!

d-ronnqvist commented Sep 24, 2025

Uh oh!

Uh oh!

Uh oh!

d-ronnqvist commented Sep 22, 2025 •

edited

Loading