Populate abbreviated declaration fragments in external render nodes

[anferbui]

Bug/issue #, if applicable: rdar://152652751&156488184

Summary

Improves the quality of the navigator titles for external render nodes. Navigator titles are derived from the item's declaration fragments when at least one of the following is met 1:

• the source language is Swift
• there is no available navigator title
• the page type is not part of a curated list 2

These declaration fragments are expected to be the abbreviated declaration fragments 3 of the symbol.

This PR helps capture the abbreviated declaration fragments of the symbol for external entities, so that they can be used in the navigator. The final goal is to have external entities in the navigator render no differently from local entities.

Implementation

There was no change needed to the LinkResolver.ExternalEntity type, as some declaration fragments were already being stored as part of LinkResolver.ExternalEntity.TopicRenderReference.fragmentsVariants 4.

The main issue was that while OutOfProcessReferenceResolver.ResolvedInformation relied on the fragments that it was receiving from the OutOfProcessReferenceResolver to be the abbreviated declaration fragments, LinkDestinationSummary was only encoding the full declaration fragments only as part of its summary:

swift-docc/Sources/SwiftDocC/LinkTargets/LinkDestinationSummary.swift

Line 445 in 1b4a185

let declaration = (symbol.declarationVariants[summaryTrait] ?? symbol.declaration).renderDeclarationTokens()

swift-docc/Sources/SwiftDocC/LinkTargets/LinkDestinationSummary.swift

Line 454 in 1b4a185

let declarationVariant = symbol.declarationVariants[trait]?.renderDeclarationTokens()

The abbreviated declaration fragments are different from the full declaration fragments and defined as a different field (subHeading 5) in the symbol graph (and therefore cannot be derived by DocC), but they're ultimately processed and captured as part of RenderMetadata.fragmentsVariants 6.

The main changes needed were to update the documentation of OutOfProcessReferenceResolver.ResolvedInformation and LinkDestinationSummary to have an extra property, subHeadingDeclarationFragments, which stores the abbreviated declaration fragments. These declaration fragments can then be used when initialising LinkResolver.ExternalEntity.TopicRenderReference.fragmentsVariants.

More detail on the implementation can be found in commit 832f1eb.

Alternatives considered

Considered modifying LinkDestinationSummary.declarationFragments instead of adding a new optional field, however the full declaration fragments are used to determine the full name 7 of the symbol for diagnostic reporting 8.

By introducing a new field we also ensure this is a non-breaking change.

Navigator comparison (using swift-docc-render):

This compares what it looks like to refer to those symbols within its local DocC bundle versus what it would look like to curate them as external links:

Local	External

And if we also bring in the symbol kind changes from #1251, it becomes even closer:

Local	External

Dependencies

N/A

Testing

Verified by using 2 bundles -- one for generating linkable entities, another for referencing an external link from the other bundle -- and verifying that the declaration fragments in the linkable entities are the abbreviated, subheading declaration fragments.

Steps:

1. Download and extract the test bundles: Example.zip
2. Build the linkable entities for the first bundle: swift run docc preview ExampleA.docc --emit-digest
   a. Look at http://localhost:8000/documentation/mykit
   b. Look at the Topics section and the navigator titles for both the class and the function
3. Modify the link resolver in the second bundle to resolves the updated linkable entities for the first bundle
4. Build the linkable entities for the second bundle: DOCC_LINK_RESOLVER_EXECUTABLE=ExampleB.docc/bin/test-data-external-resolver swift run docc preview ExampleB.docc
5. Verify that the Topics section and the navigator titles match the ones from step 2b.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• Added Updated tests
• Ran the ./bin/test script and it succeeded
• Updated documentation if necessary

[anferbui]

Some tests are failing in ExternalPathHierarchyResolverTests which look like the error diagnostics were requiring the full declaration fragments for external links. Needs investigation.

The full declaration fragments are used to determine the full name of the symbol for diagnostic reporting:

swift-docc/Sources/SwiftDocC/Infrastructure/Link Resolution/PathHierarchy+Error.swift

Lines 115 to 117 in 1b4a185

	return Solution(summary: "\(Self.replacementOperationDescription(from: foundDisambiguation, to: suggestedDisambiguation, forCollision: true)) for \n\(fullName.singleQuoted)", replacements: [
	Replacement(range: replacementRange, replacement: suggestedDisambiguation)
	])

I think we will need a new field in LinkDestinationSummary to contain the abbreviated declaration fragments rather than re-using the existing one.

[anferbui]

I've updated the implementation so that all unit tests should now pass.

[anferbui]

@swift-ci please test

[anferbui]

@swift-ci please test

[sofiaromorales]

Thanks for the incredible work on documenting this PR, Andrea!

The only question I have about the implementation is if TopicRenderReference only takes fragments, and not subheadingFragments [1], why do we need to add subHeadingDeclarationFragments into the ResolvedInformation [2]? Can't we simply keep the summarized version and default its value to the full declaration from the LinkDestinationSummary in case this is nil?

[1] https://github.com/swiftlang/swift-docc/pull/1255/files#diff-c195a5776f681e560e52fc959b22479573bbc847217e4ba4b66c0a0f29f31424L169
[2] https://github.com/swiftlang/swift-docc/pull/1255/files#diff-c195a5776f681e560e52fc959b22479573bbc847217e4ba4b66c0a0f29f31424R584

[d-ronnqvist]

Overall this looks good but I feel that we should make the public API clearer (since that's harder to change later). Also, we should add the new JSON key to the specification file.

[d-ronnqvist]

Question: Is this representative of real data? It looks odd to me that the return type isn't included.

[anferbui]

Yes, from looking at real examples I believe this is representative of real data and that the return type is not included in the subheading -- however I'm not the most familiar with Objective C symbol graphs so I might be wrong.

[anferbui]

This PR is depending on #1300 being merged, as the changes it was depending on were reverted. There are still some comments to address here so I will do that next.

I measured the performance impact of these changes using the bin/benchmark tool. As a summary, these are the changes that have been introduced to LinkDestinationSummary:

• declaration fragments are now shorter
• a plaintext declaration of the symbol is now included
• the navigator title of the symbol is now included

The overall performance impact is, over 10 iterations:

┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Metric                                   │ Change          │ 9413c599817abc9dd4f8feeed57a998b19a05a6f │ current              │
├──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Duration for 'convert-total-time'        │ -1.955%¹        │ 32.844 secs                              │ 32.202 secs          │
│ Duration for 'documentation-processing'  │ -7.115%²        │ 9.556 secs                               │ 8.876 secs           │
│ Duration for 'finalize-navigation-index' │ no change³      │ 0.239 secs                               │ 0.24 secs            │
│ Peak memory footprint                    │ no change⁴,⁵    │ 8.34 GB                                  │ 8.36 GB              │
│ Data subdirectory size                   │ no change⁶      │ 307.1 MB                                 │ 307.1 MB             │
│ Index subdirectory size                  │ no change⁷      │ 11 MB                                    │ 11 MB                │
│ Total DocC archive size                  │ -1.649%⁸        │ 472.5 MB                                 │ 464.7 MB             │
│ Topic Anchor Checksum                    │ no change       │ 20b545cce59ffb693c82ce1d12771e6e         │ 20b545cce59ffb693c82 │
│ Topic Graph Checksum                     │ no change       │ 9cacd71cfd29c7f7f622bf2c49ad48ca         │ 9cacd71cfd29c7f7f622 │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

where 9413c59 is the commit I was comparing against.

The results above show that the DocC archive size is now smaller compared to previously.

[d-ronnqvist]

Question: since this is for the navigator, should it return the navigator title?

[anferbui]

It's meant to return the subheading declaration fragments AFAICT -- for internal links in the navigator, we store this information:

swift-docc/Sources/SwiftDocC/Indexing/Navigator/RenderNode+NavigatorIndex.swift

Lines 71 to 76 in 257c62d

	var navigatorTitle: [DeclarationRenderSection.Token]? {
	wrapped.navigatorTitleVariants.value(for: traits)
	}
	var fragments: [DeclarationRenderSection.Token]? {
	wrapped.fragmentsVariants.value(for: traits)
	}

Where this type is wrapping RenderMetadata and these fragments are derived from the symbol subheading fragments:

swift-docc/Sources/SwiftDocC/Model/Rendering/RenderNode/RenderMetadata.swift

Lines 124 to 131 in 257c62d

	/// Abbreviated declaration to display in links.
	public var fragments: [DeclarationRenderSection.Token]? {
	get { getVariantDefaultValue(keyPath: \.fragmentsVariants) }
	set { setVariantDefaultValue(newValue, keyPath: \.fragmentsVariants) }
	}
	/// The variants for the fragments of a page.
	public var fragmentsVariants: VariantCollection<[DeclarationRenderSection.Token]?> = .init(defaultValue: nil)

swift-docc/Sources/SwiftDocC/Model/Rendering/RenderNodeTranslator.swift

Line 1305 in 257c62d

node.metadata.fragmentsVariants = contentRenderer.subHeadingFragments(for: documentationNode)

And then finally there's some logic when deriving the final navigator title, to choose to use either the subheading fragments or the navigator fragments:

swift-docc/Sources/SwiftDocC/Indexing/Navigator/RenderNode+NavigatorIndex.swift

Lines 141 to 157 in 257c62d

	/// Returns a navigator title preferring the fragments inside the metadata, if applicable.
	func navigatorTitle() -> String? {
	let tokens: [DeclarationRenderSection.Token]?
	// FIXME: Use `metadata.navigatorTitle` for all Swift symbols (github.com/swiftlang/swift-docc/issues/176).
	if identifier.sourceLanguage == .swift || (metadata.navigatorTitle ?? []).isEmpty {
	let pageType = navigatorPageType()
	guard !typesThatShouldNotUseNavigatorTitle.contains(pageType) else {
	return metadata.title
	}
	tokens = metadata.fragments
	} else {
	tokens = metadata.navigatorTitle
	}
	return tokens?.map(\.text).joined() ?? metadata.title
	}

[d-ronnqvist]

This looks good to me.

[anferbui]

@swift-ci please test