Fetch full documentation in code completion #2207

a7medev · 2025-07-15T19:43:33Z

When resolving documentation for code completion items, we fetch full documentation through the newly added swiftide_completion_item_get_doc_full_copy SourceKitD function, if not found we fallback to brief documentation as before using swiftide_completion_item_get_doc_brief.

Note

Unlike brief documentation, SourceKitD doesn't cache full documentation for completion results to avoid bloating memory with a lot of large strings.

As of now, SourceKit-LSP doesn't cache completion item documentation either, should we introduce a new full documentation cache (e.g. using LRUCache)?

… full doc is null

ahoppen

Very cool stuff. Excited to see this. I left a couple comments inline.

Sources/SourceKitD/sourcekitd_functions.swift

Sources/Csourcekitd/include/CodeCompletionSwiftInterop.h

ahoppen · 2025-07-16T09:17:20Z

Sources/SwiftSourceKitPlugin/ASTCompletion/CompletionSession.swift

+    session.sourcekitd.ideApi.completion_item_get_doc_full_copy(session.response, rawItem) {
+      if let cstr = $0 {
+        result = String(cString: cstr)
+        free(cstr)


Do we need to free the pointer here? I would assume that the memory is still owned by sourcedkitd since it only yields the pointer to us in a closure and is thus still responsible for its lifetime.

sourcekitd doesn't keep full documentation comments in-memory, it prints them in a temporary buffer and passes a copy through strdup that clients should free (hence the name is suffixed with _copy).

If we pass ownership to the client, I think completion_item_get_doc_full_copy should return the pointer. E.g. like swiftide_completion_result_description_copy.

I just modified completion_item_get_doc_full to not strdup the string and pass it as is to the handler, and SourceKit-LSP will copy it into a Swift String anyway and it doesn't have to manually call free.

Can you please recheck?

ahoppen · 2025-07-16T09:19:22Z

Sources/SwiftSourceKitPlugin/CompletionProvider.swift

+    if let fullDocumentation = info.fullDocumentation {
+      response.set(request.sourcekitd.keys.docFullAsXML, to: fullDocumentation)
+    } else {
+      response.set(request.sourcekitd.keys.docBrief, to: info.briefDocumentation)


Open question: Does it make sense to also return the brief documentation in addition to the full documentation? Eg. an editor might want to display a brief documentation of the symbol by default and then expand that to full documentation once the user performs an action.

Do you have a specific flow in mind? AFAIK this doesn't happen in code completion since documentation is only shown when the completion item is selected in the completions list.

For example, you could think about a UI where the brief documentation is displayed in the completion list (similar to how Xcode shows documentation) and only showing the full documentation when the user requests it.

I assume the plugin is used in SourceKit-LSP only, no? If so, then we don't have to return the brief documentation now as we don't need it. We can always add it later if we found a use case for it.

If not, it'd probably make sense to return both yes, especially if it's used by Xcode which already has this use case.

The plugin can potentially be used by any SourceKit client yes, so I think we should return both by default. We can always add options to the request for cases where the client only wants the brief or full doc

Now it makes sense, thanks for the clarification @ahoppen and @hamishknight 🙏🏼

I've changed the request to return both now.

Tests/SwiftSourceKitPluginTests/SwiftSourceKitPluginTests.swift

…ility

…tion

…kitd

ahoppen · 2025-07-17T06:14:10Z

Sources/SourceKitLSP/Swift/CodeCompletionSession.swift

        item.documentation = .markupContent(MarkupContent(kind: .markdown, value: docString))
      }
    }
    return item
  }

+  private static func documentationString(from response: SKDResponseDictionary, sourcekitd: SourceKitD) -> String? {
+    if let docFullAsXML: String = response[sourcekitd.keys.docFullAsXML] {
+      return try? xmlDocumentationToMarkdown(docFullAsXML)


Sorry, I should have noticed this before: We generally prefer to use orLog over try? because it means that the error will get logged instead of silently swallowing it, which helps debugging.

ahoppen · 2025-07-17T06:16:08Z

Tests/SourceKitLSPTests/SwiftCompletionTests.swift

+}
+
+private func sourcekitdSupportsFullDocumentation() async throws -> Bool {
+  let sourcekitdPath = await ToolchainRegistry.forTesting.default!.sourcekitd!


Could you use XCTUnwrap instead of ! to avoid crashing the test execution if for some reason one of these values is nil.

Removed now that we're using SkipUnless ✅

ahoppen · 2025-07-17T06:17:57Z

Tests/SourceKitLSPTests/SwiftCompletionTests.swift

+  line: UInt = #line
+) async throws {
+  let supportsFullDocumentation = try await sourcekitdSupportsFullDocumentation()
+  let expected = supportsFullDocumentation ? expectedFull : expectedBrief


I’m a little concerned with executing a different test path depending on whether sourcekitd supports full documentation because in Swift CI I imagine the sourcekitd will always support full documentation which means that the expectedBrief case will not get tested in Swift CI.

Instead, I would prefer to add a function to SkipUnless and skip the tests if sourcekitd doesn’t support full documentation.

Same comment for assertDocumentation for the sourcekitd plugin.

My only concern is that tests like SwiftCompletionTests.testCompletionBasic will get skipped in environments without full documentation support.
This might be accepted since it's still exercised in Swift CI anyway.

What do you think? Should we just assume full documentation exists with a SkipUnless or do you prefer duplicating the tests each skipped depending on full documentation support to ensure brief documentation is still tested if full documentation isn't supported?

I assumed we'd just skip them if full documentation is not supported with no tests for brief documentation alone (avoiding duplication & relying on the fact that Swift CI is going to have full documentation support anyway) to speedup review.

If you'd prefer another approach please let me know.

CI should be able to execute all tests. Otherwise tests can break and you don’t notice until you run them locally. So, since CI will only able to test full documentation and the brief documentation part is being phased out, we should just test full documentation support.

…c_full

…n sourcekitd" This reverts commit db44320.

ahoppen

Thanks, looks good to me, the only open question is whether we want to return both the brief and the full documentation from the plugin.

Also: Could you format your changes using swift-format as described in https://github.com/swiftlang/sourcekit-lsp/blob/main/CONTRIBUTING.md#formatting?

a7medev · 2025-07-17T19:16:44Z

I have added one more functionality change.
In the initial implementation, I converted XML documentation to markdown using the same existing flow which includes the declaration as part of the documentation.

I found that existing LSP implementations (mainly Clang and TypeScript) don't show the declaration itself as part of the full documentation in completion (contrary to the hover request!) and I think the reason for this is that the completion item already describes the declaration and typically the popup showing the documentation is small so it makes sense to show the documentation comment right away.

@ahoppen @hamishknight Can you please recheck? 🙏🏼

ahoppen

Looks great to me. Thank you 🚀

ahoppen · 2025-07-18T06:50:48Z

@swift-ci Please test

ahoppen · 2025-07-18T09:43:51Z

CI failed because Swift CI unconditionally enables all tests that are guarded by skipUnlessSupportedByToolchain (we don’t want to accidentally leave tests disabled in CI due to configuration issues). I suggest that we wait until swiftlang/swift#82464 is merged before merging this. Is that alright with you?

a7medev · 2025-07-18T09:53:17Z

@ahoppen Yes, I think that makes sense.

I wonder where in the console output did you find this though 😅

Edit: I found it by downloading the plain text output and using grep to filter it out. Let me know if there's a more efficient way to do that though 😅

ahoppen · 2025-07-18T12:51:33Z

No, I always download the log files as well.

a7medev added 3 commits July 15, 2025 22:36

Fetch full documentation in completion items

24de753

Fix completion tests to assert on full documentation

283e08c

Test completion item resolve documentation falls back to brief doc if…

fce98d5

… full doc is null

a7medev requested review from ahoppen, bnbarham, hamishknight and rintaro as code owners July 15, 2025 19:43

ahoppen reviewed Jul 16, 2025

View reviewed changes

a7medev added 3 commits July 16, 2025 22:53

Make completion_item_get_doc_full_copy optional for backward compatib…

ae3f00e

…ility

Use multiline string & turn var into let in handleCompletionDocumenta…

2b40e78

…tion

Assert appropriate completion doc based on full doc support in source…

db44320

…kitd

a7medev requested a review from ahoppen July 16, 2025 22:30

ahoppen reviewed Jul 17, 2025

View reviewed changes

a7medev added 4 commits July 17, 2025 10:36

Replace completion_item_get_doc_full_copy with completion_item_get_do…

3c66365

…c_full

Log errors on xmlDocumentationToMarkdown in documentationString

4df7ea1

Revert "Assert appropriate completion doc based on full doc support i…

ed77735

…n sourcekitd" This reverts commit db44320.

Skip tests relying on full documentation unless supported

4f8d69e

a7medev requested a review from ahoppen July 17, 2025 08:19

ahoppen reviewed Jul 17, 2025

View reviewed changes

a7medev added 4 commits July 17, 2025 20:19

Format changes with swift-format

d3ed8b1

Show full documentation without declaration in completion

9d56a93

Return both breif & full doc in codeCompleteDocumentation request

2f77852

Format files with swift format

c0b4ca2

a7medev requested a review from ahoppen July 17, 2025 19:11

ahoppen approved these changes Jul 18, 2025

View reviewed changes

Fetch full documentation in code completion #2207

Are you sure you want to change the base?

Fetch full documentation in code completion #2207

Uh oh!

Conversation

a7medev commented Jul 15, 2025

Uh oh!

ahoppen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ahoppen left a comment

Choose a reason for hiding this comment

Uh oh!

a7medev commented Jul 17, 2025

Uh oh!

ahoppen left a comment

Choose a reason for hiding this comment

Uh oh!

ahoppen commented Jul 18, 2025

Uh oh!

ahoppen commented Jul 18, 2025

Uh oh!

a7medev commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ahoppen commented Jul 18, 2025

Uh oh!

Uh oh!

a7medev commented Jul 18, 2025 •

edited

Loading