fix: refresh codebase index on config change

[shssoichiro]

Description

Updates the CodebaseIndexer to refresh the index when the assistant config is changed. The DocsService already does this, so this change copies the approach that the DocsService is using.

The biggest current issue is that the codebase indexing will begin before Continue has loaded remote assistant configs, which means it will use whatever model is available locally (usually Transformers.js in VSCode), or worse, silently fail. By refreshing after the config loads, we will ensure we are indexing the codebase using the user's configured embed model.

Checklist

• I've read the contributing guide
• The relevant docs, if any, have been updated or created
• The relevant tests, if any, have been updated or created

Screenshots

N/A This is a backend change.

Tests

Added appropriate tests to CodebaseIndexer.test.ts

[netlify]

👷 Deploy request for continuedev pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	634aafb

[recurseml]

😱 Found 1 issue. Time to roll up your sleeves! 😱

[RomneyDa]

@shssoichiro it would be great to refresh codebase indexing on config change. I think it should only update when the embeddings model or @codebase config changes. Indexing (even just computing potential changes) can be pretty expensive.

Could you add something similar to the DocsService's siteIndexingConfigsAreEqual that checks if the embeddings model config has changed OR the codebase context provider config has changed?

[shssoichiro]

Thanks, I've updated both indexers as requested and fixed the tests which were failing due to caching.

[RomneyDa]

See comment!

[RomneyDa]

return (
  embedModelsAreEqual(
    config1?.selectedModelByRole.embed,
    config2.selectedModelByRole.embed,
  ) && codebaseProvider1 === codebaseProvider2
);

This will always cause a refresh because codebaseProvider1 and codebaseProvider2 will never be equal, because the references will reset on config reload.

[shssoichiro]

What would be a good way to check for this? Looking at the ContextProviderDescription, it seems like the properties there would typically always be the same given that we are always looking at the "codebase" provider.

[RomneyDa]

@shssoichiro you are right that changes in nRetrieve and nFinal and useReranking (the only codebase params) should not trigger a refresh. I just mean that the equality expression codebaseProvider1 === codebaseProvider2 will always evaluate to false, because they are objects that are created with each config reload.

[shssoichiro]

Thanks, that makes sense. In that case it probably makes sense to remove that check and just check if the embed provider has changed. I've done that and fixed the merge conflicts.

[RomneyDa]

@shssoichiro the docConfigsAreEqual feels redundant since the 3rd param does not do anything. I don't want to nitpick too much but the docs config logic has been touchy and confusing in the past and I want to make sure we arrange it in a way that unused args aren't passed around. Let's rearrange so that the embedModelsAreEqual is only used when needed

[sestinj]

🎉 This PR is included in version 1.1.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀