Skip to content

Conversation

wale
Copy link

@wale wale commented Sep 16, 2025

πŸ”— Linked issue

Reopens and closes #3338, continues #3531

❓ Type of change

  • πŸ“– Documentation (updates to the documentation or readme)
  • 🐞 Bug fix (a non-breaking change that fixes an issue)
  • πŸ‘Œ Enhancement (improving an existing functionality like performance)
  • ✨ New feature (a non-breaking change that adds functionality)
  • ⚠️ Breaking change (fix or feature that would cause existing functionality to change)

πŸ“š Description

Creates the option of shallow cloning (--depth=1 --single-branch) a Git repository, by enabling a boolean option cloneRepository.

export default defineContentConfig({
  collections: {
    docs: defineCollection({
      type: 'page',
      source: {
        repository: 'https://github.com/nuxt/content',
        include: 'docs/content/**',
        cloneRepository: true
      },
    })
  }
})

This change also adds support for cloning repositories with Git refs (tags and branches only), with another option in the source definition; alongside support for usernames with authentication tokens, as an attempt to support Forgejo-based (and other odd) providers.

The change was required as I use a non-Github/BitBucket repository for external Markdown files, and the current solution of submodules + git-sparse-checkout was no longer tenable.

The Git source also stores .content.cache.json differently to Github/Bitbucket sources, like so:

{
  "url": "<repo-url>",
  "hash": "<git-commit-hash>",
  "createdAt": "2025-09-16T08:07:28.525Z",
  "updatedAt": "2025-09-16T08:07:28.525Z"
}

where the Git commit hash is referencing git ls-remote.

Uses isomorphic-git to avoid shelling out to the git CLI. Needs testing on bun environments.

πŸ“ Checklist

  • I have linked an issue or discussion.
  • I have updated the documentation accordingly.

Copy link

vercel bot commented Sep 16, 2025

@wale is attempting to deploy a commit to the NuxtLabs Team on Vercel.

A member of the Team first needs to authorize it.

Copy link

pkg-pr-new bot commented Sep 16, 2025

npm i https://pkg.pr.new/@nuxt/content@3542

commit: f597e2e

@farnabaz
Copy link
Member

Thanks, @wale.
I'm wonderingβ€”can’t we use the same login (with isomorphic-git) for GitHub and Bitbucket?
Ideally, removing cloneRepository and ref and add support for object syntax for repository

repository: string | { url,  branch } | { url, tag }

Also it would be awesome if we can compare performance of old logic and new one

@wale
Copy link
Author

wale commented Sep 18, 2025

Thanks, @wale. I'm wonderingβ€”can’t we use the same login (with isomorphic-git) for GitHub and Bitbucket? Ideally, removing cloneRepository and ref and add support for object syntax for repository

I had initially figured it was out of scope, but that is a good idea, the change would make it much more cohesive.

Ideally, removing cloneRepository and ref and add support for object syntax for repository

repository: string | { url,  branch } | { url, tag }

That's a good idea, although wouldn't it be more sensible to keep both the previous logic (i.e. downloading a tarball from provider) and this PR? In case one doesn't work, the other might make more sense to use; preferably as Github and Bitbucket specific sources.

@farnabaz
Copy link
Member

I'd keep only one solution if it's generic enough, So if new solution works for Github and Bitbucket lets use it. But we can provide a compatibility solution and deprecate old usage before removing them all.

As an alternative, I was thinking about using giget package to download repositories. We need to check about the coverage and performance of giget and isomorphic-git

@wale
Copy link
Author

wale commented Sep 18, 2025

I'd keep only one solution if it's generic enough, So if new solution works for Github and Bitbucket lets use it. But we can provide a compatibility solution and deprecate old usage before removing them all.

That's true, I'll keep my generic solution, and hide the previous logic behind a boolean (downloadTarball?)

As an alternative, I was thinking about using giget package to download repositories.
We need to check about the coverage and performance of giget and isomorphic-git

Personally I think giget is out of the question, because it has a similar issue of a limited amount of providers (+ not really using Git), kicking my specific issue upstream. I'm using isomorphic-git because it covers my requirements of performing a git clone operation, and users downstream aren't required to implement their own provider.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Git content source - add source variant for non-Github / Bitbucket repositories
2 participants