feat: Automate OpenAPI generation from JSON Schema #178
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Work in progress . This commit adds the core registry service implementation with methods to: - Retrieve all registry entries - List entries with cursor-based pagination - Fetch specific server details by ID
Init world. Basic registry API server <WIP>
Update README to reflect binary name change
Add count to server response and fix Makefile paths
…retrieval is validated
Add Swagger API documentation and handlers for v0
Contains about 300+ initial servers that can be used to seed a mongoDB
We need to work in multiple orgs with different private repos, this is a temporary solution until we make the main repo public.
…blishing - Added authentication service and GitHub OAuth integration. - Introduced new authentication methods and structures in the model. - Updated the API to handle authentication during the publishing process. - Created handlers for starting and checking the status of the OAuth flow. - Enhanced the publish handler to validate authentication credentials. - Updated configuration to include GitHub OAuth settings. - Added tests for the OAuth flow and publishing with authentication. - Updated documentation to reflect changes in the API and authentication process.
…e argument descriptions
…cation and update documentation
…b application and improve error handling
…ecks and introducing dynamic auth method determination
Co-authored-by: toby <[email protected]>
Co-authored-by: toby <[email protected]>
…ated documentation Co-authored-by: toby <[email protected]>
Co-authored-by: toby <[email protected]>
Co-authored-by: toby <[email protected]>
…t timestamps and show flags in --long-flag/-short format Co-authored-by: toby <[email protected]>
…#164) <!-- Provide a brief summary of your changes --> Add more information about the value field under Input in the API doc. ## Motivation and Context <!-- Why is this change needed? What problem does it solve? --> the descrioption for [value](https://github.com/modelcontextprotocol/registry/blob/81e5174a593a7488715b0bf0ae77a4dc466b4d35/docs/openapi.yaml#L219) is not clearly. ## How Has This Been Tested? <!-- Have you tested this in a real application? Which scenarios were tested? --> No need. ## Breaking Changes <!-- Will users need to update their code or configurations? --> No ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ x ] Documentation update ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. --> - [ x ] I have read the [MCP Documentation](https://modelcontextprotocol.io) - [ x ] My code follows the repository's style guidelines - [ x ] New and existing tests pass locally - [ x ] I have added appropriate error handling - [ x ] I have added or updated documentation as needed ## Additional context <!-- Add any other context, implementation notes, or design decisions -->
<!-- Provide a brief summary of your changes --> ## Motivation and Context This is in support of #159, to make it easier, I hope. My team noticed some differences in the JSON schema, most notably the "choices" property was missing. I tried to write a tool to do this but slowed down since I am not experienced with Go. This is a first pass, maybe someone else can finish the tooling. I restructured the JSON schema to break out child objects like the YAML file, and brought the descriptions and example into sync. The only differences are two extra properties in the YAML which are computed by the server: - `VersionDetail.is_latest` is not present in the JSON schema, computed by the registry based on #158 - `Server.id` is not present in the JSON schema, generated by the registry The idea is the a tool could look at ServerDetail in the YAML and then generate the schema.json with minimal fix-ups (add a root title, remove `VersionDetail.is_latest` and `Server.id`. ~Question for @tadasant - should `VersionDetail.release_date` be in the JSON schema? I think _no_ (and let it be computed) but it's there right now.~ - aligned with #167 and removed ## How Has This Been Tested? I have tested the JSON schema against the sample server.json in the repo (files and markdown blocks). ## Breaking Changes None. ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. --> - [x] I have read the [MCP Documentation](https://modelcontextprotocol.io) - [x] My code follows the repository's style guidelines - [ ] New and existing tests pass locally - [ ] I have added appropriate error handling - [x] I have added or updated documentation as needed ## Additional context <!-- Add any other context, implementation notes, or design decisions -->
Closes #155 The "generalized" `server.json` is the one that can be used in a variety of situations: - Discoverability on a centralized registry (i.e. our official Registry work) - Discoverability on a decentralized .well-known endpoint - As a response to an initialization call, so the client knows information about the MCP server to which it is connecting - As an input into crafting a [DXT file](https://www.anthropic.com/engineering/desktop-extensions) - Packaged in with the source code of an MCP server, so as to have a structured way context Of these, the official Registry is but one use case. The Registry has a unique set of concerns for which we are optimizing, like whitelisting of specific registries we trust to host MCP server source code for the public community (`npm`, `pypi`, etc). It is not necessary to have such a constraint for the generalized server.json, which may be used in, for example, an internal context where such a validation doesn't make sense. So I've done the following at a high level: - Renamed the prior `schema.json` to be `registry-schema.json` - Introduced a more generic `schema.json` to serve as the schema for the generalized `server.json` - Refactored `registry-schema.json` to be simply a constraint on the base `schema.json` I made a few small changes in this transition: - Removed `gitlab` as an option in `repository.source` for `registry-schema.json`. I'm fine going live with just GitHub support; would be happy to take a contribution from e.g. Gitlab employees or a motivated Gitlab customer to expand anytime. - Removed `homebrew` as an option in `packages.registry_name`; never really made sense, not a place where folks are publishing MCP servers - Added `docker` to `packages.runtime_hint`; was just a missing oversight - Gave both schemas a global `$id` ~- Removed `release_date` from both schemas. Certainly not something that makes sense for the generalized schema. It actually doesn't make sense for the Registry schema either, because this is the server.json that _users will submit_, i.e. the _input_ into the Registry publication API. `release_date` will be a piece of _output_ metadata. The Registry API will be free to append such extra metadata for GET /:id requests of servers, and the data can be present in the OpenAPI spec, but it does not need to be in the registry-server.json JSON schema.~ - Note this is no longer in this PR; rebased on #168 which included it - Removed unnecessary enums from the generalized `schema.json`
## Summary - Adds CLAUDE.md file to provide guidance for Claude Code instances working with this repository - Includes essential development commands and high-level architecture overview - Follows the standard format for Claude Code documentation ## Details The CLAUDE.md file contains: - Common development commands for building, running, testing, and linting - Architecture overview explaining the clean architecture pattern - Request flow documentation - Key interfaces and their implementations - Authentication flow details - Important design patterns used in the codebase This will help future Claude Code instances be more productive when working with the MCP Registry codebase. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-authored-by: Claude <[email protected]> Co-authored-by: Avinash Sridhar <[email protected]>
Closes #149 As noted in the FAQ update here, this is a helpful starting point in thinking about this: ``` ### What is the difference between "Official MCP Registry", "MCP Registry", "MCP registry", "MCP Registry API", etc? There are four underlying concepts: - "MCP Server Registry API" (or "MCP Registry API"): The OpenAPI specification defined in [openapi.yaml](./server-registry-api/openapi.yaml). This is a reusable API specification that anyone building any sort of "MCP server registry" should consider adopting / aligning with. - "Official MCP Registry" (or "MCP Registry"): The application that lives at `https://registry.modelcontextprotocol.io`. This registry currently only catalogs MCP servers, but may be extended in the future to also catalog MCP client/host apps and frameworks. - "Official MCP Registry API": The REST API that lives at `https://registry.modelcontextprotocol.io/api`, with an OpenAPI specification defined at [official-registry-openapi.yaml](./server-registry-api/official-registry-openapi.yaml) - "MCP server registry" (or "MCP registry"): A third party, likely commercial, implementation of the MCP Server Registry API or derivative specification. ``` Prior to this PR, we were just treating "MCP Server Registry API" and "Official MCP Registry API" to be the same thing. They are not the same: the latter is inherently a more constrained version of the former. A private deployment of a similar registry need not have constraints like "the only valid source code repositories are `github` and `gitlab`". This PR separates out the definitions to match that path forward. ## Notable Changes ### Base OpenAPI spec (`openapi.yaml`) - Removed official-registry-specific constraints: - No maximum limit on `/servers` endpoint - No enum constraint on `Repository.source` - No enum constraint on `Package.registry_name` - Removed `/v0` prefix from paths (it's now just baked into the `official` one's URL prefix) - Added `$id` for proper schema identification ### Official Registry OpenAPI spec (`official-registry-openapi.yaml`) - Created as a derivative of the base spec using `$ref` - Re-adds official-registry-specific constraints: - `Repository.source` enum: `[github]` - `Package.registry_name` enum: `[npm, docker, pypi, nuget]` ### Documentation updates - Added `server-registry-api/README.md` explaining the relationship between specs - Updated FAQ to clarify terminology - Reorganized docs structure
- Create server-json-to-openapi-sync tool to convert JSON Schema to OpenAPI components - Set up generation workflow with separate manual and generated files - Organize generation files in docs/server-registry-api/generation/ - Add go generate support for running the full pipeline - Preserve existing OpenAPI structure while enabling DRY schema management
9 tasks
|
@tadasant are you still working on this? |
|
@domdomegg I haven't picked it back up since I opened the PR - if you are keen, I'm happy for you to take it over (or ditch the PR and do it separately, I'm not attached to this approach - wasn't feeling great about the direction it was going anyway). |
|
I sincerely apologize for the disruption. This PR was accidentally closed due to a git history rewrite mistake. The PR has been recreated as #239 with all the original content preserved. Please continue any discussions or reviews on the new PR: #239 Again, I apologize for the inconvenience this has caused. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Just FYI @toby @joelverhagen @modelcontextprotocol/registry I am WIP on this. Don't quite love where it's at right now, will be working more on it.
This is part of #159 and the mission to make our server.json/openapi spec DRY.
Summary
This PR introduces automated generation of OpenAPI schemas from our canonical
server.jsonJSON Schema, ensuring that shared data models stay in sync between the two specifications.What's changed
tools/server-json-to-openapi-sync/- a Go tool that converts JSON Schema definitions to OpenAPI component schemasdocs/server-registry-api/generation/:openapi-manual.yaml- Hand-written API endpoints and registry-specific schemasopenapi-components-generated.yaml- Auto-generated from server.jsonopenapi.yamlis generated by bundling these togethergo generatesupport for running the generation pipelineidon Server) remain manually definedHow it works
go generate -tags generate ./...ormake -C docs/server-registry-api/generation alldocs/server-json/schema.json→ OpenAPI componentsopenapi.yamlBenefits
TODO before review