Skip to content

Add substring filtering support for TypeSpec integration test generation #3432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Aug 12, 2025

This PR enhances the --filter option in TypeSpec integration test generation commands to support partial/substring matching, making it much easier to generate specific subsets of test cases.

Problem

Previously, the --filter option only supported exact path matching. To generate test cases, you had to know the exact outputPath of each test case, which made it difficult to generate related groups of tests. For example, there was no easy way to generate all authentication-related tests or all type-related tests.

Solution

Enhanced the filtering logic in gen-cadl-ranch.js to support three matching strategies:

  1. Exact match (unchanged for backward compatibility): tsp.outputPath === pathFilter
  2. Substring match in outputPath: tsp.outputPath.includes(pathFilter)
  3. Substring match in inputPath: tsp.inputPath.includes(pathFilter)

Usage Examples

Now you can use intuitive filter keywords to generate related test cases:

# Generate all type-related specs (type/array, type/dictionary, type/union, etc.)
npm run generate-tsp-only:rlc --filter=type

# Generate all authentication-related specs
npm run generate-tsp-only:rlc --filter=authentication

# Generate all server-related specs  
npm run generate-tsp-only:rlc --filter=server

# Generate all Azure LRO specs
npm run generate-tsp-only:azure-modular --filter=lro

# Exact matching still works as before
npm run generate-tsp-only:rlc --filter=routes

Backward Compatibility

All existing functionality is preserved:

  • Exact path matching continues to work exactly as before
  • No breaking changes to existing scripts or workflows
  • Empty results when no matches are found (same behavior as before)

Testing

Comprehensively tested across all test types:

  • ✅ RLC tests (rlc tag)
  • ✅ Azure RLC tests (azure-rlc tag)
  • ✅ Modular tests (modular tag)
  • ✅ Azure Modular tests (azure-modular tag)
  • ✅ Edge cases (non-existent filters, exact matches, partial matches)

Fixes #3431.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

- Modified gen-cadl-ranch.js to support substring matching in addition to exact matching
- Filter now checks outputPath and inputPath for partial matches
- Maintains backward compatibility with exact path matching
- Tested across all test types (rlc, azure-rlc, modular, azure-modular)

Co-authored-by: MaryGao <[email protected]>
@Copilot Copilot AI changed the title [WIP] Allow to only generate one integration test case for typespec Add substring filtering support for TypeSpec integration test generation Aug 12, 2025
@Copilot Copilot AI requested a review from MaryGao August 12, 2025 04:03
Copilot finished work on behalf of MaryGao August 12, 2025 04:03
@MaryGao MaryGao added the Experiement Experiement PR/issue for AI label Aug 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Experiement Experiement PR/issue for AI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow to only generate one integration test case for typespec
2 participants