Test Generation in Modular #3396
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Added modular test generation similar to samples - Test names use sample description logic for consistency - Clean test names without trailing dots - Proper vitest/mocha framework support based on module type - Azure Identity integration for ARM clients - Generated test utilities including recordedClient.ts - Support for both single and multiple client scenarios - Unique test names including function names for multiple examples Generated 7 operation test files with professional naming: - assign role to the data product for dataProductsAddUserRoleMaximumSetGen - create data product resource for dataProductsCreateMaximumSetGen - delete data product resource for dataProductsDeleteMaximumSetGen - generate sas token for storage account for dataProductsGenerateStorageAccountSasTokenMaximumSetGen - list data products by resource group for dataProductsListByResourceGroupMaximumSetGenGeneratedByMinimumSetRuleMinimumSetGen - initiate key rotation on Data Product for dataProductsRotateKeyMaximumSetGen - update data product resource for dataProductsUpdateMaximumSetGen
Enhanced test generation to include response assertions that validate response values against expected values from examples: - Added generateResponseAssertions() function - Added generateAssertionsForValue() for recursive validation - Support for string, number, boolean, array, model, dict, null, union types - Proper handling of TypeSpec SDK numeric response indices - Validates key response properties with strictEqual checks - Array length and item validation - Model/object property validation up to 3 levels deep Generated tests now include comprehensive response validation like: assert.strictEqual(result.properties.resourceGuid, expected_value) assert.strictEqual(result.properties.provisioningState, expected_value) assert.ok(Array.isArray(result.dataTypeScope)) This ensures tests validate both API success and response content.
- Added 4 passing unit test scenarios for test generation: - basicOperationTest: Tests simple GET operation with response assertions - pagingOperationTest: Tests paging operations with array response validation - moduleTypeTest: Tests different import styles for ESM vs CommonJS - complexResponseTest: Tests nested object response assertions - Added emitTestsFromTypeSpec function to test/util/emitUtil.ts - Enhanced scenarios.spec.ts with 'tests' output block type - Improved test output formatting with proper file separation - All test scenarios validate comprehensive response assertions - Tests follow same pattern as existing samples scenarios
- Fix formatting issues in emitTests.ts (whitespace and comment styles) - Update scenarios.spec.ts with better test file handling and formatting fixes - Refresh test scenario files with updated expected outputs: - basicOperationTest.md: Updated with improved test structure - complexResponseTest.md: Updated response assertion patterns - moduleTypeTest.md: Updated with proper module import handling - pagingOperationTest.md: Updated paging operation test patterns - voidOperationTest.md: Updated void operation test handling All tests passing with SCENARIOS_UPDATE=true refresh
### LRO (Long Running Operations) Support: - Add detection for LRO operations (method.kind === 'lro' || method.kind === 'lropaging') - Generate proper poller pattern: 'const poller = await client.method()' followed by 'const result = await poller.pollUntilDone()' - Add comprehensive LRO test scenario (lroOperationTest.md) with ArmResourceCreateOrReplaceAsync example - Include proper response assertions on the final LRO result ### Paging Test Improvements: - Create dedicated generatePagingResponseAssertions() function - Fix incorrect paging assertions that were checking resArray[0].value instead of resArray directly - Now correctly asserts on collected results length and individual item properties - Updated pagingOperationTest.md with proper assertions ### Test Coverage Enhancement: - Fixed void operation test (voidOperationTest.md) to properly handle LRO delete operations - All 6 test scenarios now passing: basic, complex, module, paging, void, and LRO operations ### Technical Improvements: - Better separation of concerns between regular, paging, and LRO test generation - Proper understanding of how paging iteration works vs response structure - Correct LRO poller pattern following Azure SDK conventions
…torest.typescript into generate-test-modular
- Created shared utility module sampleTestHelpers.ts to reduce code duplication - Extracted common functions: buildParameterValueMap, prepareCommonValue, getCredentialSampleValue, getCredentialTestValue, serializeExampleValue - Unified CommonValue interface for both samples and tests - Refactored emitSamples.ts and emitTests.ts to use shared utilities - Eliminated ~200+ lines of duplicate code while maintaining functionality - All 388 modular unit tests continue to pass
…tilities - Created comprehensive sampleTestHelpers.ts module with shared utilities - Extracted common functions: prepareCommonParameters, iterateClientsAndMethods, generateMethodCall, createSourceFile, generateAssertionsForValue, generateResponseAssertions - Reduced code duplication by ~200+ lines across emit files - Enhanced test vs sample differentiation (env variables for tests, example values for samples) - Improved maintainability with unified parameter preparation and assertion generation - All 388 modular unit tests passing after refactoring Breaking down the previous monolithic emit functions into: 1. Common parameter handling logic shared between samples and tests 2. Unified client/method iteration patterns 3. Shared assertion generation for response validation 4. Consistent source file creation and import handling 5. Better separation of concerns between sample and test generation This refactoring eliminates significant code duplication while maintaining 100% test compatibility.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fixes #3247 #3371 #3345