Skip to content

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Sep 12, 2025

This PR adds support for specifying multiple Android API levels when running tests in XHarness, addressing the need for the new Helix "multi" queue that contains pre-installed emulators covering all API levels.

Changes Made

Enhanced --api-version Argument

Modified the existing ApiVersionArgument to support multiple values by extending RepeatableArgument instead of OptionalIntArgument:

# Run tests on multiple API levels
xharness android-headless test --api-version=28 --api-version=29 --api-version=30 ...

# Existing single API level support is preserved
xharness android-headless test --api-version=28 ...

The argument can now be specified multiple times to test across different API levels, with the help text automatically indicating "Can be used more than once".

Emulator Management

Extended AdbRunner with new emulator management capabilities:

  • StartEmulator(int apiLevel) - Starts an emulator for a specific API level
  • StopEmulator(int apiLevel) - Stops an emulator for a specific API level
  • StopEmulators(IEnumerable<int> apiLevels) - Stops multiple emulators

The system automatically starts emulators for specified API levels when using multiple --api-version arguments, addressing the requirement that the "multi" queue does not start emulators by default.

Distinguishable Results

When running tests on multiple API levels, results are stored in separate directories:

  • /output/api-28/ for API level 28 results
  • /output/api-29/ for API level 29 results
  • etc.

This ensures results are distinguishable when the same workitem runs on multiple API levels.

Backward Compatibility

  • Added FirstApiVersion property to ApiVersionArgument for backward compatibility with existing commands (android test, android install, android device) that only support single API versions
  • The --api-version argument maintains full backward compatibility - when used once, it behaves exactly as before
  • API level range validation (16-35) is enforced for all values

Comprehensive Testing

Added unit tests covering:

  • Single and multiple API version argument parsing
  • API level range validation
  • Command argument validation

Usage Examples

# Test on multiple API levels with emulator auto-start
xharness android-headless test \
  --test-path=/path/to/test \
  --runtime-folder=/path/to/runtime \
  --test-assembly=MyTest.dll \
  --test-script=test.sh \
  --output-directory=/path/to/output \
  --api-version=28 --api-version=29 --api-version=30

# Results will be saved to:
# /path/to/output/api-28/
# /path/to/output/api-29/  
# /path/to/output/api-30/

Fixes #1459.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

@Copilot Copilot AI changed the title [WIP] [android] Add support for running workitems on specified Android emulator API levels [android] Add support for running workitems on specified Android emulator API levels Sep 12, 2025
@Copilot Copilot AI requested a review from rmarinho September 12, 2025 13:34
Copilot finished work on behalf of rmarinho September 12, 2025 13:34
@rmarinho
Copy link
Member

Instead of adding a new property for --api-levels , can we make --api-version work for the same propose, adding one or several ?

@kotlarmilos
Copy link
Member

@copilot Instead of adding a new property for --api-levels , can we make --api-version work for the same propose, adding one or several ?

@Copilot
Copy link
Contributor Author

Copilot AI commented Oct 6, 2025

Instead of adding a new property for --api-levels , can we make --api-version work for the same propose, adding one or several ?

Done! The --api-version argument now supports multiple values. You can specify it multiple times like --api-version=28 --api-version=29 --api-version=30 to run tests on multiple API levels. The --api-levels argument has been removed. (commit f0280d5)

@copilot Instead of adding a new property for --api-levels , can we make --api-version work for the same propose, adding one or several ?

Done! The --api-version argument now supports multiple values. You can specify it multiple times like --api-version=28 --api-version=29 --api-version=30 to run tests on multiple API levels. The --api-levels argument has been removed. (commit f0280d5)

@Copilot Copilot AI requested a review from kotlarmilos October 6, 2025 12:57
Copilot finished work on behalf of kotlarmilos October 6, 2025 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[android] Add support for running workitems on specified Android emulator API levels

3 participants