Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
node_modules
build
.qaspherecli
.DS_Store
.DS_Store
.env
56 changes: 37 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

The QAS CLI is a command-line tool for submitting your test automation results to [QA Sphere](https://qasphere.com/). It provides the most efficient way to collect and report test results from your test automation workflow, CI/CD pipeline, and build servers.

The tool can upload test case results from JUnit XML files to QA Sphere test runs by matching test case names (mentions of special markers) to QA Sphere's test cases.
The tool can upload test case results from JUnit XML and Playwright JSON files to QA Sphere test runs by matching test case names (mentions of special markers) to QA Sphere's test cases.

## Installation

Expand Down Expand Up @@ -53,14 +53,14 @@ QAS_TOKEN=your_token
QAS_URL=https://qas.eu1.qasphere.com

# Example with real values:
# QAS_TOKEN=tst0000001.1CKCEtest_JYyckc3zYtest.dhhjYY3BYEoQH41e62itest
# QAS_TOKEN=qas.1CKCEtest_JYyckc3zYtest.dhhjYY3BYEoQH41e62itest
# QAS_URL=https://qas.eu1.qasphere.com
```


## Command: `junit-upload`
## Commands: `junit-upload`, `playwright-json-upload`

The `junit-upload` command creates a new test run within a QA Sphere project from your JUnit XML files or uploads results to an existing run.
The `junit-upload` and `playwright-json-upload` commands upload test results from JUnit XML and Playwright JSON reports to QA Sphere respectively. Both commands can either create a new test run within a QA Sphere project or upload results to an existing run, and they share the same set of options.

### Options

Expand Down Expand Up @@ -89,7 +89,9 @@ The `--run-name` option supports the following placeholders:

### Usage Examples

Ensure the required environment variables are defined before running these commands:
Ensure the required environment variables are defined before running these commands.

**Note:** The following examples use `junit-upload`, but you can replace it with `playwright-json-upload` and adjust the file extension from `.xml` to `.json` to upload Playwright JSON reports instead.

1. Create a new test run with default name template (`Automated test run - {MMM} {DD}, {YYYY}, {hh}:{mm}:{ss} {AMPM}`) and upload results:
```bash
Expand Down Expand Up @@ -123,36 +125,52 @@ Ensure the required environment variables are defined before running these comma
qasphere junit-upload --attachments ./test1.xml
```

7. Force upload even with missing test cases:
7. Force upload even with missing test cases or attachments:
```bash
qasphere junit-upload --force ./test-results.xml
```

## JUnit XML File Requirements
## Test Report Requirements

The QAS CLI requires test cases in your reports (JUnit XML or Playwright JSON) to reference corresponding test cases in QA Sphere. These references are used to map test results from your automation to the appropriate test cases in QA Sphere. If a report lacks these references or the referenced test case doesn't exist in QA Sphere, the tool will display an error message.

### JUnit XML

Test case names in JUnit XML reports must include a QA Sphere test case marker in the format `PROJECT-SEQUENCE`:

- **PROJECT** - Your QA Sphere project code
- **SEQUENCE** - Test case sequence number (minimum 3 digits, zero-padded if needed)

The QAS CLI tool requires JUnit XML files to have test case names that match the test case codes on QA Sphere. If your XML file doesn't contain any matching test cases, the tool will display an error message.
**Examples:**
- `PRJ-002: Login with valid credentials`
- `Login with invalid credentials: PRJ-1312`

### Test Case Naming Convention
**Note:** The project code in test names must exactly match your QA Sphere project code.

Test case names in the XML report should contain a QA Sphere test case marker (PROJECT-SEQUENCE).
### Playwright JSON

This marker is used to match test cases in the XML report with test cases in QA Sphere:
Playwright JSON reports support two methods for referencing test cases (checked in order):

- **PROJECT** is your QA Sphere project code
- **SEQUENCE** is at least a three-digit test case sequence number
1. **Test Annotations (Recommended)** - Add a [test annotation](https://playwright.dev/docs/test-annotations#annotate-tests) with:
- `type`: `"test case"` (case-insensitive)
- `description`: Full QA Sphere test case URL

Examples:
- **PRJ-312: Login with valid credentials**
- **Login with valid credentials: PRJ-312**
```typescript
test('user login', {
annotation: { type: 'test case', description: 'https://qas.eu1.qasphere.com/project/PRJ/tcase/123' }
}, async ({ page }) => {
// test code
});
```

The project code in your test names must match the project code in QA Sphere.
2. **Test Case Marker in Name** - Include the `PROJECT-SEQUENCE` marker in the test name (same format as JUnit XML)

### Development (for those who want to contribute to the tool)
## Development (for those who want to contribute to the tool)

1. Install and build: `npm install && npm run build && npm link`
2. Get test account at [qasphere.com](https://qasphere.com/) (includes demo project)
3. Configure `.qaspherecli` with credentials
4. Test with sample XML from [bistro-e2e](https://github.com/Hypersequent/bistro-e2e)
4. Test with sample reports from [bistro-e2e](https://github.com/Hypersequent/bistro-e2e)

Tests: `npm test` (Vitest) and `cd mnode-test && ./docker-test.sh` (Node.js 18+ compatibility)

Expand Down
88 changes: 81 additions & 7 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "qas-cli",
"version": "0.3.4",
"version": "0.4.0",
"description": "QAS CLI is a command line tool for submitting your automation test results to QA Sphere at https://qasphere.com/",
"type": "module",
"main": "./build/bin/qasphere.js",
Expand Down Expand Up @@ -47,6 +47,7 @@
"dotenv": "^16.5.0",
"escape-html": "^1.0.3",
"semver": "^7.7.1",
"strip-ansi": "^7.1.2",
"xml2js": "^0.6.2",
"yargs": "^17.7.2",
"zod": "^3.24.3"
Expand Down
2 changes: 1 addition & 1 deletion src/api/run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ export interface CreateRunRequest {
}

export interface CreateRunResponse {
id: string
id: number
}

export const createRunApi = (fetcher: typeof fetch) => {
Expand Down
106 changes: 0 additions & 106 deletions src/commands/junit-upload.ts

This file was deleted.

Loading