diff --git a/README.md b/README.md index a223cc6..205524d 100644 --- a/README.md +++ b/README.md @@ -58,14 +58,21 @@ QAS_URL=https://qas.eu1.qasphere.com ``` -## Commands: `junit-upload`, `playwright-json-upload` +## Commands: `junit-upload`, `playwright-json-upload`, `xcresult-upload` -The `junit-upload` and `playwright-json-upload` commands upload test results from JUnit XML and Playwright JSON reports to QA Sphere respectively. Both commands can either create a new test run within a QA Sphere project or upload results to an existing run, and they share the same set of options. +The `junit-upload`, `playwright-json-upload` and `xcresult-upload` commands upload test results from JUnit XML, Playwright JSON and Xcode reports to QA Sphere respectively. + +There are two modes for uploading results using the commands: +1. Upload to an existing test run by specifying its URL via `--run-url` flag +2. Create a new test run and upload results to it (when `--run-url` flag is not specified) ### Options -- `-r, --run-url` - Optional URL of an existing run for uploading results (a new run is created if not specified) -- `--run-name` - Optional name template for creating new test run when run url is not specified (supports `{env:VAR}`, `{YYYY}`, `{YY}`, `{MM}`, `{MMM}`, `{DD}`, `{HH}`, `{hh}`, `{mm}`, `{ss}`, `{AMPM}` placeholders). If not specified, `Automated test run - {MMM} {DD}, {YYYY}, {hh}:{mm}:{ss} {AMPM}` is used as default +- `-r`/`--run-url` - Upload results to an existing test run +- `--project-code`, `--run-name`, `--create-tcases` - Create a new test run and upload results to it (if `-r`/`--run-url` is not specified) + - `--project-code` - Project code for creating new test run. It can also be auto detected from test case markers in the results, but this is not fully reliable, so it is recommended to specify the project code explicitly + - `--run-name` - Optional name template for creating new test run. It supports `{env:VAR}`, `{YYYY}`, `{YY}`, `{MM}`, `{MMM}`, `{DD}`, `{HH}`, `{hh}`, `{mm}`, `{ss}`, `{AMPM}` placeholders (default: `Automated test run - {MMM} {DD}, {YYYY}, {hh}:{mm}:{ss} {AMPM}`) + - `--create-tcases` - Automatically create test cases in QA Sphere for results that don't have valid test case markers. A mapping file (`qasphere-automapping-YYYYMMDD-HHmmss.txt`) is generated showing the sequence numbers assigned to each new test case, use it to update your test cases to include the markers in the name, for future uploads (default: `false`) - `--attachments` - Try to detect and upload any attachments with the test result - `--force` - Ignore API request errors, invalid test cases, or attachments - `--ignore-unmatched` - Suppress individual unmatched test messages, show summary only @@ -88,40 +95,39 @@ The `--run-name` option supports the following placeholders: - `{mm}` - 2-digit minute - `{ss}` - 2-digit second -**Note:** The `--run-name` option is only used when creating new test runs (i.e., when `--run-url` is not specified). - ### Usage Examples Ensure the required environment variables are defined before running these commands. -**Note:** The following examples use `junit-upload`, but you can replace it with `playwright-json-upload` and adjust the file extension from `.xml` to `.json` to upload Playwright JSON reports instead. +**Note:** The following examples use `junit-upload`, but you can replace it with `playwright-json-upload` and `xcresult-upload` to upload Playwright JSON and Xcode reports. -1. Create a new test run with default name template (`Automated test run - {MMM} {DD}, {YYYY}, {hh}:{mm}:{ss} {AMPM}`) and upload results: +1. Upload to an existing test run: ```bash - qasphere junit-upload ./test-results.xml + qasphere junit-upload -r https://qas.eu1.qasphere.com/project/P1/run/23 ./test-results.xml ``` -2. Upload to an existing test run: +2. Create a new test run with default name template and upload results: ```bash - qasphere junit-upload -r https://qas.eu1.qasphere.com/project/P1/run/23 ./test-results.xml + qasphere junit-upload ./test-results.xml ``` + Project code is detected from test case markers in the results. 3. Create a new test run with name template without any placeholders and upload results: ```bash - qasphere junit-upload --run-name "v1.4.4-rc5" ./test-results.xml + qasphere junit-upload --project-code P1 --run-name "v1.4.4-rc5" ./test-results.xml ``` 4. Create a new test run with name template using environment variables and date placeholders and upload results: ```bash - qasphere junit-upload --run-name "CI Build {env:BUILD_NUMBER} - {YYYY}-{MM}-{DD}" ./test-results.xml + qasphere junit-upload --project-code P1 --run-name "CI Build {env:BUILD_NUMBER} - {YYYY}-{MM}-{DD}" ./test-results.xml ``` If `BUILD_NUMBER` environment variable is set to `v1.4.4-rc5` and today's date is January 1, 2025, the run would be named "CI Build v1.4.4-rc5 - 2025-01-01". -5. Create a new test run with name template using date/time placeholders and upload results: +5. Create a new test run with name template using date/time placeholders and create test cases for results without valid markers and upload results: ```bash - qasphere junit-upload --run-name "Nightly Tests {YYYY}/{MM}/{DD} {HH}:{mm}" ./test-results.xml + qasphere junit-upload --project-code P1 --run-name "Nightly Tests {YYYY}/{MM}/{DD} {HH}:{mm}" --create-tcases ./test-results.xml ``` - If the current time is 10:34 PM on January 1, 2025, the run would be named "Nightly Tests 2025/01/01 22:34". + If the current time is 10:34 PM on January 1, 2025, the run would be named "Nightly Tests 2025/01/01 22:34". This also creates new test cases in QA Sphere for any results that doesn't have a valid test case marker. A mapping file (`qasphere-automapping-YYYYMMDD-HHmmss.txt`) is generated showing the sequence numbers assigned to each newly created test case. Update your test cases to include the markers in the name, for future uploads. 6. Upload results with attachments: ```bash @@ -139,13 +145,13 @@ Ensure the required environment variables are defined before running these comma ``` This will show only a summary like "Skipped 5 unmatched tests" instead of individual error messages for each unmatched test. -9. Skip stdout/stderr for passed tests to reduce result payload size: +9. Skip stdout for passed tests to reduce result payload size: ```bash qasphere junit-upload --skip-report-stdout on-success ./test-results.xml ``` This will exclude stdout from passed tests while still including it for failed, blocked, or skipped tests. - Skip both stdout and stderr for passed tests: +10. Skip both stdout and stderr for passed tests: ```bash qasphere junit-upload --skip-report-stdout on-success --skip-report-stderr on-success ./test-results.xml ``` @@ -153,7 +159,7 @@ Ensure the required environment variables are defined before running these comma ## Test Report Requirements -The QAS CLI requires test cases in your reports (JUnit XML or Playwright JSON) to reference corresponding test cases in QA Sphere. These references are used to map test results from your automation to the appropriate test cases in QA Sphere. If a report lacks these references or the referenced test case doesn't exist in QA Sphere, the tool will display an error message. +The QAS CLI maps test results from your reports (JUnit XML or Playwright JSON) to corresponding test cases in QA Sphere using test case markers. If a test result lacks a valid marker, the CLI will display an error unless you use `--create-tcases` to automatically create test cases, or `--ignore-unmatched`/`--force` to skip unmatched results. ### JUnit XML @@ -186,6 +192,21 @@ Playwright JSON reports support two methods for referencing test cases (checked 2. **Test Case Marker in Name** - Include the `PROJECT-SEQUENCE` marker in the test name (same format as JUnit XML) +### XCode Reports + +Test case names in the XCode reports must include a QA Sphere test case marker in the format `PROJECT_SEQUENCE`: + +- **PROJECT** - Your QA Sphere project code +- **SEQUENCE** - Test case sequence number (minimum 3 digits, zero-padded if needed) + +**Examples:** +- `PRJ_002_login_with_valid_credentials` +- `login_with_valid_credentials_PRJ_1312` + +## Other Requirements + +The `xcresult-upload` command will automatically invoke `xcrun xcresulttool`, if the SQLite database is not found inside the `.xcresult` bundle. This requires **Xcode Command Line Tools** to be installed. See [Apple Developer documentation](https://developer.apple.com/xcode/resources/) for installation instructions. Alternatively, having the full Xcode application installed also provides these tools. + ## Development (for those who want to contribute to the tool) 1. Install and build: `npm install && npm run build && npm link` diff --git a/package-lock.json b/package-lock.json index 65833fe..b716f09 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,17 +1,19 @@ { "name": "qas-cli", - "version": "0.4.0", + "version": "0.4.4", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "qas-cli", - "version": "0.4.0", + "version": "0.4.4", "license": "ISC", "dependencies": { + "better-sqlite3": "^12.5.0", "chalk": "^5.4.1", "dotenv": "^16.5.0", "escape-html": "^1.0.3", + "fzstd": "^0.1.1", "semver": "^7.7.1", "strip-ansi": "^7.1.2", "xml2js": "^0.6.2", @@ -23,6 +25,7 @@ }, "devDependencies": { "@eslint/js": "^9.25.1", + "@types/better-sqlite3": "^7.6.13", "@types/escape-html": "^1.0.4", "@types/node": "^20.17.32", "@types/semver": "^7.7.0", @@ -1069,6 +1072,16 @@ "win32" ] }, + "node_modules/@types/better-sqlite3": { + "version": "7.6.13", + "resolved": "https://registry.npmjs.org/@types/better-sqlite3/-/better-sqlite3-7.6.13.tgz", + "integrity": "sha512-NMv9ASNARoKksWtsq/SHakpYAYnhBrQgGD8zkLYk/jaK8jUGn08CfEdTRgYhMypUQAfzSP8W6gNLe0q19/t4VA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/node": "*" + } + }, "node_modules/@types/cookie": { "version": "0.6.0", "resolved": "https://registry.npmjs.org/@types/cookie/-/cookie-0.6.0.tgz", @@ -1569,6 +1582,60 @@ "integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==", "dev": true }, + "node_modules/base64-js": { + "version": "1.5.1", + "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz", + "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT" + }, + "node_modules/better-sqlite3": { + "version": "12.5.0", + "resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-12.5.0.tgz", + "integrity": "sha512-WwCZ/5Diz7rsF29o27o0Gcc1Du+l7Zsv7SYtVPG0X3G/uUI1LqdxrQI7c9Hs2FWpqXXERjW9hp6g3/tH7DlVKg==", + "hasInstallScript": true, + "license": "MIT", + "dependencies": { + "bindings": "^1.5.0", + "prebuild-install": "^7.1.1" + }, + "engines": { + "node": "20.x || 22.x || 23.x || 24.x || 25.x" + } + }, + "node_modules/bindings": { + "version": "1.5.0", + "resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz", + "integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==", + "license": "MIT", + "dependencies": { + "file-uri-to-path": "1.0.0" + } + }, + "node_modules/bl": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz", + "integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==", + "license": "MIT", + "dependencies": { + "buffer": "^5.5.0", + "inherits": "^2.0.4", + "readable-stream": "^3.4.0" + } + }, "node_modules/brace-expansion": { "version": "1.1.11", "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz", @@ -1591,6 +1658,30 @@ "node": ">=8" } }, + "node_modules/buffer": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz", + "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT", + "dependencies": { + "base64-js": "^1.3.1", + "ieee754": "^1.1.13" + } + }, "node_modules/cac": { "version": "6.7.14", "resolved": "https://registry.npmjs.org/cac/-/cac-6.7.14.tgz", @@ -1645,6 +1736,12 @@ "node": ">= 16" } }, + "node_modules/chownr": { + "version": "1.1.4", + "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz", + "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==", + "license": "ISC" + }, "node_modules/cli-width": { "version": "4.1.0", "resolved": "https://registry.npmjs.org/cli-width/-/cli-width-4.1.0.tgz", @@ -1766,6 +1863,21 @@ } } }, + "node_modules/decompress-response": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz", + "integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==", + "license": "MIT", + "dependencies": { + "mimic-response": "^3.1.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, "node_modules/deep-eql": { "version": "5.0.2", "resolved": "https://registry.npmjs.org/deep-eql/-/deep-eql-5.0.2.tgz", @@ -1775,12 +1887,30 @@ "node": ">=6" } }, + "node_modules/deep-extend": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz", + "integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==", + "license": "MIT", + "engines": { + "node": ">=4.0.0" + } + }, "node_modules/deep-is": { "version": "0.1.4", "resolved": "https://registry.npmjs.org/deep-is/-/deep-is-0.1.4.tgz", "integrity": "sha512-oIPzksmTg4/MriiaYGO+okXDT7ztn/w3Eptv/+gSIdMdKsJo0u4CfYNFJPy+4SKMuCqGw2wxnA+URMg3t8a/bQ==", "dev": true }, + "node_modules/detect-libc": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz", + "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==", + "license": "Apache-2.0", + "engines": { + "node": ">=8" + } + }, "node_modules/dotenv": { "version": "16.5.0", "resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.5.0.tgz", @@ -1797,6 +1927,15 @@ "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz", "integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==" }, + "node_modules/end-of-stream": { + "version": "1.4.5", + "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz", + "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==", + "license": "MIT", + "dependencies": { + "once": "^1.4.0" + } + }, "node_modules/es-module-lexer": { "version": "1.7.0", "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-1.7.0.tgz", @@ -2064,6 +2203,15 @@ "node": ">=0.10.0" } }, + "node_modules/expand-template": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz", + "integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==", + "license": "(MIT OR WTFPL)", + "engines": { + "node": ">=6" + } + }, "node_modules/expect-type": { "version": "1.2.1", "resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.2.1.tgz", @@ -2140,6 +2288,12 @@ "node": ">=16.0.0" } }, + "node_modules/file-uri-to-path": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz", + "integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==", + "license": "MIT" + }, "node_modules/fill-range": { "version": "7.1.1", "resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.1.1.tgz", @@ -2187,6 +2341,12 @@ "integrity": "sha512-GX+ysw4PBCz0PzosHDepZGANEuFCMLrnRTiEy9McGjmkCQYwRq4A/X786G/fjM/+OjsWSU1ZrY5qyARZmO/uwg==", "dev": true }, + "node_modules/fs-constants": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz", + "integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==", + "license": "MIT" + }, "node_modules/fsevents": { "version": "2.3.3", "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", @@ -2201,6 +2361,12 @@ "node": "^8.16.0 || ^10.6.0 || >=11.0.0" } }, + "node_modules/fzstd": { + "version": "0.1.1", + "resolved": "https://registry.npmjs.org/fzstd/-/fzstd-0.1.1.tgz", + "integrity": "sha512-dkuVSOKKwh3eas5VkJy1AW1vFpet8TA/fGmVA5krThl8YcOVE/8ZIoEA1+U1vEn5ckxxhLirSdY837azmbaNHA==", + "license": "MIT" + }, "node_modules/get-caller-file": { "version": "2.0.5", "resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz", @@ -2209,6 +2375,12 @@ "node": "6.* || 8.* || >= 10.*" } }, + "node_modules/github-from-package": { + "version": "0.0.0", + "resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz", + "integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==", + "license": "MIT" + }, "node_modules/glob-parent": { "version": "6.0.2", "resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-6.0.2.tgz", @@ -2278,6 +2450,26 @@ "url": "https://github.com/sponsors/typicode" } }, + "node_modules/ieee754": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz", + "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "BSD-3-Clause" + }, "node_modules/ignore": { "version": "5.3.2", "resolved": "https://registry.npmjs.org/ignore/-/ignore-5.3.2.tgz", @@ -2312,6 +2504,18 @@ "node": ">=0.8.19" } }, + "node_modules/inherits": { + "version": "2.0.4", + "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz", + "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==", + "license": "ISC" + }, + "node_modules/ini": { + "version": "1.3.8", + "resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz", + "integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==", + "license": "ISC" + }, "node_modules/is-extglob": { "version": "2.1.1", "resolved": "https://registry.npmjs.org/is-extglob/-/is-extglob-2.1.1.tgz", @@ -2472,6 +2676,18 @@ "node": ">=8.6" } }, + "node_modules/mimic-response": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz", + "integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==", + "license": "MIT", + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, "node_modules/minimatch": { "version": "3.1.2", "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.2.tgz", @@ -2484,6 +2700,21 @@ "node": "*" } }, + "node_modules/minimist": { + "version": "1.2.8", + "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz", + "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/mkdirp-classic": { + "version": "0.5.3", + "resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz", + "integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==", + "license": "MIT" + }, "node_modules/ms": { "version": "2.1.3", "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", @@ -2561,12 +2792,39 @@ "node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1" } }, + "node_modules/napi-build-utils": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz", + "integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==", + "license": "MIT" + }, "node_modules/natural-compare": { "version": "1.4.0", "resolved": "https://registry.npmjs.org/natural-compare/-/natural-compare-1.4.0.tgz", "integrity": "sha512-OWND8ei3VtNC9h7V60qff3SVobHr996CTwgxubgyQYEpg290h9J0buyECNNJexkFm5sOajh5G116RYA1c8ZMSw==", "dev": true }, + "node_modules/node-abi": { + "version": "3.85.0", + "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.85.0.tgz", + "integrity": "sha512-zsFhmbkAzwhTft6nd3VxcG0cvJsT70rL+BIGHWVq5fi6MwGrHwzqKaxXE+Hl2GmnGItnDKPPkO5/LQqjVkIdFg==", + "license": "MIT", + "dependencies": { + "semver": "^7.3.5" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/once": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz", + "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==", + "license": "ISC", + "dependencies": { + "wrappy": "1" + } + }, "node_modules/optionator": { "version": "0.9.4", "resolved": "https://registry.npmjs.org/optionator/-/optionator-0.9.4.tgz", @@ -2717,6 +2975,32 @@ "node": "^10 || ^12 || >=14" } }, + "node_modules/prebuild-install": { + "version": "7.1.3", + "resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz", + "integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==", + "license": "MIT", + "dependencies": { + "detect-libc": "^2.0.0", + "expand-template": "^2.0.3", + "github-from-package": "0.0.0", + "minimist": "^1.2.3", + "mkdirp-classic": "^0.5.3", + "napi-build-utils": "^2.0.0", + "node-abi": "^3.3.0", + "pump": "^3.0.0", + "rc": "^1.2.7", + "simple-get": "^4.0.0", + "tar-fs": "^2.0.0", + "tunnel-agent": "^0.6.0" + }, + "bin": { + "prebuild-install": "bin.js" + }, + "engines": { + "node": ">=10" + } + }, "node_modules/prelude-ls": { "version": "1.2.1", "resolved": "https://registry.npmjs.org/prelude-ls/-/prelude-ls-1.2.1.tgz", @@ -2738,6 +3022,16 @@ "url": "https://github.com/sponsors/lupomontero" } }, + "node_modules/pump": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.3.tgz", + "integrity": "sha512-todwxLMY7/heScKmntwQG8CXVkWUOdYxIvY2s0VWAAMh/nd8SoYiRaKjlr7+iCs984f2P8zvrfWcDDYVb73NfA==", + "license": "MIT", + "dependencies": { + "end-of-stream": "^1.1.0", + "once": "^1.3.1" + } + }, "node_modules/punycode": { "version": "2.3.1", "resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.1.tgz", @@ -2773,6 +3067,44 @@ } ] }, + "node_modules/rc": { + "version": "1.2.8", + "resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz", + "integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==", + "license": "(BSD-2-Clause OR MIT OR Apache-2.0)", + "dependencies": { + "deep-extend": "^0.6.0", + "ini": "~1.3.0", + "minimist": "^1.2.0", + "strip-json-comments": "~2.0.1" + }, + "bin": { + "rc": "cli.js" + } + }, + "node_modules/rc/node_modules/strip-json-comments": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz", + "integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==", + "license": "MIT", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/readable-stream": { + "version": "3.6.2", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz", + "integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==", + "license": "MIT", + "dependencies": { + "inherits": "^2.0.3", + "string_decoder": "^1.1.1", + "util-deprecate": "^1.0.1" + }, + "engines": { + "node": ">= 6" + } + }, "node_modules/require-directory": { "version": "2.1.1", "resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz", @@ -2868,6 +3200,26 @@ "queue-microtask": "^1.2.2" } }, + "node_modules/safe-buffer": { + "version": "5.2.1", + "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz", + "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT" + }, "node_modules/sax": { "version": "1.4.1", "resolved": "https://registry.npmjs.org/sax/-/sax-1.4.1.tgz", @@ -2923,6 +3275,51 @@ "url": "https://github.com/sponsors/isaacs" } }, + "node_modules/simple-concat": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz", + "integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT" + }, + "node_modules/simple-get": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz", + "integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT", + "dependencies": { + "decompress-response": "^6.0.0", + "once": "^1.3.1", + "simple-concat": "^1.0.0" + } + }, "node_modules/source-map-js": { "version": "1.2.1", "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz", @@ -2959,6 +3356,15 @@ "integrity": "sha512-vMgjE/GGEPEFnhFub6pa4FmJBRBVOLpIII2hvCZ8Kzb7K0hlHo7mQv6xYrBvCL2LtAIBwFUK8wvuJgTVSQ5MFQ==", "dev": true }, + "node_modules/string_decoder": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz", + "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==", + "license": "MIT", + "dependencies": { + "safe-buffer": "~5.2.0" + } + }, "node_modules/string-width": { "version": "4.2.3", "resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz", @@ -3032,6 +3438,34 @@ "node": ">=8" } }, + "node_modules/tar-fs": { + "version": "2.1.4", + "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.4.tgz", + "integrity": "sha512-mDAjwmZdh7LTT6pNleZ05Yt65HC3E+NiQzl672vQG38jIrehtJk/J3mNwIg+vShQPcLF/LV7CMnDW6vjj6sfYQ==", + "license": "MIT", + "dependencies": { + "chownr": "^1.1.1", + "mkdirp-classic": "^0.5.2", + "pump": "^3.0.0", + "tar-stream": "^2.1.4" + } + }, + "node_modules/tar-stream": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz", + "integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==", + "license": "MIT", + "dependencies": { + "bl": "^4.0.3", + "end-of-stream": "^1.4.1", + "fs-constants": "^1.0.0", + "inherits": "^2.0.3", + "readable-stream": "^3.1.1" + }, + "engines": { + "node": ">=6" + } + }, "node_modules/tinybench": { "version": "2.9.0", "resolved": "https://registry.npmjs.org/tinybench/-/tinybench-2.9.0.tgz", @@ -3164,6 +3598,18 @@ "typescript": ">=4.8.4" } }, + "node_modules/tunnel-agent": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz", + "integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==", + "license": "Apache-2.0", + "dependencies": { + "safe-buffer": "^5.0.1" + }, + "engines": { + "node": "*" + } + }, "node_modules/type-check": { "version": "0.4.0", "resolved": "https://registry.npmjs.org/type-check/-/type-check-0.4.0.tgz", @@ -3257,6 +3703,12 @@ "requires-port": "^1.0.0" } }, + "node_modules/util-deprecate": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz", + "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==", + "license": "MIT" + }, "node_modules/vite": { "version": "6.3.4", "resolved": "https://registry.npmjs.org/vite/-/vite-6.3.4.tgz", @@ -3526,6 +3978,12 @@ "node": ">=8" } }, + "node_modules/wrappy": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz", + "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==", + "license": "ISC" + }, "node_modules/xml2js": { "version": "0.6.2", "resolved": "https://registry.npmjs.org/xml2js/-/xml2js-0.6.2.tgz", diff --git a/package.json b/package.json index 7bb7386..c902050 100644 --- a/package.json +++ b/package.json @@ -28,6 +28,7 @@ "homepage": "https://github.com/Hypersequent/qas-cli#readme", "devDependencies": { "@eslint/js": "^9.25.1", + "@types/better-sqlite3": "^7.6.13", "@types/escape-html": "^1.0.4", "@types/node": "^20.17.32", "@types/semver": "^7.7.0", @@ -43,9 +44,11 @@ "vitest": "^3.1.2" }, "dependencies": { + "better-sqlite3": "^12.5.0", "chalk": "^5.4.1", "dotenv": "^16.5.0", "escape-html": "^1.0.3", + "fzstd": "^0.1.1", "semver": "^7.7.1", "strip-ansi": "^7.1.2", "xml2js": "^0.6.2", diff --git a/src/api/folders.ts b/src/api/folders.ts new file mode 100644 index 0000000..866275f --- /dev/null +++ b/src/api/folders.ts @@ -0,0 +1,14 @@ +import { Folder, PaginatedRequest, PaginatedResponse, ResourceId } from './schemas' +import { appendSearchParams, jsonResponse, withJson } from './utils' + +export const createFolderApi = (fetcher: typeof fetch) => { + fetcher = withJson(fetcher) + return { + getFoldersPaginated: (projectCode: ResourceId, request: PaginatedRequest) => + fetcher( + appendSearchParams(`/api/public/v0/project/${projectCode}/tcase/folders`, request) + ).then((r) => jsonResponse>(r)), + } +} + +export type FolderApi = ReturnType diff --git a/src/api/index.ts b/src/api/index.ts index 370326a..f9cd166 100644 --- a/src/api/index.ts +++ b/src/api/index.ts @@ -1,19 +1,21 @@ +import { createFileApi } from './file' +import { createFolderApi } from './folders' import { createProjectApi } from './projects' import { createRunApi } from './run' import { createTCaseApi } from './tcases' -import { createFileApi } from './file' import { withApiKey, withBaseUrl } from './utils' const getApi = (fetcher: typeof fetch) => { - return { - projects: createProjectApi(fetcher), - runs: createRunApi(fetcher), - testcases: createTCaseApi(fetcher), - file: createFileApi(fetcher), - } + return { + files: createFileApi(fetcher), + folders: createFolderApi(fetcher), + projects: createProjectApi(fetcher), + runs: createRunApi(fetcher), + testcases: createTCaseApi(fetcher), + } } export type Api = ReturnType export const createApi = (baseUrl: string, apiKey: string) => - getApi(withApiKey(withBaseUrl(fetch, baseUrl), apiKey)) \ No newline at end of file + getApi(withApiKey(withBaseUrl(fetch, baseUrl), apiKey)) diff --git a/src/api/schemas.ts b/src/api/schemas.ts index 903ff48..ca9ad1d 100644 --- a/src/api/schemas.ts +++ b/src/api/schemas.ts @@ -2,8 +2,51 @@ export type ResourceId = string | number export type ResultStatus = 'open' | 'passed' | 'blocked' | 'failed' | 'skipped' +export interface PaginatedResponse { + data: T[] + total: number + page: number + limit: number +} + +export interface PaginatedRequest { + page?: number + limit?: number +} + +export interface TCase { + id: string + legacyId?: string + seq: number + title: string + version: number + projectId: string + folderId: number +} + +export interface CreateTCasesRequest { + folderPath: string[] + tcases: { title: string; tags: string[] }[] +} + +export interface CreateTCasesResponse { + tcases: { id: string; seq: number }[] +} + +export interface GetTCasesRequest extends PaginatedRequest { + folders?: number[] +} + +export interface GetTCasesBySeqRequest { + seqIds: string[] + page?: number + limit?: number +} + export interface Folder { id: number + parentId: number + pos: number title: string } diff --git a/src/api/tcases.ts b/src/api/tcases.ts index a3443a8..8c758c3 100644 --- a/src/api/tcases.ts +++ b/src/api/tcases.ts @@ -1,34 +1,32 @@ -import { ResourceId } from './schemas' -import { jsonResponse, withJson } from './utils' -export interface PaginatedResponse { - data: T[] - total: number - page: number - limit: number -} +import { + CreateTCasesRequest, + CreateTCasesResponse, + GetTCasesBySeqRequest, + GetTCasesRequest, + PaginatedResponse, + ResourceId, + TCase, +} from './schemas' +import { appendSearchParams, jsonResponse, withJson } from './utils' -export interface TCaseBySeq { - id: string - legacyId?: string - seq: number - version: number - projectId: string - folderId: number -} +export const createTCaseApi = (fetcher: typeof fetch) => { + fetcher = withJson(fetcher) + return { + getTCasesPaginated: (projectCode: ResourceId, request: GetTCasesRequest) => + fetcher(appendSearchParams(`/api/public/v0/project/${projectCode}/tcase`, request)).then( + (r) => jsonResponse>(r) + ), -export interface GetTCasesBySeqRequest { - seqIds: string[] - page?: number - limit?: number -} + getTCasesBySeq: (projectCode: ResourceId, request: GetTCasesBySeqRequest) => + fetcher(`/api/public/v0/project/${projectCode}/tcase/seq`, { + method: 'POST', + body: JSON.stringify(request), + }).then((r) => jsonResponse>(r)), -export const createTCaseApi = (fetcher: typeof fetch) => { - fetcher = withJson(fetcher) - return { - getTCasesBySeq: (projectCode: ResourceId, request: GetTCasesBySeqRequest) => - fetcher(`/api/public/v0/project/${projectCode}/tcase/seq`, { - method: 'POST', - body: JSON.stringify(request), - }).then((r) => jsonResponse>(r)), - } + createTCases: (projectCode: ResourceId, request: CreateTCasesRequest) => + fetcher(`/api/public/v0/project/${projectCode}/tcase/bulk`, { + method: 'POST', + body: JSON.stringify(request), + }).then((r) => jsonResponse(r)), + } } diff --git a/src/api/utils.ts b/src/api/utils.ts index 4f459db..7461557 100644 --- a/src/api/utils.ts +++ b/src/api/utils.ts @@ -48,3 +48,39 @@ export const jsonResponse = async (response: Response): Promise => { } throw new Error(response.statusText) } + +const updateSearchParams = (searchParams: URLSearchParams, obj?: T) => { + const isValidValue = (value: unknown) => { + return value || value === false || value === '' + } + + if (!obj) return + + Object.entries(obj).forEach(([key, value]) => { + if (isValidValue(value)) { + if (Array.isArray(value)) { + value.forEach((param) => { + if (isValidValue(param)) { + searchParams.append(key, String(param)) + } + }) + } else if (value instanceof Date) { + searchParams.set(key, value.toISOString()) + } else if (typeof value === 'object') { + updateSearchParams(searchParams, value) + } else { + searchParams.set(key, String(value)) + } + } + }) +} + +export const appendSearchParams = (pathname: string, obj: T): string => { + const searchParams = new URLSearchParams() + updateSearchParams(searchParams, obj) + + if (searchParams.size > 0) { + return `${pathname}?${searchParams.toString()}` + } + return pathname +} diff --git a/src/commands/main.ts b/src/commands/main.ts index d8a76b3..01c5e4f 100644 --- a/src/commands/main.ts +++ b/src/commands/main.ts @@ -13,7 +13,8 @@ Required variables: ${qasEnvs.join(', ')} ) .command(new ResultUploadCommandModule('junit-upload')) .command(new ResultUploadCommandModule('playwright-json-upload')) - .demandCommand(1, "") + .command(new ResultUploadCommandModule('xcresult-upload')) + .demandCommand(1, '') .help('h') .alias('h', 'help') .version(getVersion()) @@ -29,23 +30,26 @@ Required variables: ${qasEnvs.join(', ')} .fail((msg, err, yi) => { // if no command is provided, show help and exit if (args.length === 0) { - yi.showHelp(); - process.exit(0); + yi.showHelp() + process.exit(0) } else { if (msg) { - console.error(msg); - if (msg.startsWith('Unknown argument') || msg.startsWith('Not enough non-option arguments')) { - yi.showHelp(); - process.exit(0); + console.error(msg) + if ( + msg.startsWith('Unknown argument') || + msg.startsWith('Not enough non-option arguments') + ) { + yi.showHelp() + process.exit(0) } } else if (err && err.message) { - console.error(err.message); + console.error(err.message) } else if (err) { - console.error(String(err)); + console.error(String(err)) } else { - console.error('An unexpected error occurred.'); + console.error('An unexpected error occurred.') } - process.exit(1); + process.exit(1) } }) .parse() diff --git a/src/commands/resultUpload.ts b/src/commands/resultUpload.ts index df5cac0..28d5dde 100644 --- a/src/commands/resultUpload.ts +++ b/src/commands/resultUpload.ts @@ -1,20 +1,22 @@ import { Arguments, Argv, CommandModule } from 'yargs' import chalk from 'chalk' -import { loadEnvs } from '../utils/env' +import { loadEnvs, qasEnvFile } from '../utils/env' import { ResultUploadCommandArgs, ResultUploadCommandHandler, - UploadCommandType + UploadCommandType, } from '../utils/result-upload/ResultUploadCommandHandler' const commandTypeDisplayStrings: Record = { 'junit-upload': 'JUnit XML', 'playwright-json-upload': 'Playwright JSON', + 'xcresult-upload': 'Xcode Result Bundle', } const commandTypeFileExtensions: Record = { 'junit-upload': 'xml', 'playwright-json-upload': 'json', + 'xcresult-upload': 'xcresult', } export class ResultUploadCommandModule implements CommandModule { @@ -25,7 +27,7 @@ export class ResultUploadCommandModule implements CommandModule { @@ -36,11 +38,22 @@ export class ResultUploadCommandModule implements CommandModule + + + + + + + + + + + + diff --git a/src/tests/fixtures/playwright-json/without-markers.json b/src/tests/fixtures/playwright-json/without-markers.json new file mode 100644 index 0000000..7ca1b39 --- /dev/null +++ b/src/tests/fixtures/playwright-json/without-markers.json @@ -0,0 +1,77 @@ +{ + "suites": [ + { + "title": "ui.cart.spec.ts", + "specs": [ + { + "title": "Test cart TEST-002", + "tags": [], + "tests": [ + { + "annotations": [], + "expectedStatus": "passed", + "projectName": "chromium", + "results": [ + { + "status": "passed", + "errors": [], + "stdout": [], + "stderr": [], + "retry": 0, + "attachments": [] + } + ], + "status": "expected" + } + ] + }, + { + "title": "The cart is still filled after refreshing the page", + "tags": [], + "tests": [ + { + "annotations": [], + "expectedStatus": "passed", + "projectName": "chromium", + "results": [ + { + "status": "passed", + "errors": [], + "stdout": [], + "stderr": [], + "retry": 0, + "attachments": [] + } + ], + "status": "expected" + } + ] + }, + { + "title": "TEST-010: Cart should be cleared after making the checkout", + "tags": [], + "tests": [ + { + "annotations": [], + "expectedStatus": "passed", + "projectName": "chromium", + "results": [ + { + "status": "passed", + "errors": [], + "stdout": [], + "stderr": [], + "retry": 0, + "attachments": [] + } + ], + "status": "expected" + } + ] + } + ], + "suites": [] + } + ] +} + diff --git a/src/tests/fixtures/xcresult/Variety.xcresult/database.sqlite3 b/src/tests/fixtures/xcresult/Variety.xcresult/database.sqlite3 new file mode 100644 index 0000000..4fd3768 Binary files /dev/null and b/src/tests/fixtures/xcresult/Variety.xcresult/database.sqlite3 differ diff --git a/src/tests/junit-xml-parsing.spec.ts b/src/tests/junit-xml-parsing.spec.ts index 32c0636..6089a5e 100644 --- a/src/tests/junit-xml-parsing.spec.ts +++ b/src/tests/junit-xml-parsing.spec.ts @@ -1,16 +1,22 @@ -import { expect, test, describe } from 'vitest' +import { expect, test, describe, afterEach } from 'vitest' import { parseJUnitXml } from '../utils/result-upload/junitXmlParser' -import { readFile } from 'fs/promises' +import { createTempFile, deleteTempFile } from './utils' const xmlBasePath = './src/tests/fixtures/junit-xml' describe('Junit XML parsing', () => { - test('Should parse comprehensive test XML without exceptions', async () => { - const xmlPath = `${xmlBasePath}/comprehensive-test.xml` - const xmlContent = await readFile(xmlPath, 'utf8') + let tempXmlFile: string | null = null + + afterEach(() => { + if (tempXmlFile) { + deleteTempFile(tempXmlFile) + tempXmlFile = null + } + }) + test('Should parse comprehensive test XML without exceptions', async () => { // This should not throw any exceptions - const testcases = await parseJUnitXml(xmlContent, xmlBasePath, { + const testcases = await parseJUnitXml(`${xmlBasePath}/comprehensive-test.xml`, xmlBasePath, { skipStdout: 'never', skipStderr: 'never', }) @@ -48,10 +54,7 @@ describe('Junit XML parsing', () => { }) test('Should handle all failure/error/skipped element variations', async () => { - const xmlPath = `${xmlBasePath}/comprehensive-test.xml` - const xmlContent = await readFile(xmlPath, 'utf8') - - const testcases = await parseJUnitXml(xmlContent, xmlBasePath, { + const testcases = await parseJUnitXml(`${xmlBasePath}/comprehensive-test.xml`, xmlBasePath, { skipStdout: 'never', skipStderr: 'never', }) @@ -64,9 +67,7 @@ describe('Junit XML parsing', () => { // Verify we have the expected failure scenarios expect(failureTests.some((tc) => tc.name.includes('only type'))).toBe(true) expect(failureTests.some((tc) => tc.name.includes('type and message'))).toBe(true) - expect(failureTests.some((tc) => tc.name.includes('type, message and text content'))).toBe( - true - ) + expect(failureTests.some((tc) => tc.name.includes('type, message and text content'))).toBe(true) // Verify we have the expected error scenarios expect(errorTests.some((tc) => tc.name.includes('only type'))).toBe(true) @@ -81,10 +82,7 @@ describe('Junit XML parsing', () => { }) test('Should handle empty and similar empty tags', async () => { - const xmlPath = `${xmlBasePath}/empty-system-err.xml` - const xmlContent = await readFile(xmlPath, 'utf8') - - const testcases = await parseJUnitXml(xmlContent, xmlBasePath, { + const testcases = await parseJUnitXml(`${xmlBasePath}/empty-system-err.xml`, xmlBasePath, { skipStdout: 'never', skipStderr: 'never', }) @@ -98,10 +96,7 @@ describe('Junit XML parsing', () => { }) test('Should handle Jest failure without type attribute', async () => { - const xmlPath = `${xmlBasePath}/jest-failure-type-missing.xml` - const xmlContent = await readFile(xmlPath, 'utf8') - - const testcases = await parseJUnitXml(xmlContent, xmlBasePath, { + const testcases = await parseJUnitXml(`${xmlBasePath}/jest-failure-type-missing.xml`, xmlBasePath, { skipStdout: 'never', skipStderr: 'never', }) @@ -124,10 +119,7 @@ describe('Junit XML parsing', () => { }) test('Should extract attachments from failure/error message attributes (WebDriverIO style)', async () => { - const xmlPath = `${xmlBasePath}/webdriverio-real.xml` - const xmlContent = await readFile(xmlPath, 'utf8') - - const testcases = await parseJUnitXml(xmlContent, xmlBasePath, { + const testcases = await parseJUnitXml(`${xmlBasePath}/webdriverio-real.xml`, xmlBasePath, { skipStdout: 'never', skipStderr: 'never', }) @@ -146,10 +138,7 @@ describe('Junit XML parsing', () => { }) test('Should include stdout/stderr when skipStdout and skipStderr are set to "never"', async () => { - const xmlPath = `${xmlBasePath}/empty-system-err.xml` - const xmlContent = await readFile(xmlPath, 'utf8') - - const testcases = await parseJUnitXml(xmlContent, xmlBasePath, { + const testcases = await parseJUnitXml(`${xmlBasePath}/empty-system-err.xml`, xmlBasePath, { skipStdout: 'never', skipStderr: 'never', }) @@ -161,10 +150,7 @@ describe('Junit XML parsing', () => { }) test('Should skip stdout for passed tests when skipStdout is set to "on-success"', async () => { - const xmlPath = `${xmlBasePath}/empty-system-err.xml` - const xmlContent = await readFile(xmlPath, 'utf8') - - const testcases = await parseJUnitXml(xmlContent, xmlBasePath, { + const testcases = await parseJUnitXml(`${xmlBasePath}/empty-system-err.xml`, xmlBasePath, { skipStdout: 'on-success', skipStderr: 'never', }) @@ -177,7 +163,7 @@ describe('Junit XML parsing', () => { }) test('Should skip stderr for passed tests when skipStderr is set to "on-success"', async () => { - const xml = ` + tempXmlFile = createTempFile(` @@ -185,9 +171,9 @@ describe('Junit XML parsing', () => { stderr content -` +`, 'xml') - const testcases = await parseJUnitXml(xml, xmlBasePath, { + const testcases = await parseJUnitXml(tempXmlFile, xmlBasePath, { skipStdout: 'never', skipStderr: 'on-success', }) @@ -200,7 +186,7 @@ describe('Junit XML parsing', () => { }) test('Should include stdout/stderr for failed tests even when skip options are set to "on-success"', async () => { - const xml = ` + tempXmlFile = createTempFile(` @@ -209,9 +195,9 @@ describe('Junit XML parsing', () => { stderr from failed test -` +`, 'xml') - const testcases = await parseJUnitXml(xml, xmlBasePath, { + const testcases = await parseJUnitXml(tempXmlFile, xmlBasePath, { skipStdout: 'on-success', skipStderr: 'on-success', }) @@ -225,7 +211,7 @@ describe('Junit XML parsing', () => { }) test('Should skip both stdout and stderr for passed tests when both skip options are set to "on-success"', async () => { - const xml = ` + tempXmlFile = createTempFile(` @@ -233,9 +219,9 @@ describe('Junit XML parsing', () => { stderr content -` +`, 'xml') - const testcases = await parseJUnitXml(xml, xmlBasePath, { + const testcases = await parseJUnitXml(tempXmlFile, xmlBasePath, { skipStdout: 'on-success', skipStderr: 'on-success', }) diff --git a/src/tests/playwright-json-parsing.spec.ts b/src/tests/playwright-json-parsing.spec.ts index a88850a..c5c9250 100644 --- a/src/tests/playwright-json-parsing.spec.ts +++ b/src/tests/playwright-json-parsing.spec.ts @@ -1,19 +1,30 @@ +import { afterEach } from 'node:test' import { expect, test, describe } from 'vitest' import { parsePlaywrightJson } from '../utils/result-upload/playwrightJsonParser' -import { readFile } from 'fs/promises' +import { createTempFile, deleteTempFile } from './utils' const playwrightJsonBasePath = './src/tests/fixtures/playwright-json' describe('Playwright JSON parsing', () => { - test('Should parse comprehensive test JSON without exceptions', async () => { - const jsonPath = `${playwrightJsonBasePath}/comprehensive-test.json` - const jsonContent = await readFile(jsonPath, 'utf8') + let tempJsonFile: string | null = null + + afterEach(() => { + if (tempJsonFile) { + deleteTempFile(tempJsonFile) + tempJsonFile = null + } + }) + test('Should parse comprehensive test JSON without exceptions', async () => { // This should not throw any exceptions - const testcases = await parsePlaywrightJson(jsonContent, '', { - skipStdout: 'never', - skipStderr: 'never', - }) + const testcases = await parsePlaywrightJson( + `${playwrightJsonBasePath}/comprehensive-test.json`, + '', + { + skipStdout: 'never', + skipStderr: 'never', + } + ) // Verify that we got the expected number of test cases expect(testcases).toHaveLength(12) @@ -46,10 +57,7 @@ describe('Playwright JSON parsing', () => { }) test('Should handle empty test suite', async () => { - const jsonPath = `${playwrightJsonBasePath}/empty-tsuite.json` - const jsonContent = await readFile(jsonPath, 'utf8') - - const testcases = await parsePlaywrightJson(jsonContent, '', { + const testcases = await parsePlaywrightJson(`${playwrightJsonBasePath}/empty-tsuite.json`, '', { skipStdout: 'never', skipStderr: 'never', }) @@ -60,48 +68,51 @@ describe('Playwright JSON parsing', () => { }) test('Should use last result when there are retries', async () => { - const jsonContent = JSON.stringify({ - suites: [ - { - title: 'retry.spec.ts', - specs: [ - { - title: 'Flaky test', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'failed', - errors: [{ message: 'First attempt failed' }], - stdout: [], - stderr: [], - retry: 0, - attachments: [], - }, - { - status: 'passed', - errors: [], - stdout: [], - stderr: [], - retry: 1, - attachments: [], - }, - ], - status: 'flaky', - }, - ], - }, - ], - suites: [], - }, - ], - }) - - const testcases = await parsePlaywrightJson(jsonContent, '', { + tempJsonFile = createTempFile( + JSON.stringify({ + suites: [ + { + title: 'retry.spec.ts', + specs: [ + { + title: 'Flaky test', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'failed', + errors: [{ message: 'First attempt failed' }], + stdout: [], + stderr: [], + retry: 0, + attachments: [], + }, + { + status: 'passed', + errors: [], + stdout: [], + stderr: [], + retry: 1, + attachments: [], + }, + ], + status: 'flaky', + }, + ], + }, + ], + suites: [], + }, + ], + }), + 'json' + ) + + const testcases = await parsePlaywrightJson(tempJsonFile, '', { skipStdout: 'never', skipStderr: 'never', }) @@ -113,69 +124,72 @@ describe('Playwright JSON parsing', () => { }) test('Should handle nested suites correctly', async () => { - const jsonContent = JSON.stringify({ - suites: [ - { - title: 'parent.spec.ts', - specs: [ - { - title: 'Parent test', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'passed', - errors: [], - stdout: [], - stderr: [], - retry: 0, - attachments: [], - }, - ], - status: 'expected', - }, - ], - }, - ], - suites: [ - { - title: 'Nested Suite', - specs: [ - { - title: 'Nested test', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'passed', - errors: [], - stdout: [], - stderr: [], - retry: 0, - attachments: [], - }, - ], - status: 'expected', - }, - ], - }, - ], - suites: [], - }, - ], - }, - ], - }) - - const testcases = await parsePlaywrightJson(jsonContent, '', { + tempJsonFile = createTempFile( + JSON.stringify({ + suites: [ + { + title: 'parent.spec.ts', + specs: [ + { + title: 'Parent test', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'passed', + errors: [], + stdout: [], + stderr: [], + retry: 0, + attachments: [], + }, + ], + status: 'expected', + }, + ], + }, + ], + suites: [ + { + title: 'Nested Suite', + specs: [ + { + title: 'Nested test', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'passed', + errors: [], + stdout: [], + stderr: [], + retry: 0, + attachments: [], + }, + ], + status: 'expected', + }, + ], + }, + ], + suites: [], + }, + ], + }, + ], + }), + 'json' + ) + + const testcases = await parsePlaywrightJson(tempJsonFile, '', { skipStdout: 'never', skipStderr: 'never', }) @@ -191,53 +205,56 @@ describe('Playwright JSON parsing', () => { }) test('Should strip ANSI escape codes from errors and output', async () => { - const jsonContent = JSON.stringify({ - suites: [ - { - title: 'ansi.spec.ts', - specs: [ - { - title: 'Test with ANSI colors in error', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'failed', - errors: [ - { - message: - '\x1b[31mError: Test failed\x1b[0m\n\x1b[90m at Object.test\x1b[0m', - }, - ], - stdout: [ - { - text: '\x1b[32m✓\x1b[0m Test started\n\x1b[33mWarning:\x1b[0m Something happened', - }, - ], - stderr: [ - { - text: '\x1b[31mError output\x1b[0m\n\x1b[90mStack trace\x1b[0m', - }, - ], - retry: 0, - attachments: [], - }, - ], - status: 'unexpected', - }, - ], - }, - ], - suites: [], - }, - ], - }) - - const testcases = await parsePlaywrightJson(jsonContent, '', { + tempJsonFile = createTempFile( + JSON.stringify({ + suites: [ + { + title: 'ansi.spec.ts', + specs: [ + { + title: 'Test with ANSI colors in error', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'failed', + errors: [ + { + message: + '\x1b[31mError: Test failed\x1b[0m\n\x1b[90m at Object.test\x1b[0m', + }, + ], + stdout: [ + { + text: '\x1b[32m✓\x1b[0m Test started\n\x1b[33mWarning:\x1b[0m Something happened', + }, + ], + stderr: [ + { + text: '\x1b[31mError output\x1b[0m\n\x1b[90mStack trace\x1b[0m', + }, + ], + retry: 0, + attachments: [], + }, + ], + status: 'unexpected', + }, + ], + }, + ], + suites: [], + }, + ], + }), + 'json' + ) + + const testcases = await parsePlaywrightJson(tempJsonFile, '', { skipStdout: 'never', skipStderr: 'never', }) @@ -258,94 +275,97 @@ describe('Playwright JSON parsing', () => { }) test('Should prefix test case marker from annotations to test name', async () => { - const jsonContent = JSON.stringify({ - suites: [ - { - title: 'annotation.spec.ts', - specs: [ - { - title: 'User login test', - tags: [], - tests: [ - { - annotations: [ - { - type: 'test case', - description: 'https://qas.eu1.qasphere.com/project/PRJ/tcase/123', - }, - ], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'passed', - errors: [], - stdout: [], - stderr: [], - retry: 0, - attachments: [], - }, - ], - status: 'expected', - }, - ], - }, - { - title: 'Test without annotation', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'passed', - errors: [], - stdout: [], - stderr: [], - retry: 0, - attachments: [], - }, - ], - status: 'expected', - }, - ], - }, - { - title: 'PRJ-456: Test with marker in name and annotation', - tags: [], - tests: [ - { - annotations: [ - { - type: 'Test Case', - description: 'https://qas.eu1.qasphere.com/project/PRJ/tcase/789', - }, - ], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'passed', - errors: [], - stdout: [], - stderr: [], - retry: 0, - attachments: [], - }, - ], - status: 'expected', - }, - ], - }, - ], - suites: [], - }, - ], - }) - - const testcases = await parsePlaywrightJson(jsonContent, '', { + tempJsonFile = createTempFile( + JSON.stringify({ + suites: [ + { + title: 'annotation.spec.ts', + specs: [ + { + title: 'User login test', + tags: [], + tests: [ + { + annotations: [ + { + type: 'test case', + description: 'https://qas.eu1.qasphere.com/project/PRJ/tcase/123', + }, + ], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'passed', + errors: [], + stdout: [], + stderr: [], + retry: 0, + attachments: [], + }, + ], + status: 'expected', + }, + ], + }, + { + title: 'Test without annotation', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'passed', + errors: [], + stdout: [], + stderr: [], + retry: 0, + attachments: [], + }, + ], + status: 'expected', + }, + ], + }, + { + title: 'PRJ-456: Test with marker in name and annotation', + tags: [], + tests: [ + { + annotations: [ + { + type: 'Test Case', + description: 'https://qas.eu1.qasphere.com/project/PRJ/tcase/789', + }, + ], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'passed', + errors: [], + stdout: [], + stderr: [], + retry: 0, + attachments: [], + }, + ], + status: 'expected', + }, + ], + }, + ], + suites: [], + }, + ], + }), + 'json' + ) + + const testcases = await parsePlaywrightJson(tempJsonFile, '', { skipStdout: 'never', skipStderr: 'never', }) @@ -362,114 +382,117 @@ describe('Playwright JSON parsing', () => { }) test('Should map test status correctly', async () => { - const jsonContent = JSON.stringify({ - suites: [ - { - title: 'status.spec.ts', - specs: [ - { - title: 'Expected test', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'passed', - errors: [], - stdout: [], - stderr: [], - retry: 0, - attachments: [], - }, - ], - status: 'expected', - }, - ], - }, - { - title: 'Unexpected test', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'failed', - errors: [{ message: 'Test failed' }], - stdout: [], - stderr: [], - retry: 0, - attachments: [], - }, - ], - status: 'unexpected', - }, - ], - }, - { - title: 'Flaky test', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'failed', - errors: [], - stdout: [], - stderr: [], - retry: 0, - attachments: [], - }, - { - status: 'passed', - errors: [], - stdout: [], - stderr: [], - retry: 1, - attachments: [], - }, - ], - status: 'flaky', - }, - ], - }, - { - title: 'Skipped test', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'skipped', - projectName: 'chromium', - results: [ - { - status: 'skipped', - errors: [], - stdout: [], - stderr: [], - retry: 0, - attachments: [], - }, - ], - status: 'skipped', - }, - ], - }, - ], - suites: [], - }, - ], - }) - - const testcases = await parsePlaywrightJson(jsonContent, '', { + tempJsonFile = createTempFile( + JSON.stringify({ + suites: [ + { + title: 'status.spec.ts', + specs: [ + { + title: 'Expected test', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'passed', + errors: [], + stdout: [], + stderr: [], + retry: 0, + attachments: [], + }, + ], + status: 'expected', + }, + ], + }, + { + title: 'Unexpected test', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'failed', + errors: [{ message: 'Test failed' }], + stdout: [], + stderr: [], + retry: 0, + attachments: [], + }, + ], + status: 'unexpected', + }, + ], + }, + { + title: 'Flaky test', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'failed', + errors: [], + stdout: [], + stderr: [], + retry: 0, + attachments: [], + }, + { + status: 'passed', + errors: [], + stdout: [], + stderr: [], + retry: 1, + attachments: [], + }, + ], + status: 'flaky', + }, + ], + }, + { + title: 'Skipped test', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'skipped', + projectName: 'chromium', + results: [ + { + status: 'skipped', + errors: [], + stdout: [], + stderr: [], + retry: 0, + attachments: [], + }, + ], + status: 'skipped', + }, + ], + }, + ], + suites: [], + }, + ], + }), + 'json' + ) + + const testcases = await parsePlaywrightJson(tempJsonFile, '', { skipStdout: 'never', skipStderr: 'never', }) @@ -482,40 +505,43 @@ describe('Playwright JSON parsing', () => { }) test('Should include stdout/stderr when skipStdout and skipStderr are set to "never"', async () => { - const jsonContent = JSON.stringify({ - suites: [ - { - title: 'test.spec.ts', - specs: [ - { - title: 'Passed test with output', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'passed', - errors: [], - stdout: [{ text: 'stdout content' }], - stderr: [{ text: 'stderr content' }], - retry: 0, - attachments: [], - }, - ], - status: 'expected', - }, - ], - }, - ], - suites: [], - }, - ], - }) - - const testcases = await parsePlaywrightJson(jsonContent, '', { + tempJsonFile = createTempFile( + JSON.stringify({ + suites: [ + { + title: 'test.spec.ts', + specs: [ + { + title: 'Passed test with output', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'passed', + errors: [], + stdout: [{ text: 'stdout content' }], + stderr: [{ text: 'stderr content' }], + retry: 0, + attachments: [], + }, + ], + status: 'expected', + }, + ], + }, + ], + suites: [], + }, + ], + }), + 'json' + ) + + const testcases = await parsePlaywrightJson(tempJsonFile, '', { skipStdout: 'never', skipStderr: 'never', }) @@ -527,40 +553,43 @@ describe('Playwright JSON parsing', () => { }) test('Should skip stdout for passed tests when skipStdout is set to "on-success"', async () => { - const jsonContent = JSON.stringify({ - suites: [ - { - title: 'test.spec.ts', - specs: [ - { - title: 'Passed test with output', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'passed', - errors: [], - stdout: [{ text: 'stdout content' }], - stderr: [{ text: 'stderr content' }], - retry: 0, - attachments: [], - }, - ], - status: 'expected', - }, - ], - }, - ], - suites: [], - }, - ], - }) - - const testcases = await parsePlaywrightJson(jsonContent, '', { + tempJsonFile = createTempFile( + JSON.stringify({ + suites: [ + { + title: 'test.spec.ts', + specs: [ + { + title: 'Passed test with output', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'passed', + errors: [], + stdout: [{ text: 'stdout content' }], + stderr: [{ text: 'stderr content' }], + retry: 0, + attachments: [], + }, + ], + status: 'expected', + }, + ], + }, + ], + suites: [], + }, + ], + }), + 'json' + ) + + const testcases = await parsePlaywrightJson(tempJsonFile, '', { skipStdout: 'on-success', skipStderr: 'never', }) @@ -572,40 +601,43 @@ describe('Playwright JSON parsing', () => { }) test('Should skip stderr for passed tests when skipStderr is set to "on-success"', async () => { - const jsonContent = JSON.stringify({ - suites: [ - { - title: 'test.spec.ts', - specs: [ - { - title: 'Passed test with output', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'passed', - errors: [], - stdout: [{ text: 'stdout content' }], - stderr: [{ text: 'stderr content' }], - retry: 0, - attachments: [], - }, - ], - status: 'expected', - }, - ], - }, - ], - suites: [], - }, - ], - }) - - const testcases = await parsePlaywrightJson(jsonContent, '', { + tempJsonFile = createTempFile( + JSON.stringify({ + suites: [ + { + title: 'test.spec.ts', + specs: [ + { + title: 'Passed test with output', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'passed', + errors: [], + stdout: [{ text: 'stdout content' }], + stderr: [{ text: 'stderr content' }], + retry: 0, + attachments: [], + }, + ], + status: 'expected', + }, + ], + }, + ], + suites: [], + }, + ], + }), + 'json' + ) + + const testcases = await parsePlaywrightJson(tempJsonFile, '', { skipStdout: 'never', skipStderr: 'on-success', }) @@ -617,40 +649,43 @@ describe('Playwright JSON parsing', () => { }) test('Should include stdout/stderr for failed tests even when skip options are set to "on-success"', async () => { - const jsonContent = JSON.stringify({ - suites: [ - { - title: 'test.spec.ts', - specs: [ - { - title: 'Failed test with output', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'failed', - errors: [{ message: 'Test failed' }], - stdout: [{ text: 'stdout from failed test' }], - stderr: [{ text: 'stderr from failed test' }], - retry: 0, - attachments: [], - }, - ], - status: 'unexpected', - }, - ], - }, - ], - suites: [], - }, - ], - }) - - const testcases = await parsePlaywrightJson(jsonContent, '', { + tempJsonFile = createTempFile( + JSON.stringify({ + suites: [ + { + title: 'test.spec.ts', + specs: [ + { + title: 'Failed test with output', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'failed', + errors: [{ message: 'Test failed' }], + stdout: [{ text: 'stdout from failed test' }], + stderr: [{ text: 'stderr from failed test' }], + retry: 0, + attachments: [], + }, + ], + status: 'unexpected', + }, + ], + }, + ], + suites: [], + }, + ], + }), + 'json' + ) + + const testcases = await parsePlaywrightJson(tempJsonFile, '', { skipStdout: 'on-success', skipStderr: 'on-success', }) @@ -663,40 +698,43 @@ describe('Playwright JSON parsing', () => { }) test('Should skip both stdout and stderr for passed tests when both skip options are set to "on-success"', async () => { - const jsonContent = JSON.stringify({ - suites: [ - { - title: 'test.spec.ts', - specs: [ - { - title: 'Passed test with output', - tags: [], - tests: [ - { - annotations: [], - expectedStatus: 'passed', - projectName: 'chromium', - results: [ - { - status: 'passed', - errors: [], - stdout: [{ text: 'stdout content' }], - stderr: [{ text: 'stderr content' }], - retry: 0, - attachments: [], - }, - ], - status: 'expected', - }, - ], - }, - ], - suites: [], - }, - ], - }) - - const testcases = await parsePlaywrightJson(jsonContent, '', { + tempJsonFile = createTempFile( + JSON.stringify({ + suites: [ + { + title: 'test.spec.ts', + specs: [ + { + title: 'Passed test with output', + tags: [], + tests: [ + { + annotations: [], + expectedStatus: 'passed', + projectName: 'chromium', + results: [ + { + status: 'passed', + errors: [], + stdout: [{ text: 'stdout content' }], + stderr: [{ text: 'stderr content' }], + retry: 0, + attachments: [], + }, + ], + status: 'expected', + }, + ], + }, + ], + suites: [], + }, + ], + }), + 'json' + ) + + const testcases = await parsePlaywrightJson(tempJsonFile, '', { skipStdout: 'on-success', skipStderr: 'on-success', }) diff --git a/src/tests/result-upload.spec.ts b/src/tests/result-upload.spec.ts index fa2e1fb..720166b 100644 --- a/src/tests/result-upload.spec.ts +++ b/src/tests/result-upload.spec.ts @@ -1,10 +1,19 @@ -import { afterAll, beforeAll, expect, test, describe, afterEach } from 'vitest' -import { run } from '../commands/main' -import { setupServer } from 'msw/node' import { HttpResponse, http } from 'msw' +import { setupServer } from 'msw/node' +import { unlinkSync, readdirSync } from 'node:fs' +import { afterAll, beforeAll, beforeEach, expect, test, describe, afterEach } from 'vitest' +import { run } from '../commands/main' +import { + CreateTCasesRequest, + CreateTCasesResponse, + Folder, + PaginatedResponse, + TCase, +} from '../api/schemas' +import { DEFAULT_FOLDER_TITLE } from '../utils/result-upload/ResultUploadCommandHandler' +import { setMaxResultsInRequest } from '../utils/result-upload/ResultUploader' import { runTestCases } from './fixtures/testcases' import { countMockedApiCalls } from './utils' -import { setMaxResultsInRequest } from '../utils/result-upload/ResultUploader' const projectCode = 'TEST' const runId = '1' @@ -15,14 +24,29 @@ const runURL = `${baseURL}/project/${projectCode}/run/${runId}` process.env['QAS_TOKEN'] = 'QAS_TOKEN' process.env['QAS_URL'] = baseURL -let lastCreatedRunTitle = '' -let createRunTitleConflict = false +let lastCreatedRunTitle = '' // Stores title in the request, for the last create run API call +let createRunTitleConflict = false // If true, the create run API returns a title conflict error +let createTCasesResponse: CreateTCasesResponse | null = null // Stores mock response for the create tcases API call +let overriddenGetPaginatedTCasesResponse: PaginatedResponse | null = null // Stores overridden (non-default) response for the get tcases API call +let overriddenGetFoldersResponse: PaginatedResponse | null = null // Stores overridden (non-default) response for the get folders API call const server = setupServer( http.get(`${baseURL}/api/public/v0/project/${projectCode}`, ({ request }) => { expect(request.headers.get('Authorization')).toEqual('ApiKey QAS_TOKEN') return HttpResponse.json({ exists: true }) }), + http.get(`${baseURL}/api/public/v0/project/${projectCode}/tcase/folders`, ({ request }) => { + expect(request.headers.get('Authorization')).toEqual('ApiKey QAS_TOKEN') + return HttpResponse.json( + overriddenGetFoldersResponse || { data: [], total: 0, page: 1, limit: 50 } + ) + }), + http.get(`${baseURL}/api/public/v0/project/${projectCode}/tcase`, ({ request }) => { + expect(request.headers.get('Authorization')).toEqual('ApiKey QAS_TOKEN') + return HttpResponse.json( + overriddenGetPaginatedTCasesResponse || { data: [], total: 0, page: 1, limit: 50 } + ) + }), http.post(`${baseURL}/api/public/v0/project/${projectCode}/tcase/seq`, ({ request }) => { expect(request.headers.get('Authorization')).toEqual('ApiKey QAS_TOKEN') return HttpResponse.json({ @@ -30,6 +54,33 @@ const server = setupServer( total: runTestCases.length, }) }), + http.post(`${baseURL}/api/public/v0/project/${projectCode}/tcase/bulk`, async ({ request }) => { + expect(request.headers.get('Authorization')).toEqual('ApiKey QAS_TOKEN') + + if (!createTCasesResponse) { + return HttpResponse.json( + { + message: 'No mock response set for create tcases API call', + }, + { + status: 500, + } + ) + } + + const body = (await request.json()) as CreateTCasesRequest + if (body.tcases.length !== createTCasesResponse.tcases.length) { + return HttpResponse.json( + { + message: `${body.tcases.length} test cases in request does not match ${createTCasesResponse.tcases.length} in the mock response`, + }, + { + status: 400, + } + ) + } + return HttpResponse.json(createTCasesResponse) + }), http.post(`${baseURL}/api/public/v0/project/${projectCode}/run`, async ({ request }) => { expect(request.headers.get('Authorization')).toEqual('ApiKey QAS_TOKEN') const body = (await request.json()) as { title: string } @@ -91,6 +142,22 @@ const countFileUploadApiCalls = () => countMockedApiCalls(server, (req) => req.url.endsWith('/file')) const countResultUploadApiCalls = () => countMockedApiCalls(server, (req) => new URL(req.url).pathname.endsWith('/result/batch')) +const countCreateTCasesApiCalls = () => + countMockedApiCalls(server, (req) => new URL(req.url).pathname.endsWith('/tcase/bulk')) + +const getMappingFiles = () => + new Set( + readdirSync('.').filter((f) => f.startsWith('qasphere-automapping-') && f.endsWith('.txt')) + ) + +const cleanupGeneratedMappingFiles = (existingMappingFiles?: Set) => { + const currentFiles = getMappingFiles() + currentFiles.forEach((f) => { + if (!existingMappingFiles?.has(f)) { + unlinkSync(f) + } + }) +} const fileTypes = [ { @@ -160,7 +227,7 @@ fileTypes.forEach((fileType) => { }) }) - describe('Uploading test results', () => { + describe('Uploading test results with run URL', () => { test('Test cases on reports with all matching test cases on QAS should be successful', async () => { const numFileUploadCalls = countFileUploadApiCalls() const numResultUploadCalls = countResultUploadApiCalls() @@ -351,5 +418,103 @@ fileTypes.forEach((fileType) => { ) }) }) + + describe('Uploading test results with test case creation', () => { + let existingMappingFiles: Set | undefined = undefined + + beforeEach(() => { + existingMappingFiles = getMappingFiles() + }) + + afterEach(() => { + cleanupGeneratedMappingFiles(existingMappingFiles) + existingMappingFiles = undefined + createTCasesResponse = null + overriddenGetPaginatedTCasesResponse = null + overriddenGetFoldersResponse = null + }) + + test('Should create new test cases for results without valid markers', async () => { + const numCreateTCasesCalls = countCreateTCasesApiCalls() + const numResultUploadCalls = countResultUploadApiCalls() + + setMaxResultsInRequest(1) + createTCasesResponse = { + tcases: [ + { id: '6', seq: 6 }, + { id: '7', seq: 7 }, + ], + } + + await run( + `${fileType.command} --project-code ${projectCode} --create-tcases ${fileType.dataBasePath}/without-markers.${fileType.fileExtension}` + ) + expect(numCreateTCasesCalls()).toBe(1) + expect(numResultUploadCalls()).toBe(3) // 3 results total + }) + + test('Should not create new test case if one with same title already exists', async () => { + const numCreateTCasesCalls = countCreateTCasesApiCalls() + const numResultUploadCalls = countResultUploadApiCalls() + + setMaxResultsInRequest(1) + overriddenGetFoldersResponse = { + data: [{ id: 1, title: DEFAULT_FOLDER_TITLE, parentId: 0, pos: 0 }], + total: 1, + page: 1, + limit: 50, + } + overriddenGetPaginatedTCasesResponse = { + data: [ + { + id: '6', + seq: 6, + title: 'The cart is still filled after refreshing the page', + version: 1, + projectId: 'projectid', + folderId: 1, + }, + ], + total: 1, + page: 1, + limit: 50, + } + createTCasesResponse = { + tcases: [{ id: '7', seq: 7 }], + } + + await run( + `${fileType.command} --project-code ${projectCode} --create-tcases ${fileType.dataBasePath}/without-markers.${fileType.fileExtension}` + ) + expect(numCreateTCasesCalls()).toBe(1) + expect(numResultUploadCalls()).toBe(3) // 3 results total + }) + + test('Should not create new test cases if all results have valid markers', async () => { + const numCreateTCasesCalls = countCreateTCasesApiCalls() + const numResultUploadCalls = countResultUploadApiCalls() + + setMaxResultsInRequest(1) + await run( + `${fileType.command} --project-code ${projectCode} --create-tcases ${fileType.dataBasePath}/matching-tcases.${fileType.fileExtension}` + ) + expect(numCreateTCasesCalls()).toBe(0) + expect(numResultUploadCalls()).toBe(5) // 5 results total + }) + }) + }) +}) + +describe('Uploading XCode reports', () => { + const xcresultBasePath = './src/tests/fixtures/xcresult' + + test('Should successfully upload xcresult bundle with matching test cases', async () => { + const numFileUploadCalls = countFileUploadApiCalls() + const numResultUploadCalls = countResultUploadApiCalls() + + await run(`xcresult-upload -r ${runURL} ${xcresultBasePath}/Variety.xcresult`) + + expect(numFileUploadCalls()).toBe(0) + expect(numResultUploadCalls()).toBe(1) // 5 results total }) }) diff --git a/src/tests/utils.ts b/src/tests/utils.ts index caacea1..b557893 100644 --- a/src/tests/utils.ts +++ b/src/tests/utils.ts @@ -1,4 +1,8 @@ import { SetupServerApi } from 'msw/node' +import { randomBytes } from 'node:crypto' +import { unlinkSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' export const countMockedApiCalls = ( server: SetupServerApi, @@ -21,3 +25,24 @@ export const countMockedApiCalls = ( }) return () => count } + +/** + * Creates a temp file with the provided content in the OS temp directory and returns its path. + * @param content Content to be written to the temp file + * @param extension Extension of the file + * @returns string Path of the created temp file + */ +export function createTempFile(content: string, extension: string) { + const randomName = `tmp-${randomBytes(8).toString('hex')}.${extension}` + const tmpPath = join(tmpdir(), randomName) + writeFileSync(tmpPath, content, { encoding: 'utf-8' }) + return tmpPath +} + +/** + * Deletes the file at the given path. + * @param filePath Path to the file to delete + */ +export function deleteTempFile(filePath: string) { + unlinkSync(filePath) +} diff --git a/src/tests/xcresult-parsing.spec.ts b/src/tests/xcresult-parsing.spec.ts new file mode 100644 index 0000000..799c87b --- /dev/null +++ b/src/tests/xcresult-parsing.spec.ts @@ -0,0 +1,65 @@ +import { expect, test, describe } from 'vitest' +import { parseXCResult } from '../utils/result-upload/xcresultSqliteParser' + +const xcresultBasePath = './src/tests/fixtures/xcresult' + +describe('XCResult parsing', () => { + test('Should correctly parse all test cases from xcresult bundle', async () => { + const testcases = await parseXCResult( + `${xcresultBasePath}/Variety.xcresult`, + xcresultBasePath, + { + skipStdout: 'never', + skipStderr: 'never', + } + ) + + // Verify total count + expect(testcases).toHaveLength(5) + + // Verify each test case has required properties + testcases.forEach((tc) => { + expect(tc).toHaveProperty('name') + expect(tc).toHaveProperty('folder') + expect(tc).toHaveProperty('status') + expect(tc).toHaveProperty('message') + expect(tc).toHaveProperty('attachments') + expect(Array.isArray(tc.attachments)).toBe(true) + }) + + // Test case 1: Passed test (TEST_002) + const test1 = testcases.find((tc) => tc.name === 'test_TEST_002_AppLaunches') + expect(test1).toBeDefined() + expect(test1?.status).toBe('passed') + expect(test1?.folder).toContain('BistroAppUITests') + + // Test case 2: Failed test with failure message (TEST_003) + const test2 = testcases.find((tc) => tc.name === 'test_TEST_003_MenuShowsPizzas') + expect(test2).toBeDefined() + expect(test2?.status).toBe('failed') + expect(test2?.folder).toContain('BistroAppUITests') + expect(test2?.message).toContain('XCTAssertTrue failed') + + // Test case 3: Skipped test with skip reason (TEST_004) + const test3 = testcases.find((tc) => tc.name === 'test_TEST_004_NavigateToCart') + expect(test3).toBeDefined() + expect(test3?.status).toBe('skipped') + expect(test3?.folder).toContain('BistroAppUITests') + expect(test3?.message).toContain('Skipped Reason') + expect(test3?.message).toContain('Test not ready yet') + + // Test case 4: Another passed test (TEST_005) + const test4 = testcases.find((tc) => tc.name === 'test_TEST_005_SwitchBetweenTabs') + expect(test4).toBeDefined() + expect(test4?.status).toBe('passed') + expect(test4?.folder).toContain('BistroAppUITests') + + // Test case 5: Expected failure (blocked) with reason (TEST_006) + const test5 = testcases.find((tc) => tc.name === 'test_TEST_006_AddItemAndCheckout') + expect(test5).toBeDefined() + expect(test5?.status).toBe('blocked') + expect(test5?.folder).toContain('BistroAppUITests') + expect(test5?.message).toContain('Expected Failure') + expect(test5?.message).toContain('should fail') + }) +}) diff --git a/src/utils/misc.ts b/src/utils/misc.ts index dd70186..2f411ad 100644 --- a/src/utils/misc.ts +++ b/src/utils/misc.ts @@ -96,6 +96,10 @@ export const parseTCaseUrl = (url: string) => { } } +export const getTCaseMarker = (projectCode: string, seq: number) => { + return `${projectCode}-${seq.toString().padStart(3, '0')}` +} + export const printErrorThenExit = (e: unknown): never => { printError(e) process.exit(1) diff --git a/src/utils/result-upload/ResultUploadCommandHandler.ts b/src/utils/result-upload/ResultUploadCommandHandler.ts index 43aa3ef..c6da10a 100644 --- a/src/utils/result-upload/ResultUploadCommandHandler.ts +++ b/src/utils/result-upload/ResultUploadCommandHandler.ts @@ -1,16 +1,23 @@ import { Arguments } from 'yargs' import chalk from 'chalk' -import { readFileSync } from 'node:fs' +import { writeFileSync } from 'node:fs' import { dirname } from 'node:path' -import { parseRunUrl, printErrorThenExit, processTemplate } from '../misc' +import { + getTCaseMarker, + parseRunUrl, + printError, + printErrorThenExit, + processTemplate, +} from '../misc' import { Api, createApi } from '../../api' -import { PaginatedResponse, TCaseBySeq } from '../../api/tcases' +import { TCase } from '../../api/schemas' import { TestCaseResult } from './types' import { ResultUploader } from './ResultUploader' -import { parseJUnitXml } from './junitXmlParser' -import { parsePlaywrightJson } from './playwrightJsonParser' +import { parseJUnitXml, printJUnitMissingMarkerGuidance } from './junitXmlParser' +import { parsePlaywrightJson, printPlaywrightMissingMarkerGuidance } from './playwrightJsonParser' +import { parseXCResult, printXCResultMissingMarkerGuidance } from './xcresultSqliteParser' -export type UploadCommandType = 'junit-upload' | 'playwright-json-upload' +export type UploadCommandType = 'junit-upload' | 'playwright-json-upload' | 'xcresult-upload' export type SkipOutputOption = 'on-success' | 'never' @@ -20,31 +27,59 @@ export interface ParserOptions { } export type Parser = ( - data: string, + filePath: string, attachmentBaseDirectory: string, options: ParserOptions ) => Promise -export interface ResultUploadCommandArgs { +export type ResultUploadCommandArgs = { type: UploadCommandType - runUrl?: string - runName?: string files: string[] force: boolean attachments: boolean ignoreUnmatched: boolean skipReportStdout: SkipOutputOption skipReportStderr: SkipOutputOption -} +} & ( + | { + runUrl: string + } + | { + projectCode?: string + runName?: string + createTcases: boolean + } +) interface FileResults { file: string results: TestCaseResult[] } +interface TestCaseResultWithSeqAndFile { + seq: number | null + file: string + result: TestCaseResult +} + +const DEFAULT_PAGE_SIZE = 5000 +export const DEFAULT_FOLDER_TITLE = 'cli-import' +const DEFAULT_TCASE_TAGS = ['cli-import'] +const DEFAULT_MAPPING_FILENAME_TEMPLATE = 'qasphere-automapping-{YYYY}{MM}{DD}-{HH}{mm}{ss}.txt' + const commandTypeParsers: Record = { 'junit-upload': parseJUnitXml, 'playwright-json-upload': parsePlaywrightJson, + 'xcresult-upload': parseXCResult, +} + +export const commandTypePrintMissingMarkerGuidance: Record< + UploadCommandType, + (projectCode: string, testCaseName: string) => void +> = { + 'junit-upload': printJUnitMissingMarkerGuidance, + 'playwright-json-upload': printPlaywrightMissingMarkerGuidance, + 'xcresult-upload': printXCResultMissingMarkerGuidance, } export class ResultUploadCommandHandler { @@ -63,12 +98,11 @@ export class ResultUploadCommandHandler { return printErrorThenExit('No files specified') } - const fileResults = await this.parseFiles() - const results = fileResults.flatMap((fileResult) => fileResult.results) - + let fileResults = await this.parseFiles() let projectCode = '' let runId = 0 - if (this.args.runUrl) { + + if ('runUrl' in this.args && this.args.runUrl) { // Handle existing run URL console.log(chalk.blue(`Using existing test run: ${this.args.runUrl}`)) @@ -82,22 +116,25 @@ export class ResultUploadCommandHandler { runId = urlParsed.run projectCode = urlParsed.project } else { - // Auto-detect project from results - projectCode = this.detectProjectCode(results) - console.log(chalk.blue(`Detected project code: ${projectCode}`)) + if (this.args.projectCode) { + projectCode = this.args.projectCode as string + } else { + // Try to auto-detect project code from results. This is not fully reliable, but + // is kept for backward compatibility. Better to specify project code explicitly + projectCode = this.detectProjectCodeFromTCaseNames(fileResults) + console.log(chalk.blue(`Detected project code: ${projectCode}`)) + } - // Create a new test run if (!(await this.api.projects.checkProjectExists(projectCode))) { return printErrorThenExit(`Project ${projectCode} does not exist`) } - console.log(chalk.blue(`Creating a new test run for project: ${projectCode}`)) - const tcaseRefs = this.extractTestCaseRefs(projectCode, fileResults) - const tcases = await this.getTestCases(projectCode, tcaseRefs) - runId = await this.createNewRun(projectCode, tcases) - console.log(chalk.blue(`Test run URL: ${this.baseUrl}/project/${projectCode}/run/${runId}`)) + const resp = await this.getTCaseIds(projectCode, fileResults) + fileResults = resp.fileResults + runId = await this.createNewRun(projectCode, resp.tcaseIds) } + const results = fileResults.flatMap((fileResult) => fileResult.results) await this.uploadResults(projectCode, runId, results) } @@ -110,34 +147,40 @@ export class ResultUploadCommandHandler { } for (const file of this.args.files) { - const fileData = readFileSync(file).toString() - const fileResults = await commandTypeParsers[this.type](fileData, dirname(file), parserOptions) + const fileResults = await commandTypeParsers[this.type](file, dirname(file), parserOptions) results.push({ file, results: fileResults }) } return results } - protected detectProjectCode(results: TestCaseResult[]) { - for (const result of results) { - if (result.name) { - // Look for pattern like PRJ-123 or TEST-456 - const match = result.name.match(/([A-Za-z0-9]{1,5})-\d{3,}/) - if (match) { - return match[1] + protected detectProjectCodeFromTCaseNames(fileResults: FileResults[]) { + // Look for pattern like PRJ-123 or TEST-456 (_ is also allowed as separator) + const tcaseSeqRegex = new RegExp(/([A-Za-z0-9]{1,5})[-_]\d{3,}/) + for (const { results } of fileResults) { + for (const result of results) { + if (result.name) { + const match = tcaseSeqRegex.exec(result.name) + if (match) { + return match[1] + } } } } return printErrorThenExit( - 'Could not detect project code from test case names. Please make sure they contain a valid project code (e.g., PRJ-123)' + 'Could not detect project code from test case names. Please specify project code using --project-code flag' ) } - protected extractTestCaseRefs(projectCode: string, fileResults: FileResults[]): Set { - const tcaseRefs = new Set() + protected async getTCaseIds(projectCode: string, fileResults: FileResults[]) { const shouldFailOnInvalid = !this.args.force && !this.args.ignoreUnmatched + const tcaseSeqRegex = new RegExp(`${projectCode}[-_](\\d{3,})`) + const seqIdsSet: Set = new Set() + const resultsWithSeqAndFile: TestCaseResultWithSeqAndFile[] = [] + + // First extract the sequence numbers from the test case names for (const { file, results } of fileResults) { for (const result of results) { if (!result.name) { @@ -147,59 +190,228 @@ export class ResultUploadCommandHandler { continue } - const match = new RegExp(`${projectCode}-(\\d{3,})`).exec(result.name) + const match = tcaseSeqRegex.exec(result.name) + resultsWithSeqAndFile.push({ + seq: match ? Number(match[1]) : null, + file, + result, + }) + if (match) { - tcaseRefs.add(`${projectCode}-${match[1]}`) - continue + seqIdsSet.add(Number(match[1])) } + } + } + + // Now fetch the test cases by their sequence numbers + const apiTCasesMap: Record = {} + if (seqIdsSet.size > 0) { + const tcaseMarkers = Array.from(seqIdsSet).map((v) => getTCaseMarker(projectCode, v)) + + for (let page = 1; ; page++) { + const response = await this.api.testcases.getTCasesBySeq(projectCode, { + seqIds: tcaseMarkers, + page, + limit: DEFAULT_PAGE_SIZE, + }) - if (shouldFailOnInvalid) { - return printErrorThenExit( - `Test case name "${result.name}" in ${file} does not contain valid sequence number with project code (e.g., ${projectCode}-123)` - ) + for (const tcase of response.data) { + apiTCasesMap[tcase.seq] = tcase + } + + if (response.data.length < DEFAULT_PAGE_SIZE) { + break } } } - if (tcaseRefs.size === 0) { - return printErrorThenExit('No valid test case references found in any of the files') + // Now validate that the test cases with found sequence numbers actually exist + const tcaseIds: string[] = [] + const tcasesToCreateMap: Record = {} + for (const { seq, file, result } of resultsWithSeqAndFile) { + if (seq && apiTCasesMap[seq]) { + tcaseIds.push(apiTCasesMap[seq].id) + continue + } + + if (this.args.createTcases) { + const tcaseResults = tcasesToCreateMap[result.name] || [] + tcaseResults.push(result) + tcasesToCreateMap[result.name] = tcaseResults + continue + } + + if (shouldFailOnInvalid) { + printError( + `Test case name "${result.name}" in ${file} does not contain valid test case marker` + ) + commandTypePrintMissingMarkerGuidance[this.type](projectCode, result.name) + console.error( + chalk.yellow('Also ensure that the test cases exist in the QA Sphere project.') + ) + return process.exit(1) + } } - return tcaseRefs + // Create new test cases, if same is requested + if (Object.keys(tcasesToCreateMap).length > 0) { + const keys = Object.keys(tcasesToCreateMap) + const newTCases = await this.createNewTCases(projectCode, keys) + + for (let i = 0; i < keys.length; i++) { + const marker = getTCaseMarker(projectCode, newTCases[i].seq) + for (const result of tcasesToCreateMap[keys[i]] || []) { + // Prefix the test case markers for use in ResultUploader. The fileResults array + // containing the updated name is returned to the caller + result.name = `${marker}: ${result.name}` + } + tcaseIds.push(newTCases[i].id) + } + } + + if (tcaseIds.length === 0) { + return printErrorThenExit('No valid test cases found in any of the files') + } + + return { tcaseIds, fileResults } } - private async getTestCases(projectCode: string, tcaseRefs: Set) { - const response = await this.api.testcases.getTCasesBySeq(projectCode, { - seqIds: Array.from(tcaseRefs), - page: 1, - limit: tcaseRefs.size, + private async createNewTCases(projectCode: string, tcasesToCreate: string[]) { + console.log(chalk.blue(`Creating test cases for results with no test case markers`)) + + // First fetch the default folder ID where we are creating new test cases + let defaultFolderId = null + for (let page = 1; ; page++) { + const response = await this.api.folders.getFoldersPaginated(projectCode, { + page, + limit: DEFAULT_PAGE_SIZE, + }) + + for (const folder of response.data) { + if (folder.title === DEFAULT_FOLDER_TITLE && !folder.parentId) { + defaultFolderId = folder.id + break + } + } + + if (defaultFolderId || response.data.length < DEFAULT_PAGE_SIZE) { + break + } + } + + // If the default folder exists, fetch the test cases in it + const apiTCasesMap: Record = {} + if (defaultFolderId) { + for (let page = 1; ; page++) { + const response = await this.api.testcases.getTCasesPaginated(projectCode, { + folders: [defaultFolderId], + page, + limit: DEFAULT_PAGE_SIZE, + }) + + for (const tcase of response.data) { + apiTCasesMap[tcase.title] = tcase + } + + if (response.data.length < DEFAULT_PAGE_SIZE) { + break + } + } + } + + // Reuse existing test cases with the same title from the default folder + const ret: { id: string; seq: number }[] = [] + const idxToFill: number[] = [] + const finalTCasesToCreate: string[] = [] + for (let i = 0; i < tcasesToCreate.length; i++) { + const existingTcase = apiTCasesMap[tcasesToCreate[i]] + if (existingTcase) { + // TCase with this title already exists, reuse it + ret.push({ id: existingTcase.id, seq: existingTcase.seq }) + continue + } + + // Add a placeholder for the new test case. Will be updated later + ret.push({ id: '', seq: 0 }) + finalTCasesToCreate.push(tcasesToCreate[i]) + idxToFill.push(i) + } + + if (!finalTCasesToCreate.length) { + console.log( + chalk.blue( + `Reusing ${ret.length} test cases with same title from "${DEFAULT_FOLDER_TITLE}" folder, no new test cases created` + ) + ) + return ret + } + + // Create new test cases and update the placeholders with the actual test case IDs + const { tcases } = await this.api.testcases.createTCases(projectCode, { + folderPath: [DEFAULT_FOLDER_TITLE], + tcases: finalTCasesToCreate.map((title) => ({ title, tags: DEFAULT_TCASE_TAGS })), }) - if (response.total === 0 || response.data.length === 0) { - return printErrorThenExit('No matching test cases found in the project') + console.log( + chalk.green( + `Created ${tcases.length} new test cases in folder "${DEFAULT_FOLDER_TITLE}"${ + ret.length > tcases.length + ? ` and reused ${ret.length - tcases.length} test cases with same title` + : '' + }` + ) + ) + + for (let i = 0; i < idxToFill.length; i++) { + ret[idxToFill[i]] = tcases[i] + } + + try { + const mappingFilename = processTemplate(DEFAULT_MAPPING_FILENAME_TEMPLATE) + const mappingLines = tcases + .map((t, i) => `${getTCaseMarker(projectCode, t.seq)}: ${tcasesToCreate[i]}`) + .join('\n') + + writeFileSync(mappingFilename, mappingLines) + console.log( + chalk.yellow( + `Created mapping file for newly created test cases: ${mappingFilename}\nUpdate your test cases to include the test case markers in the name, for future uploads` + ) + ) + } catch (err) { + console.log( + chalk.yellow( + `Warning: Failed to write test case mapping file: ${ + err instanceof Error ? err.message : String(err) + }` + ) + ) } - return response + return ret } - private async createNewRun(projectCode: string, tcases: PaginatedResponse) { + private async createNewRun(projectCode: string, tcaseIds: string[]) { const title = processTemplate( - this.args.runName ?? 'Automated test run - {MMM} {DD}, {YYYY}, {hh}:{mm}:{ss} {AMPM}' + 'runName' in this.args && this.args.runName + ? (this.args.runName as string) + : 'Automated test run - {MMM} {DD}, {YYYY}, {hh}:{mm}:{ss} {AMPM}' ) + console.log(chalk.blue(`Creating a new test run for project: ${projectCode}`)) + try { const response = await this.api.runs.createRun(projectCode, { title, description: 'Test run created through automation pipeline', type: 'static_struct', - queryPlans: [ - { - tcaseIds: tcases.data.map((t: TCaseBySeq) => t.id), - }, - ], + queryPlans: [{ tcaseIds }], }) console.log(chalk.green(`Created new test run "${title}" with ID: ${response.id}`)) + console.log( + chalk.blue(`Test run URL: ${this.baseUrl}/project/${projectCode}/run/${response.id}`) + ) return response.id } catch (error) { // Check if the error is about conflicting run ID diff --git a/src/utils/result-upload/ResultUploader.ts b/src/utils/result-upload/ResultUploader.ts index f6703f0..77839c3 100644 --- a/src/utils/result-upload/ResultUploader.ts +++ b/src/utils/result-upload/ResultUploader.ts @@ -1,10 +1,14 @@ import { Arguments } from 'yargs' import chalk from 'chalk' import { RunTCase } from '../../api/schemas' -import { parseRunUrl, printError, printErrorThenExit, twirlLoader } from '../misc' +import { getTCaseMarker, parseRunUrl, printError, printErrorThenExit, twirlLoader } from '../misc' import { Api, createApi } from '../../api' import { Attachment, TestCaseResult } from './types' -import { ResultUploadCommandArgs, UploadCommandType } from './ResultUploadCommandHandler' +import { + commandTypePrintMissingMarkerGuidance, + ResultUploadCommandArgs, + UploadCommandType, +} from './ResultUploadCommandHandler' const MAX_CONCURRENT_FILE_UPLOADS = 10 let MAX_RESULTS_IN_REQUEST = 50 // Only updated from tests, otherwise it's a constant @@ -74,51 +78,17 @@ export class ResultUploader { } private printMissingTestCaseGuidance(missing: TestCaseResult[]) { - if (this.type === 'junit-upload') { - this.printJUnitGuidance() - } else if (this.type === 'playwright-json-upload') { - this.printPlaywrightGuidance(missing[0]?.name || 'your test name') - } - console.error(chalk.yellow('Also ensure that the test cases exist in the QA Sphere project and the test run (if run URL is provided).')) - } - - private printJUnitGuidance() { - console.error(` -${chalk.yellow('To fix this issue, include the test case marker in your test names:')} - - Format: ${chalk.green(`${this.project}-: Your test name`)} - Example: ${chalk.green(`${this.project}-002: Login with valid credentials`)} - ${chalk.green(`Login with invalid credentials: ${this.project}-1312`)} - - ${chalk.dim('Where is the test case number (minimum 3 digits, zero-padded if needed)')} -`) - } - - private printPlaywrightGuidance(exampleTestName: string) { - console.error(` -${chalk.yellow('To fix this issue, choose one of the following options:')} - - ${chalk.bold('Option 1: Use Test Annotations (Recommended)')} - Add a test annotation to your Playwright test: + commandTypePrintMissingMarkerGuidance[this.type](this.project, missing[0]?.name) - ${chalk.green(`test('${exampleTestName}', { - annotation: { - type: 'test case', - description: 'https://your-qas-instance.com/project/${this.project}/tcase/123' - } - }, async ({ page }) => { - // your test code - });`)} - - ${chalk.dim('Note: The "type" field is case-insensitive')} - - ${chalk.bold('Option 2: Include Test Case Marker in Name')} - Rename your test to include the marker ${chalk.green(`${this.project}-`)}: - - Format: ${chalk.green(`${this.project}-: Your test name`)} - Example: ${chalk.green(`${this.project}-1024: Login with valid credentials`)} - ${chalk.dim('Where is the test case number (minimum 3 digits, zero-padded if needed)')} -`) + if (!this.args.createTcases) { + console.error( + chalk.yellow( + `Also ensure that the test cases exist in the QA Sphere project${ + this.args.runUrl ? ' and the provided test run' : '' + }.` + ) + ) + } } private validateAndPrintMissingAttachments = (results: TCaseWithResult[]) => { @@ -181,7 +151,7 @@ ${chalk.yellow('To fix this issue, choose one of the following options:')} const uploadedAttachments = await this.processConcurrently( allAttachments, async ({ attachment, tcaseIndex }) => { - const { url } = await this.api.file.uploadFile( + const { url } = await this.api.files.uploadFile( new Blob([attachment.buffer! as BlobPart]), attachment.filename ) @@ -283,8 +253,10 @@ ${chalk.yellow('To fix this issue, choose one of the following options:')} testcaseResults.forEach((result) => { if (result.name) { const tcase = testcases.find((tcase) => { - const tcaseCode = `${this.project}-${tcase.seq.toString().padStart(3, '0')}` - return result.name.includes(tcaseCode) + const tcaseMarker = getTCaseMarker(this.project, tcase.seq) + return ( + result.name.includes(tcaseMarker) || result.name.includes(tcaseMarker.replace('-', '_')) + ) }) if (tcase) { diff --git a/src/utils/result-upload/junitXmlParser.ts b/src/utils/result-upload/junitXmlParser.ts index 45d1bc4..cfbae39 100644 --- a/src/utils/result-upload/junitXmlParser.ts +++ b/src/utils/result-upload/junitXmlParser.ts @@ -1,4 +1,6 @@ +import chalk from 'chalk' import escapeHtml from 'escape-html' +import { readFileSync } from 'node:fs' import xml from 'xml2js' import z from 'zod' import { Attachment, TestCaseResult } from './types' @@ -73,11 +75,27 @@ const junitXmlSchema = z.object({ }), }) +export const printJUnitMissingMarkerGuidance = ( + projectCode: string, + testCaseName = 'your test name' +) => { + console.error(` +${chalk.yellow('To fix this issue, include the test case marker in your test names:')} + + Format: ${chalk.green(`${projectCode}-`)}, ${chalk.dim( + 'where is the test case number (minimum 3 digits, zero-padded if needed)' + )} + Example: ${chalk.green(`${projectCode}-002: ${testCaseName}`)} + ${chalk.green(`${testCaseName}: ${projectCode}-1312`)} +`) +} + export const parseJUnitXml: Parser = async ( - xmlString: string, + filePath: string, attachmentBaseDirectory: string, options: ParserOptions ): Promise => { + const xmlString = readFileSync(filePath).toString() const xmlData = await xml.parseStringPromise(xmlString, { explicitCharkey: true, includeWhiteChars: true, diff --git a/src/utils/result-upload/playwrightJsonParser.ts b/src/utils/result-upload/playwrightJsonParser.ts index c322dfc..a1dd4bf 100644 --- a/src/utils/result-upload/playwrightJsonParser.ts +++ b/src/utils/result-upload/playwrightJsonParser.ts @@ -1,10 +1,12 @@ -import z from 'zod' +import chalk from 'chalk' import escapeHtml from 'escape-html' +import { readFileSync } from 'node:fs' import stripAnsi from 'strip-ansi' +import z from 'zod' import { Attachment, TestCaseResult } from './types' import { Parser, ParserOptions } from './ResultUploadCommandHandler' import { ResultStatus } from '../../api/schemas' -import { parseTCaseUrl } from '../misc' +import { getTCaseMarker, parseTCaseUrl } from '../misc' import { getAttachments } from './utils' // Schema definition as per https://github.com/microsoft/playwright/blob/main/packages/playwright/types/testReporter.d.ts @@ -78,11 +80,44 @@ const playwrightJsonSchema = z.object({ suites: suiteSchema.array(), }) +export const printPlaywrightMissingMarkerGuidance = ( + projectCode: string, + exampleTestName = 'your test name' +) => { + console.error(` +${chalk.yellow('To fix this issue, choose one of the following options:')} + + ${chalk.bold('Option 1: Use Test Annotations (Recommended)')} + Add a "test case" annotation to your Playwright test with the QA Sphere test case URL: + + ${chalk.green(`test('${exampleTestName}', { + annotation: { + type: 'test case', + description: 'https://your-qas-instance.com/project/${projectCode}/tcase/123' + } + }, async ({ page }) => { + // your test code + });`)} + + ${chalk.dim('Note: The "type" field is case-insensitive')} + + ${chalk.bold('Option 2: Include Test Case Marker in Name')} + Rename your test to include the test case marker: + + Format: ${chalk.green(`${projectCode}-`)}, ${chalk.dim( + 'where is the test case number (minimum 3 digits, zero-padded if needed)' + )} + Example: ${chalk.green(`${projectCode}-002: ${exampleTestName}`)} + ${chalk.green(`${exampleTestName}: ${projectCode}-1312`)} +`) +} + export const parsePlaywrightJson: Parser = async ( - jsonString: string, + filePath: string, attachmentBaseDirectory: string, options: ParserOptions ): Promise => { + const jsonString = readFileSync(filePath).toString() const jsonData = JSON.parse(jsonString) const validated = playwrightJsonSchema.parse(jsonData) const testcases: TestCaseResult[] = [] @@ -158,7 +193,7 @@ const getTCaseMarkerFromAnnotations = (annotations: Annotation[]) => { if (annotation.type.toLowerCase().includes('test case') && annotation.description) { const res = parseTCaseUrl(annotation.description) if (res) { - return `${res.project}-${res.tcaseSeq.toString().padStart(3, '0')}` + return getTCaseMarker(res.project, res.tcaseSeq) } } } @@ -183,7 +218,7 @@ const buildMessage = (result: Result, status: ResultStatus, options: ParserOptio let message = '' if (result.retry) { - message += `

Test passed in ${result.retry + 1} attempts

` + message += `

Test passed in ${result.retry + 1} attempts

` } if (result.errors.length > 0) { diff --git a/src/utils/result-upload/xcresultSqliteParser.ts b/src/utils/result-upload/xcresultSqliteParser.ts new file mode 100644 index 0000000..a171ede --- /dev/null +++ b/src/utils/result-upload/xcresultSqliteParser.ts @@ -0,0 +1,459 @@ +import Database from 'better-sqlite3' +import chalk from 'chalk' +import escapeHtml from 'escape-html' +import { decompress } from 'fzstd' +import { execSync } from 'node:child_process' +import { existsSync, readFileSync } from 'node:fs' +import path from 'node:path' +import { ResultStatus } from '../../api/schemas' +import { Attachment, TestCaseResult } from './types' +import { Parser } from './ResultUploadCommandHandler' + +// Zstandard magic bytes: 0x28 0xB5 0x2F 0xFD +const ZSTD_MAGIC = Buffer.from([0x28, 0xb5, 0x2f, 0xfd]) + +const sqliteFile = 'database.sqlite3' +const dataDir = 'data' // Contains refs and data files +const dataFilePrefix = 'data.' +const ignoredAttachmentsPrefix = 'SynthesizedEvent_' + +interface TestSuiteRow { + rowid: number + name: string | null + parentSuite_fk: number | null +} + +interface TestCaseRow { + rowid: number + testSuite_fk: number | null + name: string | null +} + +interface TestCaseRunRow { + rowid: number + testCase_fk: number | null + result: string | null + skipNotice_fk: number | null +} + +interface AttachmentRow { + rowid: number + filenameOverride: string | null + xcResultKitPayloadRefId: string | null + + // From JOIN with Activities table + testCaseRun_fk: number | null +} + +interface SkipNoticeRow { + rowid: number + message: string | null +} + +interface ExpectedFailureRow { + rowid: number + testCaseRun_fk: number | null + issue_fk: number | null + failureReason: string | null +} + +interface TestIssueRow { + rowid: number + testCaseRun_fk: number | null + compactDescription: string | null + detailedDescription: string | null + sanitizedDescription: string | null + sourceCodeContext_fk: number | null + + // From JOIN with SourceCodeContexts and SourceCodeLocations tables + filePath: string | null + lineNumber: number | null +} + +interface SourceCodeFrameRow { + rowid: number + context_fk: number | null + + // From JOIN with SourceCodeSymbolInfos table + symbolName: string | null + + // From JOIN with SourceCodeLocations table + filePath: string | null + lineNumber: number | null +} + +export const printXCResultMissingMarkerGuidance = ( + projectCode: string, + testCaseName = 'your_test_name' +) => { + console.error(` +${chalk.yellow('To fix this issue, include the test case marker in your test names:')} + + Format: ${chalk.green(`${projectCode}_`)}, ${chalk.dim( + 'where is the test case number (minimum 3 digits, zero-padded if needed)' + )} + Example: ${chalk.green(`${projectCode}_002_${testCaseName}`)} + ${chalk.green(`${testCaseName}_${projectCode}_1312`)} +`) +} + +export const parseXCResult: Parser = async (bundlePath: string): Promise => { + const dbPath = path.join(bundlePath, sqliteFile) + if (!existsSync(dbPath)) { + // Following ensures that the sqlite path exist (is generated on first run) + try { + execSync(`xcrun xcresulttool get test-results summary --path "${bundlePath}"`, { + stdio: 'ignore', + }) + } catch (error) { + const errorMessage = error instanceof Error ? error.message : String(error) + throw new Error(`Failed to get test-results summary for ${bundlePath}: ${errorMessage}`) + } + } + + const db = new Database(dbPath, { readonly: true }) + + try { + const testSuitesIdToPathMap = getTestSuitesIdToPathMap(db) + const testCasesIdToRowMap = getTestCasesIdToRowMap(db) + const skipNoticesIdToMessageMap = getSkipNoticesIdToMessageMap(db) + const testCaseRunIdToExpectedFailuresMap = getTestCaseRunIdToExpectedFailuresMap(db) + const testIssues = getTestIssues(db) + const sourceCodeContextIdToFramesMap = getSourceCodeContextIdToFramesMap(db) + const testCaseRunIdToAttachmentsMap = getTestCaseRunIdToAttachmentsMap(db, bundlePath) + + const testCaseRuns = db + .prepare('SELECT rowid, testCase_fk, result, skipNotice_fk FROM TestCaseRuns') + .all() as TestCaseRunRow[] + + const results: TestCaseResult[] = [] + for (const testCaseRun of testCaseRuns) { + const testCase = testCaseRun.testCase_fk ? testCasesIdToRowMap[testCaseRun.testCase_fk] : null + if (!testCase) { + continue + } + + const folder = testCase.testSuite_fk + ? testSuitesIdToPathMap[testCase.testSuite_fk] ?? null + : null + const status = mapResultStatus(testCaseRun.result) + const message = buildMessage( + testCaseRun.rowid, + status, + testCaseRun.skipNotice_fk + ? skipNoticesIdToMessageMap[testCaseRun.skipNotice_fk] ?? null + : null, + testCaseRunIdToExpectedFailuresMap[testCaseRun.rowid], + testIssues, + sourceCodeContextIdToFramesMap + ) + + results.push({ + name: (testCase.name ?? 'Unknown Test').split('(')[0], // Names include "()" as well + folder: folder ?? 'Unknown Suite', + status, + message, + attachments: testCaseRunIdToAttachmentsMap[testCaseRun.rowid] ?? [], + }) + } + + return results + } finally { + db.close() + } +} + +function getTestSuitesIdToPathMap(db: Database.Database): Record { + const rows = db + .prepare('SELECT rowid, name, parentSuite_fk FROM TestSuites') + .all() as TestSuiteRow[] + + const testSuitesMap: Record = {} + for (const row of rows) { + testSuitesMap[row.rowid] = row + } + + const testSuitesPathMap: Record = {} + + const getTestSuitePath = (testSuite: TestSuiteRow): string => { + if (testSuitesPathMap[testSuite.rowid]) { + return testSuitesPathMap[testSuite.rowid] + } + + const parentSuite = testSuite.parentSuite_fk ? testSuitesMap[testSuite.parentSuite_fk] : null + const parentSuitePath = parentSuite ? getTestSuitePath(parentSuite) : '' + const path = `${parentSuitePath ? `${parentSuitePath} › ` : ''}${testSuite.name ?? ''}` + + // Also store the path in the map + testSuitesPathMap[testSuite.rowid] = path + return path + } + + for (const testSuite of Object.values(testSuitesMap)) { + getTestSuitePath(testSuite) + } + + return testSuitesPathMap +} + +function getTestCasesIdToRowMap(db: Database.Database): Record { + const rows = db.prepare('SELECT rowid, name, testSuite_fk FROM TestCases').all() as TestCaseRow[] + + const map: Record = {} + for (const row of rows) { + map[row.rowid] = row + } + return map +} + +function getSkipNoticesIdToMessageMap(db: Database.Database): Record { + const rows = db.prepare('SELECT rowid, message FROM SkipNotices').all() as SkipNoticeRow[] + + const map: Record = {} + for (const row of rows) { + map[row.rowid] = row.message ?? '' + } + return map +} + +function getTestCaseRunIdToExpectedFailuresMap( + db: Database.Database +): Record { + const rows = db + .prepare( + 'SELECT rowid, issue_fk, testCaseRun_fk, failureReason FROM ExpectedFailures ORDER BY orderInOwner' + ) + .all() as ExpectedFailureRow[] + + const map: Record = {} + for (const row of rows) { + if (!row.testCaseRun_fk) { + continue + } + + const expectedFailures = map[row.testCaseRun_fk] ?? [] + expectedFailures.push(row) + map[row.testCaseRun_fk] = expectedFailures + } + return map +} + +function getTestIssues(db: Database.Database): TestIssueRow[] { + const rows = db + .prepare( + `SELECT + ti.rowid, + ti.testCaseRun_fk, + ti.compactDescription, + ti.detailedDescription, + ti.sanitizedDescription, + ti.sourceCodeContext_fk, + scl.filePath, + scl.lineNumber + FROM TestIssues AS ti + LEFT JOIN SourceCodeContexts AS scc ON scc.rowid = ti.sourceCodeContext_fk + INNER JOIN SourceCodeLocations AS scl ON scl.rowid = scc.location_fk + ORDER BY ti.testCaseRun_fk, ti.orderInOwner` + ) + .all() as TestIssueRow[] + + return rows +} + +function getSourceCodeContextIdToFramesMap( + db: Database.Database +): Record { + const rows = db + .prepare( + `SELECT + scf.rowid, + scf.context_fk, + scsi.symbolName, + scl.filePath, + scl.lineNumber + FROM SourceCodeFrames AS scf + INNER JOIN SourceCodeSymbolInfos AS scsi ON scsi.rowid = scf.symbolinfo_fk + INNER JOIN SourceCodeLocations AS scl ON scl.rowid = scsi.location_fk + WHERE scf.symbolInfo_fk IS NOT NULL + ORDER BY scf.context_fk, scf.orderInContainer` + ) + .all() as SourceCodeFrameRow[] + + const map: Record = {} + for (const row of rows) { + if (!row.context_fk) { + continue + } + + const context = map[row.context_fk] ?? [] + context.push(row) + map[row.context_fk] = context + } + return map +} + +function getTestCaseRunIdToAttachmentsMap( + db: Database.Database, + baseDir: string +): Record { + const rows = db + .prepare( + `SELECT + att.rowid, + att.filenameOverride, + att.xcResultKitPayloadRefId, + act.testCaseRun_fk + FROM Attachments AS att + INNER JOIN Activities AS act ON att.activity_fk = act.rowid` + ) + .all() as AttachmentRow[] + + const map: Record = {} + for (const row of rows) { + if (!row.testCaseRun_fk || !row.filenameOverride || !row.xcResultKitPayloadRefId) { + continue + } + + if (row.filenameOverride.startsWith(ignoredAttachmentsPrefix)) { + continue + } + + const buffer = readDataBlob(baseDir, row.xcResultKitPayloadRefId) + if (!buffer) { + continue + } + + const attachments = map[row.testCaseRun_fk] ?? [] + attachments.push({ + filename: row.filenameOverride, + buffer, + error: null, + }) + map[row.testCaseRun_fk] = attachments + } + return map +} + +function readDataBlob(baseDir: string, refId: string): Buffer | null { + const filename = `${dataFilePrefix}${refId}` + const filepath = path.join(baseDir, dataDir, filename) + + if (!existsSync(filepath)) { + return null + } + + const rawData = readFileSync(filepath) + if (isZstdCompressed(rawData)) { + return Buffer.from(decompress(rawData)) + } + + return rawData +} + +function isZstdCompressed(data: Buffer): boolean { + if (data.length < 4) return false + return data.subarray(0, 4).equals(ZSTD_MAGIC) +} + +function mapResultStatus(result: string | null): ResultStatus { + switch (result?.toLowerCase() ?? null) { + case 'success': + return 'passed' + case 'failure': + return 'failed' + case 'skipped': + return 'skipped' + case 'expected failure': + return 'blocked' + } + + return 'skipped' +} + +function buildMessage( + testCaseRunId: number, + status: ResultStatus, + skipNotice: string | null, + expectedFailures: ExpectedFailureRow[] | null, + allTestIssues: TestIssueRow[], + sourceCodeContextIdToFramesMap: Record +): string { + let message = '' + + if (status === 'skipped' && skipNotice) { + message += `

Skipped Reason: ${escapeHtml(skipNotice)}

` + } + + if (status === 'blocked' && expectedFailures) { + for (let i = 0; i < expectedFailures.length; i++) { + const expectedFailure = expectedFailures[i] + const issue = expectedFailure.issue_fk + ? allTestIssues?.find((ti) => ti.rowid === expectedFailure.issue_fk) + : null + + message += `${i > 0 ? '

' : ''}

Expected Failure: ${escapeHtml( + expectedFailure.failureReason + )}

` + if (issue) { + const issueMessage = getIssueMessage( + issue, + sourceCodeContextIdToFramesMap, + '    ' + ) + if (issueMessage) { + message += issueMessage + } + } + } + } + + if (status === 'failed') { + let addSeparation = false + const issues = allTestIssues.filter((ti) => ti.testCaseRun_fk === testCaseRunId) + + for (const issue of issues) { + const issueMessage = getIssueMessage(issue, sourceCodeContextIdToFramesMap) + if (issueMessage) { + message += `${addSeparation ? '

' : ''}

${issueMessage}

` + addSeparation = true + } + } + } + + return message +} + +function getIssueMessage( + issue: TestIssueRow, + sourceCodeContextIdToFramesMap: Record, + indent = '' +) { + let issueMessage = + issue.detailedDescription || issue.sanitizedDescription || issue.compactDescription || '' + + if (!issueMessage) { + return '' + } + + issueMessage = `${indent}${escapeHtml(issueMessage)}` + if (issue.filePath && issue.lineNumber) { + issueMessage += ` (at ${escapeHtml(issue.filePath)}:${issue.lineNumber})
` + } + + const frames = issue.sourceCodeContext_fk + ? sourceCodeContextIdToFramesMap[issue.sourceCodeContext_fk] + : null + if (frames?.length) { + for (const frame of frames) { + issueMessage += `${indent}    ${escapeHtml( + frame.symbolName ?? '??' + )}` + if (frame.filePath && frame.lineNumber) { + issueMessage += ` (at ${escapeHtml(frame.filePath)}:${frame.lineNumber})` + } + issueMessage += `
` + } + } + + return issueMessage +}