Support Playwright JSON #33

hi-rai · 2025-10-23T11:02:57Z

This fixes https://github.com/Hypersequent/tms-issues/issues/1998

gemini-code-assist · 2025-10-23T11:03:38Z

Summary of Changes

Hello @hi-rai, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the QAS CLI by introducing full support for uploading Playwright JSON test reports. This new capability is integrated through a refactored and generalized command structure, which now handles both JUnit XML and Playwright JSON reports using a common logic base. The update ensures that test results from Playwright can be seamlessly submitted to QA Sphere, complete with test case matching and attachment handling. The accompanying documentation has been revised to reflect these changes, providing clear guidance for users.

Highlights

Playwright JSON Support: The CLI now fully supports uploading Playwright JSON test reports, enabling users to integrate Playwright test results into QA Sphere.
Refactored Upload Logic: The test report upload commands (junit-upload and the new playwright-json-upload) have been refactored into a unified ResultUploadCommandModule for improved maintainability and extensibility.
Enhanced Documentation: The README.md has been updated to clearly outline the new Playwright JSON upload command, its usage, and generalized test report requirements.
Dependency Updates: Added strip-ansi to clean up console output and updated other related dependencies in package-lock.json.
API Type Alignment: The CreateRunResponse interface was updated to correctly reflect that run IDs are numerical (id: number).

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces support for Playwright JSON reports, which is a great addition. The refactoring to abstract the parsing and uploading logic is well-executed, creating a clean and extensible architecture. The introduction of a standardized TestCaseResult and a generic ResultUploadCommandHandler is a solid design choice. The new test suites, especially the parameterized tests covering both JUnit and Playwright formats, are thorough and ensure the stability of the refactored code.

I've found a potential bug in the result comment construction and a couple of minor maintainability improvements. Overall, this is a high-quality contribution.

src/utils/result-upload/playwrightJsonParser.ts

hi-rai · 2025-10-24T05:53:33Z

src/commands/resultUpload.ts

Same as older junit-upload.ts, but now support both junit xml and playwright json

Instead of deleting and copying code, renaming the file will result in better diffs.

It doesn't matter if we rename a file and modify it or delete and create a new similar one, git handles both cases in the same fashion on its own.
Git detects renames based on file similarity - if changes are too much, it would consider it as delete+create rather than rename+modify

hi-rai · 2025-10-24T05:54:32Z

src/tests/fixtures/playwright-json/comprehensive-test.json

The playwright json files are similar to junit xmls with same name. I didn't create it manually, but was generated by AI

hi-rai · 2025-10-24T05:55:49Z

src/tests/junit-xml-parsing.spec.ts

Changed from spaces to tabs and the status texts are now as per QA Sphere

hi-rai · 2025-10-24T05:56:20Z

src/tests/result-upload.spec.ts

Same as older junit-upload.spec.ts, but now support both junit xml and playwright json commands

hi-rai · 2025-10-24T05:57:40Z

src/utils/result-upload/ResultUploadCommandHandler.ts

Similar to older JUnitResultUploader.ts, but now supports both playwright json and junit xml

hi-rai · 2025-10-24T05:58:37Z

src/utils/version.ts

Space to tabs

hi-rai · 2025-10-24T05:59:02Z

src/utils/projectExtractor.ts

Is now simpler and moved to ResultUploadCommandHandler

satvik007

Thanks @hi-rai
The PR looks pretty good to me, but I spent a lot of time understanding it and wrote some optional comments.

satvik007 · 2025-10-27T09:37:30Z

src/commands/resultUpload.ts

Instead of deleting and copying code, renaming the file will result in better diffs.

satvik007 · 2025-10-27T12:08:42Z

src/utils/result-upload/playwrightJsonParser.ts

+				name: markerFromAnnotations
+					? `${markerFromAnnotations}: ${titlePrefix}${spec.title}`
+					: `${titlePrefix}${spec.title}`,


Instead of writing the annotation to the name and then again using a regex to get the tcase seq from name. Can we create a field for seq in the parsing stage itself.

The current logic is sound but I think it will be better to extract seq while parsing itself.

Had done that initially, but changed it as Prefixing it to name also helps in detectProjectCode

satvik007 · 2025-10-27T13:52:40Z

src/utils/result-upload/ResultUploader.ts

+			for (let i = 0; i < results.length; i++) {
+				const { tcase, result } = results[i]
+				let comment = result.message
+				loader.setText(`Uploading test case ${i + 1} of ${results.length}`)
+				if (this.args.attachments) {
+					const attachmentUrls: Array<{ name: string; url: string }> = []
+					for (const attachment of result.attachments) {
+						if (attachment.buffer) {
+							const { url } = await this.api.file.uploadFile(
+								new Blob([attachment.buffer]),
+								attachment.filename
+							)
+							attachmentUrls.push({ url, name: attachment.filename })
+						}
+					}
+					comment += `\n<h4>Attachments:</h4>\n${makeListHtml(attachmentUrls)}`
+				}
+
+				await this.api.runs.createResultStatus(this.project, this.run, tcase.id, {
+					status: result.status,
+					comment,
+				})
+			}


We should think of some performance improvements with parallel uploads and api calls.

Better to do it separately. I will open another PR after this is merged

satvik007 · 2025-10-27T13:54:47Z

src/utils/result-upload/ResultUploadCommandHandler.ts

+			if (conflictMatch) {
+				const existingRunId = Number(conflictMatch[1])
+				console.log(chalk.yellow(`Reusing existing test run "${title}" with ID: ${existingRunId}`))
+				return existingRunId
+			}


Another common error could be that we couldn't match the seq ids while creating test runs, can we handle that as well.

That won't happen as getTestCases only returns found tcases, which we are using here.

Currently we are silently skipping if some tcases are not found getTestCases, as long as at least one tcase is found - not sure if that is intentional or something that was missed. I haven't changed the behavior, but please let me know if we need to do that

src/utils/result-upload/ResultUploadCommandHandler.ts

AndrianBdn

That looks awesome, @hi-rai - thanks! We are going to publish the new version right away and even mention that we support two logging formats in the marketing announcements.

Support Playwright JSON

07f64bf

hi-rai requested review from AndrianBdn, ElenaGrasovskaya, ramilamparo, satvik007 and sirkostya009 October 23, 2025 11:02

hi-rai self-assigned this Oct 23, 2025

gemini-code-assist bot reviewed Oct 23, 2025

View reviewed changes

src/utils/result-upload/playwrightJsonParser.ts Show resolved Hide resolved

src/utils/result-upload/playwrightJsonParser.ts Outdated Show resolved Hide resolved

hi-rai added 2 commits October 23, 2025 16:43

Minor fixes

229d5fc

Support playwright annotations

0bdda07

hi-rai commented Oct 24, 2025

View reviewed changes

Improve

4cc6505

satvik007 approved these changes Oct 27, 2025

View reviewed changes

Update tcase marker regex in extractTestCaseRefs

882d259

AndrianBdn added the publish Publish to NPM label Oct 29, 2025

AndrianBdn approved these changes Oct 29, 2025

View reviewed changes

AndrianBdn merged commit 2286e8b into main Oct 29, 2025
1 check passed

AndrianBdn deleted the dev/himanshu/support-playwright-json branch October 29, 2025 13:37

hi-rai mentioned this pull request Oct 30, 2025

Concurrent file uploads and result creation #34

Merged

Support Playwright JSON #33

Support Playwright JSON #33

Uh oh!

Conversation

hi-rai commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist bot commented Oct 23, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

satvik007 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AndrianBdn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hi-rai commented Oct 23, 2025 •

edited

Loading