Create and configure new actions workflow for E2E tests

[tyatposit]

Cypress E2E CI Setup and Test Reliability Improvements

I think I finally got this in a reliable enough state. I plan to squash the commits since there were so many as I was troubleshooting.

Closes #2707

Intent

• Add new e2e.yaml workflow for running Cypress E2E tests in CI
• Update pull-request.yaml to trigger E2E workflow on PRs
• Configure cypress.config.js for retries and extended timeouts in CI
• Add DEBUG_CYPRESS env variable and conditional debug steps for enhanced diagnostics and video replay recordings
• Enhance failure diagnostics with detailed iframe and body HTML logging
• Add health checks and wait loops to ensure services are ready before starting tests
• Ensure VSIX extension is installed and verified before running tests
• Improve iframe detection and fallback logic for publisher webview in tests
• Add helper functions for robust waits and retries to stabilize CI runs
• Add uncaught exception handler for VS Code web, Quarto extension, and common browser errors
• Create utility script for repeatable test runs with per-spec execution and detailed summaries
• Update .gitignore and CONTRIBUTING.md for new CI workflow and local files

Type of Change

• • Bug Fix
• • New Feature
• • Breaking Change
• • Documentation
• • Refactor
• • Tooling

Approach

• Unreliable iframe loading in CI causes flakiness in E2E tests due to timing and race conditions. To address this, I had to add additional helper functions with retries and backoff logic to reliably detect and wait for the publisher webview iframe before interacting with it, improving test stability and consistency. These changes should not affect local test runs and, combined with configured retries, have reduced the likelihood of false positive failures in CI.
• There are file permissions issues that still need to be resolved that only affected the embedded-deployments.cy.js tests and they are being skipped for now in CI.
• I added a way to skip tests only in CI so Cypress.skipCI can wrap any describe or it for this. The above tests will still run and pass locally however.

User Impact

n/a

Automated Tests

• All Cypress e2e tests sans the embedded-deployments.cy.js due to permissions issues in CI (see above)

Directions for Reviewers

• e2e tests should all still run and pass locally with either npx cypress run or npx cypress open (or just dev for that matter) >> There may still be some flake on the embedded deploy tests running local that will need to be followed up on. Improve embedded_deployments.cy.js test issues #2872
• e2e tests should run in CI for any PR creation or update

Checklist

• I have updated CHANGELOG.md to cover notable changes.

[sagerb]

Thanks for taking on this work! The code changes look great, but I hit some problems when trying to validate the changes.

1. When running just dev, the embedded-deployments tests (specifically fastAPI in subdirectory of workspace) failed w/

get iframe.webview.ready
assert expected iframe.webview.ready to exist in the DOM

2. I retried the test and it worked.
3. I then exited and ran npx cypress run and it ran fine
4. I then retried running just dev and it failed again at that same test.

What I was able to notice was that it looked like the publisher extension was not actively selected from the left taskbar. So, of course, the iframe wouldn't exist. I wonder if you might need to check that logic and perhaps select it again on failure? The button was visible and able to be clicked.

Since this PR includes activating the e2e runs within CI, can you take a look at this logic once again? Please let me know when you want me to test it again. Once I can get repetitive successful runs, even with an intermittent failure, I'll be happy to approve. It was just hitting this a second time during my verification, which makes me uneasy.

[tyatposit]

Thanks for trying it @sagerb. I will take a look and see if adding that click back in helps. I may have accidentally dropped it with the helper changes. Most of my local testing was with the run command vs open, so it's possible I just missed that intermittent failure locally.

[sagerb]

From further conversations:

bill.sager:
Tim, I hit failures twice running this locally, but I noted that it looked like the extension had not been clicked upon, which is required ahead of looking for the iframe. I've added further notes within the PR. Let me know when you want me to try it again. (Retry does work, but  I did have it fail twice within 4 attempts.)

tim.yaeger:
Thanks Bill.  I’m still working to get this one more consistently stable it’s proving to be a challenge.  Did you have any of the other tests fail on you?

bill.sager:
No, that was the only one. Prior to the changes, the tests all ran successfully time and time again for me (locally).

tim.yaeger:
Ok, I’ll see if I can get this one more stable for a bit longer, but I mainly was asking because I currently have this particular test skipping in CI for a different permissions issue that only happens there and I wasn’t able to get to the bottom of that either.  So at least for now the embedded test won’t affect the CI runs due to being skipped.
I do want to get both things fixed on this test, but thought I might just do it in a separate PR/ticket.  I also have some new tests to add in another branch for our CC credentials.

bill: My theory (based on the test run recordings) is that the click on the Publisher icon isn't working to select the publisher views. I wonder if you might want to explore modifying the method in which the code activates the publisher button, to invoking the view -> Open View...  menu, and then typing in Posit Publisher w/ a return. Maybe that is more reliable than the mouse click?

As long as you have those tests skipping in CI, and that you're planning on following up on the fixes, I'm good with proceeding forward getting this PR merged. Let me update the PR.

Approving the PR and leaving it in the hands of Tim to resolve. Thanks!