diff --git a/docs/accessibility/changelog.mdx b/docs/accessibility/changelog.mdx index cf1cf4c14c..9f245b5ebd 100644 --- a/docs/accessibility/changelog.mdx +++ b/docs/accessibility/changelog.mdx @@ -9,6 +9,20 @@ sidebar_position: 200 # Changelog +## Week of 6/2/2025 + +- We now support fetching Accessibility results from Drone CI with the [Results API](/accessibility/results-api). + +## Week of 5/26/2025 + +- A new [element identification](/accessibility/core-concepts/element-identification) algorithm is now available for Cypress Accessibility. This produces improved element deduplication across different snapshots and runs, and incorporates testing-specific attributes like Test IDs into the element recognition and deduplication process. This replaces the default Axe-Core® element identification behavior. +- Two new configuration options are available in Cypress Accessibility - [`significantAttributes`](/accessibility/configuration/significantattributes) and [`attributeFilters`](/accessibility/configuration/attributefilters). These allow you to fine-tune Cypress's element identification behavior to ensure stable and relevant locators are used. +- Branch Review for Cypress Accessibility is now generally available, with an updated interface for displaying new and resolved violations, based on feedback during the Beta period. Thank you all for hitting the feedback button! [Learn more about comparing reports](/accessibility/core-concepts/compare-reports). + +## Week of 5/12/2025 + +- We now support fetching Accessibility results from AWS CodeBuild with the [Results API](/accessibility/results-api). + ## Week of 3/24/2025 - Cypress Accessibility results are now included in the [Data Extract API](/cloud/integrations/data-extract-api) so that you can retrieve data over time. diff --git a/docs/accessibility/core-concepts/cypress-rules.mdx b/docs/accessibility/core-concepts/cypress-rules.mdx new file mode 100644 index 0000000000..5132041730 --- /dev/null +++ b/docs/accessibility/core-concepts/cypress-rules.mdx @@ -0,0 +1,81 @@ +--- +title: 'Cypress Rules | Cypress Documentation' +description: 'Review the main areas to pay attention to when first reviewing an accessibility report for a Cypress run.' +sidebar_position: 70 +sidebar_label: Cypress rules +sidebar_custom_props: { 'new_label': true } +--- + + + +# Cypress rules + +In addition to running the default ruleset of the Axe-Core® library, we also create custom rules that take advantage of the addition layer of information available in a test automation context. These rules are identified by a "Cypress Rule" badge, to distinguish them from the main Axe-Core® rule set. + +The first custom rule that we have implemented is called "Interactive elements must be semantically correct." At launch, it is a non-blocking **manual-review** rule that will either pass or return an inconclusive state. + +## Interactive elements must be semantically correct + +**Rule ID**: `semantic-html-warning` + + + +This custom rule is intended to surface potential usability problems with the kinds of elements that Axe-Core® traditionally does not evaluate as interactive, such as div, span, SVG, and image elements. + +This works based on the interactions that take place during testing. Often an inaccessible custom component will have Cypress tests that perform interactions the way a user does, and that allows us to detect a mismatch between the underlying purpose of the element in code, and the usage of the element in the context of a test. + +### How to pass this rule + +You should look at the purpose and nature of the specific elements flagged by this rule and decide what interactive element is appropriate for the interactions that a user can do and the effects of the interaction. + +For example, if the element that gets flagged is a span that has been styled to look like a button and submits a form, the correct solution would be standard html button element. But if the same element was styled with underlined text, and when clicked it resets the URL to navigate to a new page, the correct element is a link. + +The goal of this rule is not to propose the correct element for your scenarios, but to surface the elements that need attention so that you can make this decision. After that, with appropriate semantics in place, the elements will then be able to be correctly parsed and analyzed by the standard Axe-Core® checks. + +:::info +**Tip**: We recommend following the [first rule of ARIA use](https://www.w3.org/TR/using-aria/#firstrule) and implementing interactive controls with standard semantic HTML where possible, relying on ARIA `role` attributes only when no suitable HTML element is available. +::: + + +### Why this matters + +In the Web Content Accessibility Guidelines (WCAG), interactive elements are required to met a wide range of Success Criteria (SCs) in order to follow the POUR principles, which stands for Perceivable, Operable, Understandable, and Robust. + +Interactive elements that are not semantically correct often don't appear in automated scans, but do appear in manual testing, because they are usually not operable with a keyboard or correctly understandable when described by a screen reader. + +Custom components implemented this way have a direct impact on the independence of your disabled users. + +### Related WCAG Success Criteria + +An incorrectly implemented interactive element may be failing multiple Success Criteria from WCAG, for example: + +- [SC 1.3.1 Info and Relationships](https://www.w3.org/TR/WCAG21/#info-and-relationships) +- [SC 2.1.1 Keyboard](https://www.w3.org/TR/WCAG21/#keyboard) +- [SC 4.1.2 Name, Role, Value](https://www.w3.org/TR/WCAG21/#name-role-value) + +### Status + +This rule is available in all Cypress Accessibility projects, though it is not turned on by default at launch. Reach out to Cypress if you would like to have this rule activated for you early. + +### Possible outcomes + +For each run, this rule can be in one of four states. Most importantly, it will never fail, which means it will not affect your accessibility score. + + +| Outcome | Description | +|---------|-------------| +| **Pass** | No interactive elements with semantic issues were detected during the test run. | +| **Inconclusive** | Interactive elements with potential semantic issues were detected. You should review each element and determine the correct solution. | +| **Not Applicable** | The test run did not interact with any elements. | +| **Ignored by configuration** | The rule was not executed during this test run because it was intentionally turned off. | + + +### False positives + +We expect a certain number of false positives to be detected during the initial rollout. If you see these, please use the "Submit Feedback" button in the product to let us know. Our goal is to follow the general Axe-Core® philosophy that minimizes false positives. + + diff --git a/docs/accessibility/get-started/introduction.mdx b/docs/accessibility/get-started/introduction.mdx index a6ae40b5eb..b257ab948a 100644 --- a/docs/accessibility/get-started/introduction.mdx +++ b/docs/accessibility/get-started/introduction.mdx @@ -3,7 +3,6 @@ sidebar_label: Introduction title: 'Accessibility Testing | Cypress Documentation' description: 'Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core® by Deque Systems.' sidebar_position: 10 -sidebar_custom_props: { 'new_label': true } --- diff --git a/docs/accessibility/results-api.mdx b/docs/accessibility/results-api.mdx index 76da044dd7..baefe6b80c 100644 --- a/docs/accessibility/results-api.mdx +++ b/docs/accessibility/results-api.mdx @@ -34,6 +34,7 @@ Fetching Accessibility results for a run supports fetching results for the follo - [GitLab](/app/continuous-integration/gitlab-ci) - [Jenkins](/app/continuous-integration/overview#Jenkins) - [AWS CodeBuild](/app/continuous-integration/aws-codebuild) +- Drone Please reach out to Cypress Support to request support for a different provider. @@ -59,6 +60,27 @@ If you check this in as a dependency, your installation will fail when we update Write a script using the `getAccessibilityResults` utility to retrieve the results and perform one or more assertions to verify if the changes are acceptable. This script will be executed in CI. +#### Basic example + +This snippet uses the `getAccessibilityResults()` helper to log out the results. It assumes your Project ID and Record Key variable are set. The following should work in any of the supported CI Providers out of the box: + +```javascript title="scripts/verifyAccessibilityResults.js" +// Assuming these environment variables are set: +// CYPRESS_PROJECT_ID=your-id +// CYPRESS_RECORD_KEY=your-record-key + +const { getAccessibilityResults } = require('@cypress/extract-cloud-results') + +getAccessibilityResults().then((results) => { + + // use `console.dir` instead of `console.log` because the data is nested + console.dir(results, { depth: Infinity }); + +}) +``` + +#### How to assert that only known rules are failing in the run + The Cypress App [repository](https://github.com/cypress-io/cypress) uses the Results API to ensure no new violations have been introduced. You can reference [this script](https://github.com/cypress-io/cypress/blob/develop/scripts/verify-accessibility-results.js) as a real example. ```javascript title="scripts/verifyAccessibilityResults.js" @@ -133,7 +155,28 @@ getAccessibilityResults({ }) ``` -#### `getAccessibilityResults` Arguments +#### How to use a previous run as a baseline + + +```javascript title="scripts/verifyAccessibilityResults.js" +// Assuming these environment variables are set: +// CYPRESS_PROJECT_ID=your-id +// CYPRESS_RECORD_KEY=your-record-key + +const { getAccessibilityResults } = require('@cypress/extract-cloud-results') + +getAccessibilityResults().then((results) => { + + // use `console.dir` instead of `console.log` + // to ensure nested data prints out correctly + console.dir(results, { depth: Infinity }); + +}) +``` + + + +### `getAccessibilityResults` Arguments `getAccessibilityResults` uses the following attributes to identify the Cypress run and return the Accessibility results: @@ -444,5 +487,30 @@ phases: + - CYPRESS_INTERNAL_ENV=staging CYPRESS_PROJECT_ID=[slug] CYPRESS_RECORD_KEY=[KEY] node ./scripts/verifyAccessibilityResults.js ``` + + + +```diff title=".drone.yaml" + kind: pipeline + name: default + + environment: + CYPRESS_PROJECT_ID: example_project_slug + CYPRESS_RECORD_KEY: + from_secret: example_record_key_secret + + steps: + - name: test + image: node:latest + commands: + - npm install + - npx cypress run --record + ++ - name: validate ++ image: node:latest ++ commands: ++ - npm install --force https://cdn.cypress.io/extract-cloud-results/v1/extract-cloud-results.tgz ++ - node ./scripts/verifyAccessibilityResults.js +``` diff --git a/docs/ui-coverage/changelog.mdx b/docs/ui-coverage/changelog.mdx index 2dc779bbd9..6df262263d 100644 --- a/docs/ui-coverage/changelog.mdx +++ b/docs/ui-coverage/changelog.mdx @@ -16,10 +16,18 @@ UI Coverage now supports defining custom commands that will count towards covera - [`additionalInteractionCommands`](/ui-coverage/configuration/additionalinteractioncommands) - [`allowedInteractionCommands`](/ui-coverage/configuration/allowedinteractioncommands) +## Week of 6/2/2025 + +- We now support fetching UI Coverage results from Drone CI with the [Results API](/ui-coverage/results-api). + ## Week of 5/26/2025 - Launched AI-powered Test Generation in Cypress Cloud, to help you quickly add tests for the untested elements detected in UI Coverage reports, in a way that follows your existing practices and conventions. For more details, read [our blog post](https://www.cypress.io/blog/add-your-missing-tests-faster-with-test-generation-in-ui-coverage). +## Week of 5/12/2025 + +- We now support fetching UI Coverage results from AWS CodeBuild with the [Results API](/ui-coverage/results-api). + ## Week of 4/1/2024 - **Shadow DOM and iFrame Support:** UI Coverage now supports Shadow DOM and iFrames for reporting on interactions. Both of these settings are off by default. Please contact your Cypress representative to opt in. diff --git a/docs/ui-coverage/get-started/introduction.mdx b/docs/ui-coverage/get-started/introduction.mdx index 241a0c4510..d60786f473 100644 --- a/docs/ui-coverage/get-started/introduction.mdx +++ b/docs/ui-coverage/get-started/introduction.mdx @@ -3,7 +3,6 @@ title: 'Test coverage with UI Coverage | Cypress Documentation' description: 'Increase your test coverage with UI Coverage, a visual test coverage report based on the interactive elements that make up your application.' sidebar_label: Introduction sidebar_position: 10 -sidebar_custom_props: { 'new_label': true } --- diff --git a/docs/ui-coverage/results-api.mdx b/docs/ui-coverage/results-api.mdx index 568a6ad9d6..ae7e839549 100644 --- a/docs/ui-coverage/results-api.mdx +++ b/docs/ui-coverage/results-api.mdx @@ -20,6 +20,8 @@ The utility supports the following CI providers. Refer to the linked guides for - [GitHub Actions](/app/continuous-integration/github-actions) - [GitLab](/app/continuous-integration/gitlab-ci) - [Jenkins](/app/continuous-integration/overview#Jenkins) +- [AWS CodeBuild](/app/continuous-integration/aws-codebuild) +- Drone For other CI providers, contact Cypress Support to request support. @@ -324,5 +326,63 @@ workflows: - run-cypress ``` + + + + +```diff title="buildspec.yaml" +phases: + install: + runtime-versions: + nodejs: latest + commands: + # Set COMMIT_INFO variables to send Git specifics to Cypress Cloud when recording + # https://docs.cypress.io/app/continuous-integration/overview#Git-information + - export COMMIT_INFO_BRANCH="$(git rev-parse HEAD | xargs git name-rev | + cut -d' ' -f2 | sed 's/remotes\/origin\///g')" + - export COMMIT_INFO_MESSAGE="$(git log -1 --pretty=%B)" + - export COMMIT_INFO_EMAIL="$(git log -1 --pretty=%ae)" + - export COMMIT_INFO_AUTHOR="$(git log -1 --pretty=%an)" + - export COMMIT_INFO_SHA="$(git log -1 --pretty=%H)" + - export COMMIT_INFO_REMOTE="$(git config --get remote.origin.url)" + - npm ci + pre_build: + commands: + - npm run cypress:verify + - npm run cypress:info + build: + commands: + - CYPRESS_INTERNAL_ENV=staging CYPRESS_PROJECT_ID=[slug] npx cypress run --record --key [KEY] ++ post_build: ++ commands: ++ - npm install --force https://cdn.cypress.io/extract-cloud-results/v1/extract-cloud-results.tgz ++ - CYPRESS_INTERNAL_ENV=staging CYPRESS_PROJECT_ID=[slug] CYPRESS_RECORD_KEY=[KEY] node ./scripts/verifyUICoverageResults.js +``` + + + + +```diff title=".drone.yaml" + kind: pipeline + name: default + + environment: + CYPRESS_PROJECT_ID: example_project_slug + CYPRESS_RECORD_KEY: + from_secret: example_record_key_secret + + steps: + - name: test + image: node:latest + commands: + - npm install + - npx cypress run --record + ++ - name: validate ++ image: node:latest ++ commands: ++ - npm install --force https://cdn.cypress.io/extract-cloud-results/v1/extract-cloud-results.tgz ++ - node ./scripts/verifyUICoverageResults.js +``` diff --git a/static/img/accessibility/core-concepts/cypress-rule.png b/static/img/accessibility/core-concepts/cypress-rule.png new file mode 100644 index 0000000000..2d64614799 Binary files /dev/null and b/static/img/accessibility/core-concepts/cypress-rule.png differ