|
1 | 1 | <!-- NOTE: This file documents a work-in-progress experimental feature. It is NOT integrated into the main Cypress error pipeline yet. --> |
2 | 2 | # Timeout Diagnostics — Smart Timeout Error Messages (WIP) |
3 | 3 |
|
| 4 | +Status: Work‑in‑progress (not yet integrated into Cypress). This document describes an experimental timeout diagnostics system that analyzes command context (selectors, network, animations, DOM mutations) and produces contextual recommendations to help users address flaky or timing-related failures. |
| 5 | + |
4 | 6 | ## Objective |
5 | 7 |
|
6 | | -Improve the developer experience when dealing with timeout errors in Cypress by providing actionable, context-aware suggestions based on analysis of the failure context. |
| 8 | +Improve the developer experience for timeout errors by producing actionable, context-aware suggestions when a command times out. Instead of a generic message like `cy.get() timed out waiting 4000ms`, diagnostics should indicate likely causes (dynamic content, pending network requests, animations, etc.) and suggest concrete fixes. |
7 | 9 |
|
8 | 10 | ## Motivation |
9 | 11 |
|
10 | | -Timeout errors are extremely common in Cypress tests, yet default messages are often generic and unhelpful: |
| 12 | +Timeout errors are a frequent source of confusion in end-to-end tests. The goal of this feature is to reduce debugging time and teach best practices inline by providing targeted suggestions and examples. |
| 13 | + |
| 14 | +## Example Enhanced Output |
11 | 15 |
|
12 | 16 | ``` |
13 | | -cy.get() timed out waiting 4000ms |
| 17 | +cy.get('.user-list') timed out waiting 4000ms |
| 18 | +
|
| 19 | +Diagnostic suggestions: |
| 20 | +
|
| 21 | +1) Selector likely targets dynamic content |
| 22 | + • Wait for the loading state: cy.get('.loading-spinner').should('not.exist') |
| 23 | + • Prefer stable data attributes: e.g. [data-cy="user-list"] |
| 24 | + • Consider intercepting the API call that populates this content |
| 25 | +
|
| 26 | +2) Network requests pending (8) |
| 27 | + • Use cy.intercept('GET', '/api/users').as('getUsers') and cy.wait('@getUsers') |
| 28 | + • If requests are expected to be slow, either wait for the specific request or increase the timeout |
| 29 | +
|
| 30 | +Learn more: https://on.cypress.io/intercept |
14 | 31 | ``` |
15 | 32 |
|
16 | | -This diagnostics system aims to enrich those messages with intelligent, actionable suggestions: |
| 33 | +## Features |
| 34 | + |
| 35 | +- Problematic selector detection (loading spinners, skeletons, dynamic IDs) |
| 36 | +- Network analysis (pending requests, slow/long-running requests) |
| 37 | +- Animation detection and guidance to reduce timing flakiness |
| 38 | +- Detection of excessive DOM mutations and recommendations to wait for stability |
| 39 | +- Command-specific suggestions (get, click, type, etc.) and doc links |
| 40 | + |
| 41 | +## Project Structure (Implementation Sketch) |
17 | 42 |
|
18 | 43 | ``` |
19 | | -cy.get() timed out waiting 4000ms |
20 | | -
|
21 | | -🔍 Diagnostic Suggestions: |
22 | | -
|
23 | | -1. The selector appears to target dynamic/loading content |
24 | | - a) Wait for the loading state to complete: cy.get('.loading-spinner').should('not.exist') |
25 | | - b) Consider using data-cy attributes instead of class names that indicate loading states |
26 | | - c) Use cy.intercept() to wait for the API request that populates this content |
27 | | - 📚 Learn more: https://on.cypress.io/best-practices#Selecting-Elements |
28 | | -
|
29 | | -2. 8 network requests are still pending |
30 | | - a) Wait for specific API calls to complete using cy.intercept() |
31 | | - <!-- NOTE: This file documents a work-in-progress experimental feature. It is NOT integrated into the main Cypress error pipeline yet. --> |
32 | | - # Timeout Diagnostics — Smart Timeout Error Messages (WIP) |
33 | | -
|
34 | | - Status: Work‑in‑progress (not yet integrated into Cypress). This document describes an experimental timeout diagnostics system that analyzes command context (selectors, network, animations, DOM mutations) and produces contextual recommendations to help users address flaky or timing-related failures. |
35 | | -
|
36 | | - Objective |
37 | | - --------- |
38 | | - Improve the developer experience for timeout errors by producing actionable, context-aware suggestions when a command times out. Instead of a generic message like `cy.get() timed out waiting 4000ms`, diagnostics should indicate likely causes (dynamic content, pending network requests, animations, etc.) and suggest concrete fixes. |
39 | | -
|
40 | | - Motivation |
41 | | - ---------- |
42 | | - Timeouts are a frequent source of confusion in end-to-end tests. The goal of this feature is to reduce debugging time and teach best practices inline by providing targeted suggestions and examples. |
43 | | -
|
44 | | - Example enhanced output |
45 | | - ----------------------- |
46 | | - ``` |
47 | | - cy.get('.user-list') timed out waiting 4000ms |
48 | | - |
49 | | - Diagnostic suggestions: |
50 | | - |
51 | | - 1) Selector likely targets dynamic content |
52 | | - • Wait for the loading state: cy.get('.loading-spinner').should('not.exist') |
53 | | - • Prefer stable data attributes: e.g. [data-cy="user-list"] |
54 | | - • Consider intercepting the API call that populates this content |
55 | | - |
56 | | - 2) Network requests pending (8) |
57 | | - • Use cy.intercept('GET', '/api/users').as('getUsers') and cy.wait('@getUsers') |
58 | | - • If requests are expected to be slow, either wait for the specific request or increase the timeout |
59 | | - |
60 | | - Learn more: https://on.cypress.io/intercept |
61 | | - ``` |
62 | | -
|
63 | | - Features |
64 | | - -------- |
65 | | - - Problematic selector detection (loading spinners, skeletons, dynamic IDs) |
66 | | - - Network analysis (pending requests, slow/long-running requests) |
67 | | - - Animation detection and guidance to reduce timing flakiness |
68 | | - - Detection of excessive DOM mutations and recommendations to wait for stability |
69 | | - - Command-specific suggestions (get, click, type, etc.) and doc links |
70 | | -
|
71 | | - Project structure (implementation sketch) |
72 | | - ---------------------------------------- |
73 | | - ``` |
74 | | - packages/driver/src/cypress/ |
75 | | - timeout_diagnostics.ts — core analysis + formatting (WIP) |
76 | | - |
77 | | - packages/driver/test/unit/cypress/ |
78 | | - timeout_diagnostics.spec.ts — unit tests |
79 | | - ``` |
80 | | -
|
81 | | - API sketch |
82 | | - ---------- |
83 | | - ```ts |
84 | | - // analyze context and produce suggestions |
85 | | - const suggestions = TimeoutDiagnostics.analyze({ |
86 | | - command: 'get', |
87 | | - selector: '.loading-spinner', |
88 | | - timeout: 4000, |
89 | | - networkRequests: 5, |
90 | | - animationsRunning: true, |
91 | | - }) |
92 | | -
|
93 | | - const formatted = TimeoutDiagnostics.formatSuggestions(suggestions) |
94 | | -
|
95 | | - // enhance base timeout message with suggestions |
96 | | - const enhanced = TimeoutDiagnostics.enhanceTimeoutError('cy.get() timed out', context) |
97 | | - ``` |
98 | | - |
99 | | - Integration notes |
100 | | - ----------------- |
101 | | - To integrate this into Cypress's existing error pipeline we would: |
102 | | - |
103 | | - 1. Extend `error_utils` (or the error creation path) to provide additional context (selector, pending network count, DOM mutation rate, animation state) when creating a timeout error. |
104 | | - 2. Call `TimeoutDiagnostics.analyze(context)` and append the returned suggestions to the error message (optionally behind a feature flag). |
105 | | - 3. Add a configuration option to opt in/out of diagnostics in CI or local environments. |
106 | | - |
107 | | - Integration example (pseudo): |
108 | | - ```ts |
109 | | - import TimeoutDiagnostics from './timeout_diagnostics' |
110 | | - |
111 | | - function createTimeoutError(cmd, ms) { |
112 | | - const context = { |
113 | | - command: cmd.get('name'), |
114 | | - selector: cmd.get('selector'), |
115 | | - timeout: ms, |
116 | | - networkRequests: getNetworkMonitor().pendingCount(), |
117 | | - animationsRunning: hasRunningAnimations(), |
118 | | - domMutations: getDOMMutationCount(), |
119 | | - } |
120 | | - |
121 | | - const baseMessage = `cy.${cmd.get('name')}() timed out waiting ${ms}ms` |
122 | | - return TimeoutDiagnostics.enhanceTimeoutError(baseMessage, context) |
123 | | - } |
124 | | - ``` |
| 44 | +packages/driver/src/cypress/ |
| 45 | + timeout_diagnostics.ts — core analysis + formatting (WIP) |
| 46 | +
|
| 47 | +packages/driver/test/unit/cypress/ |
| 48 | + timeout_diagnostics.spec.ts — unit tests |
| 49 | +``` |
125 | 50 |
|
126 | | - Tests |
127 | | - ----- |
128 | | - Run unit tests for the diagnostics module: |
| 51 | +## API Sketch |
129 | 52 |
|
130 | | - ```powershell |
131 | | - cd packages/driver |
132 | | - yarn test timeout_diagnostics.spec.ts |
133 | | - ``` |
| 53 | +```ts |
| 54 | +// analyze context and produce suggestions |
| 55 | +const suggestions = TimeoutDiagnostics.analyze({ |
| 56 | + command: 'get', |
| 57 | + selector: '.loading-spinner', |
| 58 | + timeout: 4000, |
| 59 | + networkRequests: 5, |
| 60 | + animationsRunning: true, |
| 61 | +}) |
| 62 | + |
| 63 | +const formatted = TimeoutDiagnostics.formatSuggestions(suggestions) |
| 64 | + |
| 65 | +// enhance base timeout message with suggestions |
| 66 | +const enhanced = TimeoutDiagnostics.enhanceTimeoutError('cy.get() timed out', context) |
| 67 | +``` |
134 | 68 |
|
135 | | - Notes & next steps |
136 | | - ------------------ |
137 | | - - This README documents the experimental design and examples. The feature is intentionally left out of the main error flow until heuristics and runtime metric collection are reviewed. |
138 | | - - Next: finalize analysis heuristics, expand unit tests, and implement an opt-in integration path in `error_utils`. |
| 69 | +## Integration Notes |
139 | 70 |
|
140 | | - License |
141 | | - ------- |
142 | | - MIT — consistent with the Cypress project |
| 71 | +To integrate this into Cypress's existing error pipeline we would: |
143 | 72 |
|
144 | | - If you'd like, I can also: |
145 | | - - normalize the file encoding/artifacts (there were some garbled characters in the original), and |
146 | | - - open a small PR on the `feat/timeout-diagnostics` branch with this cleaned README. |
| 73 | +1. Extend `error_utils` (or the error creation path) to provide additional context (selector, pending network count, DOM mutation rate, animation state) when creating a timeout error. |
| 74 | +2. Call `TimeoutDiagnostics.analyze(context)` and append the returned suggestions to the error message (optionally behind a feature flag). |
| 75 | +3. Add a configuration option to opt in/out of diagnostics in CI or local environments. |
| 76 | + |
| 77 | +Integration example (pseudo): |
| 78 | +```ts |
| 79 | +import TimeoutDiagnostics from './timeout_diagnostics' |
| 80 | + |
| 81 | +function createTimeoutError(cmd, ms) { |
| 82 | + const context = { |
| 83 | + command: cmd.get('name'), |
| 84 | + selector: cmd.get('selector'), |
| 85 | + timeout: ms, |
| 86 | + networkRequests: getNetworkMonitor().pendingCount(), |
147 | 87 | animationsRunning: hasRunningAnimations(), |
| 88 | + domMutations: getDOMMutationCount(), |
| 89 | + } |
| 90 | + |
| 91 | + const baseMessage = `cy.${cmd.get('name')}() timed out waiting ${ms}ms` |
| 92 | + return TimeoutDiagnostics.enhanceTimeoutError(baseMessage, context) |
| 93 | +} |
| 94 | +``` |
| 95 | + |
| 96 | +## Tests |
| 97 | + |
| 98 | +Run unit tests for the diagnostics module: |
| 99 | + |
| 100 | +```powershell |
| 101 | +cd packages/driver |
| 102 | +yarn test timeout_diagnostics.spec.ts |
| 103 | +``` |
| 104 | + |
| 105 | +## Notes & Next Steps |
| 106 | + |
| 107 | +- This README documents the experimental design and examples. The feature is intentionally left out of the main error flow until heuristics and runtime metric collection are reviewed. |
| 108 | +- Next: finalize analysis heuristics, expand unit tests, and implement an opt-in integration path in `error_utils`. |
| 109 | + |
| 110 | +## License |
| 111 | + |
| 112 | +MIT — consistent with the Cypress project |
0 commit comments