Add screen reader support #4854

andyleejordan · 2025-07-19T00:11:17Z

A new render implementation for use under screen readers that avoids redrawing the terminal buffer so as to avoid unnecessary and confusing output.

The differential rendering relies on calculating the common prefix of the buffer and previousRender strings. Nearly all necessary changes are consolidated in the new rendering function.

Features known not to be supported:

Colors: as this necessitates redrawing to insert color sequences after the input is received and the AST parsed.
Inline predictions: as this by definition changes the suffix and thus requires endless redrawing.
List view predictions: since the render implementation never calls into the prediction engine, this is not available either.
Menu completion: well, it "works" since it's not disabled and does its own rendering, but no effort has been made to improve DrawMenu(), so it's not accessible (and I'm not sure it could be given our current constraints).

Features known to be partially supported:

Text selection: mark and select commands work as intended, but provide no visual indication.
Multiple lines: as in newlines work fine, but there is no continuation prompt.
Visually wrapped lines: editing above a wrapped line redraws all subsequent lines and hence is noisy.
Status prompt based commands: what-is-key, digit-argument, and most notably, forward/backward incremental history search all render a "status prompt" on the line below the user's input buffer. This is supported; however, it can be noisy since it necessarily has to render the whole buffer when the input buffer changes, including the status prompt (and search text). But what it's reading is almost always going to be relevant.

Everything else should generally work, even Vi mode, and the tests pass. That said, this isn't perfect, and moreover the approach specifically doesn't attempt to enable things from the ground up. There may be features that are available but turn out not to be accessible (like MenuComplete) and I believe they should be left as-is.

Specifically tested with NVDA on Windows and VoiceOver on macOS within VS Code's integrated terminal, with shell integration loaded, and Code's screen reader optimizations enabled.

Is a start to resolving #2504.

Copilot

Pull Request Overview

This PR introduces prototype screen reader support by implementing a SafeRender abstraction that uses ANSI control codes when screen reader mode is enabled. The changes modify rendering behavior throughout the application to provide better accessibility for screen reader users.

Key changes:

Introduces SafeRender method that outputs ANSI escape sequences instead of full re-rendering when screen reader support is enabled
Adds screen reader detection for Windows (including Narrator) and macOS VoiceOver
Replaces many Render() calls with SafeRender() calls across editing operations, undo/redo, history, and completion functionality

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
PSReadLine/Accessibility.cs	Enhanced screen reader detection for Windows and macOS platforms
PSReadLine/Cmdlets.cs	Added ScreenReader option with automatic detection as default
PSReadLine/Options.cs	Added logic to disable predictions when screen reader is enabled
PSReadLine/Render.cs	Implemented SafeRender method using ANSI control codes
PSReadLine/UndoRedo.cs	Replaced Render calls with SafeRender for undo/redo operations
PSReadLine/PublicAPI.cs	Updated Insert and Replace methods to use SafeRender
PSReadLine/BasicEditing.cs	Modified editing operations to use SafeRender with appropriate ANSI codes
PSReadLine/History.cs	Updated history navigation and search to use SafeRender
PSReadLine/KillYank.cs	Modified kill operations to use SafeRender
PSReadLine/Completion.cs	Added TODO for menu completion screen reader testing
PSReadLine/KeyBindings.cs	Added TODO for WhatIsKey screen reader evaluation
PSReadLine/ReadLine.cs	Added TODOs for screen reader evaluation in various render calls
PSReadLine/PlatformWindows.cs	Added IsMutexPresent method for detecting Windows Narrator

PSReadLine/Accessibility.cs

PSReadLine/History.cs

PSReadLine/Accessibility.cs

PSReadLine/PlatformWindows.cs

PSReadLine/Accessibility.cs

PSReadLine/Render.cs

daxian-dbw

Review in progress. Need to stop here today and want to share my comments so far.

PSReadLine/Accessibility.cs

PSReadLine/BasicEditing.cs

PSReadLine/Cmdlets.cs

That should default to enabled when one is detected on startup, but also allows the support to be forcibly enabled.

Borrows the "better" algorithm from Electron, with attribution.

Spawns a quick `defaults` process since in .NET using the macOS events is difficult, but this is quick and easy.

PSReadLine/Accessibility.cs

PSReadLine/Cmdlets.cs

PSReadLine/Render.cs

That algorithm doesn't work for a non-windowed app like PowerShell.

Because they'll be rendered and it's useless noise.

PSReadLine/Cmdlets.cs

PSReadLine/Render.cs

andyleejordan · 2025-08-18T21:08:36Z

test/ListPredictionTest.cs

@@ -1355,8 +1373,8 @@ public void List_PluginSource_Acceptance()
                        TokenClassification.Command, "ec",
                        NextLine,
                        TokenClassification.ListPrediction, "<-/3>",
-                        TokenClassification.None, new string(' ', listWidth - 42), // 42 is the length of '<-/3>' plus '<TestPredictor(2) LongNamePredic(1)>'.
-                        dimmedColors, "<TestPredictor(2) LongNamePredic(1)>",
+                        TokenClassification.None, new string(' ', listWidth - 42), // 42 is the length of '<-/3>' plus '<TestPredictor(2) LongNamePredic…(1)>'.


Yeah, so this is funny. I'm not sure what Unicode character was used in the original file, but due to some encoding issues it seems to need to be updated. Otherwise the tests fail after re-saving this file.

test/MockConsole.cs

test/OptionsTest.cs

andyleejordan · 2025-08-18T21:12:44Z

test/UnitTestReadLine.cs

+        {
+        }
+
+        internal override bool ScreenReaderModeEnabled => true;


The PSReadLine Option is specifically not set here in the construtor because the way the tests work, it needs to be set in TestSetup, hence this new method (which isn't a field because it's C# and needs to be virtual to be overridden, I guess).

tools/helper.psm1

andyleejordan · 2025-08-18T21:13:35Z

test/KillYankTest.cs

@@ -672,8 +672,12 @@ public void SelectCommandArgument_CLIArgs()
        [SkippableFact]
        public void SelectCommandArgument_HereStringArgs()
        {
+            Skip.If(ScreenReaderModeEnabled, "We're still investigating exactly why this test fails in screen reader mode.");


This was the only test I couldn't figure out, and decided not to block on it.

andyleejordan · 2025-08-19T18:11:38Z

When this is merged, please rebase or merge, not squash 🙇

PSReadLine/Render.cs

daxian-dbw · 2025-08-19T20:42:33Z

PSReadLine/Render.cs

+            {
+                // The cursor top exceeds the buffer height and it hasn't already wrapped,
+                // (because we have exactly filled the line) so we need to scroll up the buffer by 1 line.
+                if (point.X == 0 && !currentBuffer.EndsWith("\n"))


we have exactly filled the line

I think currentBuffer.EndsWith("\n") would always be false in this case. We may just call _console.Write("\n") without this check..

Hm this is tricky. There was a reason for this. I think it was the edge case that you've filled the line and then for the last character happened to insert a newline. The cursor then moved down to (0, +1) and we needed to not repeat inserting a newline.

In fact, this is the exact change we figured out debugging together in 14fc559

Tested this: yes that edge case still requires this check.

Got it. If there is a newline at the end of input, the scrolling up should've happened. That scenario should be handled in the if (endPoint.Y >= bufferHeight) block at line 355, but it's not.

I think we need to take care of that scenario there, by adding !currentBuffer.EndsWith("\n") to the if (endPoint.X == 0) check at line 360. Note that this edge case could happen with endPoint.Y > bufferHeight at line 355.

358 // We had to scroll to render everything, update _initialY. 359 int offset = 1; // Base case to handle zero-indexing. 360 if (endPoint.X == 0 && !currentBuffer.EndsWith("\n")) 361 { 362 // The line hasn't actually wrapped yet because we have exactly filled the line. 363 offset -= 1; 364 }

Then here, we can just call _console.Write("\n") without the check.

daxian-dbw · 2025-08-19T20:48:39Z

PSReadLine/Render.cs

+                    _console.Write("\x1b[0J");
+                } // Otherwise the previous buffer is a complete prefix and we just write.
+
+                var diffData = currentBuffer.Substring(commonPrefixLength);


Write directly from the current cursor position when commonPrefixLength == previousBuffer.Length may not work in some cases. One example:

Previous line is abcdef, cursor is at the letter d

Press ctrl+v to paste defghi

In this case, the new buffer is abcdefghidef, and commonPrefixLength == previousBuffer.Length, but the cursor is still at the first letter d when code flow reaches here. Then writting out the new diff ghidef will result in abcghidef displayed in terminal.

Oh I see what you mean. Thinking about how to check for this in a way that re-renders as little as possible...I think the correct condition here is actually commonPrefixLength == current? As in, "from where we're about to write new data, can we just append?" The bug being that it was the checking the common scenario of typing characters one-by-one, not having manually moved the cursor; but in that case, current == previousBuffer.Length so it'll operate the same.

When execution reaches here, _current is already changed -- it no longer reflects where the cursor is at. This uncommon scenario is tricky, and it potentially could affect the commonPrefixLength != previousBuffer.Length case.

I suggest we leave it aside for now. The current logic works well for the most common editing scenarios. Also, after integrating with VSCode terminal, it has another layer of accessibility feature to make it works as expected to end users.

We just need to be aware of this potential problem. I will open an issue after we merged this PR.

I'll leave a note about the known bug.

daxian-dbw · 2025-08-19T20:51:40Z

PSReadLine/Render.cs

+            // Preserve the current render data.
+            var renderData = new RenderData
+            {
+                lines = new RenderedLineData[] { new(currentBuffer, isFirstLogicalLine: true) },
+                errorPrompt = (_parseErrors != null && _parseErrors.Length > 0) // Not yet used.
+            };
+            _previousRender = renderData;


Can we move this down to right before the line 395 _previousRender.UpdateConsoleInfo(...)?
The operations around _previousRender should be grouped together.

It's where it is because ReallyRender specifically saves _previousRender before calculating the offsets, and this was written to be a simplified version of that function. Do you want it moved in both functions?

It looks safe to move _previousRender = renderData; in ReallyRender too.

test/MockConsole.cs

daxian-dbw · 2025-08-19T23:08:38Z

tools/helper.psm1

@@ -210,7 +210,7 @@ function Start-TestRun

    function RunXunitTestsInNewProcess ([string] $Layout, [string] $OperatingSystem)
    {
-        $filter = "FullyQualifiedName~Test.{0}_{1}" -f ($Layout -replace '-','_'), $OperatingSystem
+        $filter = "(FullyQualifiedName~Test.{0}_{1})|(FullyQualifiedName~Test.ScreenReader)" -f ($Layout -replace '-','_'), $OperatingSystem


This mixes the regular tests with the screen reader tests. It's better to separate them out so we have clear view of how many tests get run when screen reader is on and off.

I suggest the following changes in this function:

Suggested change

$filter = "(FullyQualifiedName~Test.{0}_{1})|(FullyQualifiedName~Test.ScreenReader)" -f ($Layout -replace '-','_'), $OperatingSystem

$filter = if ($Layout) {

"FullyQualifiedName~Test.{0}_{1}" -f ($Layout -replace '-','_'), $OperatingSystem

} else {

## Today, tests for screen-reader mode only run on Windows with the 'en-US' layout.

"FullyQualifiedName~Test.ScreenReader"

}

Then make the following corresponding changes:

in the if ($env:APPVEYOR -or $env:TF_BUILD) block, add Write-Log "Testing en-US ..." at the beginning;

at the end of if ($IsWindowsEnv) block, add the following
Write-Log "Testing screen reader mode ..." RunXunitTestsInNewProcess

Honestly I have some work-in-progress to remove all this layout logic so I took the simple approach here, but I can do this for now until that's in.

Even if the layout logic is removed, I think we will still want to separate the 2 test runs -- one for regular testing and one for screen reader mode testing. In this way, it's easy to view how many tests are run for those 2 scenarios respectively, what tests are skipped in each of them, and etc.

in the if ($env:APPVEYOR -or $env:TF_BUILD) block, add Write-Log "Testing en-US ..." at the beginning;

Can you also make this change please? It adds "Testing en-US ..." similarly to the highlighted caption in the screenshot below.

It would be even better if you can prepend a new line to both those 2 messages, so as to make them easier to be noticed.

I'm not sure I follow, this is just CI (that's pending replacement). Running them as separate instances honestly makes them even harder to follow in that log. With the way I did this using a separate fixture this is how they look:

We're spending a lot of time in this PR on a system we've already identified as non-compliant and needing to be updated in another.

Like you certainly could go update RunXunitTestsInNewProcess to have a log statement like you're desiring (instead of there sometimes being a log statement before it, and sometimes not, because old legacy code), but it'll get deleted as soon as we're done with this.

What I'm trying to say is, when running tests in CI or locally from command line, the test suites for <keyboard_layout>-Windows and ScreenReader should run separately, so that instead of a combined results output like in the following screenshot, we can view the specific result for each. It makes it easy to view the detailed numbers/status about each test run scenario.

We're spending a lot of time in this PR on a system we've already identified as non-compliant and needing to be updated in another.

I'm happy to see the CI and build get modernized. But before that happens, we are still making changes based on what we have today. Since we need to run RunXunitTestsInNewProcess twice now, having a caption like "Testing en-US ..." and Testing screen reader mode ... helps to identify what is what when looking at the CI output.

daxian-dbw · 2025-08-19T23:12:49Z

When this is merged, please rebase or merge, not squash 🙇

@andyleejordan Why not squash? The commit history is not that clean (has changes reverted) and also this repo always do squash merge (like in PowerShell repo).

andyleejordan · 2025-08-20T19:16:05Z

When this is merged, please rebase or merge, not squash 🙇

@andyleejordan Why not squash? The commit history is not that clean (has changes reverted) and also this repo always do squash merge (like in PowerShell repo).

Because one single commit is going to be way too big, and the commits are clean, where reverts/changes logically follow new information. (Such as testing the DLL-loaded approach for screen readers.)

PSReadLine/Render.cs

daxian-dbw · 2025-08-22T17:46:00Z

PSReadLine/Render.cs

+
+                // We had to scroll to render everything, update _initialY.
+                int offset = 1; // Base case to handle zero-indexing.
+                if (endPoint.X == 0)


As is discussed in the comment thread below: #4854 (comment)

Suggested change

if (endPoint.X == 0)

if (endPoint.X == 0 && !currentBuffer.EndsWith("\n"))

daxian-dbw · 2025-08-22T18:11:24Z

PSReadLine/Render.cs

+                    _console.Write("\x1b[0J");
+                } // Otherwise the previous buffer is a complete prefix and we just write.
+
+                var diffData = currentBuffer.Substring(commonPrefixLength);


When execution reaches here, _current is already changed -- it no longer reflects where the cursor is at. This uncommon scenario is tricky, and it potentially could affect the commonPrefixLength != previousBuffer.Length case.

I suggest we leave it aside for now. The current logic works well for the most common editing scenarios. Also, after integrating with VSCode terminal, it has another layer of accessibility feature to make it works as expected to end users.

We just need to be aware of this potential problem. I will open an issue after we merged this PR.

PSReadLine/Render.Helper.cs

And work around helper module's test filter.

Since many of these work in screen reader mode, just without the prompt (as if it were an empty string).

It does not handle added text like in incremental history search, but it does display correctly.

We don't need to disable predictions or the continuation prompt in the options, since with tests we can verify the few places we need to check.

Only a part of it (expectedly) fails under screen reader mode.

daxian-dbw · 2025-08-25T22:10:58Z

@andyleejordan can you please simply push new changes instead of force pushing all commits? It makes it hard for me to continue the view as I cannot view the diffs between a commit that I have reviewed and the subsequent new changes.

Because one single commit is going to be way too big, and the commits are clean, where reverts/changes logically follow new information. (Such as testing the DLL-loaded approach for screen readers.)

If the cost of keeping the commit history clean is to always force push all commits, then I'm afraid I disagree this is the way to go :) Since the PR already keeps all information, I think a squash merge should be fine.

andyleejordan force-pushed the screenreader branch 4 times, most recently from 8b8a080 to 9ba544b Compare July 23, 2025 17:56

andyleejordan changed the title ~~WIP: Prototype screen reader support~~ Prototype screen reader support Jul 23, 2025

andyleejordan marked this pull request as ready for review July 23, 2025 18:39

andyleejordan requested a review from Copilot July 23, 2025 18:58

Copilot AI reviewed Jul 23, 2025

View reviewed changes

daxian-dbw reviewed Jul 23, 2025

View reviewed changes

daxian-dbw reviewed Jul 25, 2025

View reviewed changes

PSReadLine/BasicEditing.cs Outdated Show resolved Hide resolved

PSReadLine/Cmdlets.cs Outdated Show resolved Hide resolved

andyleejordan force-pushed the screenreader branch 2 times, most recently from 175096b to d89f7ca Compare August 1, 2025 03:01

andyleejordan added 3 commits July 31, 2025 20:10

Add ScreenReader support CLI option

06c70c8

That should default to enabled when one is detected on startup, but also allows the support to be forcibly enabled.

Improve screen reader detection on Windows by checking modules

4baae08

Borrows the "better" algorithm from Electron, with attribution.

Check for VoiceOver screen reader on macOS

a6dc616

Spawns a quick `defaults` process since in .NET using the macOS events is difficult, but this is quick and easy.

andyleejordan force-pushed the screenreader branch 5 times, most recently from 0f93014 to 9090264 Compare August 5, 2025 18:28

daxian-dbw reviewed Aug 5, 2025

View reviewed changes

andyleejordan changed the title ~~Prototype screen reader support~~ Add screen reader support Aug 5, 2025

andyleejordan added Issue-Enhancement It's a feature request. Area-Accessibility Label for issues related to accessibility problems or improvements labels Aug 5, 2025

andyleejordan added 2 commits August 5, 2025 16:49

Revert to system parameter screen reader check on Windows

207e358

That algorithm doesn't work for a non-windowed app like PowerShell.

Disable predictions for screen readers

c38dbae

Because they'll be rendered and it's useless noise.

andyleejordan force-pushed the screenreader branch from 9090264 to 14fc559 Compare August 5, 2025 23:52

daxian-dbw reviewed Aug 15, 2025

View reviewed changes

PSReadLine/Cmdlets.cs Outdated Show resolved Hide resolved

andyleejordan force-pushed the screenreader branch 2 times, most recently from 4ec7dc2 to b153243 Compare August 15, 2025 22:51