WIP: feat(web): Web Audio support for RDP web clients #904

gabrielbauman · 2025-07-31T19:08:16Z

Enables RDP web clients to receive and play PCM audio from remote sessions through the browser's native Web Audio API, supporting various sample rates.

Highlights:

Web Audio API backend with AudioContext management and sample buffering
PCM sample rate conversion
Extension helpers for web client session integration
Error handling for audio context creation and playback failures

Enables RDP web clients to receive and play PCM audio from remote sessions through the browser's native Web Audio API, supporting various sample rates. Highlights: - Web Audio API backend with AudioContext management and sample buffering - PCM sample rate conversion - Extension helpers for web client session integration - Error handling for audio context creation and playback failures

CBenoit · 2025-08-01T05:33:40Z

crates/ironrdp-web/src/session.rs

@@ -231,6 +241,12 @@ impl iron_remote_desktop::SessionBuilder for SessionBuilder {
                };
                self.0.borrow_mut().outbound_message_size_limit = if limit > 0 { Some(limit) } else { None };
            };
+            |enable_audio: bool| { self.0.borrow_mut().enable_audio = enable_audio };
+            |audio_sample_rate: f64| {
+                #[expect(clippy::cast_possible_truncation)] // JavaScript numbers are f64, audio uses f32


praise: Thank you for also including the reason of the suppression.

CBenoit · 2025-08-01T05:36:52Z

crates/ironrdp-web/src/session.rs

@@ -977,6 +1013,13 @@ async fn connect(
        );
    }

+    if enable_audio {
+        debug!("Enabling audio with sample rate: {:?}", audio_sample_rate);


style: It’s not bad per se, but the general preference goes for structured logging, like so

Suggested change

debug!("Enabling audio with sample rate: {:?}", audio_sample_rate);

debug!(audio_sample_rate, "Enabling audio");

It’s generally more concise while maintaining the same density of information.
Also, in theory, we could also parse the logs since it’s then structured.

CBenoit · 2025-08-01T05:37:53Z

crates/ironrdp-web/src/session.rs

+    if enable_audio {
+        debug!("Enabling audio with sample rate: {:?}", audio_sample_rate);
+        let audio_backend = WebAudioBackend::new(audio_sample_rate)
+            .map_err(|e| anyhow::Error::msg(format!("failed to initialize Web Audio backend: {e:?}")))?;


suggestion: You can use the context method provided by the anyhow::Context trait.

Suggested change

.map_err(|e| anyhow::Error::msg(format!("failed to initialize Web Audio backend: {e:?}")))?;

.context("failed to initialize Web Audio backend)?;

I’ve spotted similar patterns in the audio module as well, you may want to double check.

CBenoit · 2025-08-01T05:39:37Z

crates/ironrdp-web/src/session.rs

@@ -889,7 +921,7 @@ fn build_config(
        platform: ironrdp::pdu::rdp::capability_sets::MajorPlatformType::UNSPECIFIED,
        no_server_pointer: false,
        autologon: false,
-        no_audio_playback: true,
+        no_audio_playback: !enable_audio,


thought: For a follow up PR, no_audio_playback should be changed to enable_audio or enable_audio_playback, as I believe its a better naming.

Done in #907

CBenoit · 2025-08-01T05:40:38Z

web-client/iron-remote-desktop-rdp/src/main.ts

+/**
+ * Enable or disable audio playback for the RDP session.
+ * 
+ * When enabled, the client will negotiate audio capabilities with the server
+ * and attempt to play PCM audio through the browser's Web Audio API.
+ * 
+ * Requirements:
+ * - Modern browsers with Web Audio API support (Chrome 14+, Firefox 25+, Safari 6+)
+ * - User gesture activation (click, touch, or keypress) required by browser security policy
+ * 
+ * @param enable - Whether to enable audio playback
+ * @returns Extension for audio enablement
+ */
+export function enableAudio(enable: boolean): Extension {
+    return new Extension('enable_audio', enable);
+}
+
+/**
+ * Set the preferred sample rate for audio format negotiation.
+ * 
+ * This influences which PCM format the server is likely to choose by placing
+ * the specified sample rate first in the client's advertised format list.
+ * The implementation automatically handles sample rate conversion if the server
+ * chooses a different rate, so this is primarily an optimization.
+ * 
+ * Common sample rates:
+ * - 22050 Hz - Lower bandwidth, suitable for voice
+ * - 44100 Hz - CD quality
+ * - 48000 Hz - Professional audio, often browser native
+ * 
+ * If not specified, the browser's native sample rate is used as the preference.
+ * 
+ * @param rate - Preferred sample rate in Hz (e.g., 48000 for 48kHz)
+ * @returns Extension for sample rate preference
+ */
+export function audioSampleRate(rate: number): Extension {
+    return new Extension('audio_sample_rate', rate);


praise: Many thanks for all the documentation!

CBenoit · 2025-08-01T05:41:59Z

crates/ironrdp-web/src/error.rs

+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        f.debug_struct("IronError").field("source", &self.source).finish()
+    }
+}


question: Why was it necessary to implement Debug on IronError? The main purpose of IronError is WASM/JavaScript interop, internally we don’t really use it like the usual Rust idiomatic errors.

I take your point.

CBenoit · 2025-08-01T05:43:17Z