Skip to content

Clicking on region annotations does not synchronize with other synced components (plot and sidebar) #8603

@cloudmark

Description

@cloudmark

Describe the bug

When clicking on a region annotation (TimeSeries, Audio, etc.) either directly on the plot OR in the Regions sidebar/outliner, the view scrolls to show the region but does NOT synchronize the playback position with other synced media components. This breaks the expected behavior where all components in the same sync group should seek to the same time position when any region is selected.

Current behavior:

  • Click on ANY region annotation (TimeSeries, Audio - on the plot OR in the Regions sidebar) → Only that component's view adjusts
  • Other synced components stay at current position
  • Audio player doesn't seek to that time (if clicking on TimeSeries region)
  • TimeSeries doesn't update (if clicking on Audio region)
  • No synchronization across components

Expected behavior:

  • Click on any region annotation (on the plot OR in the Regions sidebar) → ALL synced components seek to the region's start time
  • Audio player seeks to that time
  • All TimeSeries visualizations update to show that time point

To Reproduce

Steps to reproduce the behavior:

  1. Create a labeling task with synchronized components:

    <View>
      <Audio name="audio" value="$audio" sync="sync1"/>
      <TimeSeries name="accel" value="$timeseries" sync="sync1">
        <Channel column="acceleration"/>
      </TimeSeries>
      <TimeSeriesLabels name="label" toName="accel">
        <Label value="Walking"/>
        <Label value="Running"/>
      </TimeSeriesLabels>
      <Labels name="audiolabel" toName="audio">
        <Label value="Speech"/>
        <Label value="Music"/>
      </Labels>
    </View>
  2. Load a task with synchronized audio and accelerometer data

  3. Create region annotations on ANY component (e.g., label a TimeSeries segment as "Walking" or an Audio segment as "Speech")

  4. Method 1: Click on the region directly on the visualization (TimeSeries or Audio)

    • OR Method 2: Click on the region in the Regions sidebar (outliner panel)
  5. Observe that:

    • The clicked component's view scrolls to show the region ✅
    • Other synced components do NOT seek to that time ❌
    • No synchronization happens across components ❌

Expected: All synced components should seek to the region's start time simultaneously, regardless of which region is clicked (TimeSeries, Audio) or where it's clicked (plot or Regions sidebar)


Expected behavior

When clicking on ANY region annotation (TimeSeries, Audio - either on the visualization or in the Regions sidebar/outliner), the system should:

  1. Scroll that component's view to show the region (current behavior - works)
  2. Update the cursor/playhead to the region's start time
  3. Emit a sync event so all components in the same sync group seek to that time
  4. Ensure Audio and all TimeSeries visualizations synchronize to the region's time position

This creates a cohesive experience where clicking any region anywhere (any component, plot or sidebar) immediately shows the user what was happening across ALL modalities at that moment in time.


Screenshots

Video demonstration: https://www.loom.com/share/2584716884114820a7e4b5b76a3316b3

The video demonstrates:

  • Clicking on region annotations (both on the plot and in the Regions sidebar)
  • Only the clicked component's view adjusts
  • Other synced components (Audio, TimeSeries) do NOT sync to that time
  • Expected: all synced components should seek to the region's time

Environment (please complete the following information)

  • OS: macOS (Darwin 25.0.0)
  • Label Studio Version: develop branch
  • Component: Region annotations (TimeSeries, Audio) with synchronization
  • Feature Flag: FF_TIMESERIES_SYNC must be enabled
  • Affected UI: Both direct region clicks on plot and Regions sidebar/outliner clicks

Additional context

Root Cause

The selectRegion() method in region models (TimeSeriesRegion, AudioRegion) is called whenever a region is selected (either by clicking on it directly in the visualization or by clicking on it in the Regions sidebar/outliner). However, these methods only adjust the view (scroll, highlight) but do not:

  1. Update the cursor position
  2. Emit a sync event (syncSend()) to notify other components

This is inconsistent with how clicking directly on empty plot areas works, which DOES emit sync events.

Affected Components

  • Primary: Region types
    • TimeSeriesRegion (accelerometer, velocity, any TimeSeries data)
    • AudioRegion (audio annotations)
  • Interaction Points:
    • Direct clicks on regions in any visualization
    • Clicks on regions in the Regions sidebar/outliner
  • Secondary: All synced components don't receive the seek event when regions are clicked

Impact

  • Critical: Breaks multi-modal data exploration workflows
  • High: Users cannot easily navigate synchronized data by clicking on region annotations
  • High: Affects both primary interaction methods (visualization + Regions sidebar)
  • Medium: Inconsistent behavior between clicking empty plot vs clicking on regions
  • Medium: Affects all multi-modal annotation tasks with synchronized media types

Related Issues

  • This complements the floating-point precision fix for seek events
  • Affects workflows involving multi-sensor data annotation (IoT, wearables, autonomous vehicles)
  • Related to TimeSeries synchronization with Audio/Video players
  • Impacts both primary user interaction methods (visualization and Regions sidebar navigation)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions