Skip to content

Conversation

@jaycedowell
Copy link
Member

Summary

This PR completes the Python logging integration across LSL by:
1. Extending logging support to 6 additional modules with verbose output
2. Enhancing the logger API with console/file logging and filtering capabilities
3. Creating a full-featured Tkinter GUI for log monitoring
4. Improving code quality with f-string conversion and proper documentation
5. Converting runtime warnings to use the logging system

Core Logger API Enhancements (lsl/logger.py)

New Functions

  • enable_console_logging(level=None, stream=None) - Enable logging to stdout/stderr with optional level control
  • disable_console_logging() - Disable console output
  • enable_file_logging(filename, level=None, mode='a') - Enable logging to file with append/overwrite modes
  • disable_file_logging() - Close and disable file logging
  • add_filter(pattern) - Add module pattern filters (e.g., 'lsl.imaging.*')
  • remove_filter(pattern) - Remove specific filter
  • clear_filters() - Remove all active filters

Documentation Improvements

  • Added complete Parameters/Returns sections to all function docstrings
  • NumPy-style documentation throughout
  • Enhanced inline comments

New Logger GUI Module (lsl/logger_gui.py)

A production-ready Tkinter GUI for real-time log monitoring with:

Features

  • Real-time display with color-coded log levels (DEBUG=gray, INFO=black, WARNING=orange, ERROR/CRITICAL=red)
  • Logger level control - Adjustable dropdown that changes both logger level and display filtering
  • Console output toggle - Enable/disable stderr logging via checkbox
  • File logging - Save logs to file with file dialog
  • Pattern filtering - Module-based filtering with wildcards (e.g., 'lsl.sim.*', 'lsl_logger')
  • Buffer management - Auto-trim at 10,000 lines to prevent memory issues
  • Active filters display - Shows currently active filter patterns
  • ThreadedHandler integration - Captures logs from any thread

Components

  • LoggerFrame - Display widget with auto-scrolling and level-based hiding
  • FilterFrame - Control panel for all filtering/logging options
  • LoggerGUI - Complete standalone GUI window

Documentation

  • Comprehensive module docstring with usage examples
  • Detailed class docstrings with features and parameters
  • Interactive demo mode when run standalone (python -m lsl.logger_gui)

Module Logging Extensions

Extended the logging pattern from lsl.common.sdf to 6 additional modules:

lsl/imaging/selfcal.py

  • Converted all verbose output to pure logging
  • DEBUG level for iteration progress (convergence metrics)
  • INFO level for final calibration results
  • Removed conditional if verbose: checks (kept parameter for API compatibility)

lsl/sim/vis.py

  • Beam model selection notifications
  • Julian date progress tracking
  • Source visibility logging
  • Converted positional error warning from warnings.warn() to logging.warning()

lsl/sim/dp.py

  • TBN/DRX frame simulation progress
  • Sample rate reporting
  • Frame count updates

lsl/misc/scattering.py

  • Multi-path scattering results
  • Iteration counts and convergence metrics
  • Best-fit scattering time reporting

lsl/writer/uvfits.py & lsl/writer/fitsidi.py

  • Stand ID mapping notifications
  • Geometry configuration logging

All modules follow the backward-compatible pattern:

if verbose:
    print("message")
LSL_LOGGER.info("message")

Code Quality Improvements

String Formatting Modernization

  • Converted old % formatting to f-strings throughout updated modules
  • Improved readability and maintainability
    Examples:
    "Using %d-order" % deg → f"Using {deg}-order"
    "Best Gains: %s" % tempGains → f"Best Gains: {tempGains}"

Runtime Warnings Conversion

Converted appropriate warnings.warn() calls to logging.warning():

  • lsl/sim/vis.py - Positional error warnings
  • lsl/common/sdf.py - ASP filter degeneracy warnings
  • lsl/common/_sdf_utils.py - Timezone search mode warnings

Retained warnings.warn() for API-level warnings (deprecation, usage issues).

claude and others added 17 commits November 7, 2025 18:25
Convert verbose-controlled print statements to use LSL_LOGGER across
multiple modules while maintaining backward compatibility. The pattern
used is:
- Print to console when verbose=True (backward compatible)
- Always send to LSL_LOGGER (for GUI log capture)

Modules converted:
- lsl/imaging/selfcal.py: Self-calibration iteration progress and results
- lsl/sim/vis.py: Beam model selection, JD setting, source visibility
- lsl/sim/dp.py: Frame simulation progress for TBN/DRX
- lsl/misc/scattering.py: Scattering deconvolution results
- lsl/writer/uvfits.py: Stand ID mapping notifications
- lsl/writer/fitsidi.py: Stand ID mapping notifications

This complements the existing logger support in lsl.common.sdf and
lsl.catalog, enabling error messages and status updates to be captured
by GUI applications.
Replace % formatting and .format() calls with f-strings for improved
readability in logger-enabled modules:

- lsl/imaging/selfcal.py: Iteration progress messages
- lsl/sim/vis.py: Beam model and array creation messages
- lsl/sim/dp.py: Frame simulation progress messages
- lsl/misc/scattering.py: Scattering results output
- lsl/writer/uvfits.py: Stand mapping messages
- lsl/writer/fitsidi.py: Stand mapping messages
Remove verbose parameter handling from selfcal module and convert all
logging to DEBUG level. These messages (iteration progress, convergence
metrics, final results) are primarily useful for debugging/development
rather than normal operation.

Changes:
- Remove all 'if verbose:' checks and print() calls
- Convert all logging from INFO/DEBUG to DEBUG only
- Keep verbose parameter in signatures for backward compatibility
- All messages now logged via LSL_LOGGER.debug()

Users who want to see this output can enable DEBUG level logging.
The final calibration results (Best Gains, Best Delays, Best Phase
Offsets) are actual output, not debug information. Changed from DEBUG
to INFO level so they appear in normal logging.

Iteration progress remains at DEBUG level.
Replace warnings.warn() with LSL_LOGGER.warning() for operational/runtime
conditions rather than API deprecations:

- lsl/sim/vis.py: Positional error notification during array creation
- lsl/common/sdf.py: ASP filter degeneracy validation message
- lsl/common/_sdf_utils.py: Timezone fallback search notification

These are runtime operational messages, not code deprecation warnings, so
they fit better with the logging system where users can control them via
log levels.

API usage warnings (like the STEPPED observation duration setter) remain
as warnings.warn() since they indicate code that should be fixed.
Extends logger API with:
- enable_console_logging/disable_console_logging for stdout/stderr output
- enable_file_logging/disable_file_logging for file output
- add_filter/remove_filter/clear_filters for pattern-based filtering
Complete NumPy-style documentation for all public functions:
- set_log_level: Added Parameters section
- get_log_level: Added Returns section
- add_handler: Added Parameters section
- remove_handler: Added Parameters section
- capture_warnings: Added Parameters section
New features:
- File logging: Start/stop logging to file with file dialog
- Console output toggle: Enable/disable console logging via checkbox
- Pattern filtering: Add/remove module pattern filters (e.g., 'lsl.imaging.*')
- Buffer management: Auto-trim to 10,000 lines to prevent unbounded growth
- Improved level control: Changes both logger level and display filtering
- Better level mapping: Use proper dict mapping instead of arithmetic
- Robust signal handling: Wrapped in try/except for cross-platform compatibility
- Active filters display: Shows currently active filter patterns

The GUI now leverages all the new logger.py API features including
enable_console_logging(), enable_file_logging(), and add_filter().
- Added module-level docstring with integration examples showing:
  * Basic blocking GUI usage
  * Non-blocking threaded GUI usage
  * Using individual components
- Enhanced LoggerGUI class docstring with features list and examples
- Replaced trivial __main__ with interactive demo mode that:
  * Generates example log messages at all levels
  * Provides instructions for testing GUI features
  * Runs in background thread to demonstrate real-time updates

This clarifies that logger_gui.py is meant to be imported into LSL
scripts, not run standalone for production use.
test_logger.py (16 tests):
- Test set/get log levels
- Test ThreadedHandler queue integration
- Test console logging enable/disable with optional level
- Test file logging enable/disable with append/write modes
- Test module pattern filtering (add/remove/clear)
- Test filter functionality with actual log records
- Test warning capture integration
- Test handler add/remove with formatters

test_logger_gui.py (11 tests):
- Test LoggerFrame creation and display
- Test buffer management with max_lines limit
- Test clear buffer functionality
- Test show_at_level filtering
- Test FilterFrame level changes
- Test console output toggle
- Test pattern filter add/clear
- Test complete LoggerGUI creation and messaging
- All GUI tests skip gracefully when Tkinter unavailable or no display

Both test files use unittest.skipIf decorators for headless CI compatibility.
Tests verified locally with 100% pass rate (16 OK, 11 skipped in headless env).
When enable_file_logging() is called while already logging to a file,
it now generates a RuntimeWarning before closing the existing file and
switching to the new one. This prevents silent file switching that could
confuse users.

Also added test_file_logging_switch_warning to verify the warning is
generated correctly (brings total logger tests to 17).
@codecov
Copy link

codecov bot commented Nov 24, 2025

Codecov Report

❌ Patch coverage is 84.29003% with 52 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.25%. Comparing base (e394c1d) to head (24cb267).

Files with missing lines Patch % Lines
lsl/logger_gui.py 87.50% 24 Missing ⚠️
lsl/sim/dp.py 70.00% 6 Missing ⚠️
lsl/writer/uvfits.py 33.33% 6 Missing ⚠️
lsl/sim/vis.py 70.58% 5 Missing ⚠️
lsl/imaging/selfcal.py 80.00% 4 Missing ⚠️
lsl/misc/scattering.py 66.66% 3 Missing ⚠️
lsl/writer/fitsidi.py 77.77% 2 Missing ⚠️
lsl/common/sdf.py 0.00% 1 Missing ⚠️
lsl/logger.py 98.11% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           logger     #103      +/-   ##
==========================================
+ Coverage   84.11%   84.25%   +0.13%     
==========================================
  Files          92       93       +1     
  Lines       24443    24726     +283     
==========================================
+ Hits        20561    20832     +271     
- Misses       3882     3894      +12     
Files with missing lines Coverage Δ
lsl/common/_sdf_utils.py 91.47% <100.00%> (ø)
lsl/common/sdf.py 87.23% <0.00%> (ø)
lsl/logger.py 96.62% <98.11%> (+24.40%) ⬆️
lsl/writer/fitsidi.py 87.10% <77.77%> (+0.34%) ⬆️
lsl/misc/scattering.py 95.14% <66.66%> (+0.30%) ⬆️
lsl/imaging/selfcal.py 93.05% <80.00%> (+2.64%) ⬆️
lsl/sim/vis.py 85.10% <70.58%> (+0.15%) ⬆️
lsl/sim/dp.py 93.85% <70.00%> (+0.40%) ⬆️
lsl/writer/uvfits.py 86.06% <33.33%> (-0.66%) ⬇️
lsl/logger_gui.py 87.50% <87.50%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants