Skip to content

Conversation

Copy link

Copilot AI commented Sep 12, 2025

This PR implements optional database persistence for telemetry data using SQLite and MikroORM, addressing the need for persistent storage of telemetry values and schema information.

Key Features

Opt-in Database Configuration
The telemetry persistence is completely optional and maintains full backward compatibility. Users can enable it by configuring the dev resource:

dev.with({
  port: 1337,
  maxEntries: 1000,
  database: {
    driver: "sqlite", 
    options: {
      filePath: "./telemetry.db"
    }
  }
})

Schema Persistence
The implementation saves and persists the complete introspector schema to the database. This enables runner instances to connect to an existing database and be fully self-sufficient - the GraphQL server can read the schema directly from the database without needing custom schema definitions.

Complete Telemetry Storage
All telemetry data types are persisted with full feature parity:

  • Log entries with levels, messages, and correlation data
  • Event emissions with payloads and emitter information
  • Error records with stack traces and source details
  • Task/hook execution records with timing and success metrics

MikroORM Integration
Uses the latest MikroORM v5.9.8 with proper entity definitions, automatic schema generation, and efficient querying. The database layer includes:

  • Indexed entities for optimal query performance
  • JSON column support for complex data structures
  • Automatic database schema creation and updates

API Compatibility
All existing GraphQL queries work identically regardless of storage backend. The Live interface remains unchanged, ensuring existing code continues to work without modifications.

Technical Implementation

  • Database Service: Manages MikroORM initialization and connection lifecycle
  • Database Live: Implements the Live interface with SQLite persistence
  • Schema Service: Handles introspector schema persistence and retrieval
  • Entity Models: Complete data models for all telemetry types with proper indexing
  • Async Operations: Database operations are async with synchronous compatibility layer

Testing

Comprehensive test suite covering:

  • Database initialization and schema creation
  • CRUD operations for all telemetry data types
  • Query filtering and pagination
  • Error handling and edge cases
  • Integration with the dev resource configuration

Documentation

Updated README with:

  • Configuration examples showing database setup
  • Benefits and use cases for persistence
  • Schema information and table descriptions
  • Backward compatibility guarantees
  • GraphQL query examples

The implementation allows applications to scale beyond memory limits while preserving all existing functionality and maintaining the same developer experience.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Provide a way to persist the telemetry values. And provide a strategy to work with sqlite first via an mikroorm adapter. Use latest version of mikroorm to properly handle data models. The goal is to have this opt-in. Either via dev.with({ database: { ... Add SQLite database persistence for telemetry data with MikroORM integration Sep 12, 2025
Copilot finished work on behalf of theodorDiaconu September 12, 2025 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants