Skip to content

Conversation

@JorisHeadease
Copy link
Contributor

Add distributed tracing with OTLP HTTP exporter for observability.

Components instrumented:

  • HTTP server (Echo middleware)
  • HTTP clients (outgoing requests)
  • GORM database queries
  • HashiCorp Vault client requests
  • External crypto storage client

Features:

  • W3C Trace Context propagation (traceparent headers)
  • Logs enriched with trace_id/span_id for correlation
  • Logs forwarded to OTLP endpoint when tracing enabled
  • Audit logs included in trace context

Configuration:

  • tracing.endpoint: OTLP collector endpoint (host:port)
  • tracing.insecure: use HTTP instead of HTTPS

Known limitations:

  • gRPC connections not instrumented (v5 legacy functionality)
  • Azure Key Vault uses Azure SDK which requires separate instrumentation via azotel package

Add distributed tracing with OTLP HTTP exporter for observability.

Components instrumented:
- HTTP server (Echo middleware)
- HTTP clients (outgoing requests)
- GORM database queries
- HashiCorp Vault client requests
- External crypto storage client

Features:
- W3C Trace Context propagation (traceparent headers)
- Logs enriched with trace_id/span_id for correlation
- Logs forwarded to OTLP endpoint when tracing enabled
- Audit logs included in trace context

Configuration:
- tracing.endpoint: OTLP collector endpoint (host:port)
- tracing.insecure: use HTTP instead of HTTPS

Known limitations:
- gRPC connections not instrumented (v5 legacy functionality)
- Azure Key Vault uses Azure SDK which requires separate
  instrumentation via azotel package
@qltysh
Copy link

qltysh bot commented Dec 3, 2025

Diff Coverage: The code coverage on the diff in this pull request is 32.0%.

Total Coverage: This PR will decrease coverage by 0.4%.

File Coverage Changes
PathFile Coverage ΔIndirect
audit/audit.go-1.5
core/engine.go-2.5
core/http_client.go-4.0
core/server_config.go0.2
core/tracing.go15.5
crypto/storage/external/client.go-2.5
crypto/storage/vault/vault.go-2.7
http/client/client.go-2.9
http/engine.go-3.5
network/network.go-0.3
network/transport/grpc/connection.go3.7
pki/validator.go-0.8
storage/engine.go-0.6
🛟 Help
  • Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.

  • Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.

  • File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)

    • Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds comprehensive OpenTelemetry tracing support to the Nuts node, enabling distributed tracing across HTTP requests, database operations, and external service calls. The implementation uses OTLP HTTP exporters for both traces and logs, with W3C Trace Context propagation.

Key changes:

  • Adds core tracing infrastructure with configurable OTLP endpoint and automatic instrumentation
  • Instruments all HTTP clients and servers, database queries (GORM), and external storage backends
  • Enriches logs with trace context (trace_id/span_id) and forwards logs to OTLP endpoint when enabled

Reviewed changes

Copilot reviewed 17 out of 18 changed files in this pull request and generated no comments.

Show a summary per file
File Description
core/tracing.go Core tracing setup with OpenTelemetry SDK, trace/log exporters, and logrus hooks for trace correlation
core/tracing_test.go Unit tests for tracing initialization and logrus hook functionality
core/engine.go Integrates tracing setup early in system configuration and ensures proper shutdown
core/server_config.go Adds tracing configuration structure with endpoint and insecure TLS options
core/http_client.go Wraps internal HTTP client transport with OpenTelemetry instrumentation
http/engine.go Adds Echo middleware for tracing inbound HTTP requests (skips health/metrics/status)
http/client/client.go Wraps all HTTP client transports with OpenTelemetry instrumentation
http/client/client_test.go Tests for tracing-enabled and disabled HTTP client behavior
storage/engine.go Adds GORM tracing plugin when tracing is enabled
crypto/storage/vault/vault.go Instruments Vault HTTP client with OpenTelemetry transport
crypto/storage/external/client.go Instruments external crypto storage HTTP client
pki/validator.go Instruments PKI validator HTTP client for certificate revocation checks
audit/audit.go Registers tracing hooks with audit logger to include audit logs in traces
cmd/root.go Updates comment to reflect tracing initialization in Configure()
docs/pages/deployment/monitoring.rst Comprehensive documentation for tracing configuration and capabilities
docs/pages/deployment/server_options.rst Documents new tracing configuration options
go.mod Adds OpenTelemetry dependencies and updates test/protobuf versions
go.sum Checksum updates for new and updated dependencies

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Pass request context to log calls so trace_id and span_id are
included in log output when tracing is enabled.
@JorisHeadease JorisHeadease marked this pull request as ready for review December 3, 2025 15:59
When embedded in another app, nuts-node's spans were attributed to the
host's service name because both shared the global TracerProvider.

Add GetTracerProvider() that returns nuts-node's own provider, and use
it in otelecho middleware. Only set global provider when not embedded.
@qltysh
Copy link

qltysh bot commented Dec 4, 2025

1 new issue

Tool Category Rule Count
qlty Structure Function with many returns (count = 7): SetupTracing 1

JorisHeadease added a commit to nuts-foundation/nuts-knooppunt that referenced this pull request Dec 4, 2025
Pass tracing configuration (OTLP endpoint, insecure flag) from knooppunt
to the embedded nuts-node via environment variables. Add otelhttp transport
to the reverse proxy for proper trace context propagation across the proxy
boundary.

Depends on nuts-foundation/nuts-node#3946
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants