fix: skip token/dataset validation in debug mode #176
Conversation
commit: |
There was a problem hiding this comment.
Pull request overview
This PR enhances offline development workflow by skipping validation of eval.token and eval.dataset when running in debug mode (--debug flag), allowing developers to work without network connectivity or Axiom environment variables. Additionally, it standardizes variable naming from DEBUG to isDebug throughout the codebase for consistency with boolean naming conventions.
- Wraps token/dataset validation in debug mode check to enable offline usage
- Standardizes debug variable naming from
DEBUGtoisDebugacross multiple files - Removes unused debug-specific config printing logic from reporter
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/ai/src/config/index.ts | Wraps validation of eval.token, eval.dataset, and eval.url in debug mode check to skip when AXIOM_DEBUG='true' |
| packages/ai/src/otel/withSpan.ts | Renames DEBUG variable to isDebug for consistency |
| packages/ai/src/otel/utils/wrapperUtils.ts | Renames DEBUG variable to isDebug for consistency |
| packages/ai/src/otel/initAxiomAI.ts | Renames DEBUG variable to isDebug for consistency |
| packages/ai/src/evals/reporter.ts | Removes unused printConfigEnd import and debug config printing logic; renames DEBUG to isDebug |
| packages/ai/src/evals/eval.ts | Removes duplicate local DEBUG declaration, uses existing isDebug from outer scope |
| packages/ai/src/cli/commands/eval.command.ts | Updates help text for --debug flag to clarify it prevents all network operations |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| */ | ||
| export function validateConfig(config: Partial<AxiomConfigBase>): ResolvedAxiomConfig { | ||
| const errors: string[] = []; | ||
| const debug = process.env.AXIOM_DEBUG === 'true'; |
There was a problem hiding this comment.
Variable naming inconsistency: This variable is named debug while the PR standardizes to isDebug elsewhere. For consistency, this should be const isDebug = process.env.AXIOM_DEBUG === 'true'; to match the naming convention used throughout the rest of the codebase.
| printOrphanedBaselineCases, | ||
| getCaseFingerprint, | ||
| printCaseResult, | ||
| printConfigEnd, |
There was a problem hiding this comment.
Should we remove printConfigEnd altogether? Seems like it's not used elsewhere
2 participants
When running with
--debug, skip validation ofeval.tokenandeval.datasetto allow fully offline usage without any network / axiom env var requirements. Also standardizesDEBUGvariable naming toisDebugacross the codebase.