Skip to content

comet-ml/cometx

BranchesTags

Repository files navigation

cometx

   _________  ____ ___  ___  / /__  __
  / ___/ __ \/ __ `__ \/ _ \/ __/ |/_/
 / /__/ /_/ / / / / / /  __/ /__>  <
 \___/\____/_/ /_/ /_/\___/\__/_/|_|

Open source extensions for the Comet SDK.

These extensions are created and supported by the community and are not an official project of Comet ML. We welcome contributions!

Installation

pip install cometx --upgrade

To use these command-line functions, you can set your Comet API key and URL override using command-line flags:

cometx --api-key="YOUR-COMET-API-KEY" COMMAND

If you are a Comet on-prem user, and your installation does not use smart-keys, you'll also need to set the URL override:

cometx --api-key="YOUR-COMET-API-KEY" --url-override="https://your-companys-comet.com/clientlib/" COMMAND

Usage

cometx is composed of a series of commands that are useful independently, and can be used together to create sophisticated tools for ML management.

Commands

For all commands, use the --help flag to get additional information.

Global Options

These flags are availble before a command:

  • --api-key API_KEY - Set the COMET_API_KEY
  • --url-override URL_OVERRIDE - Set the COMET_URL_OVERRIDE

This command shows the cometx version:

  • --version - Display comet_ml version

This command can be used globally, or for individual commands:

  • -h, --help - Show help message

cometx list

This command is used to:

  • get a list of all workspaces that you are a member of
  • get a list of all projects in a workspace
  • get a list of all experiments (by name or key) in a project

cometx list examples:

cometx list WORKSPACE/PROJECT/EXPERIMENT-KEY-OR-NAME
cometx list WORKSPACE/PROJECT
cometx list WORKSPACE
cometx list

Flags

  • -u, --use-name - Use experiment names for experiment folders and listings
  • --query QUERY - Only list experiments that match this Comet query string
  • --debug - Provide debug info

For more information, cometx list --help

cometx count

This command is used to:

  • count the number of workspaces you are a member of
  • count the number of projects in workspaces
  • count the number of experiments in projects
  • count the number of artifacts in workspaces

cometx count examples:

cometx count
cometx count --workspaces-only
cometx count --with-projects
cometx count --with-experiments
cometx count --count-all
cometx count --limit 10

Flags

  • --workspaces-only - Count only workspaces (fastest)
  • --with-projects - Count workspaces and projects (default)
  • --with-experiments - Count workspaces, projects, and experiments (slowest, most detailed)
  • --count-all - Count everything: workspaces, projects, artifacts, and experiments (most comprehensive)
  • --limit LIMIT - Process only the first N workspaces (useful for testing)
  • --debug - Provide debug info

For more information, cometx count --help

cometx download

This command is used to:

  • download all workspaces, projects, and experiments of workspaces that you are a member of
  • download all projects, and experiments of a given workspace
  • download all experiments of a given workspace/project
  • download artifacts and models from the registry
  • download panels

Note: For detailed information on copying experiments from one Comet installation to another, see MIGRATIONS.md.

cometx download examples:

cometx download WORKSPACE/PROJECT/EXPERIMENT-KEY-OR-NAME [RESOURCE ...] [FLAGS ...]
cometx download WORKSPACE/PROJECT [RESOURCE ...] [FLAGS ...]
cometx download WORKSPACE [RESOURCE ...] [FLAGS ...]
cometx download [RESOURCE ...] [FLAGS ...]

Where [RESOURCE ...] is zero or more of the following names:

  • run - alias for: code, git, output, graph, and requirements
  • system
  • others
  • parameters
  • metadata
  • metrics
  • assets
  • html
  • project - alias for: project_notes, project_metadata

If no RESOURCE is given it will download all of them.

Flags

  • --from from - Source of data to copy. Should be: comet, wandb, or neptune
  • -i IGNORE [IGNORE ...], --ignore IGNORE [IGNORE ...] - Resource(s) (or 'experiments') to ignore
  • -j PARALLEL, --parallel PARALLEL - The number of threads to use for parallel downloading; default (None) is based on CPUs
  • -o OUTPUT, --output OUTPUT - Output directory for downloads
  • -u, --use-name - Use experiment names for experiment folders and listings
  • -l, --list - List the items at this level (workspace, project, experiment, artifacts, or model-registry) rather than download
  • --flat - Download the files without subfolders
  • -f, --ask - Queries the user; if flag not included system will answer yes for all queries
  • --filename FILENAME - Only get resources ending with this
  • --query QUERY - Only download experiments that match this Comet query string
  • --asset-type ASSET_TYPE - Only get assets with this type
  • --sync SYNC - What level to sync at: all, experiment, project, or workspace
  • --debug - Provide debug info

To download artifacts:

cometx download WORKSPACE/artifacts/NAME [FLAGS ...]
cometx download WORKSPACE/artifacts/NAME/VERSION-OR-ALIAS [FLAGS ...]

To download models from the model registry:

cometx download WORKSPACE/model-registry/NAME [FLAGS ...]
cometx download WORKSPACE/model-registry/NAME/VERSION-OR-STAGE [FLAGS ...]

To download panels:

cometx download WORKSPACE/panels/NAME-OR-ID [FLAGS ...]
cometx download WORKSPACE/panels [FLAGS ...]

For more information, cometx download --help

cometx copy

This command is used to:

  • copy downloaded data to a new experiment
  • create a symlink from one project to existing experiments
  • copy panels

Note: For detailed information on copying experiments from one Comet installation to another, see MIGRATIONS.md.

cometx copy examples:

cometx copy SOURCE DESTINATION
cometx copy --symlink SOURCE DESTINATION
cometx copy --path /base/path SOURCE DESTINATION
cometx copy --path ~/Downloads SOURCE DESTINATION

where SOURCE is:

  • if not --symlink, "WORKSPACE/PROJECT/EXPERIMENT", "WORKSPACE/PROJECT", or "WORKSPACE" folder
  • if --symlink, then it is a Comet path to workspace or workspace/project
  • "WORKSPACE/panels" or "WORKSPACE/panels/PANEL-ZIP-FILENAME" to copy panels

where DESTINATION is:

  • WORKSPACE
  • WORKSPACE/PROJECT

Not all combinations are possible:

Destination →
Source ↓		 WORKSPACE WORKSPACE/PROJECT
WORKSPACE/*/* Copies all projects N/A
WORKSPACE/PROJ/* N/A Copies all experiments
WORKSPACE/PROJ/EXP N/A Copies experiment

Asset Types

  • 3d-image
  • 3d-points - deprecated
  • audio
  • confusion-matrix - may contain assets
  • curve
  • dataframe
  • dataframe-profile
  • datagrid
  • embeddings - may reference image asset
  • histogram2d - not used
  • histogram3d - internal only, single histogram, partial logging
  • histogram_combined_3d
  • image
  • llm_data
  • model-element
  • notebook
  • source_code
  • tensorflow-model-graph-text - not used
  • text-sample
  • video

Flags

  • -i IGNORE [IGNORE ...], --ignore IGNORE [IGNORE ...] - Resource(s) (or 'experiments') to ignore
  • -j PARALLEL, --parallel PARALLEL - The number of threads to use for parallel uploading; default (None) is based on CPUs
  • --debug - If given, allow debugging
  • --quiet - If given, don't display update info
  • --symlink - Instead of copying, create a link to an experiment in a project
  • --sync - Check to see if experiment name has been created first; if so, skip
  • --path PATH - Path to prepend to workspace_src when accessing files (supports ~ for home directory)

Using --path

The --path option allows you to specify a base directory where your workspace folders are located. This is useful when your downloaded experiments are stored in a specific directory structure.

Examples:

# Copy from experiments in /data/experiments/workspace
cometx copy --path /data/experiments workspace dest-workspace

# Copy from experiments in your home directory
cometx copy --path ~ workspace dest-workspace

# Copy from experiments in Downloads folder
cometx copy --path ~/Downloads workspace dest-workspace

For more information, cometx copy --help

cometx log

This command is used to log a resource (metrics, parameters, asset, etc) file to a specific experiment or experiments.

cometx log examples:

cometx log WORKSPACE/PROJECT/EXPERIMENT-KEY FILENAME ... --type=TYPE
cometx log WORKSPACE PANEL-ZIP-FILENAME ... --type=panel
cometx log WORKSPACE PANEL.py ... --type=panel
cometx log WORKSPACE PANEL-URL ... --type=panel
cometx log WORKSPACE/PROJECT --type=other --set "key:value"
cometx log WORKSPACE --type=other --set "key:value"

Where TYPE is one of the following names:

  • all
  • asset
  • audio
  • code
  • image
  • metrics
  • notebook
  • panel
  • tensorflow-file
  • text-sample
  • video
  • other
  • tensorboard-folder-assets

Flags

For more information, cometx log --help

cometx delete-assets

To delete experiments assets:

cometx delete-assets WORKSPACE/PROJECT --type=image
cometx delete-assets WORKSPACE/PROJECT/EXPERIMENT --type=all

Type can be valid asset type, including:

  • all
  • asset
  • audio
  • code
  • image
  • notebook
  • text-sample
  • video

Flags

  • --type TYPE - The type of item to log
  • --debug - If given, allow debugging
  • --query QUERY - Only delete experiments that match this Comet query string

For more information, cometx delete-assets --help

cometx config

To enable auto-logging of your notebooks in Jupyter environments:

cometx config --auto-log-notebook yes

To turn auto-logging of notebooks off, use:

cometx config --auto-log-notebook no

If you keep the generated experiment URLs in the notebook, but later edit the notebook, the notebooks will be updated in all of the experiments created in the notebook.

Flags

  • --debug - If given, allow debugging
  • --auto-log-notebook AUTO_LOG_NOTEBOOK - Takes a 1/yes/true, or 0/no/false

For more information, cometx config --help

cometx reproduce

cometx reproduce [-h] [--run] [--executable EXECUTABLE] COMET_PATH OUTPUT_DIR

This command is used to reproduce experiments by copying files to a specified output directory.

Flags

  • --run - Run the reproducible script
  • --executable EXECUTABLE - Run the reproducible script with specified executable

For more information, cometx reproduce --help

cometx update

cometx update [-h] [--debug] COMET_SOURCE COMET_DESTINATION

To update existing experiments.

cometx update SOURCE DESTINATION

where SOURCE is a folder:

  • "WORKSPACE/PROJECT/EXPERIMENT"
  • "WORKSPACE/PROJECT"
  • "WORKSPACE"

where DESTINATION is a Comet:

  • WORKSPACE
  • WORKSPACE/PROJECT

Flags

  • --debug - If given, allow debugging

For more information, cometx update --help

cometx admin

cometx admin [-h] [--host HOST] [--debug] ACTION [ARGUMENTS ...]

To perform admin functions

Actions

chargeback-report

Generate a chargeback report from the Comet server.

cometx admin chargeback-report [YEAR-MONTH]

Arguments:

  • YEAR-MONTH (optional, deprecated) - The YEAR-MONTH to run report for, eg 2024-09. If not provided, generates a report for all available periods.

Output:

  • Saves a JSON file: comet-chargeback-report.json (or comet-chargeback-report-{YEAR-MONTH}.json)

Examples:

cometx admin chargeback-report
<<<<<<< Updated upstream
cometx admin usage-report
=======
cometx admin chargeback-report 2024-09

Stashed changes

usage-report

Generate a usage report with experiment counts and statistics for one or more workspaces/projects.

cometx admin usage-report WORKSPACE [WORKSPACE ...]
cometx admin usage-report WORKSPACE/PROJECT [WORKSPACE/PROJECT ...]
cometx admin usage-report --app

Arguments:

  • WORKSPACE_PROJECT (required, one or more, unless using --app) - One or more WORKSPACE or WORKSPACE/PROJECT to run usage report for. If WORKSPACE is provided without a project, all projects in that workspace will be included.

Options:

  • --units {month,week,day,hour} - Time unit for grouping experiments (default: month)
    • month: Group by month (YYYY-MM format)
    • week: Group by ISO week (YYYY-WW format)
    • day: Group by day (YYYY-MM-DD format)
    • hour: Group by hour (YYYY-MM-DD-HH format)
  • --max-experiments-per-chart N - Maximum number of workspaces/projects per chart (default: 5). If more workspaces/projects are provided, multiple charts will be generated.
  • --no-open - Don't automatically open the generated PDF file after generation.
  • --app - Launch interactive Streamlit web app instead of generating PDF. When using this option, you don't need to specify workspace/project arguments.

Output:

  • PDF Report (default): Generates a PDF report containing:
    • Summary statistics (total experiments, users, run times, GPU utilization)
    • Experiment count charts by time unit
    • GPU utilization charts (if GPU data is available)
    • GPU memory utilization charts (if GPU data is available)
  • Interactive Web App (with --app): Launches a Streamlit web interface where you can:
    • Select workspace and project from dropdowns
    • View statistics and charts interactively
    • Change time units and regenerate reports
    • View "All Projects" from a workspace

Examples:

# Generate report for a single workspace
cometx admin usage-report my-workspace

# Generate report for multiple projects
cometx admin usage-report my-workspace/project1 my-workspace/project2

# Generate report with weekly grouping
cometx admin usage-report workspace1 workspace2 --units week

# Generate report with daily grouping, don't auto-open
cometx admin usage-report workspace --units day --no-open

# Launch interactive web app
cometx admin usage-report --app

gpu-report

Generate a GPU usage report for one or more workspaces/projects with detailed GPU metrics analysis.

cometx admin gpu-report WORKSPACE [WORKSPACE ...] --start-date DATE
cometx admin gpu-report WORKSPACE/PROJECT [WORKSPACE/PROJECT ...] --start-date DATE

Arguments:

  • WORKSPACE_PROJECT (required, one or more) - One or more WORKSPACE or WORKSPACE/PROJECT to run GPU report for. If WORKSPACE is provided without a project, all projects in that workspace will be included.

Options:

  • --start-date DATE (required) - Start date for the report in YYYY-MM-DD format (e.g., 2024-01-01)
  • --end-date DATE (optional) - End date for the report in YYYY-MM-DD format (e.g., 2024-12-31). If not provided, reports from start-date onwards.
  • --metrics METRIC [METRIC ...] (optional) - List of metrics to track. If not provided, uses default GPU metrics:
    • sys.gpu.0.gpu_utilization - GPU utilization percentage
    • sys.gpu.0.memory_utilization - GPU memory utilization percentage
    • sys.gpu.0.used_memory - GPU memory used in GB
    • sys.gpu.0.power_usage - GPU power usage in watts
    • sys.gpu.0.temperature - GPU temperature in Celsius
  • --open - Automatically open the generated PDF file after generation.

Output:

  • Generates a PDF report containing:
    • Summary statistics (total experiments, workspaces, metrics tracked)
    • Breakdown by workspace (if multiple workspaces)
    • Average metrics by workspace charts (bar charts)
    • Maximum metrics by month charts (time series line charts with workspace legend)
  • Generates individual PNG chart files for each metric:
    • gpu_report_avg_{metric}_by_workspace.png - Average metric value per workspace
    • gpu_report_max_{metric}_by_month.png - Maximum metric value per month over time

Examples:

# Generate report for a single workspace from a start date
cometx admin gpu-report my-workspace --start-date 2024-01-01

# Generate report with date range
cometx admin gpu-report my-workspace --start-date 2024-01-01 --end-date 2024-12-31

# Generate report for multiple projects
cometx admin gpu-report workspace1/project1 workspace2 --start-date 2024-01-01

# Generate report with custom metrics
cometx admin gpu-report my-workspace --start-date 2024-01-01 --metrics sys.gpu.0.gpu_utilization sys.gpu.0.memory_utilization

# Generate report and automatically open PDF
cometx admin gpu-report my-workspace --start-date 2024-01-01 --open

Global Flags

  • --host HOST - Override the HOST URL
  • --debug - If given, allow debugging
  • --api-key API_KEY - Set the COMET_API_KEY
  • --url-override URL_OVERRIDE - Set the COMET_URL_OVERRIDE

For more information, cometx admin --help

Note: For detailed information on admin commands, see README-ADMIN.md.

cometx smoke-test

cometx smoke-test [-h] [--exclude [EXCLUDE ...]] [--debug DEBUG] COMET_PATH [include ...]

Perform a smoke test on a Comet installation. Logs results to WORKSPACE/smoke-tests or WORKSPACE/PROJECT.

Examples:

Run all tests:

cometx smoke-test WORKSPACE    # project defaults to smoke-tests
cometx smoke-test WORKSPACE/PROJECT

Run everything except mpm tests:

cometx smoke-test WORKSPACE/PROJECT --exclude mpm

Run just optimizer tests:

cometx smoke-test WORKSPACE/PROJECT optimizer

Run just metric tests:

cometx smoke-test WORKSPACE/PROJECT metric

Items to include or exclude:

  • optimizer
  • mpm
  • panel
  • opik
  • experiment
    • metric
    • image
    • asset
    • dataset-info
    • confusion-matrix
    • embedding

Flags

  • --exclude [EXCLUDE ...] - Items to exclude; any of: asset, confusion-matrix, dataset-info, embedding, experiment, image, metric, mpm, opik, optimizer, panel
  • --debug DEBUG - Show debugging information

For more information, cometx smoke-test --help

Copy/Download Use Cases

In this section we'll explore some common scenarios.

  1. Copy a specific project from one Comet installation to another
  2. Copy all projects in workspace to a new workspace
  3. Copy specific experiments in a project to new experiments

1. Copy a specific project from one comet installation to another

A useful idiom is to set your Comet environment variables on the line of a command. In this manner, you can set the COMET_URL_OVERRIDE and COMET_API_KEY for different installations.

Of course, you don't have to set the environment variables if you are copying experiments on the same Comet installation.

Here is how you one could download the experiments in WORKSPACE/PROJECT from http://comet.a.com:

cometx --api-key=A-KEY download WORKSPACE/PROJECT

The cometx download command downloads all of the Comet experiment data into local files. Note that WORKSPACE/PROJECT refers to a workspace and project on http://comet.a.com.

One could then copy the downloaded experiment data with a similar command:

cometx --api-key=B-KEY copy WORKSPACE/PROJECT NEW-WORKSPACE/NEW-PROJECT

Note that WORKSPACE/PROJECT now refers to a directory, and NEW-WORKSPACE/NEW-PROJECT refers to a workspace and project on http://comet.b.com.

2. Copy all projects in workspace to a new workspace

Similarly, one can copy all of the projects by first downloading them:

cometx --api-key=A-KEY download WORKSPACE

and then copying them:

cometx --api-key=B-KEY copy WORKSPACE NEW-WORKSPACE

3. Copy specific experiments in a project to new experiments

Similarly, one can copy a single experiment first downloading it:

cometx --api-key=A-KEY download WORKSPACE/PROJECT/EXPERIMENT-NAME-OR-ID

and then copying it:

cometx --api-key=B-KEY copy WORKSPACE/PROJECT/EXPERIMENT-NAME-OR-ID NEW-WORKSPACE/NEW-PROJECT

Running Tests

WARNING: Running the tests will create experiments, models, assets, etc. in your default workspace if not set otherwise.

To run the tests, you can either export all of these items in the environment:

$ export COMET_USER="<USERNAME>"
$ export COMET_WORKSPACE="<WORKSPACE>"
$ export COMET_API_KEY="<API-KEY>"
$ pytest tests

Or, define workspace and api_key in your ~/.comet.config file:

$ export COMET_USER="<USERNAME>"
$ pytest tests

About

Open source contributions for integrations with comet_ml

Resources

Readme

License

MIT license
Activity
Custom properties

Stars

13 stars

Watchers

5 watching

Forks

3 forks
Report repository

Releases 25

3.2.2 Latest
Nov 24, 2025
+ 24 releases

Packages

No packages published

Contributors 7

Languages