Skip to content

Commit ad5cb33

Browse files
dmora127dmora127
committed
Updated condor_submit_quickref per PR: #905 
1 parent d86b7cc commit ad5cb33

File tree

2 files changed

+130
-1
lines changed

2 files changed

+130
-1
lines changed

‎_events‎

‎_uw-research-computing/condor_submit_quickref.md‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
---
2+
layout: guide
3+
title: "Quick reference: HTCondor Submission commands"
4+
alt_title: "Quick reference: HTCondor Submission commands"
5+
guide:
6+
category: Get started
7+
tag: htc
8+
---
9+
10+
## Introduction
11+
12+
This page lists common HTCondor commands and options for jobs. Users familiar with HTCondor and job submission on CHTC's High Throughput Computing (HTC) system can use this page as a quick reference. For users who are just starting out, we suggest reading the full guides (linked) to understand the full context of each command or option.
13+
14+
{% capture content %}
15+
- [Introduction](#introduction)
16+
- [Submit jobs](#submit-jobs)
17+
- [Basic Submit File Options](#basic-submit-file-options)
18+
- [Managing File Transfers in HTCondor](#managing-file-transfers-in-htcondor)
19+
- [Controlling Where Your Job Runs](#controlling-where-your-job-runs)
20+
- [Controlling How Your Job Runs](#controlling-how-your-job-runs)
21+
- [Queue Statement Options](#queue-statement-options)
22+
- [Glossary](#glossary)
23+
- [Related Pages](#related-pages)
24+
{% endcapture %}
25+
{% include /components/directory.html title="Table of Contents" %}
26+
27+
> **Please note the following!**
28+
>
29+
> * Bracketed items (`<>`) denote where to place your input.
30+
> Do not include the brackets in your command.
31+
> * All commands should be entered on a single line.
32+
33+
## Commands to submit jobs
34+
35+
| Command | Use | Notes and Examples |
36+
| --- | --- |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
37+
| `condor_submit` | submits job(s) as specified by `submit_file` | See [monitor your jobs](htcondor-job-submission.md) <br><br> **Example:** <br>`condor_submit helloWorld.sub` |
38+
| `condor_submit -i <submit_file>` | submits an interactive job as specified by `submit_file` | **Example:** <br>`condor_submit -i helloWorld.sub` |
39+
{:.command-table}
40+
41+
## Basic Submit File Options
42+
43+
| Option | Use | Notes and Examples |
44+
|----------------------------------| --- |----------------------------------------------------------------------------------------------------------------------------------------------------|
45+
| `executable = <executable.sh>` | path to the executable script | The executable script is automatically transferred to the Execution Point (EP) by HTCondor. <br><br> **Example:** <br>`executable = helloWorld.py` |
46+
| `arguments = "<args>"` | lists arguments to be passed to the executable as part of the command line | Arguments are wrapped by quotes(") and space separated. Embed spaces using single quotes.<br><br> **Example:** <br>`arguments = "'hello world'"` |
47+
| `environment = <parameter_list>` | lists environmental variables | Wrapped by quotes(") and space separated. <br><br> **Example:** <br>`environment = "VARIABLE1=Value1 VAR2='hello world'"` |
48+
| `log = <job.log>` | denotes the path to the log file | We always recommend specifying `log`. If `log` is not provided, no log file is written. <br><br> **Example:** <br>`log = ./log_files/job1.log` |
49+
| `output = <job.out>` | path to file capturing `stdout` screen output | Can be merged with `stderr` by denoting the same path in `error = <path>`. <br><br> **Example:** <br>`output = ./log_files/job1.out` |
50+
| `error = <job.err>` | path to file capturing `stderr` screen output | Can be merged with `stdout` by denoting the same path in `output = <path>`. <br><br> **Example:** <br>`error = ./log_files/job1.err` |
51+
{:.command-table}
52+
53+
## File transfers options
54+
55+
Visit our [file transfers guide](htc-file-transfer) for more details.
56+
57+
| Option | Use | Notes and Examples |
58+
|------------------------------------------------------------| --- |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
59+
| `transfer_input_files = ` | lists all the input files to be transferred to the Execute Point (EP) | Comma-separated list. Various file transfer protocols can be used including `file:///`, `osdf:///`, `pelican:///`, and `s3:///`. <br><br> **Examples:** <br> `transfer_input_files = osdf:///chtc/staging/...` |
60+
| `transfer_output_files = ` | explicitly lists the path to files to be returned to the Access Point (AP) | If this is not specified, HTCondor will only transfer new and changed **files** in the top-level directory of the Execution Point. <br><br> **Example:** <br><br>`transfer_output_files = results.txt`|
61+
| `transfer_output_remaps = "<path_on_EP>=<new_path_on_AP>"` | remaps output files to a new location on the AP upon job completion | File paths on the left side (e.g., ./results.txt) refer to locations on the Execution Point (EP), while remapped paths on the right side refer to the Access Point (AP). <br><br> **Example:** <br>`transfer_output_remaps = "results.txt=/staging/<user>/job1_results.txt"` |
62+
{:.command-table}
63+
64+
## Request resources
65+
66+
| Option | Use | Notes and Examples |
67+
|------------------------------------------|----------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
68+
| `request_cpus = <int>` | requests number of CPUs (cores) | **Example:** <br>`request_cpus = 4` |
69+
| `request_disk = <quantity>` | requests disk space (Default in KiB) | Can use units like `K`, `M`, `G`, or `T`. <br><br> **Example:** <br>`request_disk = 40GB` |
70+
| `request_memory = <quantity>` | requests memory for job (Default in MB) | **Example:** <br>`request_memory = 250GB` |
71+
| `request_gpus = <int>` | requests number of GPUs | If not specified, no GPUs requested. |
72+
| `requirements = <ClassAd Boolean>` | sets job execution constraints | See [ClassAd reference](https://htcondor.readthedocs.io/en/latest/classad-attributes/job-classad-attributes.html). <br><br> **Example:** <br>`requirements = (OpSysAndVer == "RedHat9")` |
73+
| `gpus_minimum_capability = <version>` | sets minimum GPU capability to specify the GPU type | **Example:** <br>`gpus_minimum_capability = 8.5` |
74+
| `gpus_maximum_capability = <version>` | sets maximum GPU capability to specify the GPU type | **Example:** <br>`gpus_maximum_capability = 9.0` |
75+
| `gpus_minimum_memory = <quantity in MB>` | specifies the minimum GPU VRAM memory to request | **Example:** <br>`gpus_minimum_memory = 3200` |
76+
77+
{:.command-table}
78+
79+
_Note: For more information on submitting GPU jobs, please visit our [Using GPUs](gpu-jobs.md) guide._
80+
81+
## Specifying the software environment
82+
83+
| Option | Use | Notes and Examples |
84+
|----------------------------------| --- |-----------------------------------------------------------------------------------------------------------------------------------|
85+
| `container_image = <image_file>` | defines container image path | Can pull from DockerHub or use a local `.sif` file. <br><br> **Example:** <br>`container_image = docker://pytorch/pytorch:latest` |
86+
{:.command-table}
87+
88+
_Note: For more information on using containers in your jobs, please visit our [Using Apptainer Containers](apptainer-htc) or [Running HTC Jobs Using Docker Containers](docker-jobs) guide._
89+
90+
## Options for submitting a list of jobs
91+
92+
| Option | Use | Notes and Examples |
93+
|------------------------------------------| --- |--------------------------------------------------------------------------------------------------------------------------------------------|
94+
| `queue` | queues a single job | If no other options specified, queues one job. |
95+
| `queue <int>` | queues multiple copies of the job | **Example:** <br>`queue 10` |
96+
| `queue <var> from <file>` | queues jobs using values from file | The `<var>` value(s) can be used as arguments/environment variables. <br><br> **Example:** <br>`queue name from ./listOfEmployeeNames.txt` |
97+
| `queue <var1>,<var2> from <file>` | queues jobs using multiple vars from file | **Example:** <br>`queue first, last from ./listOfEmployeeNames.txt` |
98+
| `queue <var> in [slice] <list>` | queues jobs using Python-style slicing | **Example:** <br>`queue name in [5:18] ./listOfEmployeeNames.txt` |
99+
| `queue <var> matching <globbing_string>` | queues jobs from file pattern matches | **Example:** <br>`queue sampleID matching ./samples/sampleID_*` |
100+
{:.command-table}
101+
102+
## Scale beyond local capacity
103+
| Option | Use | Notes and Examples |
104+
|----------------------------| --- |--------------------------------------------------------------------------------------------------------------------------------------------|
105+
| `want_campus_pools = True` | Allows jobs to run on other HTCondor pools on campus (e.g., UW Grid) | Best for short (<8hr) or checkpointable jobs. Expands matching opportunities across campus. <br><br> **May not include `HasCHTCStaging` in the requirements** |
106+
| `want_ospool = True` | Allows jobs to match to the national Open Science Pool (OSPool) | Best for short (<8hr) or checkpointable jobs. Significantly increases available compute slots. <br><br> **May not include `HasCHTCStaging` in the requirements** |
107+
{:.command-table}
108+
109+
## Glossary
110+
111+
| Term | Meaning |
112+
| --- | --- |
113+
| access point | The machine you log into for submitting jobs (e.g., CHTC login node). |
114+
| error file / standard error | The file where your job writes error messages. |
115+
| execution point | The machine where your job actually runs. |
116+
| held/hold | Job has encountered an issue and paused. |
117+
| idle | Job hasn't matched to an execution point yet. |
118+
| job ID | Unique ID made of `ClusterID.ProcID` like `12345.0`. |
119+
| log file | Tracks job events and resource usage. |
120+
| output file / standard out | File where job writes standard output (e.g., print statements). |
121+
| process ID | ID of an individual job in a job cluster. |
122+
| running | Job has matched and is currently running. |
123+
| submit file | File specifying everything needed to run your job. |
124+
125+
## Related Pages
126+
127+
* [Practice: Submit HTC Jobs using HTCondor](htcondor-job-submission)
128+
* [Monitor your jobs](condor_q)
129+
* [HTCondor manual - condor_submit](https://htcondor.readthedocs.io/en/latest/man-pages/condor_submit.html)

0 commit comments

Comments
 (0)