Skip to content

Commit b89649e

Browse files
CFLJacquetburaizu
andauthored
Update API description and SDK dataset push (#30154)
* Update API description and SDK * Capitalize boolean * Capitalize boolean --------- Co-authored-by: Bryce Eadie <[email protected]>
1 parent 6d20f42 commit b89649e

File tree

1 file changed

+17
-18
lines changed
  • content/en/llm_observability/experiments

1 file changed

+17
-18
lines changed

content/en/llm_observability/experiments/_index.md

Lines changed: 17 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -190,13 +190,12 @@ expected_output_columns=["answer"]
190190
{{% collapse-content title="Push a dataset to Datadog" level="h4" expanded=false id="dataset-push" %}}
191191

192192
```python
193-
Dataset.push(overwrite: boolean = None, new_version: boolean = None)
193+
Dataset.push(new_version: boolean = None)
194194
```
195195

196196
| Parameter | Type | Description |
197197
| --------- | ---- | ----------- |
198-
| `overwrite` | boolean | If `True`, overwrites the dataset rows of an existing version. |
199-
| `new_version` | boolean | If `True`, creates a new version of the dataset in Datadog. Defaults to `True`.<br/><br/>This flag is useful for creating a new dataset with entirely new data. |
198+
| `new_version` | Boolean | If `True`, creates a new version of the dataset in Datadog, otherwise it modifies it in place. Defaults to `True`.|
200199

201200
**Example**
202201

@@ -220,7 +219,7 @@ Dataset.as_dataframe(multiindex: bool = True) -> pd.DataFrame
220219

221220
| Parameter | Type | Description |
222221
| --------- | ---- | ----------- |
223-
| `multiindex` | boolean | If `True`, expands nested dictionaries into [MultiIndex][6] columns. Defaults to `True`. |
222+
| `multiindex` | Boolean | If `True`, expands nested dictionaries into [MultiIndex][6] columns. Defaults to `True`. |
224223

225224
**Returns**
226225

@@ -273,7 +272,7 @@ Experiment.run(jobs: int = 10, raise_errors: bool = False, sample_size: int = No
273272
| Parameter | Type | Description |
274273
| --------- | ---- | ----------- |
275274
| `jobs` | int | Number of worker threads used to run the task concurrently. Defaults to 10. |
276-
| `raise_errors` | boolean | If `True`, stops execution as soon as the first exception from the task is raised.<br/><br/>If `False`, every exception is handled, and the experiment runs continually until finished. |
275+
| `raise_errors` | Boolean | If `True`, stops execution as soon as the first exception from the task is raised.<br/><br/>If `False`, every exception is handled, and the experiment runs continually until finished. |
277276
| `sample_size` | int | Number of rows used for the experiment. You can use `sample_size` with `raise_errors` to test before you run a long experiment. |
278277

279278
**Returns**
@@ -301,7 +300,7 @@ Experiment.run_evaluations(evaluators: Optional[List[Callable]] = None, raise_er
301300
| Parameter | Type | Description |
302301
| --------- | ---- | ----------- |
303302
| `evaluators` | function[] | List of functions decorated with `@evaluator` that run against all outputs in the results. |
304-
| `raise_errors` | boolean | If `True`, stops execution as soon as the first exception from the task is raised.<br/><br/>If `False`, every exception is handled, and the experiment runs continually until finished. |
303+
| `raise_errors` | Boolean | If `True`, stops execution as soon as the first exception from the task is raised.<br/><br/>If `False`, every exception is handled, and the experiment runs continually until finished. |
305304

306305
**Returns**
307306

@@ -321,7 +320,7 @@ ExperimentResults.as_dataframe(multiindex: bool = True) -> pd.DataFrame
321320

322321
| Parameter | Type | Description |
323322
| --------- | ---- | ----------- |
324-
| `multiindex` | boolean | If `True`, expands nested dictionaries into [MultiIndex][6] columns. Defaults to `True`. |
323+
| `multiindex` | Boolean | If `True`, expands nested dictionaries into [MultiIndex][6] columns. Defaults to `True`. |
325324

326325
**Returns**
327326

@@ -668,8 +667,8 @@ List all dataset records, sorted by creation date. The most recently-created rec
668667
| ---- | ---- | ---- |
669668
| `id` | string | Unique record ID. |
670669
| `dataset_id` | string | Unique dataset ID. |
671-
| `input` | any valid JSON type (string, int, object, etc.) | Data that serves as the starting point for an experiment. |
672-
| `expected_output` | any valid JSON type (string, int, object, etc.) | Expected output |
670+
| `input` | any (string, number, Boolean, object, array) | Data that serves as the starting point for an experiment. |
671+
| `expected_output` | any (string, number, Boolean, object, array) | Expected output |
673672
| `metadata` | json | Arbitrary user-defined metadata. |
674673
| `created_at` | timestamp | Timestamp representing when the resource was created. |
675674
| `updated_at` | timestamp | Timestamp representing when the resource was last updated. |
@@ -690,8 +689,8 @@ Appends records for a given dataset.
690689

691690
| Field | Type | Description |
692691
| ---- | ---- | ---- |
693-
| `input` (_required_) | any valid JSON type (string, int, object, etc.) | Data that serves as the starting point for an experiment. |
694-
| `expected_output` | any valid JSON type (string, int, object, etc.) | Expected output |
692+
| `input` (_required_) | any (string, number, Boolean, object, array) | Data that serves as the starting point for an experiment. |
693+
| `expected_output` | any (string, number, Boolean, object, array) | Expected output |
695694
| `metadata` | json | Arbitrary user-defined metadata. |
696695

697696
**Response**
@@ -710,8 +709,8 @@ Partially update a dataset record object. Specify the fields to update in the pa
710709

711710
| Field | Type | Description |
712711
| ---- | ---- | ---- |
713-
| `input` | any valid JSON type (string, int, object, etc.) | Data that serves as the starting point for an experiment. |
714-
| `expected_output` | any valid JSON type (string, int, object, etc.) | Expected output |
712+
| `input` | any (string, number, Boolean, object, array) | Data that serves as the starting point for an experiment. |
713+
| `expected_output` | any (string, number, Boolean, object, array) | Expected output |
715714
| `metadata` | json | Arbitrary user-defined metadata. |
716715

717716
**Response**
@@ -720,8 +719,8 @@ Partially update a dataset record object. Specify the fields to update in the pa
720719
| ---- | ---- | ---- |
721720
| `id` | string | Unique record ID. |
722721
| `dataset_id` | string | Unique dataset ID. |
723-
| `input` | any valid JSON type (string, int, object, etc.) | Data that serves as the starting point for an experiment. |
724-
| `expected_output` | any valid JSON type (string, int, object, etc.) | Expected output |
722+
| `input` | any (string, number, Boolean, object, array) | Data that serves as the starting point for an experiment. |
723+
| `expected_output` | any (string, number, Boolean, object, array) | Expected output |
725724
| `metadata` | json | Arbitrary user-defined metadata. |
726725
| `created_at` | timestamp | Timestamp representing when the resource was created. |
727726
| `updated_at` | timestamp | Timestamp representing when the resource was last updated. |
@@ -891,9 +890,9 @@ Handle the ingestion of experiment spans or respective evaluation metrics.
891890
| Field | Type | Description |
892891
| ---- | ---- | ---- |
893892
| `error` | [Error](#object-error) | Captures errors. |
894-
| `input` (_required_) | any valid JSON type (string, int, object, etc.) | Input value to an operation. |
895-
| `output` (_required_) | any valid JSON type (string, int, object, etc.) | Output value to an operation. |
896-
| `expected_output` | any valid JSON type (string, int, object, etc.) | Expected output value. |
893+
| `input` (_required_) | any (string, number, Boolean, object, array) | Input value to an operation. |
894+
| `output` (_required_) | any (string, number, Boolean, object, array) | Output value to an operation. |
895+
| `expected_output` | any (string, number, Boolean, object, array) | Expected output value. |
897896
| `metadata` | json | Arbitrary user-defined metadata. |
898897

899898
#### Object: EvalMetric

0 commit comments

Comments
 (0)