Skip to content

Commit 8a7bcfd

Browse files
authored
Add tutorial for dspy-trusted-monitor using GEPA (#8938)
* add gepa_trusted_monitor notebook, copied and adapted from gepa_aime_math * clean up some naming and ordering * add some tutorial instructions * add index.py as an export to help with PR review * add more instructions, add analysis cells * run training * run evaluation and analysis * export to index.py * remove inspect outputs from notebook * add gepa_trusted_monitor link to index pages
1 parent e870ff1 commit 8a7bcfd

File tree

5 files changed

+7228
-1
lines changed

5 files changed

+7228
-1
lines changed

docs/docs/tutorials/gepa_ai_program/index.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,4 +11,7 @@ This tutorial explores how GEPA can optimize a single `dspy.ChainOfThought` base
1111
This tutorial explores how GEPA leverages predictor-level feedback to improve GPT-4.1 Nano's performance on a three-part task for structured information extraction and classification in an enterprise setting.
1212

1313
### [GEPA for Privacy-Conscious Delegation](../gepa_papillon/index.ipynb)
14-
This tutorial explores how GEPA can improve rapidly in as few as 1 iteration, while leveraging a simple feedback provided by a LLM-as-a-judge metric. The tutorial also explores how GEPA benefits from the textual feedback showing a breakdown of aggregate metrics into sub-components, allowing the reflection LM to identify what aspects of the task need improvement.
14+
This tutorial explores how GEPA can improve rapidly in as few as 1 iteration, while leveraging a simple feedback provided by a LLM-as-a-judge metric. The tutorial also explores how GEPA benefits from the textual feedback showing a breakdown of aggregate metrics into sub-components, allowing the reflection LM to identify what aspects of the task need improvement.
15+
16+
### [GEPA for Code Backdoor Classification (AI control)](../gepa_trusted_monitor/index.ipynb)
17+
This tutorial explores how GEPA can optimize a GPT-4.1 Nano classifier to identify backdoors in code written by a larger LM, using `dspy.GEPA` and a comparative metric! The comparative metric allows the prompt optimizer to create a prompt that identifies the signals in the code that are indicative of a backdoor, teasing apart positive samples from negative samples.

0 commit comments

Comments
 (0)