NetHack RL Starter

A beginner-friendly starter kit for training Reinforcement Learning agents on the NetHack Learning Environment (NLE) and MiniHack.

TL;DR

# 1) Create env
uv venv --python 3.11 || python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -e .

# 2) Sanity-check installs
python src/play_nle_random.py --env NetHackScore-v0  # headless smoke test
python src/play_minihack_random.py --env MiniHack-Room-5x5-v0

# 3) Train a tiny PPO baseline (MiniHack 5x5)
python src/train_sb3_minihack.py --env MiniHack-Room-5x5-v0 --steps 20000

If you're on Windows, use WSL2 (Ubuntu 22.04+).

This template targets Gymnasium API, NLE ≥ 1.1.0, and MiniHack ≥ 1.0.0.

Why this template?

• Minimal files to run today (random rollouts, quick PPO baseline).
• Works in Cursor/VS Code, with optional Dev Container.
• CI smoke-test ensures NLE/MiniHack import on Ubuntu (GitHub Actions).

Project layout

nethack-rl-starter/
├─ src/
│  ├─ play_nle_random.py         # rollouts in raw NLE
│  ├─ play_minihack_random.py    # rollouts in MiniHack
│  └─ train_sb3_minihack.py      # PPO baseline via Stable-Baselines3
├─ tests/
│  └─ test_envs.py               # smoke tests
├─ .github/workflows/ci.yml      # CI: install deps + run smoke test
├─ .devcontainer/                # Optional: run in container
├─ pyproject.toml                # deps + formatting config
├─ Makefile                      # handy commands
└─ README.md

Quick setup (macOS or Linux)

1. Install CMake ≥ 3.18
   • macOS: brew install cmake
   • Ubuntu 22.04/24.04: sudo apt-get update && sudo apt-get install -y cmake
2. Create a Python 3.11+ virtual env, activate it, then:

   pip install -U pip
   pip install -e .

3. Sanity check:

   python -m nle.scripts.play --mode random --env NetHackScore-v0 --no_render
   python -m minihack.scripts.play --env MiniHack-Room-5x5-v0 --mode random --no_render

4. (Optional) Train a tiny PPO baseline:

   python src/train_sb3_minihack.py --env MiniHack-Room-5x5-v0 --steps 20000

Windows note

Use WSL2 (Ubuntu 22.04+) for a smooth experience. Native Windows builds are not officially supported by Gymnasium and can be painful.

Using this with ChatGPT "Codex" and Cursor

• In Cursor, open the workspace folder and run the commands above in the integrated terminal.
• Ask ChatGPT (Codex) to:
  • write new MiniHack tasks,
  • add wrappers (e.g., observation flattening, frame-stacking),
  • create experiment configs and tensorboard plots,
  • open PRs against the GitHub repo (after you push this project).

Citations

• NLE (NetHack-LE org): https://github.com/NetHack-LE/nle
• MiniHack (new home): https://github.com/samvelyan/minihack
• Gymnasium: https://gymnasium.farama.org/