Background

At the moment, none of the MLIR activities in the MQT Core project are included in release artifacts. The mlir subdirectory isn’t built by default; a dedicated CMake option must be set to enable our MLIR infrastructure. Historically this was intentional because:

• It was unclear whether we’d pursue this direction seriously or keep it as an experiment.
• There wasn’t much end-to-end functionality yet.
• There’s uncertainty around how to package LLVM/MLIR and distribute it efficiently.

The situation has changed:

• We intend to continue and invest in MLIR-based compilation.
• We recently merged initial support to import MQT Core IR into our MLIR dialect, conversions between internal dialects, initial optimization passes, and QIR export is imminent. At that point a minimal pipeline like “import Qiskit/OpenQASM → MLIR → QIR” is conceivable.
• The remaining blocker is sustainable packaging and distribution, especially for our Python wheels.

Goal

Ship a minimal, usable MLIR toolchain by default in MQT Core releases and expose key functionality via Python, while keeping base wheels lean and managing PyPI storage constraints. Achieve this with a clear packaging strategy for LLVM/MLIR, robust CI/CD, and strict binary size budgets.

Problem

• MLIR isn’t built or shipped by default; users can’t access it out of the box.
• Packaging LLVM/MLIR into Python wheels is complex and risks ballooning size (current base wheels are ~2–3 MiB; PyPI has a 10 GiB per-project cap; we already publish ~50 wheels per release).
• CI/CD must integrate MLIR builds (cibuildwheel), ensure correct dependency bundling (auditwheel/delocate), and maintain reliability across platforms.

Proposal:

Stage the rollout and adopt a packaging strategy that balances usability and size.

1. Rollout plan

• Stage 0 (pre-releases):
  • Keep MLIR OFF by default in the top-level build, but produce experimental “MLIR-enabled” wheels as a separate companion package for pre-releases.
  • Gather feedback on usability, size, and platform coverage.
• Stage 1 (general availability):
  • Turn MLIR ON by default for source builds and developer configs.
  • Publish a companion binary wheel that adds MLIR/LLVM runtime and Python bindings, installable via an extra (e.g., pip install mqt-core[mlir]).
  • Keep base wheels lean; the extra pulls in the companion wheel only for users who need MLIR.
• Stage 2 (optional):
  • If size budgets are met, consider folding MLIR into base wheels on selected platforms, or keep the companion wheel model if it proves superior.

2. Packaging strategy (recommended: companion wheel)

• Create a separate, version-aligned PyPI package (e.g., mqt-core-mlir) that contains the MLIR/LLVM shared libraries and the minimal Python bindings to expose compiler entry points. In pyproject.toml of mqt-core, define an optional dependency group mlir that pulls in the extra package.
• Alternate/fallback modes:
  • Vendored-in wheel (single wheel bundles MLIR/LLVM) with strict size budgets and aggressive stripping/LTO.
  • System-LLVM/MLIR fallback via env var to avoid bundling on power-user systems (documented and unsupported on PyPI wheels by default).

3. Build composition and size-control

• Build minimal LLVM/MLIR:
  • Build LLVM shared monolith to reduce duplication (LLVM_LINK_LLVM_DYLIB=ON), enable only mlir in LLVM_ENABLE_PROJECTS, and prune unused MLIR components.
  • Build Release with LTO and strip symbols; exclude debug info in wheels.
• Ship only the small set of MLIR/LLVM DSOs we actually need for our pipelines (list and lock down).
• Define size budgets (e.g., per-wheel cap ≤ 50 MiB) and fail CI if exceeded.
• This ties in with 🚀 MLIR - Provide Pre-Built MLIR/LLVM Binaries for CI/CD #1124, where we should focus on providing small standalone distributions of MLIR.

4. CI/CD with cibuildwheel

• Add a dedicated MLIR job matrix that:
  • Builds the companion wheel (Linux, macOS, Windows with both x86 and arm64)
  • Uses auditwheel repair on Linux and delocate on macOS; delvewheel on Windows
  • Caches or consumes prebuilt LLVM/MLIR artifacts to keep CI fast; verify RPATH/loader behavior.
  • Runs a smoke test that compiles a tiny OpenQASM/Qiskit circuit to QIR.
• Keep the base wheel job matrix unchanged to preserve current size and reliability.

5. Python API surface (initial)

• Provide a small, documented entry point in Python, e.g.:
  • mqt.core.mlir.compile_to_qir(circuit: str | qiskit.QuantumCircuit, *, profile: str = "base") -> bytes | str
• This API should be available only when the MLIR companion wheel is installed; otherwise a clear ImportError explains how to enable MLIR support. Alternatively, the companion wheel itself might install itself into mqt.core.mlir.

6. Documentation and release notes

• Update docs/mlir.md and install docs to explain the extra, platform coverage, and size trade-offs.
• Add release notes and a migration guide for enabling MLIR in existing workflows.

Acceptance criteria:

• A top-level CMake option to enable MLIR exists and defaults to ON for release builds and source distributions; CI builds MLIR-enabled artifacts accordingly.
• A packaging decision is implemented (companion wheel recommended):
  • Companion wheel is published for all tier-1 platforms and version-aligned to mqt-core.
  • Base wheels remain ≤ current size; MLIR-enabled wheels stay within defined size caps.
• CI produces and tests MLIR-enabled wheels via cibuildwheel; wheels are repaired/audited and pass a smoke test that compiles a small circuit to QIR.
• A minimal Python API (e.g., compile_to_qir) is exposed when MLIR support is installed; documentation updated with install and usage instructions.
• Storage impact on PyPI is measured and documented; mitigation plan in place (companion wheels, pruning old releases, or limited platform coverage if necessary).