Skip to content

Faster GC --keep-derivations without complicating the Derivation data type #13573

@Ericson2314

Description

@Ericson2314

Is your feature request related to a problem?

GC with derivation scanning is currently slow. If we always parse structured attrs JSON, it will be even slower.

Proposed solution

The simplest thing to do is make structured attrs a "lazily loaded field", so we can defer parsing until its is needed. but that is gross.

Alternative solutions

Better solutions are:

  • Use a special derivation parser for GC that just gets out the output paths, since those are the only things which are needed. That would be faster than than parsing the whole derivation with or without structured attrs parsed JSON.

  • Stop using .drv files, and instead use database tables. Than we can look at e.g. a DerivationOutputs table with high throughput.

Additional context

#13263

Checklist


Add 👍 to issues you find important.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureFeature request or proposal

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions