Skip to content

nix store delete: Show why deletion fails #13421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/libstore/daemon.cc
Original file line number Diff line number Diff line change
Expand Up @@ -768,6 +768,7 @@ static void performOp(
options.action = (GCOptions::GCAction) readInt(conn.from);
options.pathsToDelete = WorkerProto::Serialise<StorePathSet>::read(*store, rconn);
conn.from >> options.ignoreLiveness >> options.maxFreed;
options.censor = !trusted;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unlike binary allow/deny authorization, this kind of use of the trusted flag is not consistent with the removal of trust by an intermediate nix daemon (untrusted cli -> daemon connection in untrusted mode -> trusted daemon connection -> root).
The intermediate daemon can deny unauthorized operations, but it can not feasibly censor messages that are returned to the untrusted cli.

For this to work correctly, the intermediate daemon needs to be able to request a demotion, so that the trusted daemon can censor as needed.

// obsolete fields
readInt(conn.from);
readInt(conn.from);
Expand Down
56 changes: 30 additions & 26 deletions src/libstore/gc.cc
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ void LocalStore::findTempRoots(Roots & tempRoots, bool censor)
while ((end = contents.find((char) 0, pos)) != std::string::npos) {
Path root(contents, pos, end - pos);
debug("got temporary root '%s'", root);
tempRoots[parseStorePath(root)].emplace(censor ? censored : fmt("{temp:%d}", pid));
tempRoots[parseStorePath(root)].emplace(censor ? censored : fmt("{nix-process:%d}", pid));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The censoring is probably a good thing, but my imagination is lacking this morning. Did you have a threat in mind for this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can reveal information about other user's processes (i.e. pid X has store path Y open).

However, since the Nix store is world-readable, I wouldn't mind getting rid of censoring altogether. It might make sense if we have store ACLs but right now it seems pointless.

pos = end + 1;
}
}
Expand Down Expand Up @@ -463,13 +463,14 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
bool gcKeepOutputs = settings.gcKeepOutputs;
bool gcKeepDerivations = settings.gcKeepDerivations;

std::unordered_set<StorePath> roots, dead, alive;
Roots roots;
std::unordered_set<StorePath> dead, alive;

struct Shared
{
// The temp roots only store the hash part to make it easier to
// ignore suffixes like '.lock', '.chroot' and '.check'.
std::unordered_set<std::string> tempRoots;
std::unordered_map<std::string, GcRootInfo> tempRoots;

// Hash part of the store path currently being deleted, if
// any.
Expand Down Expand Up @@ -580,7 +581,8 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
debug("got new GC root '%s'", path);
auto hashPart = std::string(storePath->hashPart());
auto shared(_shared.lock());
shared->tempRoots.insert(hashPart);
// FIXME: could get the PID from the socket.
shared->tempRoots.insert_or_assign(hashPart, "{nix-process:unknown}");
/* If this path is currently being
deleted, then we have to wait until
deletion is finished to ensure that
Expand Down Expand Up @@ -620,20 +622,16 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
/* Find the roots. Since we've grabbed the GC lock, the set of
permanent roots cannot increase now. */
printInfo("finding garbage collector roots...");
Roots rootMap;
if (!options.ignoreLiveness)
findRootsNoTemp(rootMap, true);

for (auto & i : rootMap)
roots.insert(i.first);
findRootsNoTemp(roots, options.censor);

/* Read the temporary roots created before we acquired the global
GC root. Any new roots will be sent to our socket. */
Roots tempRoots;
findTempRoots(tempRoots, true);
for (auto & root : tempRoots) {
_shared.lock()->tempRoots.insert(std::string(root.first.hashPart()));
roots.insert(root.first);
{
Roots tempRoots;
findTempRoots(tempRoots, options.censor);
for (auto & root : tempRoots)
_shared.lock()->tempRoots.insert_or_assign(std::string(root.first.hashPart()), *root.second.begin());
}

/* Synchronisation point for testing, see tests/functional/gc-non-blocking.sh. */
Expand Down Expand Up @@ -729,20 +727,31 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
}
};

if (options.action == GCOptions::gcDeleteSpecific && !options.pathsToDelete.count(*path)) {
throw Error(
"Cannot delete path '%s' because it's referenced by path '%s'.",
printStorePath(start),
printStorePath(*path));
}

/* If this is a root, bail out. */
if (roots.count(*path)) {
if (auto i = roots.find(*path); i != roots.end()) {
if (options.action == GCOptions::gcDeleteSpecific)
throw Error(
"Cannot delete path '%s' because it's referenced by the GC root '%s'.",
printStorePath(start),
*i->second.begin());
debug("cannot delete '%s' because it's a root", printStorePath(*path));
return markAlive();
}

if (options.action == GCOptions::gcDeleteSpecific && !options.pathsToDelete.count(*path))
return;

{
auto hashPart = std::string(path->hashPart());
auto shared(_shared.lock());
if (shared->tempRoots.count(hashPart)) {
debug("cannot delete '%s' because it's a temporary root", printStorePath(*path));
if (auto i = shared->tempRoots.find(hashPart); i != shared->tempRoots.end()) {
if (options.action == GCOptions::gcDeleteSpecific)
throw Error(
"Cannot delete path '%s' because it's in use by '%s'.", printStorePath(start), i->second);
return markAlive();
}
shared->pending = hashPart;
Expand Down Expand Up @@ -801,12 +810,7 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)

for (auto & i : options.pathsToDelete) {
deleteReferrersClosure(i);
if (!dead.count(i))
throw Error(
"Cannot delete path '%1%' since it is still alive. "
"To find out why, use: "
"nix-store --query --roots and nix-store --query --referrers",
printStorePath(i));
assert(dead.count(i));
}

} else if (options.maxFreed > 0) {
Expand Down
12 changes: 11 additions & 1 deletion src/libstore/include/nix/store/gc-store.hh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,11 @@

namespace nix {

typedef std::unordered_map<StorePath, std::unordered_set<std::string>> Roots;
// FIXME: should turn this into an std::variant to represent the
// several root types.
using GcRootInfo = std::string;

typedef std::unordered_map<StorePath, std::unordered_set<GcRootInfo>> Roots;

struct GCOptions
{
Expand Down Expand Up @@ -51,6 +55,12 @@ struct GCOptions
* Stop after at least `maxFreed` bytes have been freed.
*/
uint64_t maxFreed{std::numeric_limits<uint64_t>::max()};

/**
* Whether to hide potentially sensitive information about GC
* roots (such as PIDs).
*/
bool censor = false;
};

struct GCResults
Expand Down
1 change: 1 addition & 0 deletions tests/functional/gc-runtime.nix
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ mkDerivation {

cat > $out/program <<EOF
#! ${shell}
echo x > \$TEST_ROOT/fifo
sleep 10000
EOF

Expand Down
9 changes: 7 additions & 2 deletions tests/functional/gc-runtime.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,16 @@ nix-env -p "$profiles/test" -f ./gc-runtime.nix -i gc-runtime
outPath=$(nix-env -p "$profiles/test" -q --no-name --out-path gc-runtime)
echo "$outPath"

fifo="$TEST_ROOT/fifo"
mkfifo "$fifo"

echo "backgrounding program..."
"$profiles"/test/program &
sleep 2 # hack - wait for the program to get started
"$profiles"/test/program "$fifo" &
child=$!
echo PID=$child
cat "$fifo"

expectStderr 1 nix-store --delete "$outPath" | grepQuiet "Cannot delete path.*because it's referenced by the GC root '/proc/"

nix-env -p "$profiles/test" -e gc-runtime
nix-env -p "$profiles/test" --delete-generations old
Expand Down
4 changes: 2 additions & 2 deletions tests/functional/gc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@ if nix-store --gc --print-dead | grep -E "$outPath"$; then false; fi
nix-store --gc --print-dead

inUse=$(readLink "$outPath/reference-to-input-2")
if nix-store --delete "$inUse"; then false; fi
expectStderr 1 nix-store --delete "$inUse" | grepQuiet "Cannot delete path.*because it's referenced by path '"
test -e "$inUse"

if nix-store --delete "$outPath"; then false; fi
expectStderr 1 nix-store --delete "$outPath" | grepQuiet "Cannot delete path.*because it's referenced by the GC root "
test -e "$outPath"

for i in "$NIX_STORE_DIR"/*; do
Expand Down
10 changes: 5 additions & 5 deletions tests/functional/local-overlay-store/delete-refs-inner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@ input2=$(nix-build ../hermetic.nix --no-out-link --arg busybox "$busybox" --arg
input3=$(nix-build ../hermetic.nix --no-out-link --arg busybox "$busybox" --arg withFinalRefs true --arg seed 2 -A passthru.input3 -j0)

# Can't delete because referenced
expectStderr 1 nix-store --delete $input1 | grepQuiet "Cannot delete path"
expectStderr 1 nix-store --delete $input2 | grepQuiet "Cannot delete path"
expectStderr 1 nix-store --delete $input3 | grepQuiet "Cannot delete path"
expectStderr 1 nix-store --delete $input1 | grepQuiet "Cannot delete path.*because it's referenced by path"
expectStderr 1 nix-store --delete $input2 | grepQuiet "Cannot delete path.*because it's referenced by path"
expectStderr 1 nix-store --delete $input3 | grepQuiet "Cannot delete path.*because it's referenced by path"

# These same paths are referenced in the lower layer (by the seed 1
# build done in `initLowerStore`).
expectStderr 1 nix-store --store "$storeA" --delete $input2 | grepQuiet "Cannot delete path"
expectStderr 1 nix-store --store "$storeA" --delete $input3 | grepQuiet "Cannot delete path"
expectStderr 1 nix-store --store "$storeA" --delete $input2 | grepQuiet "Cannot delete path.*because it's referenced by path"
expectStderr 1 nix-store --store "$storeA" --delete $input3 | grepQuiet "Cannot delete path.*because it's referenced by path"

# Can delete
nix-store --delete $hermetic
Expand Down
Loading