Tilde pointed at a profile home, so the export landed elsewhere

·3 mins

The export command finished, but the expected directory was empty. Then Python reported [Errno 2] No such file or directory for a script that existed in the user’s home. The runner’s HOME pointed at a profile-specific directory. Fixing this required explicit paths for both the export destination and the Python environment.

Two homes, two plausible directory trees

An initial attempt to read a helper resolved to a profile-home path where the file did not exist. During the later export, cd ~/transfer succeeded because that directory existed too. A virtual environment had been created there, but the actual transfer script lived under the user’s expected home.

The terminal output settled the discrepancy: printing HOME returned the profile-specific home. Comparing the two transfer directories found the script only under the expected user-home root. Comparing output directories found exported data under that root and an empty directory under the profile home. The report also existed only under the expected root.

Changing directories alone would still leave Python’s Path("~/data").expanduser() following the process’s HOME. The repair replaced tilde-based output defaults with explicit absolute destinations, including separate report and log paths. The script supported overrides such as KB_OUT_ROOT. Running it from the intended transfer directory also selected that directory’s .venv/bin/python.

This was observed in a historical Hermes runner environment. It does not establish that every Hermes profile changes HOME; consult the current Hermes documentation for your setup. A configured profile directory and the home directory you expect are separate facts to check.

Check the process before searching for files

Run these inside the same terminal tool or wrapper that launches the job:

printf 'HOME=%s\n' "$HOME"
id
pwd
python3 -c 'import sys; from pathlib import Path; print(sys.executable); print(Path.home())'

Compare those paths with the expected project root. Limit directory checks to those known roots; a filesystem-wide search adds noise and can expose unrelated private files. A successful cd proves only that a directory exists, not that it contains the intended script or environment.

Shell expansion happens before the child starts

In Bash, unquoted ~/data expands in the calling shell. In env HOME=/somewhere python3 ... ~/data, the argument has already expanded before env changes the child’s environment. Single-quoted '~/data' stays literal; Python must expand it explicitly. On Unix, Path.expanduser() uses HOME when it is set.

This isolated example demonstrates both cases without touching real home directories:

SCRATCH="${TMPDIR:?Set TMPDIR to your scratch directory}/home-path-demo"
mkdir -p "$SCRATCH/caller-home" "$SCRATCH/profile-home"
PYTHON="$(command -v python3)"
export SCRATCH PYTHON
HOME="$SCRATCH/caller-home" bash --noprofile --norc <<'SH'
env HOME="$SCRATCH/profile-home" "$PYTHON" -c \
  'import sys; print(sys.argv[1])' ~/data
env HOME="$SCRATCH/profile-home" "$PYTHON" -c \
  'import sys; from pathlib import Path; print(sys.argv[1]); print(Path(sys.argv[1]).expanduser())' '~/data'
SH

The first command prints the caller’s home path. The second prints the literal ~/data, then the profile-home path.

Pin the interpreter and verify the artifact

Here is a standalone local inventory example. The program reads its input directory as read-only; the output root is explicit. It requires Python’s venv module and uses only scratch fixtures.

SCRATCH="${TMPDIR:?Set TMPDIR to your scratch directory}/home-path-demo"
mkdir -p "$SCRATCH/source" "$SCRATCH/exports"
printf 'fixture\n' > "$SCRATCH/source/sample.txt"
python3 -m venv --without-pip "$SCRATCH/venv"
export DATA_ROOT="$SCRATCH/source" KB_OUT_ROOT="$SCRATCH/exports"
"$SCRATCH/venv/bin/python" <<'PY'
import os
import sys
from pathlib import Path

source = Path(os.environ["DATA_ROOT"])
out = Path(os.environ["KB_OUT_ROOT"])
if not source.is_absolute() or not out.is_absolute():
    raise ValueError("Use absolute input and output roots")
artifact = out / "inventory.txt"
artifact.write_text("\n".join(sorted(p.name for p in source.glob("*.txt"))) + "\n")
assert artifact.read_text() == "sample.txt\n"
print("interpreter:", sys.executable)
print("verified artifact:", artifact.resolve())
PY

For a real export, validate expected contents at the selected destination and check the report there. In the failed run, a pipeline ending in tail returned zero even though Python could not open the script. Command status alone missed both the launch failure and the path mismatch. Check your job’s resolved interpreter and output root before its next run.