---
name: verify-file-transfers
description: "When copying, tarring, or syncing files between systems (Windows to VPS, Windows to Windows, any cross-system transfer), always verify the destination has the expected number of files and reasonable size before declaring success. The failure mode is that tools report exit 0 but silently drop files, especially Windows tar.exe in a loop with -C parent leaf (the staging-dir dance). Exit codes lie; file counts and sizes do not."
---

# Verify file transfers — never trust exit codes alone

## When to use this

Any time you are about to claim a file transfer, copy, tar, or sync is done. Especially:
- `tar -czf` on Windows (the BSD-derived `tar.exe` is the worst offender)
- `scp` from Windows to Linux VPS
- `Copy-Item -Recurse` in PowerShell
- `robocopy`, `rsync`, `xcopy`
- Any "I extracted the tarball" claim in a chat reply

## The rule

After every transfer, run a count + size check on both ends. Match them. If they do not match, the transfer is broken even if the exit code was 0.

## PowerShell pattern (source-side verification before pushing)

```powershell
$src = "C:\path\to\folder"
$expected = (Get-ChildItem -Recurse -File -Force $src -ErrorAction SilentlyContinue | Measure-Object).Count
Write-Host "Source: $expected files"
```

## PowerShell pattern (tarball verification after `tar -czf`)

```powershell
$dst = "C:\path\to\archive.tar.gz"
$actual = (tar -tzf "$dst" | Where-Object { $_ -notmatch '/$' } | Measure-Object).Count
Write-Host "Tarball: $actual files"
# Status: OK if $actual -eq $expected, MISMATCH otherwise
```

## Bash pattern (extraction verification after `tar -xzf` on the destination)

```bash
src_count=$(find /source -type f | wc -l)
dst_count=$(find /destination -type f | wc -l)
echo "src=$src_count dst=$dst_count"
[ "$src_count" = "$dst_count" ] && echo OK || echo MISMATCH
```

## Common gotchas discovered in real sessions

1. **Windows `tar -czf $dst -C $parent $leaf` loop silently drops files** when called multiple times to append to the same archive. Each `tar -czf` call *overwrites* the archive, not appends. If you want true append, use `tar -Af` (concatenate) or tar each project into its own file and concatenate later.
2. **Tarball listing order matters on Linux extract.** If a tarball has file entries before their parent directory entries (Windows tar does this), `tar -xzf` on Linux fails with "Cannot mkdir: No such file or directory." Workaround: extract with Python's `tarfile` module (`tar.extractall(filter='data')`) which handles out-of-order dirs gracefully.
3. **Read-only mount masquerading as BOTH a tar error AND a "successful" extraction.** This one cost three re-extract cycles in the 2026-07-14 multi-project session: on the Hostinger VPS, `/root` is mounted `ro,...` (verify with `findmnt /root`). `tar -xzf /root/bundle.tar.gz -C /root/Business_Projects/` prints `tar: ...: Cannot mkdir: No such file or directory` and exits non-zero. The mistake is treating that as a tarball corruption problem and re-extracting. The actual cause: the OS is returning **EROFS** through tar, and the "No such file or directory" is misleading because *the parent dir already exists* but the new write is being rejected. **Diagnostic protocol before blaming tar:**
   ```bash
   findmnt /root                              # should show ro,... if read-only
   touch /root/probe_$$.tmp 2>&1             # must succeed; "Read-only file system" = confirmed EROFS
   ```
   If confirmed: extract to `/root/.hermes/<path>/` (the writable overlay) instead. `/root/.hermes/` is a separate read-write mount layered over the read-only `/root`. Pre-existing files in `/root/Business_Projects/` may appear intact because they were written before the mount went read-only — but **you cannot add new files there**. Use Python's tarfile module to extract cleanly:
   ```python
   import tarfile, os
   os.makedirs("/root/.hermes/Business_Projects", exist_ok=True)
   with tarfile.open("/root/bundle.tar.gz", "r:gz") as tar:
       tar.extractall(path="/root/.hermes/Business_Projects", filter="data")
   ```
4. **Spaces in paths break `tar -C` arguments** in PowerShell unless you wrap the path with explicit double quotes in the call. The working pattern (verified on the 2026-07-14 multi-project extraction):
   ```powershell
   # Works: $parent and $leaf are quoted when passed to tar.exe
   $parent = Split-Path $stage -Parent
   $leaf   = Split-Path $stage -Leaf
   tar -czf "$dst" -C "$parent" "$leaf"
   ```
   The earlier version of this pitfall said "wrap with explicit double quotes" without showing the working call — this is the verified-good form.
5. **Scp with password auth works on Windows without SSH keys** (OpenSSH is built into Windows 10/11). First-time prompts for host key fingerprint, then password. No setup needed.

## When the verification fails

Do NOT silently retry. Diagnose:
1. Print the actual tarball listing (`tar -tzf`) to see what made it
2. Compare against source listing (`Get-ChildItem -Recurse`)
3. Identify what is missing — usually a pattern (e.g. all files in one subdirectory, all `.png` files, all files >1MB)
4. Re-bundle with a different approach (Python tarfile, individual files, or zip+scp)
5. Re-verify before declaring done