# Multi-Project Tarball — pattern and pitfalls

When you need to push more than one Windows folder to the VPS in a single upload (multiple `Project_*` subfolders of `Business_Projects/`, multiple skills, etc.), the natural reflex is to bundle them all into one tarball and push once. This works — but Windows `tar.exe` has at least two failure modes that bite when you do it the obvious way. The pattern below is the one that survived the 2026-07-14 audit (3 projects, 23 files expected, 8 actually bundled on the first try).

## The two failure modes (and the verification that catches them)

1. **The staging-dir dance silently drops files.** `tar -czf $dst -C (Split-Path $src) (Split-Path $src -Leaf)` after `Copy-Item -Recurse` of each subfolder into a single staging dir: Windows tar reports exit 0, the tarball is the right size for a few files, but most of the projects aren't in it. The exit code lies.
2. **Tarball listing order on Linux extract.** If a tarball has file entries before their parent directory entries (Windows tar does this), `tar -xzf` on Linux fails with `Cannot mkdir: No such file or directory` even though most files extract successfully. The errors are noise; verify with `find ... -type f | wc -l`.

Both fail the "exit code = success" reflex. The fix is **verify by content count, not by exit code**, at every step. See `verify-file-transfers` for the full protocol.

## Pattern 1 — tar each project in place, into its own file, concatenate

The most reliable approach. No staging dir, no `Copy-Item` to silently drop things, no need to figure out the staging-dir layout.

```powershell
$srcDir = "C:\Users\Rob\Business_Projects"
$dlDir  = "C:\Users\Rob\Downloads"
$projects = @("Project_1_Job_Seeker", "Project_2_Pipeline_Layer", "Project_3_Real_Results_Front_Desk")

$results = @()
foreach ($folder in $projects) {
  $dst = Join-Path $dlDir "$folder.tar.gz"
  $projectPath = Join-Path $srcDir $folder

  if (Test-Path $dst) { Remove-Item -Force $dst }

  $expected = (Get-ChildItem -Recurse -File -Force $projectPath -ErrorAction SilentlyContinue | Measure-Object).Count
  tar -czf "$dst" -C "$srcDir" "$folder"
  $tarExit = $LASTEXITCODE

  $actual = (tar -tzf "$dst" | Where-Object { $_ -notmatch '/$' } | Measure-Object).Count
  $size = (Get-Item $dst).Length

  $match = if ($actual -eq $expected) { "OK" } else { "MISMATCH" }
  Write-Host "$folder  expected=$expected  actual=$actual  tar-exit=$tarExit  size=$size  $match" -ForegroundColor $(if ($match -eq "OK") { "Green" } else { "Red" })

  $results += [PSCustomObject]@{
    Project = $folder
    Expected = $expected
    Actual = $actual
    SizeKB = [math]::Round($size/1KB, 1)
    Status = $match
  }
}

$results | Format-Table -AutoSize
```

**Output is a table**: each project shows expected file count, actual file count in the tarball, tar exit code, size, and OK/MISMATCH. If any project shows MISMATCH, that one didn't tar correctly and you have an actionable diagnostic instead of a vague "something's wrong."

**Why this is the most reliable pattern:**
- No `Copy-Item` step (so no chance of silent file drops)
- No staging dir to clean up
- Per-project exit code so you see WHICH project failed
- File count verification before any scp

## Pattern 2 — the staging-dir dance, with verification

If you really do want one tarball (smaller upload, one extraction step), here's the corrected form of the staging-dir pattern with the verification inline:

```powershell
$srcDir = "C:\Users\Rob\Business_Projects"
$stage  = "C:\Users\Rob\Downloads\projects_stage"
$dst    = "C:\Users\Rob\Downloads\projects.tar.gz"
$folders = @("Project_1_Job_Seeker", "Project_2_Pipeline_Layer", "Project_3_Real_Results_Front_Desk")

# Clean slate
if (Test-Path $stage) { Remove-Item -Recurse -Force $stage }
if (Test-Path $dst)   { Remove-Item -Force $dst }
New-Item -ItemType Directory -Path $stage | Out-Null

# Stage each project, count after
$stagedTotal = 0
foreach ($f in $folders) {
  $from = Join-Path $srcDir $f
  $to   = Join-Path $stage $f
  if (Test-Path $from) {
    Copy-Item -Recurse -Force $from $to
    $copied = (Get-ChildItem -Recurse -File -Force $to | Measure-Object).Count
    Write-Host "Staged $f: $copied files"
    $stagedTotal += $copied
  }
}

# Verify staging before tar
$stagedActual = (Get-ChildItem -Recurse -File -Force $stage | Measure-Object).Count
Write-Host "Staging has $stagedActual files (expected $stagedTotal)" -ForegroundColor $(if ($stagedActual -eq $stagedTotal) { "Green" } else { "Red" })

# Tar
$parent = Split-Path $stage -Parent
$leaf   = Split-Path $stage -Leaf
tar -czf "$dst" -C "$parent" "$leaf"
Write-Host "tar exit: $LASTEXITCODE"

# Verify tarball
$tarballCount = (tar -tzf "$dst" | Where-Object { $_ -notmatch '/$' } | Measure-Object).Count
Write-Host "Tarball has $tarballCount files (expected $stagedActual)" -ForegroundColor $(if ($tarballCount -eq $stagedActual) { "Green" } else { "Red" })
Get-Item $dst | Select-Object Name, Length | Format-Table -AutoSize
```

The verification is the difference between this and the version that fails. Each step counts files, and the script tells you which step dropped things.

**Common reason the staging-dir version drops files:** Windows `tar -czf` invoked in a loop with the same destination overwrites silently (it does not append). If you run `tar -czf $dst -C $srcDir $folder1` and then `tar -czf $dst -C $srcDir $folder2`, the second call REPLACES the first. Use `tar -Af` for true append, or use Pattern 1 (one tar per project) which sidesteps the issue entirely.

## Extracting on the VPS — Python fallback for out-of-order entries

The Windows tarball will have file entries listed before their parent directory entries. BSD/GNU `tar` on Linux refuses to create a file in a directory that hasn't been "announced" yet, so plain `tar -xzf` may print many `Cannot mkdir: No such file or directory` errors. The actual files usually extract fine; the errors are noise.

**Two ways to handle it:**

1. **Just verify and move on.** Run the extraction, then count files on the VPS. If the count matches, you're done. The errors are cosmetic.

   ```bash
   mkdir -p /root/Business_Projects
   tar -xzf /root/projects.tar.gz -C /root/Business_Projects/ 2>&1 | tail -5  # see only last few errors
   find /root/Business_Projects -type f | wc -l
   ```

2. **Use Python's tarfile module** if you want clean output. It handles out-of-order entries gracefully.

   ```python
   import tarfile, os
   with tarfile.open("/root/projects.tar.gz", "r:gz") as tar:
       tar.extractall(path="/root/Business_Projects", filter='data')
   ```

**If `/root` is read-only on the VPS** (Hostinger sets this — verify with `findmnt /root` showing `ro,...`), extract to `/root/.hermes/Business_Projects/` instead. The `/root/.hermes/` directory is mounted read-write as an overlay and is the right home for new content. See the main SKILL.md "Pitfalls" section for the full diagnosis.

## Decision matrix

| Scenario | Pattern | Why |
|---|---|---|
| Small number of projects, you want one upload | 2 (staging-dir) with verification | One scp, one extraction |
| Larger or more projects, or you want per-project diagnostics | 1 (per-project tarball) | Per-project OK/MISMATCH table is the most actionable failure mode |
| You need to extract quickly without debugging tar errors | Either pattern + Python tarfile | Clean output, no error noise to interpret |
| Sensitive content (PII, tokens) in any folder | Either pattern, but scrub before tar | Same risk as single-project transfers |