Xfer Files

A macOS-first copy tool that mirrors files from a source drive to a destination drive with priority-aware copying and explicit include/exclude rules.

Features

• Resumable copies that can be started, stopped, and resumed.
• Concurrent scan, hash, copy, verification, and repair stages so discovery and transfer progress can move at the same time.
• A central in-memory coordinator serves live dashboard state and batches recovery checkpoints to reduce metadata writes on the destination drive.
• High-priority folder selection for Desktop, Documents, Downloads, Pictures, Projects, and more.
• Hash-aware skip logic that leaves matching files alone and recopies same-size mismatches safely.
• Streaming-safe hashing and copying that avoid loading very large files into memory.
• Mirror-style copy layout under the destination root with a reserved state folder.
• A lightweight web UI with file progress, byte progress, throughput, queue depth, ETA, and a transfer-rate graph.
• Selectable Node.js or system-rsync transfer engines from the dashboard.

Getting started

1. Install dependencies: npm install
2. Copy the example config: cp config.example.json config.local.json
3. Edit config.local.json to point at your source folders and external backup drive.
4. Start the web dashboard: npm start
5. Open the dashboard and use its start/stop toggle button.
6. Use Recycle service after changing code to checkpoint and stop all workers, restart the Node process, and leave the backup ready for a manual start.
7. View current status with npm run backup:status

The CLI can start rsync mode directly with npm run backup:start -- --rsync.

Configuration

• sourceRoot should point to the macOS home directory or a specific folder to back up.
• destinationRoot should point to an external USB drive such as /Volumes/BackupDrive/xfer-files.
• transferEngine sets the dashboard default to node or rsync; it can still be changed before each run.
• rsyncChecksum defaults to true, making rsync compare file content and apply metadata-only timestamp changes when hashes match. Set it to false only if size/mtime quick checks are preferred over correctness.
• include controls which top-level folders are included.
• chunkedCopyThresholdMb controls when files switch to chunked, resumable copying. The default is 50.
• copyConcurrency controls how many copy workers run at once. The default is 4.
• parallelCopyMaxSizeMb limits the extra parallel workers to smaller files. The default is 4; keeping this low avoids seek contention on spinning disks.
• highPriority controls which folders are copied first.

Notes

• The example config is stored in config.example.json and is safe to commit.
• Your local overrides live in config.local.json, which is ignored by git.
• Backup state is stored under .xfer-state/state.json in the destination root.
• Per-file metadata is stored under .xfer-state in the destination root.
• Pending verification and repair work is persisted in .xfer-state/verification-queue.json and .xfer-state/repair-queue.json so it survives restarts.
• Routine state, queue, and chunk-manifest checkpoints are batched over five-second windows. A controlled stop or service recycle forces the latest state to disk before workers exit.
• Rsync mode uses the system rsync, preserves interrupted data under .xfer-state/rsync-partial, and maps its per-file progress into the dashboard. It uses rsync's native size/mtime comparison rather than the Node engine's SHA-256 verification and repair pipeline.
• Rsync progress uses the file and byte totals retained from the previous scan. Files rsync does not need to transfer are reconciled as skipped when the run finishes.
• When a completed Node scan has a saved pending queue, rsync consumes that catalog with --files-from=- instead of recursively rediscovering the source tree. The dashboard shows a timed rsync-indexing phase until the first file record arrives.
• Node copies preserve source access and modification times. Before the first rsync run after this upgrade, existing SHA-256 metadata is used for a one-time timestamp reconciliation recorded in .xfer-state/timestamps-v1.json.
• Standard Node and rsync APIs preserve modification/access timestamps but cannot reset the macOS creation/birth timestamp.
• Backup errors are appended to .xfer-state/errors.ndjson in the destination root.