article thumbnail

Tar & Archiving

The Tape-Archive Tool That Outlived the Tape

13 min read
#ostools, #tar, #archiving, #compression, #commandlinetools, #friday2

Almost every Linux download you've ever fetched — source code, a Docker layer, a Kubernetes release, a backup someone handed you — arrived as a .tar.gz, a .tar.xz, or a bare .tar. The tool behind all of them is older than most of the software it ships: tar, short for tape archive, built for a piece of hardware almost nobody uses anymore. It survived the death of magnetic tape because it solved a problem that never went away — bundling a directory tree, permissions and all, into one stream you can move as a single unit.

A Brief History

tar first appeared in 1979, in Version 7 Unix, designed to write a stream of files sequentially to a tape drive — no random access, no index up front, just files one after another, exactly the way a spinning tape has to work. That constraint is still visible in tar's behavior today: it reads and writes as a single continuous stream, which is precisely why it pipes so naturally over a network or into a compressor.

As Unix fragmented into competing vendors, tar implementations drifted apart, so POSIX standardized a common format in 1988 — ustar (Unix Standard TAR) — that most modern tar programs can still read. Ustar had a real limitation, though: filenames and paths were capped at 100–256 characters. The pax format, standardized in 2001, lifted that cap and added extended metadata. GNU tar, the version installed on virtually every Linux distribution, supports both, plus its own long-standing extensions (like sparse-file handling), and is what the rest of this article assumes.

The Core Idea: Archiving Is Not Compression

This is the distinction that confuses more people than anything else about tar: archiving and compression are two separate jobs, and tar only really does the first one.

A bare tar archive isn't compressed at all; it's the same total size as the files it contains, plus a small amount of header overhead. tar became the compressed-archive tool you know by learning to hand its output stream straight to a compressor — which is exactly what the -z/-j/-J flags below do for you in one step.

The Essential Commands

Five flags cover the overwhelming majority of real-world tar use:

Flag Meaning
-c create a new archive
-x extract an existing archive
-t list the table of contents — what's inside, without extracting
-v verbose — print each filename as it's processed
-f <file> the archive filename — almost always required, and conventionally listed last
tar -cvf backup.tar mydir/        # create backup.tar from mydir/
tar -tvf backup.tar               # list what's inside, without extracting
tar -xvf backup.tar               # extract into the current directory
tar -xvf backup.tar -C /restore/  # extract into a specific directory

Add a compression flag and tar pipes its stream through that compressor automatically:

Flag Compressor Typical extension
-z gzip .tar.gz / .tgz
-j bzip2 .tar.bz2
-J xz .tar.xz
--zstd zstd (GNU tar 1.31+) .tar.zst
tar -czvf backup.tar.gz mydir/    # create, gzip-compressed
tar -xzvf backup.tar.gz           # extract a gzip-compressed archive
tar -cJvf backup.tar.xz mydir/    # create, xz-compressed (smaller, slower)

GNU tar is smart enough to auto-detect the compression on extract from the file's contents in most cases, but naming the right flag is still good habit — and required for creation, since tar obviously can't guess what you want before the file exists.

Choosing a Compressor

Compressor Speed Ratio Use it when
gzip (-z) Fast Decent The default choice — universally supported, quick
bzip2 (-j) Slower Better Rarely the best choice anymore — mostly seen in older archives
xz (-J) Slow Best (of the classics) Distributing something once, size matters more than time
zstd (--zstd) Very fast Excellent The modern default when it's available — often beats gzip on both speed and ratio
pigz Fast (parallel) Same as gzip A drop-in, multi-core-aware gzip replacement for large archives — pair via tar -c mydir/ \| pigz > backup.tar.gz

If your tar supports --zstd and you're not tied to compatibility with very old systems, it's the best general-purpose default in 2026.

Why tar and Not Just zip?

Both bundle files, but they optimize for different things:

tar (+ gzip/xz/zstd) .zip
Compresses per-file or as one stream? One continuous stream — usually better overall ratio Per-file — you can extract or preview one file without touching the rest
Preserves Unix permissions, ownership, symlinks, hardlinks? Yes, natively Poorly to not at all
Native on Windows? No (needs a tool) Yes
Native on Linux/macOS? Yes Needs zip/unzip installed
Good for streaming/piping (network, tape)? Yes — it's designed for it Not really — needs seekable storage

Rule of thumb: use tar for anything Unix-flavored, permission-sensitive, or destined for a pipe; use .zip for anything you're handing to someone on Windows who just wants to double-click it.

Excluding Files and Working Selectively

tar -czvf project.tar.gz --exclude='*.log' --exclude='node_modules' project/

--exclude accepts glob patterns and can be repeated as many times as needed — essential for keeping build artifacts and dependency folders out of an archive you're about to email or upload.

The Pipe Trick: Copying a Directory Tree Over SSH

Because tar streams, you can archive on one machine and unpack on another without ever writing an intermediate file to disk:

tar -cf - mydir/ | ssh user@remote "tar -xf - -C /destination/"

The - in place of a filename means "stdin/stdout instead of a real file" — a small trick that turns tar into a full directory-copy tool that preserves every permission bit along the way, something scp -r doesn't reliably do.

Incremental Backups

GNU tar can track what's already been backed up and archive only what changed since:

tar --create --file=full.tar --listed-incremental=snapshot.state /data/
# ...time passes, files change...
tar --create --file=incremental1.tar --listed-incremental=snapshot.state /data/

The snapshot.state file records what tar has already seen; each subsequent run against the same snapshot file captures only the delta. Restoring means applying the full archive and then each incremental archive, in order.

Common Pitfalls

Tooling Worth Knowing

Why It Still Matters

Every Linux source release, most Docker image layers, Kubernetes Helm charts, and the overwhelming majority of Unix-world backups still move as a tar stream, compressed or not. The tape drives it was built for are museum pieces, but the design constraint they imposed — read and write as one continuous, permission-preserving stream — turned out to be exactly what modern pipes, network transfers, and container layers needed all along. Learn -czvf, -xzvf, -tvf, and --exclude, and you'll have covered the tar invocations you'll actually type for the rest of your career.

What did you learn?

Tell us what you learned from this article, or what you liked about it. We read every comment.

Comments appear after a quick review. Please no links.

Know someone who'd love it too?

Most covered topics