File Systems

What a file system does

A drive, whether a hard disk, an SSD or a USB stick, only offers a long row of numbered blocks. Nobody wants to remember that a spreadsheet lives in blocks 81,920 to 81,934. A file system is the structure and the set of rules an operating system uses to store data on a volume as named files in folders, keeping track of where each file's data lies, its metadata and which space is free. Concretely, it handles:

• Names: which characters are allowed, how long a name may be, and whether Report.docx and report.docx are the same file.
• Folders: the tree that groups files.
• Metadata: size, timestamps (created, modified, accessed), attributes such as hidden or read-only, owner and permissions.
• Allocation and free space: which blocks belong to which file, and which are still free.
• Consistency: keeping all of that correct when the power fails halfway through a change.

A file system is created on a partition or volume by formatting it. The operating system needs a driver for each file system it wants to read, which is why a disk formatted on a Mac may be unreadable on a Windows PC.

Files, folders and paths

A file is a named sequence of bytes. What those bytes mean (a photo, a document, a program) is up to the program that reads them; the file system only stores them. A folder (the Windows and macOS word) or directory (the Unix word and the technical term) is a container of files and other folders. Folders nest, which makes a tree with one root.

Paths

A path is the route through that tree to a file. Windows and Unix-like systems write it differently:

An absolute path starts at the root and means the same thing wherever you are. A relative path starts at the current folder: . is the current folder and .. the one above it. Windows does not allow the characters \ / : * ? " < > | in file names, and many older programs still fail on paths longer than 260 characters. For working with paths in code, see Files and Paths.

File extensions and hidden files

A file extension is the last part of a file name after the final dot, such as .docx, .pdf or .exe, which the operating system uses to decide which program opens the file and how to treat it. Windows relies on it completely: rename photo.jpg to photo.txt and a double-click opens Notepad, although the bytes have not changed. Linux cares much less; there the executable permission and the file's content decide what can run.

File Explorer hides extensions of known file types by default. Attackers use that: invoice.pdf.exe shows up as invoice.pdf, with a program icon that is easy to miss. Technicians turn extensions on (File Explorer, View › Show › File name extensions) and often teach users to do the same.

Hidden files work differently per system. Windows has a hidden attribute (and a system attribute for operating-system files), shown with View › Show › Hidden items. On Linux and macOS, a name starting with a dot is hidden, such as .bashrc; ls -a shows them, and in the macOS Finder Cmd+Shift+. toggles them. Hiding is a convenience, not protection: anyone who can read the folder can see hidden files.

attrib +h C:\Users\ana\notes.txt
dir /a:h C:\Users\ana

attrib +h sets the hidden attribute (-h removes it); dir /a:h lists only hidden items.

Clusters, blocks and slack space

A file system does not hand out single bytes. It allocates space in fixed units called clusters (Windows term, also allocation units) or blocks (Unix term), usually 4 KiB. A file always occupies whole clusters, so the unused end of its last cluster is wasted: that is slack space. In a file's Properties window, Windows shows it as the difference between Size and Size on disk.

Larger clusters mean less bookkeeping and slightly faster access to big files, but more slack for small files. This example shows the same four files on 4 KiB and 64 KiB clusters:

import math

files = {                       # name: size in bytes
    "notes.txt": 1_200,
    "invoice.pdf": 58_300,
    "photo.jpg": 3_145_728,
    "setup.log": 4_097,
}

def usage(size, cluster):
    clusters = math.ceil(size / cluster)     # a partly used cluster counts whole
    slack = clusters * cluster - size        # unused bytes in the last cluster
    return clusters, slack

for cluster in (4096, 65536):
    print(f"Cluster size {cluster // 1024} KiB")
    print(f"  {'file':12} {'size':>9}  {'clusters':>8}  {'slack':>6}")
    total_slack = 0
    for name, size in files.items():
        clusters, slack = usage(size, cluster)
        total_slack += slack
        print(f"  {name:12} {size:>9,}  {clusters:>8}  {slack:>6,}")
    print(f"  total slack: {total_slack:,} bytes")

Cluster size 4 KiB
  file              size  clusters   slack
  notes.txt        1,200         1   2,896
  invoice.pdf     58,300        15   3,140
  photo.jpg    3,145,728       768       0
  setup.log        4,097         2   4,095
  total slack: 10,131 bytes
Cluster size 64 KiB
  file              size  clusters   slack
  notes.txt        1,200         1  64,336
  invoice.pdf     58,300         1   7,236
  photo.jpg    3,145,728        48       0
  setup.log        4,097         1  61,439
  total slack: 133,011 bytes

One byte over a cluster boundary (setup.log, 4,097 bytes) costs a whole extra cluster. With thousands of small files, big clusters waste a lot of space; with a few huge video files they hardly matter. That is why the default (4 KiB for NTFS on normal volume sizes) is almost always right. Slack space also matters in forensics: the end of a cluster can still hold leftovers of an older, deleted file.

The common file systems

The table compares the file systems a technician meets. The size limits are those of the file system itself; tools and operating systems sometimes set lower ones.

FAT32

FAT32 (File Allocation Table, 32-bit) is a simple file system from the Windows 95 era that almost every device can read and write, but that cannot store files of 4 GiB or larger and has no permissions or journaling. Its strength is compatibility: car radios, TVs, printers, cameras and motherboard firmware-update tools often accept nothing else. The 4 GiB limit comes from the file size field, which is 32 bits. A 5 GB video simply cannot be copied to a FAT32 stick: Windows reports that the file is too large for the destination file system, even with plenty of free space.

FAT32 itself supports volumes up to 2 TiB, but the Windows format tools have long refused to create FAT32 volumes larger than 32 GB, offering exFAT or NTFS instead. (Windows 11 Insider builds since 2024 let the command-line format create FAT32 volumes up to 2 TB; the graphical tools still stop at 32 GB.) The Unified Extensible Firmware Interface (UEFI) EFI System Partition is always FAT32, because the firmware has to read it.

exFAT

exFAT (Extended File Allocation Table) is a lightweight file system designed by Microsoft for flash media that removes FAT32's 4 GiB file limit while staying simple enough for cameras and other devices. It is the required file system of SDXC and SDUC (Secure Digital Ultra Capacity) memory cards, and Windows, macOS and Linux (kernel 5.7 and later) all read and write it natively. Like FAT32, it has no permissions and no journal, so always eject the drive safely before pulling it out.

NTFS

NTFS (New Technology File System) is the standard Windows file system, and the one Windows must be installed on. It adds everything the FAT family lacks:

• Permissions: an access control list per file and folder; see NTFS permissions.
• Journaling of metadata, for fast recovery after a crash.
• Compression per file or folder, transparent to programs.
• Encrypting File System (EFS): encryption of single files and folders tied to the user's account, available in Pro, Enterprise and Education editions. BitLocker, by contrast, encrypts the whole volume and is not part of NTFS.
• Disk quotas: a maximum amount of space per user on a volume.
• Also hard links, shadow copies (used by System Restore and "Previous versions") and very large volumes.

macOS can read NTFS but not write to it without extra software; current Linux kernels read and write it.

ReFS

The Resilient File System (ReFS) is Microsoft's newer file system for servers and large storage, often combined with Storage Spaces. It checksums its metadata (and, with integrity streams, file data too) and can repair damage from a mirror copy, and it never overwrites metadata in place. It lacks some NTFS features (such as EFS and disk quotas) and Windows cannot boot from it. On Windows 11 you meet it mainly in the Dev Drive feature for developers.

ext4 and XFS

ext4 (fourth extended file system) is the default journaling file system of many Linux distributions, including Debian and Ubuntu, with Unix permissions, large volume support and a long record of reliability. Windows cannot read it without extra tools.

XFS is a high-performance journaling file system, strong with very large files and many parallel writes. It has been the default of Red Hat Enterprise Linux since version 7. An XFS volume can be enlarged but not shrunk, which matters when planning disk space. Other Linux options you may see are Btrfs (default on Fedora and openSUSE) with snapshots and checksums.

APFS and HFS+

APFS (Apple File System) is the file system of macOS since 10.13 High Sierra and of iOS and iPadOS, designed for SSDs, with snapshots, native encryption, space sharing between volumes and instant file clones. A clone is a copy that takes no extra space until one of the two is changed; a snapshot is a read-only picture of the whole volume at one moment, which Time Machine and macOS updates use. Several APFS volumes share the free space of one container (see macOS disks).

Its predecessor, HFS+ (Mac OS Extended), is journaled and still appears on older external drives and on spinning disks formatted long ago. A current Mac reads and writes it, but new internal disks use APFS.

Journaling

Saving a file changes several structures at once: the file's data, its entry in a folder, its size and timestamps, and the map of free space. If the power fails halfway, those structures disagree with each other: clusters marked as used that belong to no file, or a folder entry pointing to garbage. Without a journal, the only way to find such damage is a full scan of the file system (CHKDSK on Windows, fsck on Linux), which can take hours on a large disk.

Journaling is a technique in which the file system first writes a description of each change to a log (the journal), then makes the change, then marks it complete, so that after a crash it can replay or discard unfinished changes instead of scanning the whole volume. Recovery takes seconds, and the file system's structure is consistent again.

Most journals, including those of NTFS and ext4 in their default modes, protect metadata, not the contents of files. After a power cut the file system is intact, but the document someone was saving may still be half-written. Copy-on-write file systems such as APFS, ReFS and Btrfs reach the same goal differently: they write changes to a new place and switch over only when the write is complete, so the old version stays valid until then.

Inodes and the MFT

Under the folders, each file system keeps one record per file with its metadata and the location of its data.

In Unix-like file systems such as ext4 and XFS, that record is an inode (index node): a numbered data structure that stores a file's owner, permissions, size, timestamps and the location of its data blocks, but not its name. The name lives in the folder, which is just a list of names with inode numbers. That is why a file can have several names (hard links), and why a disk can report "No space left on device" while df -h shows free space: it has run out of inodes (on ext4 their number is fixed when the volume is formatted), which df -i reveals. It happens with millions of tiny files.

ls -li /home/ana/Documents

total 64
1835021 -rw-r--r-- 1 ana ana 58300 Oct  1 09:14 invoice.pdf
1835022 -rw-r--r-- 1 ana ana  1200 Oct  1 09:20 notes.txt

The first column is the inode number. NTFS uses the Master File Table (MFT): a special file with one record (normally 1 KiB) per file and folder, holding its name, attributes, security information and the location of its data. A very small file fits entirely inside its MFT record and uses no cluster at all. In both designs, damage to these records is what CHKDSK and fsck repair.

Choosing a file system

Most of the time the operating system chooses for you. When you do choose, compatibility with the devices that must read the disk comes first:

To see which file system a volume uses: in Windows, the File system line in the drive's Properties, or PowerShell (illustrative output):

Get-Volume

DriveLetter FriendlyName FileSystemType DriveType HealthStatus OperationalStatus SizeRemaining      Size
----------- ------------ -------------- --------- ------------ ----------------- -------------      ----
C                        NTFS           Fixed     Healthy      OK                    312.4 GB 475.68 GB
E           BACKUP       exFAT          Removable Healthy      OK                     58.1 GB  59.75 GB

On Linux, df -hT adds a Type column:

df -hT

Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/nvme0n1p2 ext4  915G  212G  657G  25% /
/dev/nvme0n1p1 vfat  511M  6.1M  505M   2% /boot/efi
/dev/sda1      xfs   3.7T  1.9T  1.8T  52% /srv/data

Linux reports FAT32 as vfat; here it is the EFI System Partition.