What Are Regular Expressions?

What a regular expression is

A regular expression (regex, sometimes regexp) is a pattern that describes a set of strings. Instead of searching for one fixed word, you describe the shape of what you want: "a ticket number", "a line that starts with a date", "an IP address after the word from". The pattern INC-\d{5}, for example, means the letters INC- followed by exactly five digits, so it finds INC-31877 and INC-40211 but not INC-12 or CHG-4410.

Working with patterns like this is called pattern matching: checking whether text, or a part of it, fits a described form instead of being equal to one fixed value. The shell's wildcards (*.log) are a much simpler form of pattern matching for file names; regular expressions are far more expressive and work on any text.

A pattern is built from two kinds of characters. Most characters are literals that match themselves: INC- matches exactly those four characters. A few are metacharacters with a special meaning, such as . (any character), * (repeat) and \d (a digit). The next pages explain each building block; this page is about the big picture.

import re

note = "Merged INC-31877 into INC-31902; CHG-4410 rolls back the change. See INC-12."
print(re.findall(r"INC-\d{5}", note))

['INC-31877', 'INC-31902']

re.findall returns every piece of the text that fits the pattern. INC-12 is not in the list because it has only two digits, and CHG-4410 does not start with INC.

Where you meet them in IT work

Regular expressions turn up in far more places than programming:

• Searching logs and files: grep, journalctl's --grep, less (/pattern) and the search boxes of most log platforms. See grep, find and regular expressions and logs and monitoring.
• Editing text in bulk: sed, awk, and the "find and replace with regex" option in editors such as VS Code, vim and Notepad++.
• Monitoring and security rules: alert filters, syslog parsing, web application firewall (WAF) rules and intrusion detection signatures are often written as regexes.
• Configuration: web servers (URL rewrite rules, nginx location ~ blocks), mail filters, and .gitignore-like include and exclude lists in some tools.
• Input checks: the HTML pattern attribute, and validation code on the server.
• Programming: Python's re module, JavaScript's RegExp, PHP's preg_ functions, and the same idea in Java, C#, Go, Rust and others.

The same idea runs everywhere. Here is one search, "every ticket number in this log", in the four tools this topic uses. The shell version writes a small log file first and then searches it:

cat <<'EOF' > portal.log
2026-11-05 08:12:44 WARN  login slow for user 4471 (INC-31902)
2026-11-05 08:13:02 INFO  meter KE-0042 reading accepted
2026-11-05 08:15:30 ERROR session store timeout (INC-31902, INC-31905)
EOF
grep -oE 'INC-[0-9]{5}' portal.log

INC-31902
INC-31902
INC-31905

import re

line = "2026-11-05 08:15:30 ERROR session store timeout (INC-31902, INC-31905)"
print(re.findall(r"INC-\d{5}", line))

['INC-31902', 'INC-31905']

const line = '2026-11-05 08:15:30 ERROR session store timeout (INC-31902, INC-31905)';
console.log(line.match(/INC-\d{5}/g));

[ 'INC-31902', 'INC-31905' ]

<?php
$line = '2026-11-05 08:15:30 ERROR session store timeout (INC-31902, INC-31905)';
preg_match_all('/INC-\d{5}/', $line, $matches);
echo implode(', ', $matches[0]), "\n";

INC-31902, INC-31905

Notice the small differences already. The shell version uses [0-9] instead of \d, because plain grep does not know \d (more on that below). Python writes the pattern as a raw string r"…", JavaScript between slashes, and PHP inside a string with its own /…/ delimiters. The page on literals, metacharacters and escaping explains why.

How a match is found

The program that reads a pattern and runs it against text is the regex engine. Python, Node.js and PHP each have one built in; grep, sed and awk have their own. Most engines first compile the pattern into an internal form (a small program or a state machine), then walk through the text.

A match is a piece of the text that the whole pattern fits, described by where it starts and where it ends. The engine looks for it in a simple, fixed way: it tries the pattern at position 0; if the pattern does not fit there, it tries again at position 1, then 2, and so on. The first position where the whole pattern fits wins. A search that wants all matches continues after the end of the previous one.

Two consequences follow. First, a regex knows nothing about words or meaning: ERR is just E, R, R, so it also matches inside ERRATIC. Second, the leftmost possible match always wins, even when a "better" one comes later. Python's re.finditer shows every match with its span (start included, end excluded, like a slice):

import re

line = "WARN meter KE-0042 ERRATIC values; ERROR 42 on feeder 7"
for m in re.finditer(r"ERR", line):
    word = line[m.start():].split()[0]  # the text from the match to the next space
    print(m.group(), m.span(), word)

ERR (19, 22) ERRATIC
ERR (35, 38) ERROR

The first match is the start of ERRATIC, not the word ERROR. To match whole words only you need a word boundary, such as \bERROR\b. How the engine continues when part of a pattern fails halfway (it goes back and tries another choice) is explained with backtracking.

The building blocks at a glance

Every regex is made of a handful of ideas, each with its own page in this topic. The table is a map, not something to memorize now.

Where they came from

Regular expressions started as mathematics. In a 1951 RAND memorandum, published in 1956 in the book Automata Studies, the mathematician Stephen Kleene described "regular events": sets of sequences that can be built from single symbols with three operations, one after the other (concatenation), one or the other (alternation) and "repeated zero or more times". That last operation, written *, is still called the Kleene star.

In formal language theory, a regular language is a set of strings that can be described with exactly those three operations, or, equivalently, recognized by a finite automaton: a machine with a fixed number of states and no other memory. That is where the word "regular" comes from. It also explains a limit: a machine with no memory cannot count, so a true regular expression cannot check that brackets are balanced or that HTML tags are properly nested.

The jump to computers came from Ken Thompson. His 1968 article "Regular expression search algorithm" (Communications of the ACM) compiled a pattern into machine code, and he built regexes into his QED editor and then into the Unix editor ed. In ed, the command g/re/p meant "globally, find lines matching the regular expression re, and print them"; the GNU grep manual confirms that this is where the name grep comes from. grep appeared in Unix in the early 1970s, followed by sed and awk.

Later engines added features that go beyond the mathematics. Henry Spencer's widely copied regex library (mid-1980s) and the Perl language (from the late 1980s) made regexes central to scripting and spread the backtracking style (already used inside the early Unix ed and grep) that most programming languages still use. PCRE (Perl Compatible Regular Expressions, by Philip Hazel) appeared in 1997; its successor PCRE2 followed in 2015 and is at version 10.49 (a security release, 28 September 2026). PHP and grep -P use PCRE2 today.

One idea, many dialects

Because so many tools implemented regexes independently, there is no single regex language. Each implementation is a flavor: the core (literals, ., *, […], ^, $) is shared, but the details differ. A classic trap: \d means "a digit" in Python, JavaScript, PHP and grep -P, but plain GNU grep does not know it. In the GNU grep 3.8 build that Debian ships, it then silently matches the letter d:

printf '%s\n' 'meter 4471 ok' 'meter idle' 'no digits here' > meters.txt
echo "--- grep -E '\d' (not a digit class here):"
grep -E '\d' meters.txt
echo "--- grep -E '[0-9]':"
grep -E '[0-9]' meters.txt
echo "--- grep -P '\d' (Perl-compatible flavor):"
grep -P '\d' meters.txt

--- grep -E '\d' (not a digit class here):
meter idle
no digits here
--- grep -E '[0-9]':
meter 4471 ok
--- grep -P '\d' (Perl-compatible flavor):
meter 4471 ok

Upstream GNU grep 3.8 prints a warning about the "stray backslash"; Debian's package switches that warning off, so nothing tells you that the pattern is wrong. Always test a pattern in the tool and version where it will run, and check its documentation for the flavor. The topic shows Python, JavaScript, PHP and the GNU command-line tools side by side wherever they differ.

When a regex is the wrong tool

Regexes are powerful, but they are also compact, easy to get subtly wrong and hard to read later. Reach for something simpler when you can:

JSON is a good example. A regex written for one layout of the data breaks when the spacing changes, which is perfectly valid JSON; the parser does not care:

import json
import re

pretty = '{"meter": "KE-0042", "kwh": 318.4}'
compact = '{"meter":"KE-0042","kwh":318.4}'

for text in (pretty, compact):
    found = re.search(r'"meter": "([^"]+)"', text)
    print("regex: ", found.group(1) if found else None)
    print("parser:", json.loads(text)["meter"])

regex:  KE-0042
parser: KE-0042
regex:  None
parser: KE-0042

The same goes for addresses. A quick pattern for "four numbers with dots" also accepts 256.1.1.1, which is not an IPv4 address; Python's ipaddress module knows the rules:

import ipaddress
import re

for text in ["192.0.2.10", "256.1.1.1", "198.51.100.300"]:
    looks_ok = bool(re.fullmatch(r"[0-9]{1,3}(\.[0-9]{1,3}){3}", text))
    try:
        ipaddress.IPv4Address(text)
        real = True
    except ValueError:
        real = False
    print(f"{text:15} regex={looks_ok!s:5} ipaddress={real}")

192.0.2.10      regex=True  ipaddress=True
256.1.1.1       regex=True  ipaddress=False
198.51.100.300  regex=True  ipaddress=False

A sensible split is: use a regex to find candidates in messy text (a log line, an email body), then hand each candidate to a parser that knows the real rules. The page on practical patterns uses that approach throughout, and input validation in security work relies on the same idea.