Concurrency Basics

Why concurrency?

A simple program does one thing after another. That is easy to reason about, but it wastes time in three common situations:

• Waiting for I/O. Reading from a network, a disk or a database takes milliseconds to seconds — millions of processor cycles. A program that downloads ten web pages one by one spends almost all its time waiting. Starting all ten downloads and handling each answer as it arrives is far faster.
• Using multiple cores. Almost every processor today has several cores. A program that runs on one core leaves the others idle, even for heavy work such as resizing thousands of photos.
• Staying responsive. An app with a user interface must keep reacting to clicks while it saves a file or loads data. If the long job runs on the same path as the interface, the window freezes.

Concurrency is the ability of a program to have several tasks in progress during the same period of time, instead of finishing each one before starting the next. It is a way of structuring a program. Whether those tasks also run at the same instant is a separate question.

Concurrency versus parallelism

Parallelism means that several computations really execute at the same moment, on different processor cores (or different machines). Concurrency does not need that: one core can switch between tasks so quickly that all of them make progress, like a cook who stirs one pan, chops some onions, then stirs again. Parallelism is having two cooks.

A program can be concurrent without being parallel (one core taking turns), and a well-structured concurrent program can become parallel when more cores are available. Which one you need depends on the bottleneck:

Processes and threads

The operating system offers two basic building blocks for running things at the same time.

A process is a running instance of a program, with its own private memory and its own resources such as open files. When you start a program twice, you get two processes that cannot see each other's variables. The operating system protects them from each other: if one crashes, the other keeps running. That isolation has a price: starting a process is relatively slow, and processes must exchange data through explicit channels such as pipes, sockets or files.

A thread is a single sequence of instructions that runs inside a process. Every process starts with one thread; it can start more. All threads of a process share the same memory — the same global variables, the same objects — but each has its own call stack and its own place in the code. Threads are cheaper to create than processes and can share data directly, which is exactly what makes them dangerous, as the next section shows.

Which thread runs when is decided by the operating system's scheduler. It can pause a thread between almost any two machine instructions and switch to another. You cannot predict or control exactly where those switches happen — keep that in mind for everything that follows.

Race conditions

A race condition is a bug in which the result depends on the unpredictable timing or order in which concurrent tasks access shared data. The classic example is the lost update. Two threads each want to add 1 to a shared counter. In code this looks like a single step, counter = counter + 1, but the processor does it in three: read the value, add 1, write the result back. If the scheduler switches threads between the read and the write, this can happen:

Two increments, but the counter is 1: thread B worked with a value that was already out of date, and its write overwrote A's. If A had finished all three steps before B started, the result would have been 2. Both orders are possible, so the program is sometimes right and sometimes wrong.

Because real thread switches are unpredictable, a demonstration with real threads would not reliably show the bug. The program below simulates the two threads instead, following a fixed schedule of steps, so you can see both interleavings side by side:

def run(schedule):
    """Simulate threads that each do: read counter, then write read value + 1."""
    counter = 0
    seen = {}  # the value each thread read
    for thread, action in schedule:
        if action == "read":
            seen[thread] = counter
        else:
            counter = seen[thread] + 1
    return counter


one_after_another = [("A", "read"), ("A", "write"), ("B", "read"), ("B", "write")]
interleaved = [("A", "read"), ("B", "read"), ("A", "write"), ("B", "write")]

print("One after another:", run(one_after_another))
print("Interleaved:      ", run(interleaved))

One after another: 2
Interleaved:       1

Critical sections and locks

A critical section is a piece of code that accesses shared data and must not be executed by more than one thread at a time — like the read-add-write of the counter. The standard tool to protect it is a lock, also called a mutex (from mutual exclusion): an object that at most one thread can hold at a time. A thread acquires the lock before the critical section and releases it afterward. Any other thread that tries to acquire it meanwhile has to wait.

In Python, threading.Lock is used with a with statement, which releases the lock even if an error occurs inside the block. Here four threads each add 1 to a shared counter 10,000 times:

import threading

counter = 0
counter_lock = threading.Lock()


def add(times):
    global counter
    for _ in range(times):
        with counter_lock:  # only one thread at a time gets past this line
            counter += 1


threads = [threading.Thread(target=add, args=(10_000,)) for _ in range(4)]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()  # wait until the thread has finished

print(counter)

40000

With the lock, the result is always 40000, no matter how the threads are scheduled. Note the join() calls: without them, the main thread could print the counter while the others are still working.

Locks have costs. Threads waiting for a lock do no useful work, so keep critical sections short — do slow work such as I/O outside the lock. And every piece of code that touches the shared data must use the same lock; one forgotten access brings the race back.

Deadlock

As soon as a program uses more than one lock, a new problem appears. A deadlock is a situation in which two or more threads each wait for a resource that another one holds, so none of them can ever continue. Imagine a bank program that locks both accounts during a transfer:

THREAD 1: transfer from checking to savings
    ACQUIRE lock_checking
    ACQUIRE lock_savings      ← waits: thread 2 holds it
    move the money
    RELEASE both locks

THREAD 2: transfer from savings to checking
    ACQUIRE lock_savings
    ACQUIRE lock_checking     ← waits: thread 1 holds it
    move the money
    RELEASE both locks

If thread 1 gets the first lock and thread 2 gets its first lock before either takes a second one, each waits for the other — forever. The program does not crash; it simply stops making progress, which can be even harder to diagnose.

A deadlock can only occur when four conditions hold at the same time (the Coffman conditions):

1. Mutual exclusion — a resource can be held by only one thread.
2. Hold and wait — a thread holds one resource while it waits for another.
3. No preemption — a resource cannot be taken away from the thread that holds it.
4. Circular wait — there is a cycle of threads, each waiting for a resource held by the next.

Break any one condition and deadlock is impossible. The most practical rule breaks the circular wait: always acquire locks in the same global order. If every transfer first locks the account with the lower account number, then both threads above ask for the same lock first; one gets it, the other waits without holding anything, and no cycle can form. Other defenses are acquiring a lock with a timeout and backing off, and — best of all — designing so that a thread never needs two locks at once.

Asynchronous programming

Threads are not the only way to wait for many things at once. Asynchronous programming is a style in which a task starts a slow operation, such as a network request, and gives up control while it waits, so other tasks can run on the same thread in the meantime. Nothing runs in parallel; the program simply never sits idle while there is other work to do.

The heart of it is an event loop: a loop that keeps a list of tasks, runs whichever one is ready until it has to wait, and then moves on to the next ready task. Because a task only gives up control at points the programmer marks explicitly, a switch can never happen in the middle of counter += 1. That removes most race conditions — as long as no task blocks the loop with slow, non-asynchronous work.

async and await in Python

In Python, async def defines a coroutine: a function that can pause. Inside it, await marks the places where it pauses until a result is ready, letting the event loop run other tasks. asyncio.run() starts the event loop. In this example, each await asyncio.sleep(0) hands control back to the loop, so two tasks take turns on a single thread:

import asyncio


async def worker(name, steps):
    for step in range(1, steps + 1):
        print(f"{name}: step {step}")
        await asyncio.sleep(0)  # pause here and let another task run


async def main():
    await asyncio.gather(worker("A", 3), worker("B", 2))


asyncio.run(main())

A: step 1
B: step 1
A: step 2
B: step 2
A: step 3

The real use is waiting for I/O. asyncio.gather() runs several coroutines concurrently and returns their results in the order you passed them, not in the order they finish. Below, three simulated downloads take 0.6, 0.2 and 0.4 seconds; one after another that would be 1.2 seconds, but concurrently the total is about 0.6 seconds — the longest one:

import asyncio
import time


async def fetch(source, seconds):
    await asyncio.sleep(seconds)  # stands in for a slow network request
    return f"{source}: {seconds} s"


async def main():
    start = time.perf_counter()
    results = await asyncio.gather(
        fetch("weather", 0.6),
        fetch("news", 0.2),
        fetch("prices", 0.4),
    )
    elapsed = time.perf_counter() - start
    for line in results:
        print(line)
    print("Faster than one after another:", elapsed < 1.0)


asyncio.run(main())

weather: 0.6 s
news: 0.2 s
prices: 0.4 s
Faster than one after another: True

async and await in JavaScript

JavaScript was built around an event loop from the start: a web page has a single main thread, so every slow operation is asynchronous. A promise stands for a result that will arrive later, and await pauses an async function until the promise is fulfilled. Promise.all() plays the role of asyncio.gather():

function fetchPrice(item, ms, price) {
  return new Promise((resolve) => {
    setTimeout(() => resolve(`${item}: ${price.toFixed(2)}`), ms);
  });
}

async function main() {
  console.log("Start");
  const prices = await Promise.all([
    fetchPrice("coffee", 200, 3.5),
    fetchPrice("tea", 100, 2.75),
  ]);
  for (const line of prices) {
    console.log(line);
  }
  console.log("Done");
}

main();
console.log("main() is waiting; the rest of the program keeps running");

Start
main() is waiting; the rest of the program keeps running
coffee: 3.50
tea: 2.75
Done

Look at the order of the output. main() runs until its first await, then returns control, so the last line of the program prints before the prices. The results again come back in the order they were requested, although tea is ready first.

Threads in Python and the GIL

CPython, the standard Python implementation, has a Global Interpreter Lock (GIL): an internal lock that lets only one thread at a time execute Python bytecode. It keeps the interpreter's own data simple and safe, but it has two consequences:

• Threads help with I/O, not with computation. A thread that waits for the network or the disk releases the GIL, so other threads can run. But several threads doing pure-Python calculations still use only one core at a time. For CPU-bound work, use several processes (multiprocessing or ProcessPoolExecutor), or libraries such as NumPy whose C code releases the GIL while it works.
• The GIL does not prevent race conditions. A thread can be switched out between the bytecode instructions that make up counter += 1, so you still need locks around shared data.

This is changing slowly. Python 3.13 added an optional, experimental "free-threaded" build of CPython without a GIL (see PEP 703); since Python 3.14 it is officially supported (PEP 779). It is still a separate build that you have to install on purpose, and not all third-party packages support it yet; the regular Python you download still has the GIL. Other languages, such as Java, C# and C, have no GIL: their threads run truly in parallel on multiple cores.

Practical advice

Hand-written threads and locks are the assembly language of concurrency: powerful, and easy to get subtly wrong. Prefer the highest-level tool that does the job:

• Share less. The safest shared data is no shared data. Give each task its own input and combine the results at the end.
• Use a pool. Python's concurrent.futures module runs functions on a pool of worker threads (ThreadPoolExecutor) or processes (ProcessPoolExecutor) and hands back the results. You never start, join or lock anything yourself.
• Pass messages through a queue. When tasks must cooperate, let one put work items into a thread-safe queue (queue.Queue, or asyncio.Queue in async code) and another take them out. The queue does the locking.
• Pick the model that fits the bottleneck: asynchronous code or threads for waiting, processes for computation.

from concurrent.futures import ThreadPoolExecutor


def word_count(text):
    return len(text.split())


reviews = [
    "Great coffee and friendly staff",
    "Too noisy",
    "The croissants were fresh and warm this morning",
]

with ThreadPoolExecutor(max_workers=3) as pool:
    counts = list(pool.map(word_count, reviews))  # results keep the input order

print(counts)

[5, 2, 8]

Switching this to ProcessPoolExecutor would run the work in separate processes, in parallel on several cores — without changing anything else. Whatever tool you choose, test concurrent code carefully (see Debugging and Testing), and remember that a passing test cannot prove the absence of a race.