Floating-Point Numbers

Scientific notation in base 2

In science you write very large and very small numbers with a fixed number of significant digits and a power of ten: the speed of light is about 2.998 \times 10^{8} meters per second, and a nanosecond is 1 \times 10^{-9} seconds. The digits stay the same length; only the exponent changes. That is what lets one format hold both huge and tiny values: the decimal point "floats" to wherever the exponent puts it.

Computers do exactly the same in base 2. The number 6.5, for example, is 110.1 in binary (4 + 2 + ½), which is 1.101_2 \times 2^{2} when you move the point until one digit is left in front of it. The page Number Systems and Data Representation introduces the bit layout with 6.5 as a worked example; this page adds the mathematics: how much precision you get, where the errors come from and how big they can be.

Almost all hardware and programming languages follow IEEE 754, the technical standard of the Institute of Electrical and Electronics Engineers (IEEE) for floating-point arithmetic. It was first published in 1985; the current edition is IEEE 754-2019. It defines the bit formats, the special values and, just as important, how every result must be rounded, so that the same calculation gives the same answer on every conforming machine.

A stored number has three parts. The significand holds the significant bits (the 1.101 part, also called the mantissa in older texts). The exponent says by which power of two to scale it. The sign bit says whether the number is positive or negative. In one formula, for a normal 64-bit number:

Read it as: the sign s is 0 (plus) or 1 (minus); the 52 stored fraction bits f, read as a whole number, give the digits after the binary point; and the 11 stored exponent bits e, minus 1023, give the power of two. The next section takes each part in turn.

The bit layout

IEEE 754 defines several binary formats. Two of them cover nearly all everyday use:

Why is the significand one bit longer than the stored fraction? In binary, a normalized number always starts with 1. — the only nonzero digit there is. Because that leading 1 is always there, the format does not store it: it is the implicit or hidden bit, and you get one bit of precision for free.

The exponent bias

The exponent must be able to go negative (for numbers smaller than 1), but it is stored as an unsigned whole number. The trick is the exponent bias: a fixed number that is added before storing and subtracted when reading. For binary64 the bias is 1023, so a stored exponent of 1019 means 1019 - 1023 = -4, and a stored 1024 means +1. The two extreme patterns, all zeros and all ones, are reserved for zero, the very small subnormal numbers, infinity and NaN (see special values).

Worked example: decoding 0.1

Python can show the 64 bits that are stored for 0.1:

import struct
from decimal import Decimal

bits = struct.unpack(">Q", struct.pack(">d", 0.1))[0]
print(f"{bits:064b}")
print(hex(bits))
print((0.1).hex())
print(Decimal(0.1))

0011111110111001100110011001100110011001100110011001100110011010
0x3fb999999999999a
0x1.999999999999ap-4
0.1000000000000000055511151231257827021181583404541015625

Reading the pattern step by step:

1. Sign. The first bit is 0, so the number is positive.
2. Exponent. The next 11 bits, 01111111011, are 1019 in decimal. Subtract the bias: 1019 - 1023 = -4. The number is scaled by 2^{-4} = \frac{1}{16}.
3. Significand. The last 52 bits are 999999999999a in hex. Put the hidden 1 in front: 1 + \frac{f}{2^{52}} \approx 1.6. The pattern 1001 would repeat forever (0.1 has no finite binary form, just as ⅓ has no finite decimal form), so the hardware rounded the last bits up to 1010.
4. Value. 1.6000000000000000888\ldots \times 2^{-4} is 1.6000000000000000888… divided by 16, which is 0.1000000000000000055…: the closest double to 0.1, and slightly too big.

The same function works for any value. The tariff of €0.27 per kWh that Kestrel Energy's billing system uses is stored with exponent −2 (because 0.27 lies between ¼ and ½), and a negative number only flips the sign bit:

import struct


def fields(x):
    """Split a double into sign, stored exponent and stored fraction."""
    bits = struct.unpack(">Q", struct.pack(">d", x))[0]
    sign = bits >> 63
    exponent = (bits >> 52) & 0x7FF          # 11 bits
    fraction = bits & ((1 << 52) - 1)       # 52 bits
    return sign, exponent, fraction


for x in (0.1, 0.27, -2.5, 1.0):
    s, e, f = fields(x)
    print(f"{x:>5}: sign={s} exponent={e} ({e - 1023:+d}) fraction=0x{f:013x}")

  0.1: sign=0 exponent=1019 (-4) fraction=0x999999999999a
 0.27: sign=0 exponent=1021 (-2) fraction=0x147ae147ae148
 -2.5: sign=1 exponent=1024 (+1) fraction=0x4000000000000
  1.0: sign=0 exponent=1023 (+0) fraction=0x0000000000000

The shifts and masks in fields() are explained on Binary Arithmetic and Bitwise Operations. Note that 1.0 has a fraction of all zeros: the hidden 1 is the whole significand.

Precision: the gaps between floats

A double has 53 significant bits, so between two consecutive powers of two there are exactly 2^{52} doubles, evenly spaced. Between 1 and 2 they are 2^{-52} apart. Between 2 and 4 the same number of values must cover twice the distance, so they are twice as far apart, and so on. The gaps double at every power of two.

A toy format makes this visible. Suppose the significand had only two bits after the point. Then between 1 and 2 you can store 1, 1.25, 1.5 and 1.75; between 2 and 4 only 2, 2.5, 3 and 3.5; and between 4 and 8 only the whole numbers:

The gap just above 1 has a name. The machine epsilon, \varepsilon, is the distance from 1 to the next larger representable number. For doubles:

It is a measure of relative precision: any double is stored with an error of at most half a gap, which is at most \varepsilon / 2 times its size. Python reports it in sys.float_info, and math.ulp(x) (unit in the last place) gives the gap at any value:

import math
import sys

print(sys.float_info.epsilon)
print(sys.float_info.dig, sys.float_info.mant_dig)   # decimal digits, significand bits
print(sys.float_info.max)

for x in (1.0, 1000.0, 1e6, 2.0 ** 53):
    print(x, math.ulp(x))

print(2.0 ** 53 + 1 == 2.0 ** 53)

2.220446049250313e-16
15 53
1.7976931348623157e+308
1.0 2.220446049250313e-16
1000.0 1.1368683772161603e-13
1000000.0 1.1641532182693481e-10
9007199254740992.0 2.0
True

At 2^{53} = 9{,}007{,}199{,}254{,}740{,}992 the gap has grown to 2, so 2^{53} + 1 simply cannot be stored and rounds back to 2^{53}. Every integer up to 2^{53} is exact; beyond it, some are skipped. That is why JavaScript, whose only ordinary number type is a double, has a largest safe integer of 2^{53} - 1, and why APIs send large database IDs in JSON as strings.

Worked example: a counter that gets stuck

Single precision has only 24 significant bits. A monitoring dashboard at Kestrel Energy keeps a running total of delivered energy in a 32-bit float and adds 0.25 kWh for each reading. Here is what happens from 4,194,000 kWh onward, simulated in Python by rounding every intermediate result to binary32:

import struct


def to_float32(x):
    """Round a Python float (binary64) to the nearest binary32 value."""
    return struct.unpack("f", struct.pack("f", x))[0]


total32 = to_float32(4_194_000.0)
total64 = 4_194_000.0
for reading in range(4000):
    total32 = to_float32(total32 + 0.25)
    total64 = total64 + 0.25

print("expected:", 4_194_000 + 4000 * 0.25)
print("binary64:", total64)
print("binary32:", total32)
print("0.27 as binary32:", to_float32(0.27))

expected: 4195000.0
binary64: 4195000.0
binary32: 4194304.0
0.27 as binary32: 0.27000001072883606

1. Between 2^{21} and 2^{22} (2,097,152 to 4,194,304) the binary32 gap is 2^{21-23} = 0.25, so adding 0.25 still works.
2. From 2^{22} = 4{,}194{,}304 the gap is 0.5. The exact sum 4,194,304.25 lies exactly halfway between 4,194,304 and 4,194,304.5.
3. A tie is rounded to the neighbor whose last bit is 0 (see round half to even), which is 4,194,304. The total never moves again: 696 kWh disappear without any error message.

Doubles have the same problem, only much later. The lesson for time-series databases and metrics agents: know the type of a counter, and prefer 64-bit integers for totals that only grow.

Where rounding errors come from

A rounding error is the difference between the exact result of a calculation and the value that is actually stored. With floats it arises in two places:

• Converting input. The decimal number in your source code or CSV file is rounded to the nearest double. 0.1, 0.2 and 0.27 are not stored exactly; 0.5, 0.25 and 6.5 are, because they are sums of powers of two.
• Every operation. IEEE 754 requires each addition, subtraction, multiplication, division and square root to give the exact result, rounded once to the nearest float. So each step adds at most a tiny relative error:

Read it as: the computed sum is the true sum times a factor that is off from 1 by at most 2^{-53}, about one part in 10^{16}. One step is harmless. The trouble starts when errors pile up, or when a subtraction throws away the correct digits. The famous 0.1 + 0.2 example shows the first effect in one step; summing shows it growing:

import math

print(0.1 + 0.2)
print(0.1 + 0.2 == 0.3)
print(sum([0.1] * 10), math.fsum([0.1] * 10))
print(1e16 + 1 - 1e16)
print(123456789.12 - 123456789.02)

0.30000000000000004
False
0.9999999999999999 1.0
0.0
0.10000000894069672

• Accumulation. Ten additions of 0.1 drift to 0.9999999999999999. math.fsum tracks the lost low bits and returns the correctly rounded total.
• Absorption. At 10^{16} the gap between doubles is 2, so adding 1 changes nothing and the 1 is lost completely.
• Catastrophic cancellation. Two meter readings of about 123 million kWh each carry an error in their last bits. Subtract them and the leading digits cancel; what is left is mostly that error: 0.10000000894069672 instead of 0.1, correct to only eight digits. Subtracting nearly equal numbers is the most common way to lose precision.

Round half to even

When an exact result lies exactly halfway between two floats, it has to go one way. IEEE 754's default rule is round half to even: pick the neighbor whose last digit is even (for binary: whose last bit is 0). It is also called banker's rounding. Unlike "always round .5 up", it does not push sums upward on average, because half the ties go down and half go up.

Python's built-in round() uses the same rule for decimal places, which surprises people who learned "5 rounds up" at school:

print(round(0.5), round(1.5), round(2.5), round(3.5))
print(round(2.675, 2))
print(f"{2.675:.20f}")

0 2 2 4
2.67
2.67499999999999982236

The first line is round half to even at work: 0.5 and 2.5 go down to the even 0 and 2, 1.5 and 3.5 go up to the even 2 and 4. The second line is a different effect. 2.675 is not a tie at all: the stored double is slightly below 2.675, so 2.67 is simply the nearest two-decimal value.

Special values: infinity, NaN and zero

The reserved exponent patterns give IEEE 754 a few values that are not ordinary numbers:

• Positive and negative infinity (exponent all ones, fraction zero). A result too large for the format overflows to infinity instead of wrapping around: the largest double is about 1.8 \times 10^{308}, and ten times that is inf.
• NaN, "not a number" (exponent all ones, fraction not zero), the result of an undefined operation such as \infty - \infty. NaN is not equal to anything, including itself. The term is explained for JavaScript in Numbers, Math and Dates.
• Signed zero. There is a +0.0 and a −0.0. They compare equal, but the sign survives (for example a tiny negative result that underflowed to zero).
• Subnormal numbers (exponent all zeros). Below the smallest normal double, about 2.2 \times 10^{-308}, the hidden bit becomes 0 and precision fades out gradually instead of jumping straight to zero.

import math

inf, nan = float("inf"), float("nan")
print(inf + 1, -inf, inf - inf)
print(nan == nan, math.isnan(nan))
print(0.0 == -0.0, math.copysign(1, -0.0))
print(1e308 * 10)
print(sorted([3.0, nan, 1.0]))
try:
    print(1 / 0.0)
except ZeroDivisionError as e:
    print("ZeroDivisionError:", e)

inf -inf nan
False True
True -1.0
inf
[3.0, nan, 1.0]
ZeroDivisionError: float division by zero

Two results deserve a second look. Sorting a list that contains NaN silently fails: every comparison with NaN is false, so the sort cannot place it and the list can come out unsorted. And IEEE 754 defines 1 / 0 as infinity, but Python chooses to raise ZeroDivisionError instead; JavaScript returns Infinity. A missing sensor value that turns into NaN can therefore wreck averages, sorting and alert thresholds far from where it entered.

Comparing floats

Since almost every float result carries a small error, two calculations that "should" agree often differ in the last bits. Comparing with == then gives the wrong answer. The fix is to accept values as equal when they are close enough. There are two ways to say "close":

• Absolute tolerance: |a - b| \le 0.001. Good when you know the scale, such as temperatures in degrees.
• Relative tolerance: the difference is small compared to the numbers themselves, such as one part in a billion. Good when values can be tiny or huge.

Python's math.isclose(a, b) combines both. It returns true when

with a default relative tolerance of 10^{-9} and a default absolute tolerance of 0. That default has a catch: when one of the numbers is 0, the allowed difference is 10^{-9} times the other number, which is always smaller than the difference itself, so only an exact 0 passes.

import math

voltage_a = 0.1 + 0.2
voltage_b = 0.3
print(math.isclose(voltage_a, voltage_b))

drift = 1e-12                     # a sensor offset that should count as zero
print(math.isclose(drift, 0))
print(math.isclose(drift, 0, abs_tol=1e-9))

True
False
True

Choose the tolerance from the problem, not from habit: a power reading in kW might be "equal" within 0.001, a GPS coordinate within 10^{-7} degrees. The standard library function is described in The Standard Library.

When to use decimals instead

Floats are the right choice for measurements, physics, graphics and machine learning, where the input is approximate anyway and speed matters. They are the wrong choice when values are defined in decimal and must come out exact: money, tax, tariffs, quantities on an invoice. There, use one of these:

• Integers in the smallest unit: store €12.34 as 1234 cents. Fast, exact and supported everywhere.
• A decimal type: Python's decimal.Decimal, Java's BigDecimal, C#'s decimal, SQL's DECIMAL(p, s). These store base-10 digits, so 0.1 is exactly 0.1, and they let you choose the rounding rule.
• Fractions (Python's fractions.Fraction) when you need exact ratios such as ⅓; rare in business code.

Worked example: one invoice line

Joost Bakker, an engineer in Kestrel Energy's billing team, checks a line on a customer's invoice: 1003.5 kWh at €0.27 per kWh.

1. Exact amount: 1003.5 \times 0.27 = 270.945. That is exactly halfway between €270.94 and €270.95, so the rounding rule decides.
2. The contract says amounts are rounded half up, so the invoice must say €270.95.
3. The float calculation 1003.5 * 0.27 prints 270.945, but the stored double is a little below it, so round(…, 2) gives 270.94: a cent off.
4. With Decimal built from strings, the amount is exactly 270.945 and the rounding rule is explicit.

from decimal import Decimal, ROUND_HALF_EVEN, ROUND_HALF_UP

kwh = Decimal("1003.5")
tariff = Decimal("0.27")
amount = kwh * tariff
cent = Decimal("0.01")

print(amount)
print(amount.quantize(cent, rounding=ROUND_HALF_UP))
print(amount.quantize(cent, rounding=ROUND_HALF_EVEN))
print(1003.5 * 0.27, round(1003.5 * 0.27, 2))

print(Decimal("0.1") + Decimal("0.2") == Decimal("0.3"))
print(Decimal(0.1) == Decimal("0.1"))

270.945
270.95
270.94
270.945 270.94
True
False

Note the last line: Decimal(0.1) first creates the float 0.1, with its binary error, and then copies that error exactly into the decimal. Always build decimals from strings or integers.