Safety and Reliability

Keeping people and plants safe: functional safety, safety instrumented systems and SIL levels, why safety and security depend on each other, and designing for reliability.

Functional safety

Industrial processes store dangerous energy: pressure in a vessel, heat in a furnace, flammable gas in a pipeline, 380,000 volts on a line. Functional safety is the part of overall safety that depends on a system working correctly in response to its inputs: a pressure switch that opens a relief path, a controller that shuts a fuel valve, a relay that trips a breaker. It is not about hard hats or guard rails (those are other kinds of safety); it is about automatic protective functions that must work when they are needed.

IEC 61508 and risk reduction

IEC 61508 is the basic international standard for the functional safety of electrical, electronic and programmable electronic safety-related systems. It is generic; sector standards apply it to their field, such as IEC 61511 for the process industry, IEC 62061 for machinery and ISO 26262 for road vehicles.

The core idea is risk reduction. A hazard analysis estimates how often a hazardous event would happen without protection and how bad it would be. The organization decides what risk it can tolerate. The gap between the two is the risk reduction that safety functions must provide. If an overpressure would occur once in 10 years without protection and the tolerable frequency is once in 10,000 years, the protection layers together must reduce the risk by a factor of 1,000.

The standard also covers the whole safety lifecycle: from hazard analysis and requirements, through design, verification and installation, to operation, proof testing, modification and decommissioning. Many safety failures are not random hardware faults but systematic errors: a wrong requirement, a software bug, an unreviewed change. The lifecycle exists to catch those.

Safety instrumented systems

A safety instrumented system (SIS) performs one or more safety functions with three parts:

  • Sensors that measure the hazardous condition: a pressure transmitter, a flame detector, a gas detector.
  • A logic solver, usually a safety PLC, that decides whether the condition requires action.
  • Final elements that act: a shutdown valve, a motor contactor, a breaker.

The normal control system, often called the basic process control system (BPCS), keeps the process at its setpoint. The SIS does nothing as long as everything is normal; it only acts when the process leaves its safe operating range. That is why it must be independent of the control system: separate sensors, separate logic, separate valves. If both shared one pressure transmitter and that transmitter failed, the controller would let the pressure rise and the SIS would never see it.

A pressure vessel with two independent loops. The control loop at the top: pressure transmitter PT-101 to the process control system to control valve PV-101, which keeps pressure at its setpoint. The safety loop at the bottom: a separate pressure transmitter PT-201 to the safety logic solver to shutdown valve XV-201, which closes only when pressure is too high. Vessel process PT-101 Control system PV-101 PT-201 Safety logic solver XV-201 control: continuous safety: only on demand
The control loop and the safety loop share the process, and nothing else: own sensor, own logic, own valve.

The SIS is one of several layers of protection. A typical stack, from inside out: good process design, the control system, alarms and operator response, the SIS, mechanical relief devices such as relief valves and rupture discs, physical containment (bunds, blast walls) and emergency response. Each layer must be independent of the others to count. Note that mechanical relief valves need no electronics at all, so they are also a protection that cyberattacks cannot reach.

Safety integrity levels

How reliable must a safety function be? A safety integrity level (SIL) expresses that, from SIL 1 (lowest) to SIL 4 (highest). For a function that is called on rarely (low-demand mode, the usual case in process plants), the measure is the average probability of failure on demand (PFDavg): the chance that, when the hazard occurs, the function does not work. Its inverse is the risk reduction factor (RRF).

SIL PFDavg (low demand) Risk reduction factor
SIL 4 10−5 to less than 10−4 more than 10,000 up to 100,000
SIL 3 10−4 to less than 10−3 more than 1,000 up to 10,000
SIL 2 10−3 to less than 10−2 more than 100 up to 1,000
SIL 1 10−2 to less than 10−1 more than 10 up to 100

In the earlier example, a factor of 1,000 that the SIS must provide on its own would put it at the top of SIL 2 or into SIL 3. SIL 4 is rare in the process industry; when a design seems to need it, the usual answer is to change the process or add other layers. For functions that are demanded often or run continuously (high-demand mode, common in machinery), SIL is expressed as a probability of dangerous failure per hour instead.

A SIL is a property of a safety function (sensor, logic and final element together), not of a single box. A "SIL 3 certified" logic solver does not make a function SIL 3 if its valve is never tested. Reaching a SIL also requires architectural constraints (enough redundancy and diagnostics) and protection against systematic errors, not only a low enough number.

From failure rate to SIL in Python

A common simplified formula for a single-channel function (one sensor, one logic solver, one valve, "1oo1") is PFDavg ≈ λDU × TI / 2: the rate of dangerous failures that diagnostics cannot detect, times the proof-test interval, halved because on average a hidden failure has existed for half the interval. The example shows how the proof-test interval moves a function between SIL bands. 

# Low-demand mode: which SIL band does an average probability of failure
# on demand (PFDavg) fall into, and what risk reduction does it give?
SIL_BANDS = [  # (SIL, lowest PFDavg, highest PFDavg) - IEC 61508 low demand
    (4, 1e-5, 1e-4),
    (3, 1e-4, 1e-3),
    (2, 1e-3, 1e-2),
    (1, 1e-2, 1e-1),
]


def sil_for(pfd_avg):
    for sil, low, high in SIL_BANDS:
        if low <= pfd_avg < high:
            return f"SIL {sil}"
    return "outside SIL 1-4"


def pfd_single_channel(failure_rate_per_hour, test_interval_hours):
    # Simplified formula for one channel (1oo1): PFDavg = lambda_DU * TI / 2
    return failure_rate_per_hour * test_interval_hours / 2


HOURS_PER_YEAR = 8760
lambda_du = 2e-6  # dangerous undetected failures per hour (example value)

for years in (0.5, 1, 2):
    pfd = pfd_single_channel(lambda_du, years * HOURS_PER_YEAR)
    rrf = 1 / pfd
    print(f"proof test every {years:>3} yr: PFDavg {pfd:.2e}  "
          f"RRF {rrf:>5.0f}  {sil_for(pfd)}")
proof test every 0.5 yr: PFDavg 4.38e-03  RRF   228  SIL 2
proof test every   1 yr: PFDavg 8.76e-03  RRF   114  SIL 2
proof test every   2 yr: PFDavg 1.75e-02  RRF    57  SIL 1

The lesson for operations: proof tests are part of the safety design. Postponing the annual test of a shutdown valve to save a production stop quietly lowers the protection the plant was designed for. Real calculations also include detected failures, repair times, redundant channels and common cause failures; they are done by safety engineers with dedicated tools.

IEC 61511 in the process industry

IEC 61511 (in the US adopted as ANSI/ISA-61511, earlier ISA-84) applies IEC 61508 to safety instrumented systems in the process industry: chemicals, oil and gas, refining, pharmaceuticals. Where IEC 61508 is written for manufacturers of safety equipment, IEC 61511 is written for those who design, install and operate the systems at a plant. Its lifecycle typically runs:

  1. Hazard and risk assessment, often a HAZOP (hazard and operability study) workshop that walks through the process line by line asking "what if".
  2. Allocation of safety functions to protection layers, often with LOPA (layer of protection analysis), which assigns each function its required SIL.
  3. A safety requirements specification (SRS) per function: what to measure, the trip point, the action, the response time, the SIL, the proof-test interval.
  4. Design, installation, validation, and then operation and maintenance: proof tests, managed bypasses during maintenance, and management of change for every modification.

Edition 2 (2016) of IEC 61511 also requires a security risk assessment of the SIS. Safety engineers and security teams therefore have to work together; see safety and security below.

Fail-safe design

A fail-safe design ensures that the most likely failures, such as loss of power or a broken wire, drive the system to its safe condition. The classic technique is de-energize to trip: during normal operation, the sensor loop carries current, the logic solver output is energized and the shutdown valve is held open against a spring. A trip removes the energy, and the spring closes the valve. A cut wire, a blown fuse, a dead power supply or a failed output card all remove energy too, so they also close the valve.

What is "safe" depends on the process. For a fuel valve on a furnace, closed is safe. For cooling water to a reactor, open is safe, so that valve fails open. For a breaker in the grid, "safe" usually means open (no current), but cutting supply to a hospital or to a pumping station has consequences too. Deciding the safe state is an engineering decision for each function.

Fail-safe has a cost: spurious trips. A loose terminal shuts down the plant. Designers balance this with redundancy and voting:

  • 1oo2 (one out of two): two sensors, either can trip. Very safe, but twice the spurious trips.
  • 2oo3 (two out of three): three sensors, two must agree. Safe even if one sensor fails, and one faulty sensor cannot trip the plant on its own.

Reliability and availability

Safety asks "does it work when needed?". Reliability asks "how often does it fail?" and availability asks "what fraction of the time is it working?". Three numbers connect them.

  • Mean time between failures (MTBF) is the average operating time between failures of a repairable item, measured or predicted over a large population in its useful life.
  • The mean time to repair (MTTR) is the average time to bring a failed item back into service: getting there, diagnosing, getting the spare, replacing, testing. It differs from the IT service management metric mean time to restore, which runs from the start of an outage until the service works again, including detection.
  • Availability = MTBF / (MTBF + MTTR).

Availability in Python

Two remote terminal units with the same MTBF, one with a spare on site and one whose spare must be shipped. The pair assumes either unit can do the job and that they fail independently. 

# Availability from mean time between failures (MTBF) and
# mean time to repair (MTTR), for one unit and for a redundant pair.
HOURS_PER_YEAR = 8760


def availability(mtbf_hours, mttr_hours):
    return mtbf_hours / (mtbf_hours + mttr_hours)


units = [
    ("RTU, spare on site", 50_000, 4),
    ("RTU, spare shipped", 50_000, 72),
]
for name, mtbf, mttr in units:
    single = availability(mtbf, mttr)
    pair = 1 - (1 - single) ** 2  # either unit can do the job (independent)
    print(f"{name}: MTBF {mtbf} h, MTTR {mttr} h")
    print(f"  single: {single:.5%}  "
          f"down {(1 - single) * HOURS_PER_YEAR:5.1f} h/yr")
    print(f"  pair:   {pair:.7%}  "
          f"down {(1 - pair) * HOURS_PER_YEAR * 3600:5.1f} s/yr")
RTU, spare on site: MTBF 50000 h, MTTR 4 h
  single: 99.99200%  down   0.7 h/yr
  pair:   99.9999994%  down   0.2 s/yr
RTU, spare shipped: MTBF 50000 h, MTTR 72 h
  single: 99.85621%  down  12.6 h/yr
  pair:   99.9997932%  down  65.2 s/yr

Two lessons. First, the repair time matters as much as the failure rate: a spare on the shelf is cheap availability. Second, the numbers for the pair are optimistic. Two identical units in the same cabinet share power, temperature, firmware bugs and the same mistaken configuration change: common cause failures that independence ignores. The same parallel and series reasoning appears in cloud high availability.

Safety and security together

Safety engineering assumes failures are random or accidental: a valve sticks, an engineer makes a mistake. An attacker is neither. An attacker can make several "independent" layers fail at the same time, can hide a dangerous condition from operators, and can wait for the worst moment. In OT, a security incident can therefore become a safety incident. The OT security fundamentals page develops this interplay; IEC TR 63069, a technical report, describes how to apply IEC 61508 and IEC 62443 together.

Triton: an attack on a safety system

In 2017, malware known as Triton (also Trisis or HatMan) targeted Schneider Electric Triconex safety controllers at a petrochemical facility in the Middle East (Saudi Arabia). According to CISA, the safety systems initiated automatic shutdowns, and the refinery was shut down for several days. In 2022 the US Department of Justice unsealed charges against an employee of TsNIIKhM, a research institute of the Russian Ministry of Defense. The threats and notable incidents page places it among other OT attacks.

Triton matters because it targeted the last automated layer meant to protect people. The defensive lessons:

  • Keep the SIS on its own network segment, with no path from the business network and tightly controlled access from engineering workstations.
  • Where controllers have a physical mode key, leave it in run mode so programs cannot be changed remotely; CISA recommends this for PLCs in general.
  • Monitor and alert on every connection to a safety controller and every program download.
  • Treat an unexplained trip as a question for both the safety engineers and the security team. In Triton, the fail-safe design did its job: the plant tripped to a safe state.

Why "turn it off and on again" is dangerous

In IT, restarting a server is often the quickest fix. In OT, it can be the start of an incident:

  • Outputs go to their fail state. When a PLC stops, its outputs usually de-energize (some can be configured to hold their last value). Valves close, pumps stop, motors coast down. Fail-safe, yes, but also a plant trip, lost production, and thermal and mechanical stress on equipment.
  • State is lost. A controller may be halfway through a batch or a start-up sequence. After a cold restart it no longer knows where it was, and resuming may need manual steps.
  • Some processes cannot just stop. A glass furnace that cools solidifies; a chemical reaction may run away without cooling; a substation without its protection relay must be taken out of service.
  • Operators lose their view. Restarting an HMI or a SCADA server during a disturbance leaves operators blind just when they need information.
  • Start-up is the riskiest phase. Many process accidents happen during start-up and shutdown, not during steady operation.