How Machines Learn

Rules versus learning

Think about a spam filter. The traditional way to build one is to write rules: if the subject contains "FREE!!!", if there are more than ten links, if the sender's domain is on a blocklist, then it is spam. That works for a while. Then spammers change their wording, the rules multiply, and nobody remembers why rule 312 exists.

Machine learning turns the problem around. Instead of writing the rules, you collect thousands of emails that people already marked as spam or not spam, and let an algorithm find the patterns that separate them. When spammers change tactics, you add fresh examples and train again.

Traditional programming:  rules + data      →  answers
Machine learning:         data + answers    →  rules (a model)

The same applies on a service desk. You could write keyword rules to route tickets (see the example on What is AI?), or you could learn the routing from the last two years of tickets, each of which already records the queue that eventually solved it.

What a model is

In machine learning, a model is a function that maps an input to an output — a ticket to a queue, an email to "spam" or "not spam", a temperature to an expected energy use — whose behavior was learned from data rather than written by hand. A trained model is usually stored as a file: a description of the function's shape plus a long list of numbers.

Model parameters

Those numbers are the model parameters: the internal values a model adjusts during training and then keeps fixed when it is used. A simple model may have one or two. Take the rule "flag an account as suspicious if it has at least t failed logins in an hour". The shape (compare a count with a threshold) is chosen by a person; the value of t is the parameter, and the computer can learn it. A straight-line model y = a·x + b has two parameters, a and b. A large language model has billions; GPT-3, for example, had 175 billion.

Features and labels

A model cannot read a ticket the way a person does; it needs the input as measurable values. A feature is one measurable property of an example that the model uses as input: the number of failed logins, the hour of the day, whether the text contains the word "VPN", the length of an email. Choosing and preparing good features used to be most of the work in machine learning, and for small tabular data it still is.

A label is the correct answer attached to a training example — the thing the model should learn to predict. For ticket routing the label is the queue that actually solved the ticket; for a spam filter it is "spam" or "not spam".

Notice the fourth row: it mentions VPN but was an account problem. A keyword rule would get it wrong; a model that sees enough examples can learn that "VPN" together with "password" usually means account. Where labels come from, and why they are often the hardest part, is covered on Data for AI.

Training and inference

A model has two phases in its life, and they have very different costs.

Training is the process of adjusting a model's parameters, using example data, so that its outputs match the desired outputs as closely as possible. It usually works as a loop: make predictions, measure how wrong they are, nudge the parameters to be less wrong, repeat. Training a small model takes seconds on a laptop; training a large language model takes weeks on thousands of GPUs. It happens once, or whenever the model is refreshed.

Inference is using a trained model to produce an output for new input: classifying today's tickets, answering a chat message. The parameters do not change during inference. Each inference is cheap compared with training, but it happens every time the model is used, so for a popular service the total inference cost can exceed the training cost.

In cloud AI services this split shows up directly: you pay for compute time to train or fine-tune, and then per request (or per token) to run the deployed model.

A tiny training loop

Here is the whole idea in its simplest form. A security analyst has labeled sixteen accounts as suspicious (1) or normal (0), together with the number of failed logins each had in one hour. The model is the rule "suspicious if failed logins ≥ threshold"; the threshold is its only parameter. Training means trying candidate thresholds and keeping the one that is right most often.

# Failed logins in one hour per account, labeled by an analyst
# (1 = suspicious, 0 = normal).
data = [(0, 0), (1, 0), (2, 0), (3, 0), (2, 0), (4, 0), (6, 1), (5, 0),
        (8, 1), (12, 1), (7, 1), (25, 1), (3, 0), (9, 1), (6, 0), (1, 0)]


def accuracy(threshold):
    """Share of examples the rule 'failed >= threshold' gets right."""
    correct = 0
    for failed, label in data:
        prediction = 1 if failed >= threshold else 0
        if prediction == label:
            correct += 1
    return correct / len(data)


# Training: try candidate thresholds and keep the most accurate one.
best = None
for threshold in range(2, 11):
    acc = accuracy(threshold)
    print(f"threshold {threshold:2}: accuracy {acc:.2f}")
    if best is None or acc > accuracy(best):
        best = threshold
print("learned threshold:", best)

# Inference: apply the learned rule to accounts it has never seen.
for failed in (2, 6, 40):
    verdict = "suspicious" if failed >= best else "normal"
    print(f"{failed:2} failed logins -> {verdict}")

threshold  2: accuracy 0.56
threshold  3: accuracy 0.69
threshold  4: accuracy 0.81
threshold  5: accuracy 0.88
threshold  6: accuracy 0.94
threshold  7: accuracy 0.94
threshold  8: accuracy 0.88
threshold  9: accuracy 0.81
threshold 10: accuracy 0.75
learned threshold: 6
 2 failed logins -> normal
 6 failed logins -> suspicious
40 failed logins -> suspicious

A few things to notice, because they hold for every model, however large:

• The rule came from the data. Nobody typed "6". Give it different labeled data and it learns a different threshold.
• It is not perfect. Even the best threshold gets one example wrong: a normal user with six failed logins (someone who kept typing an old password). Real data is noisy, and a model that fits every training example exactly is usually suspicious.
• Ties need a policy. Thresholds 6 and 7 score the same; the code keeps the first. Real algorithms make such choices too, often to prefer the simpler model.
• Trying every option does not scale. With one parameter and nine candidates, brute force is fine. With millions of parameters it is impossible, so real training uses smarter search, mostly gradient descent, explained on Neural networks.

In practice you would use a library such as scikit-learn or PyTorch, which does the same thing much faster and with better algorithms. Building it by hand once shows that there is no magic inside.

Supervised learning

Supervised learning is machine learning from labeled examples: each training example comes with the correct answer, and the model learns to predict that answer for new inputs. The labels act as a teacher who checks the work. The example above is supervised learning, and so is most machine learning in business use. It comes in two kinds, depending on what you predict:

A quick test: if the answer is one of a list of names, it is classification; if it is a quantity on a scale, it is regression. "Will this disk be full within 30 days?" is classification (yes or no); "how full will it be in 30 days?" is regression. The algorithms for both are on Classic machine learning.

Unsupervised learning

Unsupervised learning finds structure in data that has no labels: groups of similar items, unusual items, or a simpler way to describe the data. There is no teacher and no single right answer, so a person has to judge whether what it found is useful. Two common uses in IT:

• Clustering: grouping similar items. Grouping ten thousand untagged tickets into a few dozen clusters can reveal a recurring problem nobody had named yet, such as a cluster of "printer offline after update" tickets.
• Anomaly detection: learning what normal looks like and flagging what does not fit, such as a server whose CPU pattern suddenly differs from its usual week, or a login from an unusual place at an unusual time. See AI in IT operations.

Unsupervised learning is attractive because labels are expensive. Its weakness is that it finds patterns, not meaning: a cluster may be "tickets submitted on Mondays" rather than anything useful.

Reinforcement learning

Reinforcement learning (RL) is learning by trial and error: an agent takes actions in an environment, receives a reward (or a penalty) for the result, and learns a strategy that maximizes its total reward over time. Nobody tells it the right move; it only learns how good the outcome was, often long after the move that mattered.

Games are the classic example. AlphaGo, which beat Lee Sedol at Go in 2016, learned partly by playing millions of games against itself, with "win" as the reward. Other uses are robot control, and tuning systems where you can try actions safely, such as a data center cooling setup in simulation.

Reinforcement learning matters for language models too. After a model has learned to predict text, it is tuned with reinforcement learning from human feedback, in which people's preferences between answers act as the reward. That step is a big part of why chat assistants follow instructions and refuse some requests. It is explained on Transformers and large language models.

Generalization: the real goal

A model that simply memorized the sixteen accounts would score as well as possible on them and be useless on the seventeenth. What you want is generalization: good performance on new data the model has never seen. That is why you always measure a model on data that was kept apart from training, and why a perfect training score is a warning sign rather than a success. The two ways to fail — learning too little (underfitting) and memorizing noise (overfitting) — and how to measure honestly are covered on Evaluating models. How to set data aside properly is on Data for AI.