Data and information

Data are recorded facts: the number 249, the text 2026-07-01, the word shipped. On their own they say little. Information is data put in context so that it answers a question: "order 104, placed on 1 July 2026, has been shipped" or "we have only seven 27-inch monitors left". Organizations collect data to get information out of it, and the quality of the information can never be better than the quality of the data.

A database is an organized collection of related data, stored so that it can be found, changed and combined reliably. "Organized" is the important word: the data has a known structure (customers have an email address, every order belongs to one customer), so a program can ask precise questions and get precise answers.

Data is often grouped by how much structure it has, a distinction you will meet in certification exams:

Kind What it looks like Typical storage
Structured Fixed fields in rows: every customer has a name, email and country Relational database tables
Semi-structured Self-describing records whose fields can vary, such as JSON documents or XML Document databases, files, data lakes
Unstructured No internal fields: photos, video, PDFs, free text Files or object storage, often with a database holding the metadata

Why not just files or spreadsheets?

A small shop could keep its orders in a spreadsheet, and many do, at first. The problems start when several people need the data at once, when the data must follow rules, and when it grows. A database is built for exactly those situations:

Concern Files and spreadsheets A database
Concurrency Two people editing the same file overwrite each other's changes, or the file is locked for everyone else Hundreds of users and programs read and write at the same time; the DBMS keeps their changes apart
Integrity Any cell accepts anything: a price of −5, an order for a customer who does not exist, a typo in a date Rules (data types, constraints, keys) are enforced on every change; invalid data is refused
Size Spreadsheets slow down with many rows and have a hard row limit (about one million in Excel) Billions of rows; indexes find one row without reading the rest
Security Whoever can open the file sees and changes everything in it Permissions per user, per table and even per row; access can be logged
Querying Filters and formulas by hand; combining several sheets is fragile One language (SQL) answers any question about the data, combining tables as needed
Recovery A crash during saving can leave a half-written, corrupted file; you restore yesterday's copy Changes are all-or-nothing and logged; after a crash the database restarts in a consistent state, and backups can be restored to a point in time

Spreadsheets remain excellent for personal analysis, quick calculations and sharing a snapshot of results. The rule of thumb: when data is shared, must stay correct, or is the system of record for the business, it belongs in a database.

The database management system

Applications never touch the database files directly. They talk to a database management system (DBMS): the software that stores the data, accepts requests, checks them against the rules and returns results. People often say "the database" when they mean the DBMS, the data, or both; the context usually makes it clear. A DBMS that implements the relational model is called a relational database management system (RDBMS).

Whatever the product, a DBMS does the same jobs:

  • Storage — laying data out efficiently on disk and caching the busiest parts in memory.
  • Query processing — reading a request, choosing a fast way to answer it (the query plan, see Indexes and Query Performance) and executing it.
  • Integrity — enforcing data types and constraints so that invalid data cannot get in.
  • Concurrency control — letting many users work at once without seeing each other's half-finished changes (see Transactions and ACID).
  • Security — authenticating users and checking their permissions (see Database Security).
  • Recovery — logging changes so the data survives crashes, and supporting backups and replicas (see Backup, Recovery and Replication).

The example below shows integrity at work. The shop's products table has the rule that a price must be greater than zero, so the DBMS refuses a product that costs nothing, and nothing is stored:

INSERT INTO products (product_id, name, category, price, stock)
VALUES (11, 'Mouse pad', 'Accessories', 0, 50);
Error: CHECK constraint failed: price > 0

Relational databases and products

A relational database stores data in tables of rows and columns and links tables through shared key values: an order row holds the number of the customer who placed it. The idea comes from Edgar F. Codd's 1970 paper on the relational model, and it has been the dominant way to store business data since the 1980s. How tables, keys and relationships work is the subject of The Relational Model.

The products you are most likely to meet:

Product License Architecture Typical use
SQLite Public domain Embedded Inside apps, phones, browsers and devices; local storage; learning and testing (all examples in this topic)
PostgreSQL Open source (PostgreSQL License) Client-server General-purpose server database with a reputation for standards compliance and extensions
MySQL / MariaDB Open source (GPL); MySQL also has commercial editions from Oracle Client-server Web applications and content management systems; MariaDB is a community fork of MySQL
Microsoft SQL Server Commercial (free Express and Developer editions) Client-server Organizations built on Microsoft technology; the engine behind Azure SQL Database
Oracle Database Commercial (free edition available) Client-server Large enterprises, banking, ERP systems

Embedded or client-server

An embedded database such as SQLite is a library inside the application: there is no separate server process, and the whole database is one ordinary file. That makes it ideal for a phone app or a desktop program, and less suited to many users writing at once over a network.

A client-server database runs as a service on a server (see the client-server model). Applications connect over the network, log in with a database account and send SQL; the server manages concurrency, permissions and memory for all of them. In the cloud the same engines are offered as managed databases, where the provider runs the server for you.

sqlite3 shop.db < shop.sql      # create shop.db from the sample script
sqlite3 shop.db                   # open an interactive SQL prompt on it

Those two commands are all it takes to follow along with this topic: download shop.sql, install the sqlite3 command-line tool and load the sample shop. A client-server product needs an installed (or hosted) server, a user account and a connection string before you can type the first query.

SQL, the language

SQL (Structured Query Language) is the standard language for defining, querying and changing data in a relational database. It was developed at IBM in the 1970s, became an ANSI and ISO standard in 1986–1987 and is still revised; the current edition is SQL:2023. Every relational product speaks SQL, each with its own extensions and small differences (its dialect): Microsoft's is called T-SQL, Oracle's procedural extension PL/SQL.

SQL is a declarative language: you describe what you want, not how to get it. You never write "open the file, loop over the lines, keep the ones where…". You write the result you want, and the DBMS decides which indexes to use and in which order to work. Here is a first query on the sample shop: the cables, cheapest first.

SELECT product_id, name, category, price
FROM products
WHERE category = 'Cables'
ORDER BY price;
product_id  name                category  price
----------  ------------------  --------  -----
10          Ethernet cable 5 m  Cables    7
1           USB-C cable 2 m     Cables    9
2           HDMI cable 3 m      Cables    12

Read it almost as English: select these four columns, from the products table, where the category is Cables, ordered by price. The result is itself a small table. The full grammar of SELECT follows in Querying with SELECT.

SQL statements fall into a few families:

Family Statements Purpose
Querying SELECT Read data (sometimes counted as part of DML)
Data manipulation language (DML) INSERT, UPDATE, DELETE Change the rows (see Changing Data)
Data definition language (DDL) CREATE, ALTER, DROP Change the structure (see Designing Tables)
Data control language (DCL) GRANT, REVOKE Manage permissions
Transaction control BEGIN, COMMIT, ROLLBACK Group changes into all-or-nothing units

Other kinds of databases

Relational databases are not the only option. Since the late 2000s a family of non-relational databases, usually called NoSQL, has grown for workloads where fixed tables are a poor fit: enormous scale spread over many servers, data whose shape varies from record to record, or questions about connections between things.

Kind Stores Examples
Key-value A value looked up by a unique key Redis, Amazon DynamoDB
Document JSON-like documents with varying fields MongoDB, Azure Cosmos DB
Wide-column Rows with flexible, very many columns Apache Cassandra
Graph Nodes and the relationships between them Neo4j

They trade some of the guarantees and the query flexibility of SQL for scale and flexibility of shape. The details are in NoSQL Databases. Many systems use both: a relational database for orders and payments, a key-value store as a fast cache.

Transactional and analytical use

Databases also differ in the work they are tuned for. A transactional (operational) database runs the business day to day: many small reads and writes, each touching a few rows, such as placing an order or updating a stock level. An analytical database answers questions about the business: fewer, much larger queries that scan millions of rows, such as "revenue per category per month for the last three years". The two are usually kept apart, with data copied from the first to the second. These workloads are called OLTP and OLAP; see Data Warehousing and Analytics.

The schema

A schema is the structure of a database: which tables exist, which columns each one has, their data types and the rules that connect them. It is defined once (with DDL statements such as CREATE TABLE) and changes rarely; the data inside changes all the time. The schema is a contract: every program that uses the database can rely on orders.order_date existing and holding a date.

The sample shop's schema is in shop.sql: five CREATE TABLE statements, followed by the data. A database also describes its own schema, so you can ask it which tables it has. In SQLite that information lives in the built-in sqlite_schema table:

SELECT name
FROM sqlite_schema
WHERE type = 'table'
ORDER BY name;
name
-----------
customers
employees
order_items
orders
products

Other products expose the same kind of information through the standard information_schema views (PostgreSQL, MySQL, SQL Server) or their own system catalogs.