Index

Module 01 · Database InternalsOne Database, to the Bolt

The sentence that says what

The Query Planner
The Query Planner · Open board

The machine inside a database that turns a question you type into a way of fetching the answer. This lesson is about the question you type; the rest of the chapter opens the machine.

Before you start. You have used an app that saves things: a sign-up form, a bank balance, a list of past orders. That is all this lesson needs; it comes first in the chapter and teaches the language every later lesson reads.

May 1974: a language for accountants and urban planners

In May 1974, at a workshop in Ann Arbor, Michigan, two researchers from IBM’s lab in San Jose, Donald Chamberlin and Raymond Boyce, presented a paper called “SEQUEL: A Structured English Query Language”. Four years earlier their IBM colleague Edgar Codd had proposed keeping data as plain tables, and the San Jose lab was starting to build a database on that idea. What it lacked was a way for people to ask it things.

Their first try, SQUARE, looked like a maths formula and was hard to type. So they started again, in words.

The paper’s first example: Find the names of employees in the toy department. And the answer, in their new language:

SELECT NAME
FROM   EMP
WHERE  DEPT = 'TOY'

The paper says who it was for: people who are “not computer specialists”, and it names them, “accountants, engineers, architects, and urban planners”. It also describes how a person does the job by hand: they “might look down the DEPT column of the EMP table, finding ‘TOY’ entries and making a list of the corresponding NAME entries”. The new sentence says none of that. It names the answer and stops.

Boyce did not see what came of it. He died of an aneurysm in June 1974, a month after the workshop, aged 27. SEQUEL later became SQL, the language nearly every database since has spoken, including Postgres, the free database this chapter opens up.

Ask the librarian: say what, never how

Picture a big library. You can walk the aisles yourself, checking every spine for the author you want. Or you can go to the desk and say: “Every book by Ursula Le Guin, please.” You have said what you want, not how to find it. The librarian knows the building and picks the fastest way.

A database keeps data in tables. A table is a grid: each row is one record (one user), each column is one field of it (a name, a city). This chapter runs on one small database you can build on your laptop in two commands (the second read shows how): a users table with 100,000 rows and an accounts table with 106,667, the same rows on every machine. Here is a question to it, written the way Chamberlin and Boyce wrote theirs:

SELECT id, name, city FROM users WHERE id <= 5;

What the database handed back, from a real run on 2026-09-28:

 id |  name   |   city    
----+---------+-----------
  1 | bob_1   | Pune
  2 | carol_2 | Kolkata
  3 | dev_3   | Pune
  4 | esha_4  | Hyderabad
  5 | farid_5 | Hyderabad
(5 rows)

SELECT names the columns you want, FROM names the table, WHERE says which rows count. A question written this way is a query. Ask how many users live in Pune and the answer is 10,065. The query never said how to find them: walk all 100,000 rows, or jump straight to the Pune ones. That choice was left to the database, the way the shelves were left to the librarian.

Two tables, one answer: the join

Users and accounts live in separate tables. Each row of accounts carries a user_id, the id of the user who owns it, like a borrowing slip carrying a reader’s card number. To see a user beside their accounts, you ask for rows from both tables where that number matches:

flowchart LR
  U["users row: id 3, dev_3"] -->|"accounts.user_id = 3"| A1["account: credit, 17903"]
  U -->|"accounts.user_id = 3"| A2["account: credit, 34384"]
  F["users row: id 5, farid_5"] -.->|"no account has user_id 5"| X["nothing to pair with"]

That pairing is a join. For the first six users it printed seven rows: user 3 twice (two accounts), user 6 twice, and user 5 not at all. The practice data behind these runs was built so that every fifth user has no account, and a plain join shows only pairs that exist. Write LEFT JOIN instead and every user comes back, with farid_5 standing next to an empty slot where the account would be. Which one you want is part of the answer, so SQL makes you say it.

Folding rows into groups: GROUP BY

“How many users live in each city?” is a question about piles, not rows. You tell the database to sort the users into a pile per city and count each pile:

SELECT city, count(*) FROM users GROUP BY city ORDER BY count(*) DESC;

That is GROUP BY: one output row per pile. It printed ten rows. Nine were cities, each with roughly ten thousand users, Delhi on top at 10,185. The tenth row had a count of 9,937 and nothing at all in the city column.

The blank that is not a value: NULL

In 2009, at the QCon conference in London, the computer scientist Tony Hoare gave a talk called “Null References: The Billion Dollar Mistake”. In 1965, designing a programming language called ALGOL W, he had added a way for a slot to hold nothing, “simply because it was so easy to implement”. He guessed it had since caused a billion dollars of errors and crashes. His was a programming language’s nothing, not SQL’s, but SQL’s bites the same way.

Those 9,937 users signed up without giving a city. Their city is not an empty word; it is a slot marked “unknown”, and SQL calls that mark NULL. Now ask a question that must surely cover everyone: users whose city is Pune, or whose city is not Pune. The count came back 90,063, not 100,000.

Here is why. Is an unknown city equal to Pune? The database will not say yes, and it will not say no. It says “unknown”, a third answer beside true and false, and WHERE keeps only the rows that come out true. So every NULL row falls out of both halves of the question, and 9,937 users vanish. The same rule catches city = NULL, which looks like the obvious way to find them and returned 0 rows: nothing is ever known to equal an unknown, not even another unknown. SQL has a separate test for the mark itself, city IS NULL, and that one returned all 9,937.

A librarian holding a book with no author on it would not file it under “not Le Guin” either. She would say she does not know.

Who picks the how: the planner

Every query so far said what it wanted and left the how alone. Something inside the database has to decide it, fresh, for every query. That part is the planner, and it is the librarian of this story.

flowchart TD
  Q["your query: says WHAT"] --> P["the planner: decides HOW"]
  P -->|"one user, by id"| A["jump to that user, then check each account"]
  P -->|"every Pune user"| B["read both tables whole and match them up"]
  A --> R["the rows you asked for"]
  B --> R

Two queries of the same shape show it. Joining users to their accounts for one user by id, the planner went straight to that one user through a lookup the table keeps on its ids. Joining for every Pune user, about 10,000 of them, it read both tables from end to end instead, because touching ten thousand scattered users one at a time would cost more than one pass over everything. Same sentence, two routes, and you asked for neither.

This is where the library stops matching. A librarian learns one building over years and fetches a few books a day. The planner chooses in a moment, for tables of any size, from a short summary of what each table holds; how it guesses, and how it guesses wrong, is the planner lesson later in this chapter. Everything between here and there is the shelving: how rows sit on disk, what is kept close at hand, what makes one route cheaper. You will keep writing the same kind of sentence the whole way through.

You now know

  • SQL is a language for asking a database questions; a query names the answer you want and never the steps to find it.
  • A join matches rows from two tables on a shared value, such as putting each account beside the user whose id it carries. A plain join leaves out a user with no account; a left join keeps that user and leaves the account columns blank.
  • GROUP BY sorts rows into piles by a column and returns one row per pile, such as a count of users per city.
  • NULL marks a value as unknown; any comparison with it comes out unknown rather than true or false, so = NULL finds nothing and IS NULL is the test that works.
  • Inside the database, a part called the planner picks how to answer each query, and two queries of the same shape can be answered in completely different ways.

Stop here if you want. What follows is the rest of the language in real runs, then twelve exercises on the same two tables, each with its answer.

Second read: every clause, then twelve exercises

Your bench

Every query in this lesson ran on the chapter’s bench: a throwaway Postgres 16 in Docker, filled by a script that makes the same rows on every machine. The Pages and heap files lesson, later in this chapter, uses the same bench and says more about it. Start the database. Docker runs a program in a sealed box on your machine; this line fetches Postgres 16, names the box dbbench, sets its password to bench, and makes it reachable on port 5433 (a numbered door on your machine):

docker run -d --name dbbench -e POSTGRES_PASSWORD=bench -p 5433:5432 postgres:16

Then fill it:

cd teaching/code/di-bench
python3 seed.py
Run it yourself: full source file
  1. Install Python 3. Check it with python3 --version.
  2. Make the folder: mkdir -p teaching/code/di-bench. Save the file below at the path shown above it, including any subfolders.
  3. Run the commands in the block above from the folder where you ran mkdir.
teaching/code/di-bench/seed.py
"""Seed the Database Internals bench: one Postgres every Try it in the track runs on.

Creates two tables, dropping them first, so running it again resets them:

  users     100,000 rows  id (primary key), name, email, city, signed_up_at
            city is NULL for about one user in ten.
  accounts  106,667 rows  id (primary key), user_id, kind, balance, opened_at
            users whose id is a multiple of 5 have no account; the rest have
            one, and those whose id is also a multiple of 3 have two.
            kind is 'checking', 'savings' or 'credit'.

Every value is derived from the row number (md5 of it, and modulo), never
random(), so every reader's bench holds the same rows. The only indexes are
the two primary keys, users_pkey and accounts_pkey, plus nothing on
accounts.user_id on purpose: a missing index has to show up somewhere.

Run it against a postgres:16 container (the name defaults to dbbench):

  docker run -d --name dbbench -e POSTGRES_PASSWORD=bench -p 5433:5432 postgres:16
  python3 seed.py            # or: python3 seed.py <container-name>

Python 3 standard library only. It pipes SQL into psql inside the container
with `docker exec -i <container> psql -U postgres`.
"""

import subprocess
import sys
import time

CONTAINER = sys.argv[1] if len(sys.argv) > 1 else "dbbench"
USERS = 100_000

# h(x) is a 32-bit number read off md5(x): the same on every machine, every run.
SEED_SQL = f"""
DROP TABLE IF EXISTS accounts;
DROP TABLE IF EXISTS users;

CREATE TABLE users (
    id           int         NOT NULL,
    name         text        NOT NULL,
    email        text        NOT NULL,
    city         text,
    signed_up_at timestamptz NOT NULL
);

INSERT INTO users (id, name, email, city, signed_up_at)
SELECT g,
       n.first || '_' || g,
       n.first || '_' || g || '@example.com',
       (ARRAY['Pune', 'Delhi', 'Mumbai', 'Bengaluru', 'Chennai',
              'Kolkata', 'Jaipur', 'Hyderabad', 'Lucknow', NULL])[1 + h.v % 10],
       timestamptz '2024-01-01 00:00:00+00'
         + (h.v % 1000) * interval '1 day'
         + (g % 1440)   * interval '1 minute'
FROM generate_series(1, {USERS}) AS g
CROSS JOIN LATERAL (
    SELECT ('x' || substr(md5(g::text), 1, 8))::bit(32)::bigint AS v
) AS h
CROSS JOIN LATERAL (
    SELECT (ARRAY['alice', 'bob', 'carol', 'dev', 'esha',
                  'farid', 'gita', 'hari'])[1 + g % 8] AS first
) AS n
ORDER BY g;

ALTER TABLE users ADD CONSTRAINT users_pkey PRIMARY KEY (id);

CREATE TABLE accounts (
    id        int         NOT NULL,
    user_id   int         NOT NULL,
    kind      text        NOT NULL,
    balance   int         NOT NULL,
    opened_at timestamptz NOT NULL
);

INSERT INTO accounts (id, user_id, kind, balance, opened_at)
SELECT row_number() OVER (ORDER BY u.id, k),
       u.id,
       (ARRAY['checking', 'savings', 'credit'])[1 + h.v % 3],
       (h.v % 50000)::int,
       u.signed_up_at + (h.v % 90) * interval '1 day'
FROM users AS u
CROSS JOIN LATERAL generate_series(
    1,
    CASE WHEN u.id % 5 = 0 THEN 0      -- no account at all
         WHEN u.id % 3 = 0 THEN 2      -- two accounts
         ELSE 1 END
) AS k
CROSS JOIN LATERAL (
    SELECT ('x' || substr(md5('acct' || u.id || '-' || k), 1, 8))::bit(32)::bigint AS v
) AS h
ORDER BY u.id, k;

ALTER TABLE accounts ADD CONSTRAINT accounts_pkey PRIMARY KEY (id);
ALTER TABLE accounts ADD CONSTRAINT accounts_user_id_fkey
    FOREIGN KEY (user_id) REFERENCES users (id);

VACUUM ANALYZE users;
VACUUM ANALYZE accounts;
"""

SUMMARY_SQL = """
SELECT label, n FROM (
          SELECT 1 AS ord, 'users' AS label, count(*) AS n FROM users
UNION ALL SELECT 2, 'accounts', count(*) FROM accounts
UNION ALL SELECT 3, 'users with no account', count(*) FROM users u
          WHERE NOT EXISTS (SELECT 1 FROM accounts a WHERE a.user_id = u.id)
UNION ALL SELECT 4, 'users with city NULL', count(*) FROM users WHERE city IS NULL
UNION ALL SELECT 5, 'accounts, kind = ' || kind, count(*) FROM accounts GROUP BY kind
) s ORDER BY ord, label;
"""


def psql(sql, *flags):
    return subprocess.run(
        ["docker", "exec", "-i", CONTAINER,
         "psql", "-U", "postgres", "-X", "-v", "ON_ERROR_STOP=1", *flags],
        input=sql, text=True, capture_output=True,
    )


def wait_for_postgres():
    # The image starts a temporary server to initialise, then restarts. The
    # temporary one does not listen on TCP, so asking over 127.0.0.1 waits
    # for the real one.
    for _ in range(60):
        r = subprocess.run(
            ["docker", "exec", CONTAINER, "pg_isready", "-h", "127.0.0.1", "-U", "postgres"],
            capture_output=True, text=True,
        )
        if r.returncode == 0:
            return
        time.sleep(1)
    sys.exit(f"Postgres in container {CONTAINER!r} did not come up in 60s")


def main():
    wait_for_postgres()
    r = psql(SEED_SQL, "-q")
    if r.returncode != 0:
        sys.exit(r.stderr)
    r = psql(SUMMARY_SQL, "-At", "-F", "\t")
    if r.returncode != 0:
        sys.exit(r.stderr)
    print("bench seeded:")
    for line in r.stdout.splitlines():
        label, count = line.split("\t")
        print(f"  {label:<26} {int(count):>7,}")


if __name__ == "__main__":
    main()

What it printed (measured 2026-09-28, on a fresh container):

bench seeded:
  users                      100,000
  accounts                   106,667
  users with no account       20,000
  users with city NULL         9,937
  accounts, kind = checking   35,544
  accounts, kind = credit     35,261
  accounts, kind = savings    35,862

users has id, name, email, city (NULL for about one user in ten) and signed_up_at. accounts has id, user_id, kind (checking, savings or credit), balance and opened_at. Users whose id is a multiple of 5 have no account; the rest have one, and those whose id is also a multiple of 3 have two. Open a prompt with:

docker exec -it dbbench psql -U postgres

Then type \pset null '(null)' once. Postgres prints NULL as blank by default, which is how the story’s GROUP BY row looked empty; with this setting it prints (null), and every output below except the city count was captured with it on. docker rm -f dbbench throws the bench away when you are done.

The clauses, in the order the database reads them

You write SELECT … FROM … WHERE … GROUP BY … HAVING … ORDER BY … LIMIT. The database reads them in a different order: FROM (and any join), then WHERE, then GROUP BY, then HAVING, then SELECT, then ORDER BY, then LIMIT. That order explains most of the rules below. WHERE runs before the piles exist, so it cannot filter on a count; HAVING runs after, so it can.

The story’s plain join, for the first six users. Each row is one user paired with one of their accounts; user 5 is missing:

SELECT u.id, u.name, a.kind, a.balance
FROM users u JOIN accounts a ON a.user_id = u.id
WHERE u.id <= 6 ORDER BY u.id, a.id;
 id |  name   |   kind   | balance 
----+---------+----------+---------
  1 | bob_1   | savings  |   27304
  2 | carol_2 | savings  |    6529
  3 | dev_3   | credit   |   17903
  3 | dev_3   | credit   |   34384
  4 | esha_4  | savings  |    7676
  6 | gita_6  | checking |     233
  6 | gita_6  | checking |   27549
(7 rows)

The same with LEFT JOIN. Farid has no account, so his account columns are NULL:

SELECT u.id, u.name, a.kind, a.balance
FROM users u LEFT JOIN accounts a ON a.user_id = u.id
WHERE u.id <= 6 ORDER BY u.id, a.id;
 id |  name   |   kind   | balance 
----+---------+----------+---------
  1 | bob_1   | savings  |   27304
  2 | carol_2 | savings  |    6529
  3 | dev_3   | credit   |   17903
  3 | dev_3   | credit   |   34384
  4 | esha_4  | savings  |    7676
  5 | farid_5 | (null)   |  (null)
  6 | gita_6  | checking |     233
  6 | gita_6  | checking |   27549
(8 rows)

The story’s city count, every pile, captured before \pset null was set; the blank-city pile is the seventh row:

   city    | count 
-----------+-------
 Delhi     | 10185
 Lucknow   | 10100
 Bengaluru | 10089
 Kolkata   | 10086
 Pune      | 10065
 Jaipur    |  9948
           |  9937
 Hyderabad |  9898
 Mumbai    |  9852
 Chennai   |  9840
(10 rows)

ORDER BY and LIMIT. A table has no order of its own; without ORDER BY, rows come back in whatever order the chosen route produced them, and that can change. ORDER BY sorts, LIMIT keeps the first few. The five biggest balances, ties broken by account id:

SELECT id, user_id, kind, balance FROM accounts ORDER BY balance DESC, id LIMIT 5;
  id   | user_id |   kind   | balance 
-------+---------+----------+---------
 72053 |   67549 | savings  |   49999
 31711 |   29728 | checking |   49998
 89208 |   83632 | savings  |   49998
 92148 |   86388 | savings  |   49998
   751 |     703 | credit   |   49997
(5 rows)

GROUP BY with more than a count. Each pile can be summarised any way: count, sum, avg, min, max. Columns read pile, how many accounts in it, their average balance rounded to a whole number:

SELECT kind, count(*) AS accounts, avg(balance)::int AS avg_balance
FROM accounts GROUP BY kind ORDER BY kind;
   kind   | accounts | avg_balance 
----------+----------+-------------
 checking |    35544 |       24940
 credit   |    35261 |       24830
 savings  |    35862 |       25068
(3 rows)

HAVING filters piles. Cities whose users hold more than 10,700 accounts between them:

SELECT u.city, count(*) AS accounts
FROM users u JOIN accounts a ON a.user_id = u.id
GROUP BY u.city HAVING count(*) > 10700 ORDER BY accounts DESC;
  city   | accounts 
---------+----------
 Delhi   |    10895
 Pune    |    10823
 Lucknow |    10782
 Kolkata |    10721
(4 rows)

NULL, three more ways it bites

The three answers. Each column below is one comparison; t is true, and (null) here is the answer “unknown”:

SELECT NULL = NULL AS equal, NULL IS NULL AS is_null,
       1 = NULL AS one_eq_null, NOT (1 = NULL) AS negated;
 equal  | is_null | one_eq_null | negated 
--------+---------+-------------+---------
 (null) | t       | (null)      | (null)
(1 row)

Not-unknown is still unknown. That is why “Pune or not Pune” lost 9,937 rows: city <> 'Pune' alone returned 79,998, which is 100,000 − 10,065 Pune − 9,937 unknown (checked with python3).

COUNT(*) and COUNT(column) are different questions. count(*) counts rows. count(city) counts rows whose city is not NULL:

SELECT count(*) AS every_row, count(city) AS rows_with_a_city FROM users;
 every_row | rows_with_a_city 
-----------+------------------
    100000 |            90063
(1 row)

NOT IN with a NULL in the list returns nothing. city NOT IN ('Pune', 'Delhi') returned 69,813. Add a NULL to the list and it returned 0:

SELECT count(*) FROM users WHERE city NOT IN ('Pune', 'Delhi', NULL);
 count 
-------
     0
(1 row)

NOT IN (a, b, NULL) means “not a, and not b, and not unknown”. The last part is unknown for every row, so no row comes out true. In practice the NULL arrives from a subquery (a query inside another query) over a column that allows NULL; NOT EXISTS or a LEFT JOIN … IS NULL does not have this trap.

Naming a step: WITH

A CTE (common table expression) gives a query’s result a name, so the next step can read it like a table. First each user’s total balance, then how many totals pass 60,000:

WITH totals AS (
  SELECT user_id, sum(balance) AS total FROM accounts GROUP BY user_id
)
SELECT count(*) AS users_over_60k FROM totals WHERE total > 60000;
 users_over_60k 
----------------
           8595
(1 row)

It says nothing about how. The planner is free to run it as one piece.

A window: ranking without folding

GROUP BY folds a pile into one row. A window function computes across a set of rows and keeps every row. rank() OVER (ORDER BY balance DESC) gives each account its place among the others; here, the accounts of users 1 to 12:

SELECT user_id, kind, balance, rank() OVER (ORDER BY balance DESC) AS place
FROM accounts WHERE user_id <= 12 ORDER BY place;
 user_id |   kind   | balance | place 
---------+----------+---------+-------
      11 | checking |   38460 |     1
       3 | credit   |   34384 |     2
      12 | savings  |   29168 |     3
       6 | checking |   27549 |     4
       1 | savings  |   27304 |     5
       9 | savings  |   26397 |     6
       7 | savings  |   25946 |     7
       9 | checking |   24564 |     8
      12 | checking |   18818 |     9
       3 | credit   |   17903 |    10
       4 | savings  |    7676 |    11
       8 | savings  |    7239 |    12
       2 | savings  |    6529 |    13
       6 | checking |     233 |    14
(14 rows)

Add PARTITION BY kind inside the OVER and the ranking restarts for each kind; exercise 10 does that.

The how, printed

EXPLAIN in front of a query prints the planner’s chosen route instead of running it. The story’s two joins. Every Pune user: read both tables whole (Seq Scan), build a lookup of the Pune users in memory (Hash), and match accounts against it (Hash Join):

EXPLAIN SELECT u.name, a.balance FROM users u JOIN accounts a ON a.user_id = u.id WHERE u.city = 'Pune';
                                QUERY PLAN                                 
---------------------------------------------------------------------------
 Hash Join  (cost=2406.34..4538.02 rows=10781 width=15)
   Hash Cond: (a.user_id = u.id)
   ->  Seq Scan on accounts a  (cost=0.00..1851.67 rows=106667 width=8)
   ->  Hash  (cost=2280.00..2280.00 rows=10107 width=15)
         ->  Seq Scan on users u  (cost=0.00..2280.00 rows=10107 width=15)
               Filter: (city = 'Pune'::text)
(6 rows)

One user by id: jump to it through the primary key’s index (Index Scan using users_pkey), then check accounts for its id (Nested Loop):

EXPLAIN SELECT u.name, a.balance FROM users u JOIN accounts a ON a.user_id = u.id WHERE u.id = 42;
                                   QUERY PLAN                                    
---------------------------------------------------------------------------------
 Nested Loop  (cost=0.29..2126.66 rows=1 width=15)
   ->  Index Scan using users_pkey on users u  (cost=0.29..8.31 rows=1 width=15)
         Index Cond: (id = 42)
   ->  Seq Scan on accounts a  (cost=0.00..2118.34 rows=1 width=8)
         Filter: (user_id = 42)
(5 rows)

The rows= figures are the planner’s guesses from a random sample of the table (10,107 Pune users guessed; 10,065 real), so your run may print slightly different guesses and costs. The shape of each plan is the part that should match. Notice the second plan still reads all of accounts to find user 42’s rows: the bench deliberately has no index on accounts.user_id. Reading plans like these is the planner lesson’s job.

Twelve exercises

Each is a question you can answer with what is above. Write your query, run it on the bench, then open the answer. Every answer was run on the bench on 2026-09-28 with \pset null '(null)' on.

1. How many users live in Mumbai?

Answer
SELECT count(*) FROM users WHERE city = 'Mumbai';
 count 
-------
  9852
(1 row)

2. The five users who signed up most recently: id, name, sign-up time.

Answer
SELECT id, name, signed_up_at FROM users ORDER BY signed_up_at DESC, id LIMIT 5;
  id   |    name     |      signed_up_at      
-------+-------------+------------------------
  4304 | alice_4304  | 2026-09-26 23:44:00+00
 71948 | esha_71948  | 2026-09-26 23:08:00+00
 96378 | carol_96378 | 2026-09-26 22:18:00+00
 11413 | farid_11413 | 2026-09-26 22:13:00+00
 63247 | hari_63247  | 2026-09-26 22:07:00+00
(5 rows)

The id after DESC breaks ties, so the answer is the same every run.

3. For each kind of account, how many hold more than 40,000?

Answer
SELECT kind, count(*) FROM accounts WHERE balance > 40000 GROUP BY kind ORDER BY kind;
   kind   | count 
----------+-------
 checking |  7063
 credit   |  6922
 savings  |  7266
(3 rows)

WHERE drops the small accounts before the piles are made.

4. How many users have no account at all?

Answer
SELECT count(*) FROM users u LEFT JOIN accounts a ON a.user_id = u.id WHERE a.id IS NULL;
 count 
-------
 20000
(1 row)

The left join keeps every user; the ones with no partner have a NULL account id. That is one user in five, the multiples of 5.

5. How many users have exactly two accounts?

Answer
SELECT count(*) FROM (
  SELECT user_id FROM accounts GROUP BY user_id HAVING count(*) = 2
) AS two;
 count 
-------
 26667
(1 row)

Multiples of 3 that are not multiples of 5: 33,333 − 6,666 = 26,667. With 80,000 users holding at least one, 80,000 + 26,667 = 106,667 accounts, the seed’s total.

6. Total balance held by each city’s users, biggest first, with the no-city users labelled.

Answer
SELECT coalesce(u.city, '(no city)') AS city, sum(a.balance) AS total
FROM users u JOIN accounts a ON a.user_id = u.id
GROUP BY u.city ORDER BY total DESC;
   city    |   total   
-----------+-----------
 Delhi     | 271600877
 Pune      | 271101090
 Lucknow   | 269134849
 Jaipur    | 265729188
 Bengaluru | 264733192
 Kolkata   | 264651402
 Mumbai    | 264615045
 (no city) | 264505917
 Hyderabad | 264156377
 Chennai   | 260774961
(10 rows)

coalesce(x, y) returns x unless it is NULL, then y. GROUP BY puts all the NULLs in one pile, even though no NULL “equals” another.

7. Pune has 10,065 users and Delhi 10,185. Why does WHERE city NOT IN ('Pune', 'Delhi') return 69,813 and not 79,750? Fix it.

Answer
SELECT count(*) FROM users WHERE city NOT IN ('Pune', 'Delhi') OR city IS NULL;
 count 
-------
 79750
(1 row)

For the 9,937 users with no city, “not Pune and not Delhi” is unknown, so WHERE drops them: 79,750 − 9,937 = 69,813. Say you want them, with OR city IS NULL.

8. How many users signed up during 2025?

Answer
SELECT count(*) FROM users
WHERE signed_up_at >= '2025-01-01' AND signed_up_at < '2026-01-01';
 count 
-------
 36457
(1 row)

A half-open range (>= start, < next start) catches every moment of 31 December without guessing its last second. The dates are read in the bench’s time zone, UTC.

9. Among users whose accounts total more than 60,000, which three cities have the most?

Answer
WITH totals AS (
  SELECT user_id, sum(balance) AS total FROM accounts GROUP BY user_id
)
SELECT u.city, count(*) AS rich_users
FROM totals t JOIN users u ON u.id = t.user_id
WHERE t.total > 60000
GROUP BY u.city ORDER BY rich_users DESC LIMIT 3;
  city  | rich_users 
--------+------------
 Delhi  |        901
 (null) |        893
 Pune   |        878
(3 rows)

The no-city users are the second-biggest pile here. A report that dropped NULL cities would have got the ranking wrong.

10. The two biggest accounts of each kind.

Answer
WITH ranked AS (
  SELECT kind, id, balance,
         rank() OVER (PARTITION BY kind ORDER BY balance DESC) AS place
  FROM accounts
)
SELECT kind, id, balance, place FROM ranked WHERE place <= 2 ORDER BY kind, place, id;
   kind   |  id   | balance | place 
----------+-------+---------+-------
 checking | 31711 |   49998 |     1
 checking | 64608 |   49997 |     2
 credit   |   751 |   49997 |     1
 credit   | 52866 |   49992 |     2
 savings  | 72053 |   49999 |     1
 savings  | 89208 |   49998 |     2
 savings  | 92148 |   49998 |     2
(7 rows)

Savings has two accounts tied at 49,998, and rank() gives both place 2, so “top two” returns three rows. row_number() would pick one of them arbitrarily. The CTE is needed because WHERE runs before the window is computed.

11. For users 1 to 4, each account next to that user’s total, without losing the account rows.

Answer
SELECT user_id, kind, balance, sum(balance) OVER (PARTITION BY user_id) AS user_total
FROM accounts WHERE user_id <= 4 ORDER BY user_id, id;
 user_id |  kind   | balance | user_total 
---------+---------+---------+------------
       1 | savings |   27304 |      27304
       2 | savings |    6529 |       6529
       3 | credit  |   17903 |      52287
       3 | credit  |   34384 |      52287
       4 | savings |    7676 |       7676
(5 rows)

GROUP BY user_id would have folded user 3 into one row. The window keeps both and writes 17,903 + 34,384 = 52,287 beside each.

12. Sign-ups per month for January to June 2024, with a running total.

Answer
WITH monthly AS (
  SELECT date_trunc('month', signed_up_at)::date AS month, count(*) AS signups
  FROM users WHERE signed_up_at < '2024-07-01' GROUP BY 1
)
SELECT month, signups, sum(signups) OVER (ORDER BY month) AS running_total
FROM monthly ORDER BY month;
   month    | signups | running_total 
------------+---------+---------------
 2024-01-01 |    3119 |          3119
 2024-02-01 |    2906 |          6025
 2024-03-01 |    2964 |          8989
 2024-04-01 |    2902 |         11891
 2024-05-01 |    3072 |         14963
 2024-06-01 |    3001 |         17964
(6 rows)

A sum() OVER (ORDER BY month) adds up every row from the first to the current one. The bench’s sign-ups start on 1 January 2024. Doing it in one query, with sum(count(*)) OVER (ORDER BY date_trunc('month', signed_up_at)), fails on the bench with column "users.signed_up_at" must appear in the GROUP BY clause: the window orders by the raw column, which no longer exists once rows are folded into months. Naming the monthly step first is the clean way out.

Try it

  1. Build the bench from the second read (two commands, about a minute) and run the twelve exercises before opening any answer.
  2. Break a join on purpose: write the users-to-accounts join and leave off the ON a.user_id = u.id, replacing it with ON true, for u.id <= 3 only. Count the rows and explain the number from the table sizes.
  3. Put EXPLAIN in front of exercise 4 and exercise 9. Name the operator at the top of each plan. Then run CREATE INDEX ON accounts (user_id);, run EXPLAIN on the one-user join from the second read again, and see which line changed. python3 seed.py rebuilds both tables without the index when you are done.
  4. Rewrite exercise 4 with NOT EXISTS and with NOT IN (SELECT user_id FROM accounts). All three should agree here. Say why NOT IN is safe on this column (hint: accounts.user_id is declared NOT NULL) and would not be on users.city.
  5. docker rm -f dbbench when you are done.

Go deeper

Database Internals - the deep-dive set · Pages & Heap Files · The Query Planner

The full Database Internals list is in CURRICULUM.

Retrieval practice

7 questions. Answer each one before you open it.

What does it mean that SQL says “what” and not “how”?

A query names the answer: which columns, from which tables, which rows, how grouped and sorted. It never says the steps: which table to read first, whether to scan every row or jump through an index, how to pair rows for a join. The database’s planner picks those steps fresh for every query, so the same sentence can run by two completely different routes, as the Pune join (read both tables whole, hash join) and the one-user join (index scan, nested loop) did on the bench.

On the bench, a join of users to accounts for users 1 to 6 returned seven rows and no row for user 5. Why, and how do you get user 5 back?

A plain (inner) join returns only pairs that exist: user 3 and user 6 have two accounts each, so they appear twice, and user 5 has no account, so there is nothing to pair him with. A LEFT JOIN keeps every row of the left table (users) and fills the missing account columns with NULL, which returns eight rows with farid_5 beside two NULLs.

Why did “city = ‘Pune’ OR city <> ‘Pune’” count 90,063 users and not 100,000?

9,937 users have a NULL city, meaning unknown. Comparing unknown to ‘Pune’ gives neither true nor false but unknown, for both halves, and unknown OR unknown is still unknown. WHERE keeps only rows that come out true, so all 9,937 fall out: 100,000 − 9,937 = 90,063.

Why does WHERE city = NULL return no rows, and what do you write instead?

Any comparison with NULL, including NULL = NULL, comes out unknown, and WHERE drops unknown rows, so nothing ever matches. IS NULL is a separate test that asks “is this slot marked unknown?” and answers true or false; on the bench it returned 9,937 rows.

count(*) on users gave 100,000 and count(city) gave 90,063. What is each one counting?

count(*) counts rows. count(column) counts rows where that column is not NULL. The gap, 9,937, is the users with no city. Which one you want depends on the question; mixing them up makes an average or a percentage quietly wrong.

Why does NOT IN ('Pune', 'Delhi', NULL) return zero rows, and what is the safe alternative?

NOT IN means “not equal to each item”, joined with AND. For every row, “not equal to NULL” is unknown, and anything AND unknown can never become true, so WHERE keeps nothing. The NULL usually sneaks in from a subquery over a column that allows NULL. NOT EXISTS, or a LEFT JOIN … WHERE other.id IS NULL, asks the same question without the trap.

What is the difference between GROUP BY and a window function, and why can’t HAVING be replaced with WHERE?

GROUP BY folds each pile of rows into one output row. A window function (rank() OVER (…), sum() OVER (PARTITION BY …)) computes across a set of rows but keeps every row, writing the result beside each. WHERE runs before the piles exist, so it cannot test a count or a sum; HAVING runs after grouping, so it can. The same timing is why filtering on a window’s rank needs a CTE around it.

Lab

Your query has an index and Postgres refuses to use it. At what fraction of the table does reading all of it in order start to win?

Drag What fraction of the table the query actually matches upward one notch at a time, and watch The plan it picks.

Open The selectivity crossover →

Your sort spills to disk. Give it more memory and it stops. Why does nobody set work_mem to a gigabyte?

Click The default, then A big desk and The whole room, drag Sessions running it at once, and watch The whole box.

Open The work_mem dial →

Related paper

Review Database Internals →

Saved in this browser, and to your account when you sign in.