README.md

@@ -178,47 +178,91 @@ allowlisted source hosts for open access.
 
 ## Design
 
+### Concepts
+
+These terms are the names the code uses:
+
+- **Source**: the image at one upstream URL, named by its host, path and query.
+- **Source key**: the SHA-256 of a source's host, path and query, which names
+  the source's file and its row in the database.
+- **Transform**: the output size, format, quality and fit an image request asks
+  for; with the format `auto`, pixa chooses the output format from the `Accept`
+  header.
+- **Image request**: a source and a transform, which each image route reads from
+  its URL.
+- **Variant**: the image pixa serves for an image request, made from its source
+  with its transform.
+- **Variant key**: the SHA-256 of a variant's source, size, format, quality and
+  fit, which names the variant's file and its row in the database, and gives its
+  `ETag`.
+- **Signed URL**: a `/v1/image/` URL whose `sig` signs its image request and its
+  expiry `exp` with `signing_key`.
+- **Encrypted URL**: a `/v1/e/` URL whose token holds its image request and any
+  expiry, encrypted with a key derived from `signing_key`.
+- **Login session**: an encrypted cookie that keeps a browser logged in to the
+  generator page for 30 days, or until a logout replaces the random value pixa
+  keeps in the `logout` table (see Encrypted URLs).
+- **Negative cache**: the source URLs pixa does not fetch again for 5 minutes,
+  after one of these failed fetches:
+    - the upstream answered with an error status
+    - the body's first bytes were not those of the format its `Content-Type`
+      names
+- **Image cache**: the part of pixa that keeps the sources, the variants and the
+  negative cache (see Storage).
+- **Disk cache limit** (`cache_max_bytes`): the size in bytes that the cached
+  sources and variants may reach together.
+- **Image proxy**: the part of pixa that serves a variant from the image cache
+  when it can, and otherwise makes it from the source.
+
 ### Storage
 
-- **Source images**:
-  `<state_dir>/cache/sources/<ab>/<cd>/<sha256 of host, path and query>`, one
-  file for each source URL
+- **Sources**: `<state_dir>/cache/sources/<ab>/<cd>/<source key>`, one file for
+  each source URL
 - **Database**: `<state_dir>/state.sqlite3` (SQLite)
-- **Transformed images**:
-  `<state_dir>/cache/variants/<ab>/<cd>/<sha256 of host, path, query, size, format, quality and fit>`
+- **Variants**: `<state_dir>/cache/variants/<ab>/<cd>/<variant key>`
 
 `<ab>` and `<cd>` are the first and second pairs of characters of the file's
 name.
 
-Each source image and each transformed image has one row in the database,
-holding its size and when it was last used. pixa reads a cached source image's
-format from its file's first bytes. When they match none of the source formats,
-pixa fetches the source again. Two source URLs that serve the same bytes are
-stored twice, once under each URL. A source image is stored by writing its file
-and then its row; when the row cannot be written, the file is removed again.
+The image cache keeps these tables in the database:
 
-Source and transformed image files are written under a temporary name, synced to
-disk, and only then renamed into place, so a crash leaves no file under its own
-name with part of its data. When pixa starts, and every five minutes after, it
-removes each cached source or transformed image whose file is empty, or not the
-size the database records for it, with a warning naming the file; the next
-request for it fetches the source or converts the image again. Looking for those
-files reads the database once for every cached file, so pixa skips each
-five-minute look that would start less than ten times as long after the last
-look ended as that look took. After a look that took one minute, the next one
-starts 10 to 15 minutes after that look ended.
+- `source_content`: a row for each source, by source key, with its size and when
+  it was last used
+- `variant_content`: a row for each variant, by variant key (column
+  `cache_key`), with its size and when it was last used
+- `cache_usage`: the total size of the cached sources and variants, which
+  eviction reads
+- `negative_cache`: a row for each source URL in the negative cache, with when
+  it expires
+
+pixa reads a cached source image's format from its file's first bytes. When they
+match none of the source formats, pixa fetches the source again. Two source URLs
+that serve the same bytes are stored twice, once under each URL. A source image
+is stored by writing its file and then its row; when the row cannot be written,
+the file is removed again.
+
+Each source and variant file is written under a temporary name and synced to
+disk before it is renamed into place. So a crash never leaves part of a file
+under its own name. When pixa starts, and every five minutes after, it removes
+each source or variant whose file is empty, or not the size its row records. It
+logs a warning naming the file. A later request for that image fetches the
+source or makes the variant again. Looking for those files reads the database
+once for every cached file, so pixa skips each five-minute look that would start
+less than ten times as long after the last look ended as that look took. After a
+look that took one minute, the next one starts 10 to 15 minutes after that look
+ended.
 
 pixa's target is 1-5k r/s, which has not been measured at that rate (see Load
-Test). Toward it, a cache hit opens the transformed image's file and nothing
-else, and makes no database call. The content type comes from the file's first
-bytes, which are those of the format pixa wrote it in; a file whose first bytes
-are not those of a format pixa writes is not served, and the image is converted
-again. A hit hands the time it was served to a background writer and does not
-wait for the writer. Eviction orders by that time. The hit is counted in memory.
-A miss that uses a cached source image also hands that writer the time it used
-the image. Once a second that writer writes the waiting times in one
-transaction. While 10,000 times of transformed images, or 10,000 of source
-images, wait to be written, a further one is dropped, and that image only looks
+Test). Toward it, a cache hit opens the variant's file and nothing else, and
+makes no database call. The content type comes from the file's first bytes,
+which are those of the format pixa wrote it in; a file whose first bytes are not
+those of a format pixa writes is not served, and the image is converted again. A
+hit hands the time it was served to a background writer and does not wait for
+the writer. Eviction orders by that time. The hit is counted in memory. A miss
+that uses a cached source image also hands that writer the time it used the
+image. Once a second that writer writes the waiting times in one transaction. Up
+to 10,000 access times of variants, and as many of sources, can wait to be
+written. A time handed over beyond that is dropped. Its image then only looks
 less recently used to eviction. `script/bench` measures how many hits pixa
 serves per second (see Load Test).
 
@@ -880,9 +924,25 @@ See `configs/config.example.yml` for all options with defaults.
 
 ### Architecture
 
-- **Dependency injection**: Uber fx. `internal/app` builds every part of pixad
-  from the config in one fx graph. Each setting's default is declared once, in
-  `internal/config`.
+- **Dependency injection**: Uber fx. `internal/app` puts every part of pixad in
+  one fx graph. `internal/providers` builds the image cache, the image proxy and
+  the other parts that serve images and logins. Each setting's default is
+  declared once, in `internal/config`.
+- **Packages**: these packages under `internal/` work with the concepts above:
+    - `handlers`: the routes, which read the image request from the URL, check
+      the signed or encrypted URL, and make the login session
+    - `imagerequest`: the image request, with the source key and the variant key
+    - `imageproxy`: the image proxy
+    - `imgcache`: the image cache, which keeps its sources in one `FileStore`
+      and its variants in another
+    - `httpfetcher`: fetching sources from upstream hosts
+    - `magic`: reading an image's format from its first bytes
+    - `imageprocessor`: making a variant from a source with libvips
+    - `signature`: checking the `sig` of a signed URL
+    - `encurl`: making and reading the tokens of encrypted URLs
+    - `seal`: the keys derived from `signing_key`, and the encryption of those
+      tokens
+    - `session`: the login session
 - **HTTP router**: go-chi
 - **Image processing**: govips (CGO wrapper for libvips)
 - **Database**: SQLite via modernc.org/sqlite

TODO.md

@@ -30,6 +30,13 @@ P2: security: per-origin rate limiting
 
 # Completed Steps
 
+- 2026-10-26 `README.md`'s Design section starts with a Concepts list (closes
+  #321). Each concept gets one sentence, under the name the code gives it.
+    - Storage calls the transformed images variants, in shorter sentences.
+    - Storage's paths end in the source key and the variant key.
+    - Storage lists the image cache's tables in the database.
+    - Architecture names `internal/providers`, and lists the packages that work
+      with the concepts.
 - 2026-10-26 one table in `internal/config` lists every setting once (closes
   #320). `Config.settings` gives each setting's key, its environment variable
   and the function that loads it.