Make the README's storage paths, formats, TLS and metrics match the code (closes #74) #172
@@ -71,13 +71,17 @@ prevent abuse, and allowlisted source hosts for open access.
|
|||||||
### Storage
|
### Storage
|
||||||
|
|
||||||
- **Source content**:
|
- **Source content**:
|
||||||
`<statedir>/cache/src-content/<ab>/<cd>/<sha256 of source content>`
|
`<state_dir>/cache/sources/<ab>/<cd>/<sha256 of source content>`
|
||||||
- **Source metadata**:
|
- **Source metadata**:
|
||||||
`<statedir>/cache/src-metadata/<hostname>/<sha256 of path>.json`
|
`<state_dir>/cache/metadata/<hostname>/<sha256 of path and query>.json`
|
||||||
(fetch time, original headers, request, content hash)
|
(host, path and query, content hash, upstream status and headers, fetch time)
|
||||||
- **Database**: `<statedir>/state.sqlite3` (SQLite)
|
- **Database**: `<state_dir>/state.sqlite3` (SQLite)
|
||||||
- **Output documents**:
|
- **Transformed images**:
|
||||||
`<statedir>/cache/dst-content/<ab>/<cd>/<sha256 of output content>`
|
`<state_dir>/cache/variants/<ab>/<cd>/<sha256 of host, path, query, size, format, quality and fit>`,
|
||||||
|
each with a `.meta` file beside it holding its content type
|
||||||
|
|
||||||
|
`<ab>` and `<cd>` are the first and second pairs of characters of the file's
|
||||||
|
name.
|
||||||
|
|
||||||
Multiple source paths may reference the same content blob; the
|
Multiple source paths may reference the same content blob; the
|
||||||
database tracks references rather than using filesystem refcounting.
|
database tracks references rather than using filesystem refcounting.
|
||||||
@@ -92,12 +96,15 @@ the metadata file stored beside it.
|
|||||||
/v1/image/<host>/<path>/<size>.<format>?sig=<signature>&exp=<expiration>
|
/v1/image/<host>/<path>/<size>.<format>?sig=<signature>&exp=<expiration>
|
||||||
```
|
```
|
||||||
|
|
||||||
Images are only fetched from origins using TLS with valid certificates.
|
Images are only fetched from origins using TLS with valid certificates, unless
|
||||||
|
`allow_http` is set: then pixa fetches every image over plain HTTP, which is for
|
||||||
|
testing only.
|
||||||
|
|
||||||
A request whose query string cannot be decoded, or gives any parameter more
|
A request whose query string cannot be decoded, or gives any parameter more
|
||||||
than once, is refused with 400.
|
than once, is refused with 400.
|
||||||
|
|
||||||
- `<format>`: one of `orig`, `png`, `jpeg`, `webp`
|
- `<format>`: one of `orig` (or `original`), `jpeg` (or `jpg`), `png`, `webp`,
|
||||||
|
`avif`, `gif`
|
||||||
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
|
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
|
||||||
|
|
||||||
An image is served with `Cache-Control: public, max-age=<seconds>, immutable`.
|
An image is served with `Cache-Control: public, max-age=<seconds>, immutable`.
|
||||||
@@ -173,7 +180,8 @@ Where:
|
|||||||
- `query` — source query string, empty string if none
|
- `query` — source query string, empty string if none
|
||||||
- `width` — requested width in pixels, `0` for original
|
- `width` — requested width in pixels, `0` for original
|
||||||
- `height` — requested height in pixels, `0` for original
|
- `height` — requested height in pixels, `0` for original
|
||||||
- `format` — output format (jpeg, png, webp, avif, gif, orig)
|
- `format` — output format, one of those listed under Routes, with `original`
|
||||||
|
signed as `orig` and `jpg` as `jpeg`
|
||||||
- `expiration` — the URL's `exp` query parameter, the Unix timestamp when
|
- `expiration` — the URL's `exp` query parameter, the Unix timestamp when
|
||||||
the signature expires; a request whose `exp` is not a whole number, an
|
the signature expires; a request whose `exp` is not a whole number, an
|
||||||
empty `exp=` included, is refused with 400
|
empty `exp=` included, is refused with 400
|
||||||
@@ -330,7 +338,10 @@ See `config.example.yml` for all options with defaults.
|
|||||||
- **Image processing**: govips (CGO wrapper for libvips)
|
- **Image processing**: govips (CGO wrapper for libvips)
|
||||||
- **Database**: SQLite via modernc.org/sqlite
|
- **Database**: SQLite via modernc.org/sqlite
|
||||||
- **Static assets**: embedded via `//go:embed`
|
- **Static assets**: embedded via `//go:embed`
|
||||||
- **Metrics**: Prometheus
|
- **Metrics**: Prometheus, at `/metrics`: generic HTTP request metrics
|
||||||
|
(duration, response size, requests in flight) and the Go runtime and process
|
||||||
|
metrics; requests are measured and `/metrics` is served only when
|
||||||
|
`metrics.username` and `metrics.password` are set
|
||||||
- **Logging**: stdlib slog
|
- **Logging**: stdlib slog
|
||||||
|
|
||||||
## Entrypoints
|
## Entrypoints
|
||||||
|
|||||||
@@ -29,6 +29,14 @@ P2: security: referer blacklist
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-04 `README.md` matches the code (closes #74): "Storage" names the
|
||||||
|
cache directories pixa uses (`cache/sources`, `cache/metadata`,
|
||||||
|
`cache/variants`) and how files are named in each, and the comments in
|
||||||
|
`001_schema.sql` name the same paths; the routes and the signature section
|
||||||
|
list the same output formats, `jpg` and `original` included; the TLS
|
||||||
|
sentence names `allow_http` as its exception; "Metrics" says only generic
|
||||||
|
HTTP and Go runtime metrics exist, measured and served only when the metrics
|
||||||
|
username and password are set.
|
||||||
- 2026-10-03 shutdown sets the exit code and waits for image processing
|
- 2026-10-03 shutdown sets the exit code and waits for image processing
|
||||||
(closes #86): fx alone handles SIGINT and SIGTERM, and the server's own
|
(closes #86): fx alone handles SIGINT and SIGTERM, and the server's own
|
||||||
signal handler is gone; fx's `Run` in `cmd/pixad` exits with the shutdown's
|
signal handler is gone; fx's `Run` in `cmd/pixad` exits with the shutdown's
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
-- Creates all tables for the pixa caching image proxy
|
-- Creates all tables for the pixa caching image proxy
|
||||||
|
|
||||||
-- Source content blobs
|
-- Source content blobs
|
||||||
-- Files stored at: cache/src-content/<ab>/<cd>/<sha256>
|
-- Files stored at: cache/sources/<ab>/<cd>/<sha256>
|
||||||
-- last_accessed_at is NULL until the first LRU touch; eviction falls
|
-- last_accessed_at is NULL until the first LRU touch; eviction falls
|
||||||
-- back to fetched_at for rows that have never been touched.
|
-- back to fetched_at for rows that have never been touched.
|
||||||
CREATE TABLE IF NOT EXISTS source_content (
|
CREATE TABLE IF NOT EXISTS source_content (
|
||||||
@@ -16,7 +16,7 @@ CREATE INDEX IF NOT EXISTS idx_source_content_last_accessed
|
|||||||
ON source_content(last_accessed_at);
|
ON source_content(last_accessed_at);
|
||||||
|
|
||||||
-- Source URL metadata - maps URLs to content hashes
|
-- Source URL metadata - maps URLs to content hashes
|
||||||
-- JSON stored at: cache/src-metadata/<hostname>/<path_hash>.json
|
-- JSON stored at: cache/metadata/<hostname>/<path_hash>.json
|
||||||
CREATE TABLE IF NOT EXISTS source_metadata (
|
CREATE TABLE IF NOT EXISTS source_metadata (
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
source_host TEXT NOT NULL,
|
source_host TEXT NOT NULL,
|
||||||
@@ -56,7 +56,8 @@ CREATE INDEX IF NOT EXISTS idx_variant_content_last_accessed
|
|||||||
ON variant_content(last_accessed_at);
|
ON variant_content(last_accessed_at);
|
||||||
|
|
||||||
-- Output/transformed content blobs
|
-- Output/transformed content blobs
|
||||||
-- Files stored at: cache/dst-content/<ab>/<cd>/<sha256>
|
-- Not written: transformed images are stored in cache/variants and
|
||||||
|
-- tracked in variant_content above.
|
||||||
CREATE TABLE IF NOT EXISTS output_content (
|
CREATE TABLE IF NOT EXISTS output_content (
|
||||||
content_hash TEXT PRIMARY KEY,
|
content_hash TEXT PRIMARY KEY,
|
||||||
content_type TEXT NOT NULL,
|
content_type TEXT NOT NULL,
|
||||||
|
|||||||
Reference in New Issue
Block a user