Adds httptest-server tests for the risk-bearing paths that only had
helper-level coverage: the CheckRedirect validator (a 302 to a
link-local address is refused with ErrSSRFBlocked and never dialed,
while a redirect chain and a redirect to a public target still
succeed), per-host semaphore release on the error, full-read-close and
partial-read-close paths (proven by saturating a one-slot host), the
MaxResponseSize limit through Fetch, non-2xx and disallowed
content-type rejection, and ssrfSafeDialer's block of a private target.
The upstream host is a TEST-NET-1 literal (192.0.2.10) that isPrivateIP
treats as public and that resolves with no DNS, with a recording dialer
routing it to the local server so the real SSRF checks still run. Full
DNS-rebinding simulation is out of scope; the dial-time private-target
block that closes that window is tested directly.
Model: opus-4-8
Accumulating milestone branch. One squashed commit per closed issue; `next` is kept green and mergeable to `main` at any time without notice.
Landed so far:
- `chore: update golangci-lint to v2.12.2 with canonical config` (#54) — canonical v2-schema `.golangci.yml`, pins bumped in `Dockerfile` and `script/bootstrap`, tree at `0 issues.`. Three behaviour deltas are recorded in that PR's body: `Cache.StoreVariant` takes a context, `MetadataStorage.Store` no longer leaks temp files on failure, and the `signing_key` too-short error text gained a `value too short:` prefix.
Sequencing for the milestone is tracked in #103.
Reviewed-on: #105
Co-authored-by: clawbot <clawbot@noreply.example.org>
Extracts the HTTP fetcher concern out of `internal/imgcache/` into its own `internal/httpfetcher/` package, per the plan in [issue #39](#39).
This is one of four planned extractions; only the fetcher is moved here so the diff stays reviewable. The remaining three (signature, magic, urlparser) will land in separate PRs.
## What moved
From `internal/imgcache/fetcher.go` and `internal/imgcache/mock_fetcher.go` into `internal/httpfetcher/`:
- `HTTPFetcher` type, its `New` constructor, and `Config` (formerly `FetcherConfig`) with `DefaultConfig`
- SSRF-safe dialer, `validateURL`, `isPrivateIP`, `isLocalhost`, `extractHost`
- Per-host connection semaphore (rate limiting) and `limitedReader`
- Content-type validation (`isAllowedContentType`, `detectContentTypeFromPath`)
- All related error values (`ErrSSRFBlocked`, `ErrUpstreamError`, `ErrUpstreamTimeout`, `ErrPayloadTooLarge`, `ErrDisallowedContentType`)
- All related constants (`DefaultFetchTimeout`, `DefaultMaxPayloadBytes`, `DefaultMaxConnectionsPerHost`, etc.)
- The `Fetcher` interface and `FetchResult` type (moved here to keep the import edge one-way: `imgcache` depends on `httpfetcher`, never the reverse)
- `MockFetcher` test helper
## Renames (no stuttering)
- `NewHTTPFetcher` → `httpfetcher.New`
- `FetcherConfig` → `httpfetcher.Config`
- `DefaultFetcherConfig` → `httpfetcher.DefaultConfig`
- `NewMockFetcher` → `httpfetcher.NewMock`
The `ServiceConfig.FetcherConfig` field name is retained — it describes what kind of config the field holds (not a stutter).
## Behavior
Pure refactor. No behavior changes. All existing tests pass; unit tests for the new package are included.
`docker build .` passes (fmt-check, lint, test, build).
refs #39
Co-authored-by: clawbot <clawbot@eeqj.de>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #43
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
sneak
clawbot
