Stop a transcode whose request has ended before it encodes (closes #243) #271
@@ -327,16 +327,17 @@ the first request's) stops waiting for a processing slot, or, once it has one,
|
|||||||
gives it back without beginning to encode. A request waiting for a shared
|
gives it back without beginning to encode. A request waiting for a shared
|
||||||
transcode that stops this way is answered 503 with the error
|
transcode that stops this way is answered 503 with the error
|
||||||
`server busy, try again later`, though its own `downstream_timeout` may not have
|
`server busy, try again later`, though its own `downstream_timeout` may not have
|
||||||
ended. libvips cannot stop an encode once it has begun, so an encode keeps its
|
ended. govips gives pixa no way to stop libvips once an encode has begun, so an
|
||||||
slot until it finishes, which can be after `downstream_timeout`. The limit of
|
encode keeps its slot until it finishes, which can be after
|
||||||
8192 on each side of the source and of the output bounds how long that takes.
|
`downstream_timeout`. The limit of 8192 on each side of the source and of the
|
||||||
With one libvips thread, as pixad runs it, encoding 8192x8192 random pixels
|
output bounds how long that takes. With one libvips thread, as pixad runs it,
|
||||||
takes up to about 1 second as JPEG, 15 as PNG, 16 as GIF, 63 as WebP, 110 as
|
encoding 8192x8192 random pixels takes up to about 1 second as JPEG, 15 as PNG,
|
||||||
AVIF and 96 as JPEG XL, the slowest of 8 or 16 bits per sample, with or without
|
16 as GIF, 63 as WebP, 110 as AVIF and 96 as JPEG XL, the slowest of 8 or 16
|
||||||
an alpha channel. WebP, AVIF and JPEG XL are that slow only with an alpha
|
bits per sample, with or without an alpha channel. WebP, AVIF and JPEG XL are
|
||||||
channel: 8-bit RGB takes about 44, 53 and 28 seconds. Decoding such a source
|
that slow only with an alpha channel: 8-bit RGB takes about 44, 53 and 28
|
||||||
adds up to about 11 seconds. A WebP, AVIF or JPEG XL encode can therefore hold
|
seconds. Decoding such a source adds up to about 11 seconds. A WebP, AVIF or
|
||||||
its slot past the default `downstream_timeout` of 60 seconds.
|
JPEG XL encode can therefore hold its slot past the default `downstream_timeout`
|
||||||
|
of 60 seconds.
|
||||||
|
|
||||||
The login form (`POST /`) is limited to 5 attempts per minute per client
|
The login form (`POST /`) is limited to 5 attempts per minute per client
|
||||||
address, counting an IPv6 client by its /64; an attempt over the limit is
|
address, counting an IPv6 client by its /64; an attempt over the limit is
|
||||||
|
|||||||
@@ -65,16 +65,16 @@ P2: security: per-origin rate limiting
|
|||||||
slot back, when the context has ended by the time it has the slot or before it
|
slot back, when the context has ended by the time it has the slot or before it
|
||||||
encodes. A request that waited for a shared transcode stopped at the first
|
encodes. A request that waited for a shared transcode stopped at the first
|
||||||
request's deadline is answered 503 `server busy, try again later`, not 500,
|
request's deadline is answered 503 `server busy, try again later`, not 500,
|
||||||
though its own `downstream_timeout` may not have ended. libvips cannot stop an
|
though its own `downstream_timeout` may not have ended. govips gives pixa no
|
||||||
encode once it has begun, so that encode keeps its slot until it finishes; the
|
way to stop libvips once an encode has begun, so that encode keeps its slot
|
||||||
limit of 8192 on each side of the source and of the output bounds how long
|
until it finishes; the limit of 8192 on each side of the source and of the
|
||||||
that takes. With one libvips thread, 8192x8192 random pixels take up to about
|
output bounds how long that takes. With one libvips thread, 8192x8192 random
|
||||||
1 second to encode as JPEG, 15 as PNG, 16 as GIF, 63 as WebP, 110 as AVIF and
|
pixels take up to about 1 second to encode as JPEG, 15 as PNG, 16 as GIF, 63
|
||||||
96 as JPEG XL, the last three with an alpha channel, plus up to about 11
|
as WebP, 110 as AVIF and 96 as JPEG XL, the last three with an alpha channel,
|
||||||
seconds to decode the source; `README.md` gives these times and no longer says
|
plus up to about 11 seconds to decode the source; `README.md` gives these
|
||||||
the work stops when `downstream_timeout` ends. The settings of the WebP, AVIF
|
times and no longer says the work stops when `downstream_timeout` ends. The
|
||||||
and JPEG XL encoders, which can hold a slot past the default
|
settings of the WebP, AVIF and JPEG XL encoders, which can hold a slot past
|
||||||
`downstream_timeout` of 60 seconds, are unchanged.
|
the default `downstream_timeout` of 60 seconds, are unchanged.
|
||||||
- 2026-10-15 a cached variant is served while its source URL is in the negative
|
- 2026-10-15 a cached variant is served while its source URL is in the negative
|
||||||
cache (closes #245): `Get` checked the negative cache before the variant
|
cache (closes #245): `Get` checked the negative cache before the variant
|
||||||
cache, so once a fetch of the source failed, as one does when another variant
|
cache, so once a fetch of the source failed, as one does when another variant
|
||||||
|
|||||||
@@ -209,8 +209,9 @@ func New(params Params) *ImageProcessor {
|
|||||||
// MaxConcurrentProcessing images are already being processed, it waits up
|
// MaxConcurrentProcessing images are already being processed, it waits up
|
||||||
// to ProcessingWaitTimeout for one to finish, then fails with
|
// to ProcessingWaitTimeout for one to finish, then fails with
|
||||||
// ErrTooManyImages. When ctx has ended once Process has a slot, or before it
|
// ErrTooManyImages. When ctx has ended once Process has a slot, or before it
|
||||||
// encodes, it returns ctx's error and gives the slot back. libvips cannot stop
|
// encodes, it returns ctx's error and gives the slot back. govips gives pixa
|
||||||
// an encode once it has begun, so an encode keeps its slot until it finishes.
|
// no way to stop libvips once an encode has begun, so an encode keeps its
|
||||||
|
// slot until it finishes.
|
||||||
func (p *ImageProcessor) Process(
|
func (p *ImageProcessor) Process(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
input io.Reader,
|
input io.Reader,
|
||||||
@@ -532,9 +533,9 @@ func (p *ImageProcessor) resize(
|
|||||||
|
|
||||||
const defaultQuality = 85
|
const defaultQuality = 85
|
||||||
|
|
||||||
// encode encodes an image to the specified format. libvips cannot stop an
|
// encode encodes an image to the specified format. govips gives pixa no way
|
||||||
// encode once it has begun, so when ctx has ended encode returns ctx's error
|
// to stop libvips once an encode has begun, so when ctx has ended encode
|
||||||
// without beginning one.
|
// returns ctx's error without beginning one.
|
||||||
func (p *ImageProcessor) encode(
|
func (p *ImageProcessor) encode(
|
||||||
ctx context.Context, img *vips.ImageRef, format Format, quality int,
|
ctx context.Context, img *vips.ImageRef, format Format, quality int,
|
||||||
) ([]byte, error) {
|
) ([]byte, error) {
|
||||||
@@ -644,10 +645,11 @@ func exportWebP(img *vips.ImageRef, quality int) ([]byte, error) {
|
|||||||
|
|
||||||
// avifEffort is the AVIF effort, from 0 (fastest) to 9; 1 is the lowest
|
// avifEffort is the AVIF effort, from 0 (fastest) to 9; 1 is the lowest
|
||||||
// govips can set. With one thread, as pixad runs libvips, 1 takes about 51
|
// govips can set. With one thread, as pixad runs libvips, 1 takes about 51
|
||||||
// seconds to save an 8192x8192 image of random pixels, the worst case,
|
// seconds to save an 8192x8192 RGB image of random pixels, and about 110
|
||||||
// against the default downstream_timeout of 60 seconds. On an image of
|
// with an alpha channel, the worst case, against the default
|
||||||
// milder noise, which 1 saves in about 12 seconds, 2 takes nearly a minute
|
// downstream_timeout of 60 seconds. On an image of milder noise, which 1
|
||||||
// and libvips' default, 4, takes minutes.
|
// saves in about 12 seconds, 2 takes nearly a minute and libvips' default,
|
||||||
|
// 4, takes minutes.
|
||||||
const avifEffort = 1
|
const avifEffort = 1
|
||||||
|
|
||||||
// avifBitdepth is the AVIF bit depth, 8 bits per sample for every image.
|
// avifBitdepth is the AVIF bit depth, 8 bits per sample for every image.
|
||||||
|
|||||||
Reference in New Issue
Block a user