HTTP API: register, list and remove webhooks, kept across restarts (closes #6)
check / check (push) Successful in 1m19s

An external app can register webhooks on a chat (`POST /api/v1/chats/{id}/webhooks` with a URL), list them, and remove one. Registering the same URL again returns the existing registration. Registrations are kept in `$DATA_DIR/webhooks.json`, rewritten whole on each change through a file created 0600, synced and renamed, so a crash leaves the old file or the new one; a file present but unreadable stops startup. Delivery of incoming messages is the next unit.

Disclosures: the JSON body reading is now one helper shared with the send endpoint; gosec G304 is suppressed on reading the webhooks file; the 0600-from-creation claim rests on `os.CreateTemp`'s source, not a system-call trace.

Model: opus-5-5
This commit was merged in pull request #17.
This commit is contained in:
2026-09-29 08:38:13 +02:00
parent f53b666119
commit 397fc95149
11 changed files with 1018 additions and 47 deletions
+3 -3
View File
@@ -109,9 +109,9 @@ WORKDIR /app
COPY --from=builder /build/bin/simplexcalc /app/simplexcalc COPY --from=builder /build/bin/simplexcalc /app/simplexcalc
# Data directory: the SimpleX database, which holds the bot's profile, # Data directory: the SimpleX database, which holds the bot's profile,
# its keys, its address and its contacts. Mount a volume over it; # its keys, its address and its contacts, and the API's webhooks. Mount
# without one, the bot gets a new address every time the container is # a volume over it; without one, the bot gets a new address and loses
# recreated. # its webhooks every time the container is recreated.
RUN mkdir -p /var/lib/simplexcalc && \ RUN mkdir -p /var/lib/simplexcalc && \
chown simplexcalc:simplexcalc /var/lib/simplexcalc chown simplexcalc:simplexcalc /var/lib/simplexcalc
+101 -9
View File
@@ -73,8 +73,10 @@ development.
is never quietly replaced by the default. Defaults apply only to is never quietly replaced by the default. Defaults apply only to
variables that are absent. variables that are absent.
- `DATA_DIR` — where the SimpleX database lives: the bot's profile, its - `DATA_DIR` — where the bot keeps what it must not lose: the SimpleX
keys, its address and its contacts. Default `./data`; the image sets database, which holds the bot's profile, its keys, its address and its
contacts, and `webhooks.json`, which holds the webhooks registered
through the [API](#api). Default `./data`; the image sets
`/var/lib/simplexcalc`. `/var/lib/simplexcalc`.
- `DEBUG` — `true` or `false`, default `false`. `true` logs every event - `DEBUG` — `true` or `false`, default `false`. `true` logs every event
the chat client sends. the chat client sends.
@@ -89,7 +91,8 @@ variables that are absent.
## API ## API
An HTTP API beside the chat client lets another program read the bot's An HTTP API beside the chat client lets another program read the bot's
chats and send messages in them. It speaks JSON on `PORT`. chats, send messages in them and register webhooks on them. It speaks
JSON on `PORT`.
**Authentication.** Every request carries the credential from **Authentication.** Every request carries the credential from
`API_TOKEN_FILE`: `API_TOKEN_FILE`:
@@ -218,6 +221,88 @@ Nothing is sent, and the answer is:
- `413` if the body is over 64 KiB, or the text is too long for one - `413` if the body is over 64 KiB, or the text is too long for one
SimpleX message, which holds about 15,000 bytes. SimpleX message, which holds about 15,000 bytes.
### `POST /api/v1/chats/{id}/webhooks`
Registers the `url` in the body as a webhook on the chat `id`, and
answers `201` with the webhook. If the chat already has a webhook with
the same `url`, character for character, the answer is `200` with that
webhook instead.
```sh
curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d '{"url":"https://example.com/hook"}' \
http://127.0.0.1:8080/api/v1/chats/3/webhooks
```
```json
{
"id": "5f0c7a1e9b2d4c8e3a6f1b7d2e9c4a80",
"chat_id": 3,
"url": "https://example.com/hook"
}
```
A webhook has:
- `id` — 16 random bytes, written as 32 hexadecimal digits.
- `chat_id` — the chat it is registered on.
- `url` — the URL, as it was registered.
The bot keeps the webhooks in `webhooks.json` in `DATA_DIR`, with mode
0600, and reads that file at startup, so they survive a restart. Without
the file there are none; a file that cannot be read, or that holds
anything but webhooks as the bot writes them, aborts startup, as
configuration that cannot be parsed does. The bot does not post anything
to a webhook yet.
Nothing is registered, and the answer is:
- `400` if the body is not JSON of that shape, or `url` is not an
absolute `http` or `https` URL with a host, at most 2048 bytes long;
- `404` if `GET /api/v1/chats` does not list `id`;
- `413` if the body is over 64 KiB;
- `500` if `webhooks.json` cannot be written.
### `GET /api/v1/chats/{id}/webhooks`
The webhooks registered on the chat `id`, in the order they were
registered; `404` if `GET /api/v1/chats` does not list `id`. A chat's
webhooks stay after its contact deletes the chat (`contact_deleted` is
`true`), until they are removed.
```sh
curl -H "Authorization: Bearer $TOKEN" \
http://127.0.0.1:8080/api/v1/chats/3/webhooks
```
```json
{
"webhooks": [
{
"id": "5f0c7a1e9b2d4c8e3a6f1b7d2e9c4a80",
"chat_id": 3,
"url": "https://example.com/hook"
}
]
}
```
### `DELETE /api/v1/chats/{id}/webhooks/{webhook_id}`
Removes the webhook `webhook_id` from the chat `id`, and answers `204`
with no body.
```sh
curl -X DELETE -H "Authorization: Bearer $TOKEN" \
http://127.0.0.1:8080/api/v1/chats/3/webhooks/5f0c7a1e9b2d4c8e3a6f1b7d2e9c4a80
```
Nothing is removed, and the answer is:
- `404` if `GET /api/v1/chats` does not list `id`, or the chat has no
webhook `webhook_id`;
- `500` if `webhooks.json` cannot be written.
## Entrypoints ## Entrypoints
This repo adheres to the This repo adheres to the
@@ -315,6 +400,12 @@ container.
the one that delivers events, which also delivers the chat client's the one that delivers events, which also delivers the chat client's
answers. When the bot stops, requests in progress get 5 seconds to answers. When the bot stops, requests in progress get 5 seconds to
finish before the chat client is stopped. finish before the chat client is stopped.
- **Webhooks** (`internal/api`) are read from `$DATA_DIR/webhooks.json`
before the chat client starts. Changes are made one at a time, and
each rewrites the file whole: the bot writes a temporary file beside
it, named `webhooks.json.` and digits, and renames it over the old
one. A crash leaves the old file or the new one, never part of one,
and at worst a stray temporary file, which the bot ignores.
- **Replies**: for each text message a contact sends in a direct chat, - **Replies**: for each text message a contact sends in a direct chat,
the bot sends back the result, as a reply quoting the message. Group the bot sends back the result, as a reply quoting the message. Group
messages, files and the bot's own messages are ignored. messages, files and the bot's own messages are ignored.
@@ -359,12 +450,13 @@ container.
## Operating it ## Operating it
**Backup.** Everything durable is on the volume: the SimpleX database, **Backup.** Everything durable is on the volume: the SimpleX database,
`simplex_chat.db` and `simplex_agent.db`, and the API credential, `simplex_chat.db` and `simplex_agent.db`; the API credential,
`api-token`. Stop the container before copying the database, since a `api-token`; and the webhooks, `webhooks.json`. Stop the container
copy taken from under the running client can be inconsistent. The before copying the database, since a copy taken from under the running
database holds the bot's keys, so a copy lets its holder answer as the client can be inconsistent. The database holds the bot's keys, so a copy
bot, and the credential lets its holder use the API; keep both as lets its holder answer as the bot; the credential lets its holder use
private as the running instance. the API; and a webhook's URL can hold a secret of the program it points
at. Keep all three as private as the running instance.
**Upgrade.** Rebuild the image and recreate the container with the same **Upgrade.** Rebuild the image and recreate the container with the same
volume. The chat client migrates its database on start. A newer volume. The chat client migrates its database on start. A newer
+2
View File
@@ -27,6 +27,8 @@ with no deprecation warning.
# Completed Steps # Completed Steps
- 2026-09-29 `POST`, `GET` and `DELETE` on a chat's webhooks, under
`/api/v1/chats/{id}/webhooks`, kept in `$DATA_DIR/webhooks.json`
- 2026-09-29 `GET` and `POST /api/v1/chats/{id}/messages`: a chat's - 2026-09-29 `GET` and `POST /api/v1/chats/{id}/messages`: a chat's
latest messages, and sending one latest messages, and sending one
- 2026-09-29 Added the HTTP API in `internal/api`: the server on `PORT`, - 2026-09-29 Added the HTTP API in `internal/api`: the server on `PORT`,
+43 -2
View File
@@ -1,5 +1,6 @@
// Package api is the bot's HTTP API, through which another program // Package api is the bot's HTTP API, through which another program
// reads the bot's chats and sends messages in them. // reads the bot's chats, sends messages in them and registers webhooks
// on them.
// //
// Every request must carry the credential, as "Authorization: Bearer // Every request must carry the credential, as "Authorization: Bearer
// {credential}"; with no credential configured, every request is // {credential}"; with no credential configured, every request is
@@ -14,6 +15,8 @@ import (
"context" "context"
"crypto/subtle" "crypto/subtle"
"encoding/json" "encoding/json"
"errors"
"io"
"log/slog" "log/slog"
"net/http" "net/http"
"strconv" "strconv"
@@ -75,6 +78,10 @@ type Params struct {
// Token is the credential every request must carry. Empty refuses // Token is the credential every request must carry. Empty refuses
// every request. // every request.
Token string Token string
// Webhooks holds the webhooks registered on the chats; ReadWebhooks
// makes it.
Webhooks *Webhooks
} }
// New returns the API's server. The caller starts it with // New returns the API's server. The caller starts it with
@@ -84,7 +91,13 @@ func New(p Params) *http.Server {
p.Log.Warn("API_TOKEN_FILE is not set, so the API refuses every request") p.Log.Warn("API_TOKEN_FILE is not set, so the API refuses every request")
} }
h := &handlers{log: p.Log, client: p.Client, userID: p.UserID, token: p.Token} h := &handlers{
log: p.Log,
client: p.Client,
userID: p.UserID,
token: p.Token,
webhooks: p.Webhooks,
}
router := chi.NewRouter() router := chi.NewRouter()
router.Use(securityHeaders, h.authenticate, router.Use(securityHeaders, h.authenticate,
@@ -99,6 +112,9 @@ func New(p Params) *http.Server {
r.Get("/chats", h.handleChats()) r.Get("/chats", h.handleChats())
r.Get("/chats/{id}/messages", h.handleMessages()) r.Get("/chats/{id}/messages", h.handleMessages())
r.Post("/chats/{id}/messages", h.handleSend()) r.Post("/chats/{id}/messages", h.handleSend())
r.Get("/chats/{id}/webhooks", h.handleWebhooks())
r.Post("/chats/{id}/webhooks", h.handleRegister())
r.Delete("/chats/{id}/webhooks/{webhook_id}", h.handleRemove())
}) })
return &http.Server{ return &http.Server{
@@ -123,6 +139,7 @@ type handlers struct {
client ChatClient client ChatClient
userID int64 userID int64
token string token string
webhooks *Webhooks
} }
// authenticate lets a request through only if it carries the // authenticate lets a request through only if it carries the
@@ -165,6 +182,30 @@ func (h *handlers) respondError(w http.ResponseWriter, status int, sentence stri
}{sentence}) }{sentence})
} }
// decodeBody decodes the request's JSON body into v. If the body is too
// large, or is not JSON of v's shape, it answers the request itself, 413
// or 400 naming example as the shape wanted, and returns false.
func (h *handlers) decodeBody(
w http.ResponseWriter, r *http.Request, v any, example string,
) bool {
body, err := io.ReadAll(r.Body)
var tooLarge *http.MaxBytesError
if errors.As(err, &tooLarge) {
h.respondError(w, http.StatusRequestEntityTooLarge, "the body is too large")
return false
}
if err != nil || json.Unmarshal(body, v) != nil {
h.respondError(w, http.StatusBadRequest, "the body must be JSON such as "+example)
return false
}
return true
}
// securityHeaders go on every response. The API returns JSON to // securityHeaders go on every response. The API returns JSON to
// programs, so a browser may not frame, sniff, cache or refer from it, // programs, so a browser may not frame, sniff, cache or refer from it,
// nor give it the camera, microphone or location, and must reach it // nor give it the camera, microphone or location, and must reach it
+7
View File
@@ -23,6 +23,7 @@ const (
chatsPath = "/api/v1/chats" chatsPath = "/api/v1/chats"
messagesPath = "/api/v1/chats/3/messages" messagesPath = "/api/v1/chats/3/messages"
webhooksPath = "/api/v1/chats/3/webhooks"
unauthorized = `{"error":"unauthorized"}` + "\n" unauthorized = `{"error":"unauthorized"}` + "\n"
) )
@@ -197,6 +198,12 @@ func TestNoPathIsExempt(t *testing.T) {
http.MethodPost, messagesPath, "", http.MethodPost, messagesPath, "",
http.StatusUnauthorized, unauthorized, http.StatusUnauthorized, unauthorized,
}, },
{http.MethodGet, webhooksPath, "", http.StatusUnauthorized, unauthorized},
{http.MethodPost, webhooksPath, "", http.StatusUnauthorized, unauthorized},
{
http.MethodDelete, webhooksPath + "/" + unknownID, "",
http.StatusUnauthorized, unauthorized,
},
} { } {
rec := request(t, srv, tc.method, tc.path, tc.auth) rec := request(t, srv, tc.method, tc.path, tc.auth)
if rec.Code != tc.want || rec.Body.String() != tc.body { if rec.Code != tc.want || rec.Body.String() != tc.body {
+1 -16
View File
@@ -1,9 +1,7 @@
package api package api
import ( import (
"encoding/json"
"errors" "errors"
"io"
"net/http" "net/http"
"net/url" "net/url"
"strconv" "strconv"
@@ -124,21 +122,8 @@ func (h *handlers) handleSend() http.HandlerFunc {
return return
} }
body, err := io.ReadAll(r.Body)
var tooLarge *http.MaxBytesError
if errors.As(err, &tooLarge) {
h.respondError(w, http.StatusRequestEntityTooLarge, "the body is too large")
return
}
var req request var req request
if !h.decodeBody(w, r, &req, `{"text":"hello"}`) {
if err != nil || json.Unmarshal(body, &req) != nil {
h.respondError(w, http.StatusBadRequest,
`the body must be JSON such as {"text":"hello"}`)
return return
} }
+306
View File
@@ -0,0 +1,306 @@
package api
import (
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io/fs"
"net/http"
"net/url"
"os"
"path/filepath"
"slices"
"strconv"
"sync"
"github.com/go-chi/chi/v5"
)
const (
// webhooksFile, in the data directory, keeps the webhooks.
webhooksFile = "webhooks.json"
// idBytes is how many random bytes make a webhook's id.
idBytes = 16
// maxURLBytes caps a webhook's URL.
maxURLBytes = 2048
)
// errWebhooksFile is the error for a webhooks file that does not hold
// webhooks as the bot writes them.
var errWebhooksFile = errors.New("not a list of webhooks as the bot writes it")
// webhook is a URL registered on a chat.
type webhook struct {
ID string `json:"id"`
ChatID int64 `json:"chat_id"`
URL string `json:"url"`
}
// valid reports whether w is a webhook that registering could make.
func (w webhook) valid() bool {
id, err := hex.DecodeString(w.ID)
return err == nil && len(id) == idBytes && w.ChatID > 0 && validURL(w.URL)
}
// webhookList is how a list of webhooks is written: in the answer to GET,
// and in the webhooks file.
type webhookList struct {
Webhooks []webhook `json:"webhooks"`
}
// Webhooks holds the webhooks registered on every chat, and keeps them in
// the webhooks file. It is safe for concurrent use.
type Webhooks struct {
path string
// mu is held across each change, the write of the file included, so
// the changes reach the file one at a time and in order.
mu sync.Mutex
// all is in the order the webhooks were registered. It is never nil,
// which would be written as null, a file ReadWebhooks refuses.
all []webhook
}
// ReadWebhooks returns the webhooks kept in dir, and none if dir has no
// webhooks file. A file that cannot be read, or that holds anything but
// webhooks as the bot writes them, is an error, so that the bot neither
// starts without them nor later writes over them.
func ReadWebhooks(dir string) (*Webhooks, error) {
path := filepath.Join(dir, webhooksFile)
b, err := os.ReadFile(path) //nolint:gosec // G304: the data directory's own file.
if errors.Is(err, fs.ErrNotExist) {
return &Webhooks{path: path, all: []webhook{}}, nil
}
if err != nil {
return nil, fmt.Errorf("reading %s: %w", path, err)
}
var file webhookList
err = json.Unmarshal(b, &file)
if err != nil {
return nil, fmt.Errorf("reading %s: %w", path, err)
}
// The list is nil only if the file has none, as in {} or null; [] is
// an empty list.
if file.Webhooks == nil {
return nil, fmt.Errorf("reading %s: %w", path, errWebhooksFile)
}
for _, w := range file.Webhooks {
if !w.valid() {
return nil, fmt.Errorf("reading %s: %w", path, errWebhooksFile)
}
}
return &Webhooks{path: path, all: file.Webhooks}, nil
}
// register registers hookURL on the chat chatID, and returns the new
// webhook and true; or, if that chat has a webhook with that URL already,
// that webhook and false.
func (s *Webhooks) register(chatID int64, hookURL string) (webhook, bool, error) {
s.mu.Lock()
defer s.mu.Unlock()
i := slices.IndexFunc(s.all, func(w webhook) bool {
return w.ChatID == chatID && w.URL == hookURL
})
if i >= 0 {
return s.all[i], false, nil
}
id := make([]byte, idBytes)
// Never fails: crypto/rand ends the program instead.
_, _ = rand.Read(id)
added := webhook{ID: hex.EncodeToString(id), ChatID: chatID, URL: hookURL}
all := append(slices.Clone(s.all), added)
err := s.write(all)
if err != nil {
return webhook{}, false, err
}
s.all = all
return added, true, nil
}
// list returns the webhooks registered on the chat chatID, in the order
// they were registered.
func (s *Webhooks) list(chatID int64) []webhook {
s.mu.Lock()
defer s.mu.Unlock()
hooks := make([]webhook, 0, len(s.all))
for _, w := range s.all {
if w.ChatID == chatID {
hooks = append(hooks, w)
}
}
return hooks
}
// remove removes the webhook id from the chat chatID, and returns false if
// that chat has no such webhook.
func (s *Webhooks) remove(chatID int64, id string) (bool, error) {
s.mu.Lock()
defer s.mu.Unlock()
all := slices.DeleteFunc(slices.Clone(s.all), func(w webhook) bool {
return w.ChatID == chatID && w.ID == id
})
if len(all) == len(s.all) {
return false, nil
}
err := s.write(all)
if err != nil {
return false, err
}
s.all = all
return true, nil
}
// write replaces the webhooks file with one holding all. It writes a
// temporary file in the same directory and renames it over the old one,
// so a crash leaves the old file or the new one, never part of one, and a
// failure leaves the old file as it was. os.CreateTemp creates the
// temporary file with mode 0600, which the file keeps.
func (s *Webhooks) write(all []webhook) error {
b, err := json.MarshalIndent(webhookList{Webhooks: all}, "", " ")
if err != nil {
return fmt.Errorf("writing %s: %w", s.path, err)
}
tmp, err := os.CreateTemp(filepath.Dir(s.path), webhooksFile+".*")
if err != nil {
return fmt.Errorf("writing %s: %w", s.path, err)
}
// Synced before the rename, or a crash could leave the new file
// without its contents.
_, err = tmp.Write(append(b, '\n'))
err = errors.Join(err, tmp.Sync(), tmp.Close())
if err == nil {
err = os.Rename(tmp.Name(), s.path)
}
if err != nil {
_ = os.Remove(tmp.Name())
return fmt.Errorf("writing %s: %w", s.path, err)
}
return nil
}
// validURL reports whether s can be a webhook's URL: an absolute http or
// https URL with a host, at most maxURLBytes long.
func validURL(s string) bool {
if len(s) > maxURLBytes {
return false
}
u, err := url.Parse(s)
return err == nil && (u.Scheme == "http" || u.Scheme == "https") &&
u.Hostname() != ""
}
// handleWebhooks lists a chat's webhooks.
func (h *handlers) handleWebhooks() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
chatID, ok := h.chatID(w, r)
if !ok {
return
}
h.respond(w, http.StatusOK, webhookList{Webhooks: h.webhooks.list(chatID)})
}
}
// handleRegister registers the URL in the request's body on a chat, and
// answers with the webhook: 201 if it is new, 200 if the chat had it.
func (h *handlers) handleRegister() http.HandlerFunc {
type request struct {
URL string `json:"url"`
}
return func(w http.ResponseWriter, r *http.Request) {
chatID, ok := h.chatID(w, r)
if !ok {
return
}
var req request
if !h.decodeBody(w, r, &req, `{"url":"https://example.com/hook"}`) {
return
}
if !validURL(req.URL) {
h.respondError(w, http.StatusBadRequest, "url must be an http or https URL "+
"with a host, at most "+strconv.Itoa(maxURLBytes)+" bytes long")
return
}
hook, added, err := h.webhooks.register(chatID, req.URL)
if err != nil {
h.log.Error("registering a webhook", "error", err)
h.respondError(w, http.StatusInternalServerError,
"the webhooks could not be saved")
return
}
status := http.StatusOK
if added {
status = http.StatusCreated
}
h.respond(w, status, hook)
}
}
// handleRemove removes a webhook from a chat.
func (h *handlers) handleRemove() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
chatID, ok := h.chatID(w, r)
if !ok {
return
}
removed, err := h.webhooks.remove(chatID, chi.URLParam(r, "webhook_id"))
if err != nil {
h.log.Error("removing a webhook", "error", err)
h.respondError(w, http.StatusInternalServerError,
"the webhooks could not be saved")
return
}
if !removed {
h.respondError(w, http.StatusNotFound, "no such webhook")
return
}
w.WriteHeader(http.StatusNoContent)
}
}
+510
View File
@@ -0,0 +1,510 @@
package api_test
import (
"bytes"
"encoding/json"
"errors"
"io/fs"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"sync"
"testing"
"sneak.berlin/go/simplexcalc/internal/api"
"sneak.berlin/go/simplexcalc/internal/simplex"
)
const (
hookURL = "https://example.com/hook"
// unknownID is shaped like a webhook's id, and no webhook has it.
unknownID = "00112233445566778899aabbccddeeff"
)
// hook is a webhook as the API answers with it.
type hook struct {
ID string `json:"id"`
ChatID int64 `json:"chat_id"`
URL string `json:"url"`
}
// String returns h as the API writes it.
func (h hook) String() string {
return `{"id":"` + h.ID + `","chat_id":` + strconv.FormatInt(h.ChatID, 10) +
`,"url":"` + h.URL + `"}`
}
// twoChats returns the bot's contacts in these tests: chat 3, which
// webhooksPath names, and chat 4, whose contact has deleted it.
func twoChats() []simplex.Contact {
return append(oneChat(), simplex.Contact{ContactID: 4, Status: "deleted"})
}
// readWebhooks returns the webhooks kept in dir.
func readWebhooks(t *testing.T, dir string) *api.Webhooks {
t.Helper()
webhooks, err := api.ReadWebhooks(dir)
if err != nil {
t.Fatal(err)
}
return webhooks
}
// webhookAPI returns an API with webhooks, whose chats are those of
// twoChats.
func webhookAPI(webhooks *api.Webhooks) *http.Server {
return api.New(api.Params{
Log: slog.New(slog.DiscardHandler),
Client: &fakeClient{contacts: twoChats()},
UserID: 1,
Port: 8080,
Token: credential,
Webhooks: webhooks,
})
}
// register registers u on the chat whose webhooks are at path, and
// returns the webhook answered with. The answer must have status.
func register(t *testing.T, srv *http.Server, path, u string, status int) hook {
t.Helper()
rec := post(t, srv, path, `{"url":"`+u+`"}`)
var h hook
err := json.Unmarshal(rec.Body.Bytes(), &h)
if err != nil || rec.Code != status || rec.Body.String() != h.String()+"\n" {
t.Fatalf("registering %s: %d %q, want %d and a webhook",
u, rec.Code, rec.Body.String(), status)
}
return h
}
// listed fails the test unless the chat whose webhooks are at path lists
// want, in that order.
func listed(t *testing.T, srv *http.Server, path string, want ...hook) {
t.Helper()
hooks := make([]string, 0, len(want))
for _, h := range want {
hooks = append(hooks, h.String())
}
body := `{"webhooks":[` + strings.Join(hooks, ",") + "]}\n"
rec := request(t, srv, http.MethodGet, path, bearer)
if rec.Code != http.StatusOK || rec.Body.String() != body {
t.Errorf("GET %s: %d %q, want 200 %q", path, rec.Code, rec.Body.String(), body)
}
}
// eachEndpoint lists the webhooks of the chat whose webhooks are at path,
// registers one and removes one, and returns the three answers.
func eachEndpoint(
t *testing.T, srv *http.Server, path string,
) []*httptest.ResponseRecorder {
t.Helper()
return []*httptest.ResponseRecorder{
request(t, srv, http.MethodGet, path, bearer),
post(t, srv, path, `{"url":"`+hookURL+`"}`),
request(t, srv, http.MethodDelete, path+"/"+unknownID, bearer),
}
}
// onlyFile fails the test unless dir holds the webhooks file and nothing
// else: no temporary file is left.
func onlyFile(t *testing.T, dir string) {
t.Helper()
entries, err := os.ReadDir(dir)
if err != nil || len(entries) != 1 || entries[0].Name() != "webhooks.json" {
t.Errorf("the directory holds %v (%v), want webhooks.json alone", entries, err)
}
}
// noFile fails the test if dir has a webhooks file.
func noFile(t *testing.T, dir string) {
t.Helper()
_, err := os.Stat(filepath.Join(dir, "webhooks.json"))
if !errors.Is(err, fs.ErrNotExist) {
t.Errorf("webhooks.json: %v, want it not written", err)
}
}
// TestWebhooks: a URL registered on a chat gets 201 and a new id of 32
// hexadecimal digits; the same URL again gets 200 and the same webhook.
// Each chat lists its own, in the order registered, the chat of a contact
// who deleted it included.
func TestWebhooks(t *testing.T) {
t.Parallel()
srv := webhookAPI(readWebhooks(t, t.TempDir()))
listed(t, srv, webhooksPath)
first := register(t, srv, webhooksPath, hookURL, http.StatusCreated)
again := register(t, srv, webhooksPath, hookURL, http.StatusOK)
second := register(t, srv, webhooksPath, hookURL+"/2", http.StatusCreated)
other := register(t, srv, "/api/v1/chats/4/webhooks", hookURL, http.StatusCreated)
if !regexp.MustCompile(`^[0-9a-f]{32}$`).MatchString(first.ID) ||
first.ChatID != 3 || first.URL != hookURL {
t.Errorf("registered %v, want a new id, chat 3 and %s", first, hookURL)
}
if again != first {
t.Errorf("registered again: %v, want %v", again, first)
}
if second.ID == first.ID || other.ID == first.ID || other.ID == second.ID {
t.Errorf("ids repeat: %v, %v, %v", first, second, other)
}
listed(t, srv, webhooksPath, first, second)
listed(t, srv, "/api/v1/chats/4/webhooks", other)
}
// TestRemoveWebhook: a removed webhook gets 204 with no body and is no
// longer listed. Removing it again, removing a webhook through a chat it
// is not on, or removing an id no webhook has, gets 404.
func TestRemoveWebhook(t *testing.T) {
t.Parallel()
srv := webhookAPI(readWebhooks(t, t.TempDir()))
first := register(t, srv, webhooksPath, hookURL, http.StatusCreated)
second := register(t, srv, webhooksPath, hookURL+"/2", http.StatusCreated)
rec := request(t, srv, http.MethodDelete, webhooksPath+"/"+first.ID, bearer)
if rec.Code != http.StatusNoContent || rec.Body.Len() != 0 {
t.Errorf("removing: %d %q, want 204 and no body", rec.Code, rec.Body.String())
}
noSuchWebhook := `{"error":"no such webhook"}` + "\n"
for _, path := range []string{
webhooksPath + "/" + first.ID,
"/api/v1/chats/4/webhooks/" + second.ID,
webhooksPath + "/" + unknownID,
} {
rec := request(t, srv, http.MethodDelete, path, bearer)
if rec.Code != http.StatusNotFound || rec.Body.String() != noSuchWebhook {
t.Errorf("DELETE %s: %d %q, want 404 %q",
path, rec.Code, rec.Body.String(), noSuchWebhook)
}
}
listed(t, srv, webhooksPath, second)
}
// TestRegisterRefused: a body that is not JSON with a URL a webhook can
// have, or that is over 64 KiB, registers nothing. A URL of 2048 bytes
// is registered.
func TestRegisterRefused(t *testing.T) {
t.Parallel()
notJSON := `{"error":"the body must be JSON such as ` +
`{\"url\":\"https://example.com/hook\"}"}` + "\n"
badURL := `{"error":"url must be an http or https URL with a host, ` +
`at most 2048 bytes long"}` + "\n"
longest := hookURL + "/" + strings.Repeat("a", 2048-len(hookURL)-1)
for name, tc := range map[string]struct {
body string
status int
answer string
}{
"no body": {"", http.StatusBadRequest, notJSON},
"a bare URL": {hookURL, http.StatusBadRequest, notJSON},
"url not a string": {`{"url":5}`, http.StatusBadRequest, notJSON},
"no url": {`{}`, http.StatusBadRequest, badURL},
"relative": {`{"url":"/hook"}`, http.StatusBadRequest, badURL},
"no scheme": {`{"url":"example.com/hook"}`, http.StatusBadRequest, badURL},
"another scheme": {`{"url":"ftp://example.com/"}`, http.StatusBadRequest, badURL},
"no host": {`{"url":"https:///hook"}`, http.StatusBadRequest, badURL},
"a port, no host": {`{"url":"http://:80/hook"}`, http.StatusBadRequest, badURL},
"not a URL": {`{"url":"https://a b/"}`, http.StatusBadRequest, badURL},
"over 2048 bytes": {`{"url":"` + longest + `a"}`, http.StatusBadRequest, badURL},
"over 64 KiB": {
`{"url":"` + hookURL + "/" + strings.Repeat("a", 64<<10) + `"}`,
http.StatusRequestEntityTooLarge, `{"error":"the body is too large"}` + "\n",
},
} {
t.Run(name, func(t *testing.T) {
t.Parallel()
dir := t.TempDir()
rec := post(t, webhookAPI(readWebhooks(t, dir)), webhooksPath, tc.body)
if rec.Code != tc.status || rec.Body.String() != tc.answer {
t.Errorf("response = %d %q, want %d %q",
rec.Code, rec.Body.String(), tc.status, tc.answer)
}
noFile(t, dir)
})
}
register(t, webhookAPI(readWebhooks(t, t.TempDir())), webhooksPath, longest,
http.StatusCreated)
}
// TestWebhooksNoSuchChat: an id that GET /api/v1/chats does not list gets
// 404 from each webhook endpoint, 1 and 2 included, and nothing is
// written. When the chats cannot be read to look the id up, the answer
// is 500.
func TestWebhooksNoSuchChat(t *testing.T) {
t.Parallel()
for _, id := range []string{
"1", "2", "5", "tester", "0", "-3", "3.5", "99999999999999999999",
} {
dir := t.TempDir()
for _, rec := range eachEndpoint(t, webhookAPI(readWebhooks(t, dir)),
"/api/v1/chats/"+id+"/webhooks") {
if rec.Code != http.StatusNotFound || rec.Body.String() != noSuchChat {
t.Errorf("chat %q: %d %q, want 404 %q",
id, rec.Code, rec.Body.String(), noSuchChat)
}
}
noFile(t, dir)
}
srv := api.New(api.Params{
Log: slog.New(slog.DiscardHandler),
Client: &fakeClient{contactsErr: errChat},
Token: credential,
Webhooks: readWebhooks(t, t.TempDir()),
})
want := `{"error":"the chats could not be read"}` + "\n"
for _, rec := range eachEndpoint(t, srv, webhooksPath) {
if rec.Code != http.StatusInternalServerError || rec.Body.String() != want {
t.Errorf("chats unreadable: %d %q, want 500 %q",
rec.Code, rec.Body.String(), want)
}
}
}
// TestWebhooksKept: a new read of the directory finds the webhooks
// registered, and none once they are removed.
func TestWebhooksKept(t *testing.T) {
t.Parallel()
dir := t.TempDir()
srv := webhookAPI(readWebhooks(t, dir))
first := register(t, srv, webhooksPath, hookURL, http.StatusCreated)
second := register(t, srv, "/api/v1/chats/4/webhooks", hookURL, http.StatusCreated)
srv = webhookAPI(readWebhooks(t, dir))
listed(t, srv, webhooksPath, first)
listed(t, srv, "/api/v1/chats/4/webhooks", second)
for _, path := range []string{
webhooksPath + "/" + first.ID, "/api/v1/chats/4/webhooks/" + second.ID,
} {
rec := request(t, srv, http.MethodDelete, path, bearer)
if rec.Code != http.StatusNoContent {
t.Fatalf("DELETE %s: %d %q, want 204", path, rec.Code, rec.Body.String())
}
}
srv = webhookAPI(readWebhooks(t, dir))
listed(t, srv, webhooksPath)
listed(t, srv, "/api/v1/chats/4/webhooks")
}
// TestWebhooksFile: the file has mode 0600, and a change replaces it with
// a new file rather than writing into it: through a second name, the old
// file still holds what it held. No temporary file is left.
func TestWebhooksFile(t *testing.T) {
t.Parallel()
dir := t.TempDir()
file := filepath.Join(dir, "webhooks.json")
srv := webhookAPI(readWebhooks(t, dir))
register(t, srv, webhooksPath, hookURL, http.StatusCreated)
info, err := os.Stat(file)
if err != nil || info.Mode() != 0o600 {
t.Errorf("webhooks.json: %v (%v), want mode 0600", info, err)
}
before, err := os.ReadFile(file) //nolint:gosec // G304: the test's own file.
if err != nil {
t.Fatal(err)
}
old := filepath.Join(t.TempDir(), "old")
err = os.Link(file, old)
if err != nil {
t.Fatal(err)
}
register(t, srv, webhooksPath, hookURL+"/2", http.StatusCreated)
after, err := os.ReadFile(old) //nolint:gosec // G304: the test's own file.
if err != nil || !bytes.Equal(after, before) {
t.Errorf("the old file holds %q (%v), want %q as before", after, err, before)
}
onlyFile(t, dir)
}
// TestWebhooksWriteFailure: a change whose file cannot be replaced gets
// 500 and is not made, and its temporary file is removed. A directory
// stands where the file goes, which a rename cannot replace; a directory
// without write permission would not do, as tests may run as root.
func TestWebhooksWriteFailure(t *testing.T) {
t.Parallel()
dir := t.TempDir()
file := filepath.Join(dir, "webhooks.json")
srv := webhookAPI(readWebhooks(t, dir))
first := register(t, srv, webhooksPath, hookURL, http.StatusCreated)
err := os.Remove(file)
if err == nil {
err = os.Mkdir(file, 0o700)
}
if err != nil {
t.Fatal(err)
}
want := `{"error":"the webhooks could not be saved"}` + "\n"
for _, rec := range []*httptest.ResponseRecorder{
post(t, srv, webhooksPath, `{"url":"`+hookURL+`/2"}`),
request(t, srv, http.MethodDelete, webhooksPath+"/"+first.ID, bearer),
} {
if rec.Code != http.StatusInternalServerError || rec.Body.String() != want {
t.Errorf("response = %d %q, want 500 %q", rec.Code, rec.Body.String(), want)
}
}
listed(t, srv, webhooksPath, first)
onlyFile(t, dir)
}
// TestReadWebhooksRefuses: a webhooks file that holds anything but
// webhooks as the bot writes them cannot be read, and the error names
// it. An absent file is no webhooks.
func TestReadWebhooksRefuses(t *testing.T) {
t.Parallel()
record := func(id, chatID, url string) string {
return `{"webhooks":[{"id":"` + id + `","chat_id":` + chatID +
`,"url":"` + url + `"}]}`
}
for name, contents := range map[string]string{
"empty file": "",
"a word": "webhooks",
"cut short": `{"webhooks":[`,
"more after it": `{"webhooks":[]} {}`,
"null": "null",
"no list": "{}",
"null list": `{"webhooks":null}`,
"list alone": "[]",
"empty webhook": `{"webhooks":[{}]}`,
"id not hex": record("not-an-id", "3", hookURL),
"id too short": record("0011", "3", hookURL),
"chat 0": record(unknownID, "0", hookURL),
"url of another kind": record(unknownID, "3", "ftp://example.com/"),
} {
t.Run(name, func(t *testing.T) {
t.Parallel()
dir := t.TempDir()
err := os.WriteFile(filepath.Join(dir, "webhooks.json"), []byte(contents), 0o600)
if err != nil {
t.Fatal(err)
}
_, err = api.ReadWebhooks(dir)
if err == nil || !strings.Contains(err.Error(), "webhooks.json") {
t.Errorf("ReadWebhooks = %v, want an error naming webhooks.json", err)
}
})
}
listed(t, webhookAPI(readWebhooks(t, t.TempDir())), webhooksPath)
}
// TestWebhooksConcurrent: registrations at the same time, among listings
// and removals, each get their own webhook, all kept; the same URL
// registered at the same time is registered once.
func TestWebhooksConcurrent(t *testing.T) {
t.Parallel()
const n = 20
dir := t.TempDir()
webhooks := readWebhooks(t, dir)
answers := make([]*httptest.ResponseRecorder, 2*n)
var wg sync.WaitGroup
for i := range n {
wg.Go(func() {
answers[i] = post(t, webhookAPI(webhooks), webhooksPath,
`{"url":"`+hookURL+"/"+strconv.Itoa(i)+`"}`)
})
wg.Go(func() {
answers[n+i] = post(t, webhookAPI(webhooks), webhooksPath,
`{"url":"`+hookURL+`"}`)
})
wg.Go(func() {
srv := webhookAPI(webhooks)
request(t, srv, http.MethodGet, webhooksPath, bearer)
request(t, srv, http.MethodDelete, webhooksPath+"/"+unknownID, bearer)
})
}
wg.Wait()
created := 0
ids := map[string]bool{}
for _, rec := range answers {
var h hook
_ = json.Unmarshal(rec.Body.Bytes(), &h)
ids[h.ID] = true
if rec.Code == http.StatusCreated {
created++
}
}
if created != n+1 || len(ids) != n+1 {
t.Errorf("%d registered, %d ids; want %d of each", created, len(ids), n+1)
}
rec := request(t, webhookAPI(readWebhooks(t, dir)),
http.MethodGet, webhooksPath, bearer)
var got struct {
Webhooks []hook `json:"webhooks"`
}
err := json.Unmarshal(rec.Body.Bytes(), &got)
if err != nil || len(got.Webhooks) != n+1 {
t.Errorf("read again: %d webhooks (%v), want %d", len(got.Webhooks), err, n+1)
}
}
+14 -6
View File
@@ -54,12 +54,12 @@ const (
var errExited = errors.New("simplex-chat exited") var errExited = errors.New("simplex-chat exited")
// Run starts the chat client with its database in cfg.DataDir, serving // Run reads the webhooks kept in cfg.DataDir, starts the chat client with
// its API on localhost at chatPort, connects to it, sets up the bot's // its database there, serving its API on localhost at chatPort, connects
// address, then answers messages and serves the bot's API until ctx is // to it, sets up the bot's address, then answers messages and serves the
// cancelled — which is a clean stop and returns nil — or until the chat // bot's API until ctx is cancelled — which is a clean stop and returns
// client, the connection to it or the API's listener fails, which // nil — or until the chat client, the connection to it or the API's
// returns the error. // listener fails, which returns the error.
func Run( func Run(
ctx context.Context, log *slog.Logger, cfg *config.Config, chatPort int, ctx context.Context, log *slog.Logger, cfg *config.Config, chatPort int,
) error { ) error {
@@ -68,6 +68,13 @@ func Run(
return fmt.Errorf("creating data directory: %w", err) return fmt.Errorf("creating data directory: %w", err)
} }
// Before the chat client starts, so that a file that cannot be read
// stops the bot as configuration that cannot be read does.
webhooks, err := api.ReadWebhooks(cfg.DataDir)
if err != nil {
return err
}
// Cancelling this stops the chat client; the deferred wait makes // Cancelling this stops the chat client; the deferred wait makes
// Run return only once it has exited, whatever path Run takes. // Run return only once it has exited, whatever path Run takes.
// Cancelling ctx does not reach it, so that the API, stopped first, // Cancelling ctx does not reach it, so that the API, stopped first,
@@ -105,6 +112,7 @@ func Run(
UserID: user.UserID, UserID: user.UserID,
Port: cfg.Port, Port: cfg.Port,
Token: cfg.APIToken, Token: cfg.APIToken,
Webhooks: webhooks,
}) })
served := make(chan error, 1) served := make(chan error, 1)
+19
View File
@@ -182,6 +182,25 @@ func TestStopDuringRequest(t *testing.T) {
} }
} }
// TestUnreadableWebhooks: a webhooks file that cannot be read stops Run
// before it starts the chat client.
func TestUnreadableWebhooks(t *testing.T) {
// No chat client on PATH: starting one would fail with another error.
t.Setenv("PATH", t.TempDir())
cfg := &config.Config{DataDir: t.TempDir(), Port: freePort(t)}
err := os.WriteFile(filepath.Join(cfg.DataDir, "webhooks.json"), []byte("{"), 0o600)
if err != nil {
t.Fatal(err)
}
err = bot.Run(t.Context(), slog.New(slog.DiscardHandler), cfg, freePort(t))
if err == nil || !strings.Contains(err.Error(), "webhooks.json") {
t.Errorf("Run = %v, want an error naming webhooks.json", err)
}
}
// freePort returns a TCP port that nothing listens on at the moment. // freePort returns a TCP port that nothing listens on at the moment.
func freePort(t *testing.T) int { func freePort(t *testing.T) int {
t.Helper() t.Helper()
+3 -2
View File
@@ -64,8 +64,9 @@ var ErrInvalidConfig = errors.New("invalid configuration")
// later, so there is exactly one moment at which configuration can be // later, so there is exactly one moment at which configuration can be
// wrong, and it is before anything starts. // wrong, and it is before anything starts.
type Config struct { type Config struct {
// DataDir holds the SimpleX Chat database: the bot's profile, its // DataDir holds the SimpleX Chat database, with the bot's profile,
// address and its contacts. Losing it loses the address. // its address and its contacts, and the API's webhooks. Losing it
// loses the address.
DataDir string DataDir string
Debug bool Debug bool