ipabusedb-docker-reporter: rename from ipabusedb-docker-ssh, add Dockerfile with runit service
Build on git.eeqj.de/sneak/baseimage:2026-11-05 with the docker CLI copied from docker:cli. A runit service sleeps INTERVAL seconds (default 1200), then runs the reporter with --since INTERVAL --submit; DOCKER_CONTAINER_NAME and IPABUSEDB_API_KEY come from the container environment. Add .dockerignore, make image, and README section; fix the binary name in .gitignore.
This commit is contained in:
sneak committed
1 parent
b8c36689ae
commit
1aa7d84401
16 files changed
+135
-11
No files matched your search
@@ -0,0 +1,6 @@
|
||||
.env
|
||||
.git
|
||||
*.csv
|
||||
/ipabusedb-docker-reporter
|
||||
deploy.sh
|
||||
example_parser_script.py
|
||||
@@ -0,0 +1,5 @@
|
||||
# Copy to .env (gitignored). Variables already set in your shell win.
|
||||
IPABUSEDB_API_KEY=
|
||||
DOCKER_CONTAINER_NAME=sshd
|
||||
# Point the docker CLI at a remote daemon; credentials above stay local.
|
||||
DOCKER_HOST=ssh://user@sshd-host.example.com
|
||||
@@ -0,0 +1,3 @@
|
||||
/ipabusedb-docker-reporter
|
||||
/reports.csv
|
||||
/.env
|
||||
@@ -0,0 +1,39 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
#
|
||||
# Runs ipabusedb-docker-reporter as a runit service under the baseimage's
|
||||
# runsvinit: sleep INTERVAL seconds, report the last INTERVAL seconds of
|
||||
# sshd brute-force attempts to AbuseIPDB, repeat.
|
||||
#
|
||||
# Run with the host's docker socket bind-mounted and the config in the
|
||||
# container environment:
|
||||
#
|
||||
# docker run -d --restart unless-stopped \
|
||||
# -v /var/run/docker.sock:/var/run/docker.sock \
|
||||
# -e DOCKER_CONTAINER_NAME=sshd \
|
||||
# -e IPABUSEDB_API_KEY=... \
|
||||
# -e INTERVAL=1200 \
|
||||
# ipabusedb-docker-reporter
|
||||
|
||||
FROM git.eeqj.de/sneak/baseimage:2026-11-05
|
||||
|
||||
# The reporter shells out to "docker logs", so it needs the docker CLI.
|
||||
# The static binary from the official CLI image avoids pulling the whole
|
||||
# docker.io package (and daemon) in via apt.
|
||||
COPY --from=docker:cli /usr/local/bin/docker /usr/local/bin/docker
|
||||
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN --mount=type=cache,target=/root/go/pkg/mod go mod download
|
||||
COPY *.go ./
|
||||
RUN --mount=type=cache,target=/root/go/pkg/mod \
|
||||
--mount=type=cache,target=/root/.cache/go-build \
|
||||
CGO_ENABLED=0 go build -o /usr/local/bin/ipabusedb-docker-reporter .
|
||||
|
||||
COPY sv/ipabusedb-docker-reporter/run /etc/sv/ipabusedb-docker-reporter/run
|
||||
RUN chmod +x /etc/sv/ipabusedb-docker-reporter/run \
|
||||
&& ln -s /etc/sv/ipabusedb-docker-reporter /etc/service/ipabusedb-docker-reporter
|
||||
|
||||
# Seconds to sleep between runs; also the --since window of each run.
|
||||
ENV INTERVAL=1200
|
||||
|
||||
WORKDIR /
|
||||
@@ -0,0 +1,26 @@
|
||||
BIN := ipabusedb-docker-reporter
|
||||
|
||||
.PHONY: build test lint fmt fmt-check check clean image
|
||||
|
||||
build:
|
||||
go build -o $(BIN) .
|
||||
|
||||
image:
|
||||
docker build -t $(BIN) .
|
||||
|
||||
test:
|
||||
go test ./...
|
||||
|
||||
fmt:
|
||||
gofmt -w *.go
|
||||
|
||||
fmt-check:
|
||||
@test -z "$$(gofmt -l .)" || { gofmt -l .; echo "gofmt: files need formatting"; exit 1; }
|
||||
|
||||
lint: fmt-check
|
||||
go vet ./...
|
||||
|
||||
check: lint test build
|
||||
|
||||
clean:
|
||||
rm -f $(BIN) *.csv
|
||||
@@ -0,0 +1,97 @@
|
||||
# ipabusedb-docker-ssh
|
||||
|
||||
Reads the logs of a dockerised sshd with `docker logs -t`, extracts
|
||||
brute-force / invalid-user login attempts, and emits a CSV in the format
|
||||
expected by the [AbuseIPDB bulk-report
|
||||
endpoint](https://www.abuseipdb.com/bulk-report). Optionally submits it.
|
||||
|
||||
What gets reported:
|
||||
|
||||
- `Invalid user X from IP port N`
|
||||
- `User X from IP not allowed because not listed in AllowUsers`
|
||||
- `Failed password for invalid user X from IP port N`
|
||||
- `Connection closed by invalid user X IP port N` and
|
||||
`Disconnected from invalid user X IP port N`
|
||||
|
||||
Attempts with the username `git` are excluded (so people poking a git
|
||||
host with `git@` are not reported). Private, loopback and link-local
|
||||
source addresses are skipped. Each IP appears once in the CSV, dated with
|
||||
its most recent attempt and annotated with the number of matching lines.
|
||||
Categories are `18,22` (Brute-Force, SSH).
|
||||
|
||||
## Configuration
|
||||
|
||||
A `.env` file in the working directory is loaded at startup (see
|
||||
`.env.example`; `.env` is gitignored). Variables already set in the
|
||||
shell take precedence over the file.
|
||||
|
||||
| Variable | Purpose |
|
||||
| ----------------------- | -------------------------------------------------------- |
|
||||
| `DOCKER_CONTAINER_NAME` | container whose logs are read (required) |
|
||||
| `IPABUSEDB_API_KEY` | AbuseIPDB API key (required only with `--submit`) |
|
||||
| `DOCKER_HOST` | passed through to the `docker` CLI to target a remote daemon |
|
||||
|
||||
Because the program shells out to the `docker` CLI with its own
|
||||
environment, `DOCKER_HOST` (and `DOCKER_CONTEXT`, `DOCKER_TLS_VERIFY`,
|
||||
`DOCKER_CERT_PATH`) work exactly as they do for `docker` itself. Set
|
||||
`DOCKER_HOST=ssh://user@host` in `.env` to read logs from a remote
|
||||
daemon while the AbuseIPDB key never leaves the local machine.
|
||||
|
||||
Flags:
|
||||
|
||||
| Flag | Purpose |
|
||||
| --------------- | --------------------------------------------------------- |
|
||||
| `--since 24h` | only consider lines newer than now minus this duration |
|
||||
| `-o FILE` | write the CSV to `FILE` instead of stdout |
|
||||
| `--input FILE` | parse `FILE` (or `-` for stdin) instead of `docker logs` |
|
||||
| `--submit` | POST the CSV to the AbuseIPDB bulk-report endpoint |
|
||||
| `--env FILE` | env file to autoload (default `.env`; missing is ignored) |
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
$ make build
|
||||
$ export DOCKER_CONTAINER_NAME=sshd IPABUSEDB_API_KEY=...
|
||||
$ ./ipabusedb-docker-ssh --since 24h -o reports.csv
|
||||
$ ./ipabusedb-docker-ssh --since 24h --submit
|
||||
```
|
||||
|
||||
Submitting manually with curl instead of `--submit`:
|
||||
|
||||
```
|
||||
$ ./ipabusedb-docker-ssh --since 24h > reports.csv && curl https://api.abuseipdb.com/api/v2/bulk-report -F csv=@reports.csv -H "Key: $IPABUSEDB_API_KEY"
|
||||
```
|
||||
|
||||
## Running as a container
|
||||
|
||||
The `Dockerfile` builds on `git.eeqj.de/sneak/baseimage` and installs the
|
||||
program as a runit service (`sv/ipabusedb-docker-reporter/run`) that
|
||||
sleeps `INTERVAL` seconds, then runs the reporter with
|
||||
`--since ${INTERVAL}s --submit`, and repeats. The docker CLI is copied in
|
||||
from the `docker:cli` image so the program can read logs through the
|
||||
host's docker socket.
|
||||
|
||||
| Variable | Purpose |
|
||||
| ----------------------- | -------------------------------------------------- |
|
||||
| `DOCKER_CONTAINER_NAME` | container whose logs are read (required) |
|
||||
| `IPABUSEDB_API_KEY` | AbuseIPDB API key (required) |
|
||||
| `INTERVAL` | seconds between runs, and the report window (1200) |
|
||||
|
||||
```
|
||||
$ make image
|
||||
$ docker run -d --restart unless-stopped \
|
||||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||
-e DOCKER_CONTAINER_NAME=sshd \
|
||||
-e IPABUSEDB_API_KEY=... \
|
||||
-e INTERVAL=1200 \
|
||||
ipabusedb-docker-reporter
|
||||
```
|
||||
|
||||
Each run's CSV and the AbuseIPDB response go to the container log. If a
|
||||
required variable is missing the service logs an error, sleeps a minute
|
||||
and exits so runit can retry.
|
||||
|
||||
`make check` runs vet, the tests and the build. `loglines_example.txt`
|
||||
holds sample `docker logs -t` output used by the tests;
|
||||
`example_parser_script.py` is the reference script from AbuseIPDB that
|
||||
this program replaces.
|
||||
Executable
+14
@@ -0,0 +1,14 @@
|
||||
#!/bin/bash
|
||||
|
||||
BIN="ipabusedb-docker-reporter"
|
||||
|
||||
DH="root@fsn1app1"
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
|
||||
cd "$DIR"
|
||||
|
||||
rsync -avP --delete ./ "$DH":/tmp/reporterdeploy/
|
||||
|
||||
#ssh $DH "cd /tmp/reporterdeploy && go get -v && go build -o ./${BIN} . && ls -tla"
|
||||
ssh $DH "cd /tmp/reporterdeploy && go get -v && go build . && ls -tla && mv ./${BIN} /usr/local/sbin"
|
||||
@@ -0,0 +1,76 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import os
|
||||
import sys
|
||||
import argparse
|
||||
import re
|
||||
import csv
|
||||
from datetime import datetime, timezone
|
||||
import pytz
|
||||
|
||||
def main(arguments):
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument('infile', help="Input file", type=argparse.FileType('r'))
|
||||
parser.add_argument('-o', '--outfile', help="Output file",
|
||||
default=sys.stdout, type=argparse.FileType('w'))
|
||||
|
||||
args = parser.parse_args(arguments)
|
||||
|
||||
# Define field names.
|
||||
fieldnames = ['IP', 'Categories', 'Comment', 'ReportDate']
|
||||
# Begin CSV output.
|
||||
writer = csv.DictWriter(args.outfile, fieldnames=fieldnames)
|
||||
writer.writeheader()
|
||||
|
||||
# Initialize empty list to hold addresses
|
||||
ipv4_addresses = list()
|
||||
|
||||
for line in args.infile:
|
||||
# !! Match this format to your system's format.
|
||||
timestamp = "([a-zA-Z]+\s+[0-9]+ [0-9]+:[0-9]+:[0-9]+)"
|
||||
ipv4 = "([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})"
|
||||
comment = "(Invalid user [a-zA-Z0-9]+ from " + ipv4 + " port [0-9]+)"
|
||||
|
||||
# The regex of the line we're looking for, built up from component regexps.
|
||||
combined_re = timestamp + " .* " + comment
|
||||
|
||||
# Run the regexp.
|
||||
matches = re.findall(combined_re, line)
|
||||
# If this line is in the format we're looking for,
|
||||
if matches:
|
||||
# Pull the tuple out of the list.
|
||||
matches_flat = matches[0]
|
||||
|
||||
# Remove duplicate addresses from the report.
|
||||
if matches_flat[2] not in ipv4_addresses:
|
||||
ipv4_addresses.append(matches_flat[2])
|
||||
else:
|
||||
continue
|
||||
|
||||
### !!! You may need to update this. ###
|
||||
# Parse log datetime to Python datetime object so we can update the timezone.
|
||||
# The format string should must your log files. Here we use the default in Debian/Redhat distros.
|
||||
attack_datetime = datetime.strptime(matches_flat[0], '%b %d %H:%M:%S')
|
||||
# Assume year is the current year.
|
||||
attack_datetime = attack_datetime.replace(datetime.now().year)
|
||||
# !! Set tzinfo to your system timezone using timezone.
|
||||
my_tz = pytz.timezone('America/New_York')
|
||||
attack_datetime = attack_datetime.replace(tzinfo=my_tz)
|
||||
|
||||
# Format to ISO 8601 to make it universal and portable.
|
||||
attack_datetime_iso = attack_datetime.isoformat()
|
||||
|
||||
# We'll add the categories column statically at this step.
|
||||
# Output as a CSV row.
|
||||
writer.writerow({
|
||||
'IP': matches_flat[2],
|
||||
'Categories': "18,22",
|
||||
'Comment': matches_flat[1],
|
||||
'ReportDate': attack_datetime_iso
|
||||
})
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
@@ -0,0 +1,5 @@
|
||||
module sneak.berlin/go/hacks/ipabusedb-docker-reporter
|
||||
|
||||
go 1.26.5
|
||||
|
||||
require github.com/joho/godotenv v1.5.1
|
||||
@@ -0,0 +1,2 @@
|
||||
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
|
||||
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
|
||||
@@ -0,0 +1,13 @@
|
||||
2026-10-06T12:04:24.554951297Z User root from 41.93.28.11 not allowed because not listed in AllowUsers
|
||||
2026-10-06T12:04:24.595412750Z Connection closed by invalid user root 41.93.28.11 port 51904 [preauth]
|
||||
2026-10-06T12:04:24.687263829Z Invalid user production from 45.191.4.252 port 38924
|
||||
2026-10-06T12:04:24.927218990Z Connection closed by invalid user production 45.191.4.252 port 38924 [preauth]
|
||||
2026-10-06T12:04:28.005424065Z Timeout before authentication for connection from 95.78.250.88 to 172.19.0.29, pid = 307434
|
||||
2026-10-06T12:04:29.009224164Z User root from 164.77.161.180 not allowed because not listed in AllowUsers
|
||||
2026-10-06T12:04:29.494569018Z Connection closed by invalid user root 164.77.161.180 port 33916 [preauth]
|
||||
2026-10-06T12:04:30.222573149Z Invalid user production from 212.128.3.152 port 60626
|
||||
2026-10-06T12:04:30.266239421Z Connection closed by invalid user production 212.128.3.152 port 60626 [preauth]
|
||||
2026-10-06T12:04:38.873013329Z Timeout before authentication for connection from 212.19.8.30 to 172.19.0.29, pid = 308210
|
||||
2026-10-06T12:04:39.622456198Z Received disconnect from 172.19.0.1 port 53534:11: disconnected by user
|
||||
2026-10-06T12:04:41.066933472Z Received disconnect from 172.19.0.1 port 53540:11: disconnected by user
|
||||
2026-10-06T12:04:41.413322464Z Received disconnect from 172.19.0.1 port 53552:11: disconnected by user
|
||||
@@ -0,0 +1,398 @@
|
||||
// ipabusedb-docker-ssh reads the logs of a dockerised sshd via
|
||||
// "docker logs -t", extracts brute-force / invalid-user attempts, and
|
||||
// emits a CSV suitable for the AbuseIPDB bulk-report endpoint.
|
||||
//
|
||||
// Environment (a .env file in the working directory is loaded first;
|
||||
// variables already set in the process environment take precedence):
|
||||
//
|
||||
// DOCKER_CONTAINER_NAME container whose logs are read (required)
|
||||
// IPABUSEDB_API_KEY AbuseIPDB API key (required with --submit)
|
||||
// DOCKER_HOST passed through to the docker CLI, so e.g.
|
||||
// ssh://user@host reads logs from a remote daemon
|
||||
//
|
||||
// Flags:
|
||||
//
|
||||
// --since 24h only consider log lines newer than now()-duration
|
||||
// -o FILE write CSV to FILE instead of stdout
|
||||
// --input FILE parse FILE (or "-" for stdin) instead of docker logs
|
||||
// --submit POST the CSV to the AbuseIPDB bulk-report endpoint
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/csv"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"os"
|
||||
"os/exec"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/joho/godotenv"
|
||||
)
|
||||
|
||||
const (
|
||||
bulkReportURL = "https://api.abuseipdb.com/api/v2/bulk-report"
|
||||
// AbuseIPDB categories: 18 = Brute-Force, 22 = SSH.
|
||||
categories = "18,22"
|
||||
excludedUser = "git"
|
||||
// AbuseIPDB rejects bulk reports with more than this many rows.
|
||||
bulkReportMaxRows = 10000
|
||||
)
|
||||
|
||||
// patterns match sshd messages that evidence an unauthorised login
|
||||
// attempt. Group 1 is the username, group 2 the source address. Order
|
||||
// is priority: the earliest matching pattern supplies the report comment.
|
||||
var patterns = []*regexp.Regexp{
|
||||
regexp.MustCompile(`Invalid user (\S+) from (\S+) port \d+`),
|
||||
regexp.MustCompile(`User (\S+) from (\S+) not allowed because not listed in AllowUsers`),
|
||||
regexp.MustCompile(`Failed password for invalid user (\S+) from (\S+) port \d+`),
|
||||
regexp.MustCompile(`Connection closed by invalid user (\S+) (\S+) port \d+`),
|
||||
regexp.MustCompile(`Disconnected from invalid user (\S+) (\S+) port \d+`),
|
||||
}
|
||||
|
||||
type event struct {
|
||||
ip netip.Addr
|
||||
user string
|
||||
comment string
|
||||
prio int // index into patterns; lower is more descriptive
|
||||
when time.Time
|
||||
count int
|
||||
}
|
||||
|
||||
// parser accumulates one event per source IP, keeping the most recent
|
||||
// matching message and counting how many matched in total.
|
||||
type parser struct {
|
||||
cutoff time.Time
|
||||
|
||||
mu sync.Mutex
|
||||
events map[netip.Addr]*event
|
||||
skipped int
|
||||
// unparsed keeps the last few lines without a valid timestamp; the
|
||||
// docker CLI writes its own errors to stderr without one.
|
||||
unparsed []string
|
||||
}
|
||||
|
||||
const keepUnparsed = 5
|
||||
|
||||
func newParser(cutoff time.Time) *parser {
|
||||
return &parser{cutoff: cutoff, events: map[netip.Addr]*event{}}
|
||||
}
|
||||
|
||||
// handleLine parses one "docker logs -t" line: an RFC3339Nano timestamp,
|
||||
// a space, then the message.
|
||||
func (p *parser) handleLine(line string) {
|
||||
line = strings.TrimRight(line, "\r\n")
|
||||
ts, msg, ok := strings.Cut(line, " ")
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
when, err := time.Parse(time.RFC3339Nano, ts)
|
||||
if err != nil {
|
||||
p.mu.Lock()
|
||||
p.skipped++
|
||||
p.unparsed = append(p.unparsed, line)
|
||||
if len(p.unparsed) > keepUnparsed {
|
||||
p.unparsed = p.unparsed[1:]
|
||||
}
|
||||
p.mu.Unlock()
|
||||
return
|
||||
}
|
||||
if !p.cutoff.IsZero() && when.Before(p.cutoff) {
|
||||
return
|
||||
}
|
||||
for prio, re := range patterns {
|
||||
m := re.FindStringSubmatch(msg)
|
||||
if m == nil {
|
||||
continue
|
||||
}
|
||||
user, addr := m[1], m[2]
|
||||
if user == excludedUser {
|
||||
return
|
||||
}
|
||||
ip, err := netip.ParseAddr(addr)
|
||||
if err != nil || !reportable(ip) {
|
||||
return
|
||||
}
|
||||
p.record(ip, user, strings.TrimSpace(m[0]), prio, when)
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
func reportable(ip netip.Addr) bool {
|
||||
return ip.IsGlobalUnicast() && !ip.IsPrivate()
|
||||
}
|
||||
|
||||
// record merges an observation into the per-IP event: the report date is
|
||||
// the latest observation, the comment comes from the most descriptive
|
||||
// pattern (and, within the same pattern, the latest observation).
|
||||
func (p *parser) record(ip netip.Addr, user, comment string, prio int, when time.Time) {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
ev, ok := p.events[ip]
|
||||
if !ok {
|
||||
p.events[ip] = &event{ip: ip, user: user, comment: comment, prio: prio, when: when, count: 1}
|
||||
return
|
||||
}
|
||||
ev.count++
|
||||
newer := when.After(ev.when)
|
||||
if newer {
|
||||
ev.when = when
|
||||
}
|
||||
if prio < ev.prio || (prio == ev.prio && newer) {
|
||||
ev.prio, ev.user, ev.comment = prio, user, comment
|
||||
}
|
||||
}
|
||||
|
||||
func (p *parser) ingest(r io.Reader) error {
|
||||
sc := bufio.NewScanner(r)
|
||||
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
|
||||
for sc.Scan() {
|
||||
p.handleLine(sc.Text())
|
||||
}
|
||||
return sc.Err()
|
||||
}
|
||||
|
||||
func (p *parser) sorted() []*event {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
out := make([]*event, 0, len(p.events))
|
||||
for _, ev := range p.events {
|
||||
out = append(out, ev)
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].when.Before(out[j].when) })
|
||||
return out
|
||||
}
|
||||
|
||||
func writeCSV(w io.Writer, events []*event) error {
|
||||
cw := csv.NewWriter(w)
|
||||
if err := cw.Write([]string{"IP", "Categories", "Comment", "ReportDate"}); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, ev := range events {
|
||||
comment := ev.comment
|
||||
if ev.count > 1 {
|
||||
comment = fmt.Sprintf("%s (%d events)", ev.comment, ev.count)
|
||||
}
|
||||
row := []string{
|
||||
ev.ip.String(),
|
||||
categories,
|
||||
comment,
|
||||
ev.when.UTC().Format(time.RFC3339),
|
||||
}
|
||||
if err := cw.Write(row); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
cw.Flush()
|
||||
return cw.Error()
|
||||
}
|
||||
|
||||
// readDockerLogs runs "docker logs -t [--since D] NAME" and feeds both
|
||||
// stdout and stderr (sshd logs to stderr) through the parser.
|
||||
func readDockerLogs(ctx context.Context, p *parser, container string, since time.Duration) error {
|
||||
args := []string{"logs", "-t"}
|
||||
if since > 0 {
|
||||
args = append(args, "--since", since.String())
|
||||
}
|
||||
args = append(args, container)
|
||||
// cmd.Env is left nil so the docker CLI inherits the full process
|
||||
// environment, including DOCKER_HOST / DOCKER_CONTEXT / DOCKER_TLS_*
|
||||
// loaded from .env, and can therefore target a remote daemon.
|
||||
cmd := exec.CommandContext(ctx, "docker", args...)
|
||||
if host := os.Getenv("DOCKER_HOST"); host != "" {
|
||||
slog.Info("reading logs via docker", "DOCKER_HOST", host, "container", container)
|
||||
}
|
||||
|
||||
stdout, err := cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
stderr, err := cmd.StderrPipe()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := cmd.Start(); err != nil {
|
||||
return fmt.Errorf("starting docker: %w", err)
|
||||
}
|
||||
|
||||
var wg sync.WaitGroup
|
||||
errs := make(chan error, 2)
|
||||
for _, r := range []io.Reader{stdout, stderr} {
|
||||
wg.Add(1)
|
||||
go func(r io.Reader) {
|
||||
defer wg.Done()
|
||||
if err := p.ingest(r); err != nil {
|
||||
errs <- err
|
||||
}
|
||||
}(r)
|
||||
}
|
||||
wg.Wait()
|
||||
close(errs)
|
||||
if err := cmd.Wait(); err != nil {
|
||||
p.mu.Lock()
|
||||
detail := strings.Join(p.unparsed, "\n")
|
||||
p.mu.Unlock()
|
||||
if detail != "" {
|
||||
return fmt.Errorf("docker logs %s: %w\n%s", container, err, detail)
|
||||
}
|
||||
return fmt.Errorf("docker logs %s: %w", container, err)
|
||||
}
|
||||
for err := range errs {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func readInput(p *parser, path string) error {
|
||||
if path == "-" {
|
||||
return p.ingest(os.Stdin)
|
||||
}
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
return p.ingest(f)
|
||||
}
|
||||
|
||||
func submit(ctx context.Context, apiKey string, csvData []byte) ([]byte, error) {
|
||||
var body bytes.Buffer
|
||||
mw := multipart.NewWriter(&body)
|
||||
part, err := mw.CreateFormFile("csv", "reports.csv")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err := part.Write(csvData); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := mw.Close(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, bulkReportURL, &body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Key", apiKey)
|
||||
req.Header.Set("Accept", "application/json")
|
||||
req.Header.Set("Content-Type", mw.FormDataContentType())
|
||||
|
||||
client := &http.Client{Timeout: 2 * time.Minute}
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
respBody, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.StatusCode/100 != 2 {
|
||||
return respBody, fmt.Errorf("abuseipdb returned %s", resp.Status)
|
||||
}
|
||||
return respBody, nil
|
||||
}
|
||||
|
||||
func run() error {
|
||||
since := flag.Duration("since", 0, "only report events newer than now()-duration (e.g. 24h)")
|
||||
outPath := flag.String("o", "", "write CSV to this file instead of stdout")
|
||||
inPath := flag.String("input", "", "parse this file (or - for stdin) instead of running docker logs")
|
||||
doSubmit := flag.Bool("submit", false, "POST the CSV to the AbuseIPDB bulk-report endpoint")
|
||||
envFile := flag.String("env", ".env", "env file to autoload (missing file is ignored)")
|
||||
flag.Parse()
|
||||
|
||||
// Load .env without overriding variables already in the environment.
|
||||
if err := godotenv.Load(*envFile); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return fmt.Errorf("loading %s: %w", *envFile, err)
|
||||
}
|
||||
|
||||
apiKey := os.Getenv("IPABUSEDB_API_KEY")
|
||||
container := os.Getenv("DOCKER_CONTAINER_NAME")
|
||||
if *doSubmit && apiKey == "" {
|
||||
return errors.New("IPABUSEDB_API_KEY must be set in the environment to --submit")
|
||||
}
|
||||
if *inPath == "" && container == "" {
|
||||
return errors.New("DOCKER_CONTAINER_NAME must be set in the environment")
|
||||
}
|
||||
|
||||
var cutoff time.Time
|
||||
if *since > 0 {
|
||||
cutoff = time.Now().Add(-*since)
|
||||
}
|
||||
p := newParser(cutoff)
|
||||
|
||||
ctx := context.Background()
|
||||
var err error
|
||||
if *inPath != "" {
|
||||
err = readInput(p, *inPath)
|
||||
} else {
|
||||
err = readDockerLogs(ctx, p, container, *since)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
events := p.sorted()
|
||||
if len(events) > bulkReportMaxRows {
|
||||
slog.Warn("truncating to AbuseIPDB bulk report limit", "limit", bulkReportMaxRows, "found", len(events))
|
||||
events = events[len(events)-bulkReportMaxRows:]
|
||||
}
|
||||
slog.Info("parsed logs", "abusers", len(events), "unparseable_lines", p.skipped)
|
||||
|
||||
var buf bytes.Buffer
|
||||
if err := writeCSV(&buf, events); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
out := io.Writer(os.Stdout)
|
||||
if *outPath != "" {
|
||||
f, err := os.Create(*outPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
out = f
|
||||
}
|
||||
if _, err := out.Write(buf.Bytes()); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if !*doSubmit {
|
||||
return nil
|
||||
}
|
||||
if len(events) == 0 {
|
||||
slog.Info("nothing to submit")
|
||||
return nil
|
||||
}
|
||||
resp, err := submit(ctx, apiKey, buf.Bytes())
|
||||
if len(resp) > 0 {
|
||||
fmt.Fprintln(os.Stderr, string(resp))
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
slog.Info("submitted bulk report", "rows", len(events))
|
||||
return nil
|
||||
}
|
||||
|
||||
func main() {
|
||||
slog.SetDefault(slog.New(slog.NewTextHandler(os.Stderr, nil)))
|
||||
if err := run(); err != nil {
|
||||
slog.Error(err.Error())
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"net/netip"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestParseExampleLines(t *testing.T) {
|
||||
data, err := os.ReadFile("loglines_example.txt")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
p := newParser(time.Time{})
|
||||
if err := p.ingest(bytes.NewReader(data)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
want := map[string]int{
|
||||
"41.93.28.11": 2,
|
||||
"45.191.4.252": 2,
|
||||
"164.77.161.180": 2,
|
||||
"212.128.3.152": 2,
|
||||
}
|
||||
if len(p.events) != len(want) {
|
||||
t.Fatalf("got %d events, want %d", len(p.events), len(want))
|
||||
}
|
||||
for ipStr, count := range want {
|
||||
ev, ok := p.events[netip.MustParseAddr(ipStr)]
|
||||
if !ok {
|
||||
t.Errorf("missing event for %s", ipStr)
|
||||
continue
|
||||
}
|
||||
if ev.count != count {
|
||||
t.Errorf("%s: count %d, want %d", ipStr, ev.count, count)
|
||||
}
|
||||
}
|
||||
|
||||
var out bytes.Buffer
|
||||
if err := writeCSV(&out, p.sorted()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lines := strings.Split(strings.TrimSpace(out.String()), "\n")
|
||||
if lines[0] != "IP,Categories,Comment,ReportDate" {
|
||||
t.Errorf("bad header: %q", lines[0])
|
||||
}
|
||||
if len(lines) != 5 {
|
||||
t.Fatalf("got %d csv lines, want 5:\n%s", len(lines), out.String())
|
||||
}
|
||||
if !strings.Contains(lines[1], "41.93.28.11,\"18,22\",") || !strings.HasSuffix(lines[1], "2026-10-06T12:04:24Z") {
|
||||
t.Errorf("unexpected first row: %q", lines[1])
|
||||
}
|
||||
// The descriptive "Invalid user ... from" line should win over the
|
||||
// later "Connection closed by invalid user" line for the comment.
|
||||
if !strings.Contains(lines[2], ",Invalid user production from 45.191.4.252 port 38924 (2 events),") {
|
||||
t.Errorf("unexpected comment in row: %q", lines[2])
|
||||
}
|
||||
// AllowUsers rejections are reported with their own message.
|
||||
if !strings.Contains(lines[1], "User root from 41.93.28.11 not allowed because not listed in AllowUsers (2 events)") {
|
||||
t.Errorf("unexpected comment in row: %q", lines[1])
|
||||
}
|
||||
}
|
||||
|
||||
func TestExclusions(t *testing.T) {
|
||||
p := newParser(time.Time{})
|
||||
for _, line := range []string{
|
||||
"2026-10-06T12:04:24.687263829Z Invalid user git from 1.2.3.4 port 38924",
|
||||
"2026-10-06T12:04:24.687263829Z User git from 1.2.3.5 not allowed because not listed in AllowUsers",
|
||||
"2026-10-06T12:04:24.687263829Z Invalid user bob from 172.19.0.1 port 38924",
|
||||
"2026-10-06T12:04:24.687263829Z Invalid user bob from 10.0.0.9 port 38924",
|
||||
"2026-10-06T12:04:24.687263829Z Invalid user bob from 127.0.0.1 port 38924",
|
||||
"2026-10-06T12:04:24.687263829Z Invalid user bob from fd00::1 port 38924",
|
||||
"2026-10-06T12:04:24.687263829Z Invalid user bob from not-an-ip port 38924",
|
||||
"2026-10-06T12:04:24.687263829Z Received disconnect from 8.8.8.8 port 53534:11: disconnected by user",
|
||||
"garbage line without timestamp",
|
||||
} {
|
||||
p.handleLine(line)
|
||||
}
|
||||
if len(p.events) != 0 {
|
||||
t.Errorf("expected no events, got %v", p.events)
|
||||
}
|
||||
if p.skipped != 1 {
|
||||
t.Errorf("skipped = %d, want 1", p.skipped)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSinceCutoff(t *testing.T) {
|
||||
cutoff := time.Date(2026, 10, 6, 12, 4, 29, 0, time.UTC)
|
||||
p := newParser(cutoff)
|
||||
p.handleLine("2026-10-06T12:04:24.687263829Z Invalid user production from 45.191.4.252 port 38924")
|
||||
p.handleLine("2026-10-06T12:04:30.222573149Z Invalid user production from 212.128.3.152 port 60626")
|
||||
if len(p.events) != 1 {
|
||||
t.Fatalf("got %d events, want 1", len(p.events))
|
||||
}
|
||||
if _, ok := p.events[netip.MustParseAddr("212.128.3.152")]; !ok {
|
||||
t.Error("expected only the post-cutoff event to remain")
|
||||
}
|
||||
}
|
||||
|
||||
func TestIPv6AndLatestWins(t *testing.T) {
|
||||
p := newParser(time.Time{})
|
||||
p.handleLine("2026-10-06T12:04:24Z Invalid user a from 2001:db8::1 port 1")
|
||||
p.handleLine("2026-10-06T12:04:26Z Invalid user b from 2001:db8::1 port 2")
|
||||
p.handleLine("2026-10-06T12:04:25Z Invalid user c from 2001:db8::1 port 3")
|
||||
ev := p.events[netip.MustParseAddr("2001:db8::1")]
|
||||
if ev == nil || ev.count != 3 || ev.user != "b" {
|
||||
t.Fatalf("unexpected event: %+v", ev)
|
||||
}
|
||||
var out bytes.Buffer
|
||||
if err := writeCSV(&out, p.sorted()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(out.String(), "(3 events)") {
|
||||
t.Errorf("expected event count in comment:\n%s", out.String())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
#!/bin/sh
|
||||
# runit service: sleep INTERVAL seconds (default 1200), then report the
|
||||
# last INTERVAL seconds of sshd brute-force attempts from the logs of
|
||||
# DOCKER_CONTAINER_NAME to AbuseIPDB. runit restarts this script when it
|
||||
# exits, so one invocation is one iteration of the loop.
|
||||
#
|
||||
# Configuration comes from the container environment:
|
||||
# DOCKER_CONTAINER_NAME container whose logs are read (required)
|
||||
# IPABUSEDB_API_KEY AbuseIPDB API key (required)
|
||||
# INTERVAL seconds between runs (default 1200)
|
||||
exec 2>&1
|
||||
|
||||
: "${INTERVAL:=1200}"
|
||||
|
||||
for var in DOCKER_CONTAINER_NAME IPABUSEDB_API_KEY; do
|
||||
eval "val=\${$var:-}"
|
||||
if [ -z "$val" ]; then
|
||||
echo "ipabusedb-docker-reporter: $var is not set in the environment" >&2
|
||||
# runit would otherwise retry every second and flood the log.
|
||||
sleep 60
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# Sleep in the background so that SIGTERM from "sv stop" (which runsvinit
|
||||
# sends on "docker stop") interrupts the wait instead of orphaning sleep.
|
||||
trap 'kill "$sleeper" 2>/dev/null; exit 0' TERM INT
|
||||
sleep "$INTERVAL" &
|
||||
sleeper=$!
|
||||
wait "$sleeper"
|
||||
trap - TERM INT
|
||||
|
||||
exec /usr/local/bin/ipabusedb-docker-reporter --since "${INTERVAL}s" --submit
|
||||
Reference in new issue
Block a user