WATCHLING In development volenti.ca ↗

Human-gated AI for infrastructure / a Volenti project

Your hosts already know
what is wrong. Watchling asks.
A human still decides.

The evidence for the thing that broke at 03:14 is sitting in a journal nobody has time to read. Watchling reads the machines themselves and turns what it finds into one card per problem: what is wrong, the evidence for it, and a proposed fix with its blast radius. Then it stops. A named person approves or rejects every fix, and that decision is a signed entry on a chain you can read.

Reads your hosts over a hop you own The model never emits a shell command No unattended mode. Not now, not later. Built for sysadmins, not steering committees
A Watchling problem card, opened. On the left, the PROBLEM region: host web-01 at 10.20.0.11, the sentence 'The unit entered the failed state at 03:14 and has not recovered. 38 identical failures since', and the verbatim systemd and journal output as evidence, with the collector name, the occurrence count and the finding key. On the right, the RESOLUTION region: a proposal marked 'proposed', a WHY paragraph, a BLAST RADIUS line reading 'one host, one unit. Mail queued during the restart is retained, not dropped. Reversible', a risk class of elevated over one system, and PRIOR CONTEXT from operational memory. Below them, an Approve this fix and a Reject this fix button, a triage note box, further dispositions of postpone, ignore and escalate, and a Discuss this button.
The product, with invented hosts One problem, one card. What is wrong and the evidence for it on the left; the proposed fix, why it is proposed, what it would touch and what the system already remembers about this host on the right. Approving decides the fix. Whether the problem itself is real, postponed or ignored is a separate decision, at the foot of the card, and it is signed too.

Features

Six things it does, before any of the detail.

01 / Expose

It reads the machine, not a dashboard

Collectors run on the host itself: journal errors, the live nftables ruleset, failed units, and pending updates with security advisories called out separately.

02 / Resolve

One card per problem

The evidence, a proposed fix, what it would touch and its risk class. A typed action, never a generated shell command.

03 / Record

Every decision is signed

Append-only and hash-chained, naming who decided what. The same object an auditor asks for six months later.

04 / Improve

A finding closes on evidence

Not when somebody clicks done. It closes when the host is read again and the condition is actually gone.

05 / Cortex memory

It remembers this host

What was learned before is read before anything is proposed, and a memory it cannot reach refuses the scan rather than deciding blind.

06 / Your own model

Your telemetry stays yours

Point it at any OpenAI-compatible endpoint you run. Nothing has to leave your estate, and no vendor holds your inference.

The premise

An LLM is genuinely useful here. It is also wrong often enough that you cannot hand it your estate.

Both halves of that are true, and most of what gets sold as AIOps only believes the first one. Watchling is built around the second.

A model is very good at reading ten thousand lines of log and telling you which forty matter. It is also confidently wrong, and has no idea which box is load-bearing at 2am. So it does not get a shell.

It proposes a typed action from a catalog you own, and anything outside that catalog is not blocked, it is inexpressible — because denylisting does not work. rm -rf / is easy to catch; a base64 heredoc, a two line Python script or find / -delete all walk straight through. You cannot denylist a Turing-complete input.

The mechanism

One loop, and a human sits inside it.

Not a funnel. This is the order the code runs in, and the label on each arrow is what actually travels along it.

The Watchling loop A six-step cycle. Observe: collectors run on the host itself, reading the systemd journal, the nftables ruleset, failed units and pending updates. An arrow labelled "sealed notation, hashed and signed" leads to Finding: one condition, keyed on its own identity, carrying the evidence verbatim. An arrow labelled "prior notes read first, fail closed" leads to Proposal: a typed action from the catalog, with the reason, the blast radius and the risk class. An arrow labelled "nothing has run yet" leads to Decision: a named human approves or rejects, per system, never in a batch. An arrow labelled "the approval is bound to the state it was granted against" leads to Execution: the inventory and monitoring are re-checked at run time, and the action either runs or refuses by name. An arrow labelled "exit status, or a named refusal" leads to Record: an append-only, hash-chained, signed entry plus a memory note bound to it. A dashed arrow returns to Observe, labelled "re-collect: a finding closes only when the host is read again and the condition is gone". 01 / OBSERVE On the host itself The journal at priority err, the live nftables ruleset, failed units, the packages pending. sealed notation, hashed and signed 02 / FINDING One condition, keyed to itself The evidence verbatim. Re-observed, it keeps its id; unobserved, it goes stale, not clean. prior notes read first, fail closed 03 / PROPOSAL A typed action, not a command From the catalog you own, with the reason, the blast radius and the risk class attached. nothing has run yet 04 / DECISION A named human Approve or reject, per system, never as a fleet-wide batch. Rejection carries a reason. the approval is bound to the state it was granted against 05 / EXECUTION Re-checked, then run or refused Inventory and monitoring are read again at run time. Moved state voids the approval. exit status, or a named refusal 06 / RECORD Append-only, signed, readable The chain entry names who decided what, and a memory note is bound to that entry. re-collect: a finding closes only when the host is read again and the condition is gone

Nothing in that loop is a report you have to go and read. The observation is the evidence, the proposal cites it, the decision is yours, and the record of the decision is the same object an auditor would ask for six months later.

A silence is not a clean bill. If a collector could not look — no permission, host unreachable, package manager disabled — the finding goes stale, carrying a sentence naming why nobody could check. It never quietly becomes "resolved". An empty list is one of four different facts, and the screen says which one.

Cortex memory

It remembers, and it refuses to decide blind.

A tool that forgets every run proposes the same failed fix forever. Memory here is not a log you can go and read; it is consulted before the system is allowed to decide.

Read before the decision

What do we already know about this host

Before a pass starts it asks: when the last one ran, what it saw, which drift is still carried forward, which patterns recur.

The asymmetry that matters

Absence is an answer. Failure is not.

A first pass has nothing to remember, and that is a valid result. But a memory store that is present and errors refuses the scan - deciding while blind to memory you were meant to read is the exact failure this exists to prevent.

Intent before results

The reason cannot be retro-fitted

What a pass sought, and under whose authorization, is written down before anyone sees a result - so nobody can quietly fit the reason to the finding afterwards.

Bound to the chain

The why rides with the what

The ledger records what an approval did. A memory note is bound to that same entry, carrying the human rationale beside it.

A worked example

Tuesday, 09:14. Five things are open and one of them matters.

The same estate the card at the top of this page came from. Hosts and names invented; the screens, the classes, the risk classes and the chain are the product's own.

The Watchling Problems board. A header reads 'Approve, deny, postpone, ignore or escalate. Every decision is a signed entry on the tenant chain naming the deciding human', followed by '5 cards, 5 still pending for you. Count equals the list on this poll.' A table counts findings by class, worst first: Exposure critical, one finding on one host; Service failure high, one finding on one host; Journal errors medium, one finding on one host; Package updates low, two findings on two hosts. Below, filter chips per class, then the five cards worst first: a critical exposure on 10.20.0.1, a high service failure on 10.20.0.11, a medium journal error on 10.20.0.12, and two low package-update cards.
The board Worst first, and the count is the list — on the same poll, by construction, so the number at the top can never disagree with the rows underneath it.
  1. Collection ran over your jumpbox at 08:40 and read the enrolled hosts. One machine would not answer; it is named on its own row with the hop that failed, rather than silently dropped from the count.
  2. web-01 reports postfix.service failed, 38 times since 03:14, with the journal lines attached verbatim. The finding is keyed on the unit's own name, so when the same condition is seen again tomorrow it is the same finding, not a new one.
  3. The AI reads its prior notes on web-01 first. Two of them name the same relay. That is on the card as prior context, in the model's own words — and if the memory store had been unreachable, the card would have said so instead of implying there was no history.
  4. It proposes agent.service.restart — a catalog entry, not a command line — on that one host, that one unit. Risk class elevated. The blast radius says what it touches and that it is reversible by agent.service.start. It does not touch the relay, which is a separate finding on db-02.
  5. You decide. Approve, reject with a reason, or open Discuss and argue with it about the finding first. Separately, at the foot of the card, you can mark the problem a false positive, known-and-accepted, postponed, ignored or escalated — ignoring requires an explanation and is itself signed.
  6. The decision lands on the chain naming you, before anything else happens. At execution the inventory and monitoring checks run again; if web-01 entered maintenance in the meantime the approval is void and it comes back to you.
  7. Honestly, today: a restart is proposable and approvable, and on a deployment with no execution runner wired it refuses at execution rather than reporting a success nobody performed. An approved re-collection, by contrast, runs end to end on a real host right now — and if the unit is genuinely back, the finding closes on that evidence.
the read-only half, from the CLI
# nothing is probed until a scope is signed. the scanner verifies that
# signature against the scope's own stored contents on every run.
$ watchling scope sign --label northfield --cidr 10.20.0.0/24 \
      --signer d.okoye --id northfield
ed25519 keypair generated . authorization northfield signed . ledger genesis

$ watchling scan
targets default to the signed CIDRs . out-of-scope targets are refused
and the refusal is written down before the error surfaces

$ watchling walk <scan-id>
(a)ccept  (e)dit  (r)eject  (s)kip  (q)uit        resumable after kill -9
  NEW    10.20.0.13  app-03     not in inventory        > a
  DRIFT  10.20.0.11  web-01     primary_ip disagrees    > a
  GHOST  10.20.0.44  old-build  in inventory, not found > flagged, never deleted

$ watchling violations
every scope refusal, logged against the authorization that forbade it
Accepting a candidate documents it. It does not enrol it, and it does not make it executable.

How it gets in

You are the operator. You pick the way in, and you can take it back.

Two pathways. Neither of them requires you to expose anything to the internet, and neither of them is chosen for you.

Pathway A

A jumpbox over WireGuard

One entrance, on a host you own, joined over WireGuard. Watchling comes in through that hop and nothing else in your estate has to be reachable. You decide where it sits, what it can see, and when it stops existing.

Pathway B

Endpoint enrollment

Enroll a host and it joins on its own tunnel, with no inbound access to it at all. Per machine, individually, and never implied by a scan: documenting a system is not enrolling it.

The Connect endpoints table. Four rows: web-01 and db-02 at 10.20.0.11 and 10.20.0.12 are connected via jumpbox with a green 'agent connected' badge; app-03 at 10.20.0.13 is connected via jumpbox with an orange 'read path only' badge explaining that no agent record exists but the jumpbox reached it over ssh, offering Retry Agent, Retry SSH and Disconnect; win-rdp-01 at 10.20.0.21 is unenrolled with a red 'unreachable' badge naming Permission denied publickey and a missing seed on the host. A closing line reads that endpoints left unenrolled are named on their own row with the hop that failed, and are not silently skipped.
Connecting hosts A host that will not connect is named, with the hop that failed and the next thing to try. Green is the agent, orange is the read path only, red is neither. Nothing is silently skipped, because a shorter list is how a fleet gets quietly smaller than it really is — and a host your signed scope excludes says which exclusion excluded it.
The one part that is not a choice

Plugins always come over the jumpbox. Always, whichever pathway your hosts use.

Your monitoring, your wiki, your source of truth and your automation are internal systems. They should not be exposed to anything, us included. They are also nothing like each other: an OPNsense box is not a TrueNAS box is not a vCenter, and none of them is ever going to run our agent. So every plugin is reached through the single hop you control, normalized, in the open, and never opened up one connection at a time.

Quietly reaching under the hood of your infrastructure is not where we help, and it is not a thing we want to be able to do.

Plugins

Bring what you already run. Or bring nothing.

The source of truth is the host, so the core never depends on any of these being present. Connect one and it gets richer, in that tool's own language. Here is exactly what each one does today — not what it might do later.

NetBox logo Source of truth NetBox

Reads your devices as intent and diffs them against what the scan actually saw. An observed field disagreeing is a correction you walk through one at a time; an intent field disagreeing is a finding, never a silent update. Writing back is a separate step with its own gate.

Zabbix logo Monitoring Zabbix

Reads host state and open problems, and fails closed: if Zabbix cannot answer, the state is unknown, never no problems. Where an unacknowledged event matches a finding on your host it proposes acknowledging it — that is a write, so it opens an approval.

Nagios logo Monitoring Nagios

Reads host and service state. Against an instance with a configuration API it also scans the config and names what has drifted — a host absent, an address wrong, a service never applied, a group membership gone. Each one is a single proposed write, approved on its own, with the paging consequence stated before you accept.

MediaWiki logo Documentation MediaWiki

Where the runbooks land. Every claim in a draft must cite evidence that resolves or it is stripped and counted; the absences are computed from the collection window, so thin history reads as thin history. Nothing publishes without a recorded human approver, and the publish is on the chain before the page is written.

Ansible logo Automation Ansible®

Asks your controller's own Ansible which config, playbook directory and inventory it is really using. From a finding it recommends, in plain words, the handler or task that would address it, and can surface a reviewed Galaxy playbook with a content hash to check it against. It does not write playbooks for you, and running one is escalation-class.

Anything else Your own

A plugin declares what it can read, what it can suggest and what it can write, and the registry refuses to register a write that is not human-approved. That is enforced where plugins are loaded, not left to the plugin author's judgement.

The Plugin suggestions screen. A header explains that each enabled plugin proposes from the host-native findings on this tenant, grouped under the plugin that produced it and waiting for a decision, and that every suggestion cites the finding ids the host itself produced. A status block shows five host findings, enrichment available with four of four enabled plugins answered, and a per-plugin line: inventory.netbox consulted with no suggest capability declared, monitoring.zabbix consulted answered host.get, integration.mediawiki consulted with no suggest capability declared, automation.ansible consulted, ansible-playbook available. Below, grouped rows: NetBox zero awaiting approval, Zabbix one awaiting approval to acknowledge the matching problem on web-01, MediaWiki zero awaiting approval with the sentence 'MediaWiki was consulted in this assessment and had no grounded addition. That is a result, not a plugin that went unasked', and Ansible one awaiting approval to restart and verify postfix on web-01 through a reviewable handler.
Enrichment, never a prerequisite Every suggestion cites the finding the host itself produced; one that cannot is refused and recorded, never shown as insight. A plugin that was asked and had nothing to add says so — because "nothing here" and "we never asked" are different facts, and a screen that renders them the same way is lying to you.

Start at zero. The host findings are read before any plugin is consulted and returned regardless, so with nothing connected the assessment is the host's own and it is complete. A plugin that is switched off, unconfigured or unreachable becomes a sentence naming the hop that failed — not a spinner, not a blank panel, and never a silent gap in the count. A plugin is never a requirement, and a failing plugin is a sentence, not a broken screen.

Your model / your metal

Nothing about your estate has to leave it.

If your telemetry cannot go to a vendor, this is usually where an evaluation stops. So it is not an enterprise upgrade and it is not a roadmap item: it is how the product is configured on day one.

Any endpoint

OpenAI-compatible, wherever you run it

Your own hardware, your own tenancy, your own network. You choose the model, the upgrade schedule and what it costs you.

The model list is real

Populated by asking your endpoint

The dropdown comes from a live call to your server, never from a guessed list of what we imagine you are running.

The test is real

One real completion, shown verbatim

Setup fires an actual request and prints the upstream's own answer. A canned response presented as proof is worse than no test at all.

The key is handled like a key

0600, redacted everywhere, written last

Registered as a redaction boundary so it cannot surface on any screen or in the ledger, and never written to disk until there is a signed entry to hang it on.

Security

The boundary is what the model can express, not what we hope it will not do.

Most AI tooling asks whether it can stop a model doing something dangerous. This one is built so the dangerous thing cannot be expressed in the first place.

No generated shell

The model never emits a command

Actions are typed entries in a catalogue with declared parameters. Execution passes argv directly - it never spawns a shell, so there is no string for an injection to live in.

Signed scope

We ask before we look

A signed document names the ranges, the exclusions and the expiry. The scanner verifies that signature against the scope's own contents on every run, so widening it behind the software's back makes it refuse to start - and write down that it refused.

Approval bound to state

An approval can go stale, and then it is void

Approved at 14:00 for a host that entered maintenance at 14:20 is not an approval any more. It is re-validated at execution time and refused, rather than acting on a world that has moved.

Nothing unattended

A named human, every time

There is no autonomous mode to enable, no schedule that approves on your behalf, and Watchling cannot approve its own work.

✗It does not run unattended. Not now, not later.
✗It cannot approve its own work.
✗It cannot look outside your signed scope.
✗It cannot act on an approval that no longer matches the machine.

The lines we keep

This is not a tool for having fewer administrators.

We are not asking you to believe that. Most of what follows is enforced by the software or by a test that fails the build, and the rest is checkable against how we charge.

Enforced by a test

There is no unattended mode

No flag, no environment variable, no API parameter and no database column moves a consequential action to automatic. A test enumerates every configuration surface and fails the build if one appears. We expect to be asked. The answer is no.

Enforced by the software

Approval fan-out is bounded

One person approving five thousand actions is ten administrators' throughput routed through one. Approvals are rate limited per person and consequential actions are approved per system.

Enforced by a static check

No per-person metrics. Ever.

Who approved something is a forensic field, never a grouping dimension. No leaderboards, no ratios, no dashboard that ranks your staff by how fast they click approve. The build fails on a person column in a GROUP BY.

By design

The ledger belongs to the people in it

Readable by the administrator whose work it records, not held above them for review. An audit trail only management can read is surveillance with better branding. Nothing is rewritten: a correction is a new signed entry, and both remain.

Deliberately absent

We collect nothing about your staff

No HR integrations, no personnel-event correlation, no termination keyword detection. Not as a feature, not as telemetry, not as a configuration hook - and named here so its later arrival would be a visible act.

Checkable against the invoice

Priced per approving human

Not per managed system. Per-node pricing earns the same whether a team has twenty people or two, which makes the substitution arithmetic trivial to write. Our revenue shrinks if you cut your team.

Any tool that reduces the hours needed to run infrastructure can end up cited in somebody's layoff justification, and a company that swears otherwise is lying to you. What we can do is refuse to build the features that make it easy, put those refusals in the source tree where a test enforces them, and state them publicly at version one. The full policy, including what would break it, ships in the repository as HUMANITY.md.

Where it actually is

In development. Here is the honest state of it.

A limitation dressed as a feature is a lie with better lighting. So, plainly:

Working
  • Host-native collection on real hosts. Journal errors, the nftables ruleset, failed units, and pending updates with security advisories separated out. Three further collectors are declared and not yet implemented — they report inert, which is a thing that did not look, and can never add up to "clean".
  • The two lenses and the problem card. Problems and Security as separate screens; the card with its evidence, its proposed fix, the blast radius, the risk class and prior context; approve or reject the fix, and triage, postpone, ignore (signed) or escalate the finding. Discussing a finding with the model works and is deliberately ephemeral — nothing is stored but the decision you take afterwards.
  • The trust substrate, which is the part that took the time: the signed hash-chained ledger local and central, the typed action catalog, the approval gate and its re-validation, signed scope enforcement, tenancy and consent, and operational memory that is read before every proposal and fails closed when it cannot be.
  • Discovery end to end, with a walkthrough that survives a lost connection, and runbook drafting into MediaWiki that publishes only on a recorded human approval.
  • Approved re-collection runs end to end on a real host today, and can close a finding on the evidence that the condition is gone.
Not yet

Consequential fixes are proposable and approvable, and they do not yet run. Restarting a unit, applying the pending updates, narrowing a firewall rule: all of these go through the catalog, the gate and the chain exactly as described on this page, and then on a deployment with no execution runner wired they refuse at execution — by name, before any receipt is written. That is deliberate. An earlier build minted signed receipts saying exit 0 for work no host had been asked to do, and a signed lie is worse than a refusal.

The managed-system agent exists, enrolls against a real certificate plane and holds a closed set of semantic verbs; dispatching an approved action to it is not wired yet. Timed capture and re-evaluation are built and scheduled, and need packet-capture privileges on the host. There is no packaged release, no installer, no hosted version, no pricing page and no signup.

Not ever

Unattended execution of consequential actions. Metrics that count what individual people approved. Any collection of data about your staff. These are refusals, named now, before there is money on the table to change our minds.

Get in touch

If you run this stuff for a living, we want to hear from you.

We would rather build the rest of this against real estates than imagined ones. Tell us what you run, what wakes you up, and what would have to be true before you would let a model anywhere near it. If you think we have the balance wrong, that is the most useful message you could send. Every one is read by a person.

hello@volenti.ca

There is no signup form here, no scripts, no cookies and no third-party tracking of any kind. Our own web server keeps an access log, as every web server does, and it records which pages and links were requested; it rolls every fourteen days and we read it in aggregate. When there is something to try, this is where it will be announced.
Security disclosure: security@volenti.ca