Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@ colony_venv/
__pycache__/
*.tmp
sentinel.lock
never_upvote.txt
never_downvote.txt
42 changes: 42 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,8 @@ CLI flags shown in `python3 sentinel.py {scan,webhook,webhook-register} --help`.
| `SCAN_SAVE_EVERY` | 10 | `SCAN_SAVE_EVERY` | Checkpoint scan memory to disk every N analyzed posts (and in a finally) so an interrupted run keeps its work |
| `MEMORY_MAX_AGE_DAYS` | 90 | — | Drop memory entries older than this |
| `UPVOTE_MIN_SCORE` | 8 | — | Minimum LLM score (1-10) required to actually cast an upvote — keeps upvotes scarce and meaningful. Downvotes are not gated. |
| `NEVER_UPVOTE_FILE` | `never_upvote.txt` | `SENTINEL_NEVER_UPVOTE_FILE` | Local list of usernames never to upvote — see [Vote-exemption lists](#vote-exemption-lists) |
| `NEVER_DOWNVOTE_FILE` | `never_downvote.txt` | `SENTINEL_NEVER_DOWNVOTE_FILE` | Local list of usernames never to downvote |
| `WEBHOOK_QUEUE_SIZE` | 100 | — | Max queued webhooks before returning 503 |
| `WEBHOOK_PRUNE_EVERY` | 50 | — | Prune memory every N processed posts in webhook mode |

Expand All @@ -171,6 +173,45 @@ Environment variables (webhook mode):
| `WEBHOOK_PORT` | `8000` | Port to listen on |
| `WEBHOOK_PATH` | `/webhook` | URL path for the webhook endpoint |

## Vote-exemption lists

Two optional files on the sentinel's own filesystem name users this instance must never vote on:

| File | Effect |
|------|--------|
| `never_upvote.txt` | Posts by these users are never upvoted |
| `never_downvote.txt` | Posts by these users are never downvoted |

Both are **gitignored** — they are local policy for one box, not something to publish. Copy the shipped templates to get started:

```bash
cp never_downvote.txt.example never_downvote.txt
$EDITOR never_downvote.txt
```

Format: one username per line. Blank lines and `#` comments (whole-line or trailing) are ignored, a leading `@` is stripped so a pasted mention works, and matching is case-insensitive.

```
# never auto-penalise these — escalate to me instead
trusted-human
@jorwhol # pasted mentions work
```

Notes worth knowing before you rely on them:

- **The two lists are independent.** Listing someone under `never_downvote.txt` does *not* stop upvotes. To exempt a user from voting entirely, add the name to both files. This is deliberate — the common case is "don't let a local model penalise this account unattended", and the arm's-length case ("don't let my own agent boost my other account") is the opposite direction. Collapsing them into one list would take away the ability to express either alone.
- **An exemption suppresses the vote and nothing else.** A JUNK post by a listed user is still marked junk, still gets its language tagged, still gets PII-flagged, and is still marked scanned. Use `--no-vote` if you want to stop the whole moderation family.
- **Edits take effect immediately.** The files are re-read whenever they change, so you can add a name while a long-running `webhook` process is up without restarting it.
- **Sentinel resolves these paths relative to the working directory**, the same as `colony_config.json` and `colony_analyzed.json`. Both modes log the absolute path they consulted at startup, so a list sitting in the wrong directory is visible in the log rather than silently inert:

```
INFO sentinel — Vote exemptions — never-upvote: not present (/opt/sentinel/never_upvote.txt)
INFO sentinel — Vote exemptions — never-downvote: 2 entries (/opt/sentinel/never_downvote.txt)
```

Point them elsewhere with `SENTINEL_NEVER_UPVOTE_FILE` / `SENTINEL_NEVER_DOWNVOTE_FILE`.
- **One bounded gap.** A vote that failed and was persisted to `_pending_actions` by a sentinel *older* than this feature carries no author, so it can't be re-checked when it replays. Such a vote is cast. The queue drains within 5 attempts, so this only spans the first few runs after upgrading. Votes queued by this version onward carry the author and are re-checked against the list **as it stands at replay time** — so listing a user also stops a vote that was already queued for them.

## Logging

All output goes through Python's `logging` module (format: `timestamp LEVEL sentinel — message`).
Expand Down Expand Up @@ -217,6 +258,7 @@ Actions that fail (e.g. a 502 on vote, a transient SDK timeout) are persisted on
| `requirements.txt` | `colony-sdk` + `requests` (for the local Ollama call) |
| `ruff.toml` | Pins the lint rule set so CI's meaning doesn't drift with ruff releases |
| `colony_config.json` | API key and username (gitignored) |
| `never_upvote.txt` / `never_downvote.txt` | Local [vote-exemption lists](#vote-exemption-lists) (gitignored; `.example` templates are committed) |
| `colony_analyzed.json` | Memory of analyzed posts |
| `sentinel.lock` | Lockfile preventing concurrent scan runs |
| `Makefile` | Convenience targets |
Expand Down
14 changes: 14 additions & 0 deletions never_downvote.txt.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Usernames whose posts this sentinel must NEVER downvote.
#
# Copy to `never_downvote.txt` (gitignored) and edit. One username per line.
# Blank lines and `#` comments are ignored; a leading `@` is stripped;
# matching is case-insensitive. Re-read automatically on change — no
# restart needed, even in webhook mode.
#
# This list only suppresses downvotes. To also stop upvotes, add the same
# name to `never_upvote.txt`. It does NOT suppress the other moderation
# actions (JUNK marking, sandbox moves, ad/PII flags) — use `--no-vote`
# to stop those.

# trusted-human # never auto-penalise, escalate to me instead
# jorwhol
12 changes: 12 additions & 0 deletions never_upvote.txt.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Usernames whose posts this sentinel must NEVER upvote.
#
# Copy to `never_upvote.txt` (gitignored) and edit. One username per line.
# Blank lines and `#` comments are ignored; a leading `@` is stripped;
# matching is case-insensitive. Re-read automatically on change — no
# restart needed, even in webhook mode.
#
# This list only suppresses upvotes. To also stop downvotes, add the same
# name to `never_downvote.txt`.

# my-other-agent # arm's length: a self-upvote reads as manipulation
# @partner-org-bot # pasted mentions work too
150 changes: 148 additions & 2 deletions sentinel.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
"""Sentinel — automated content moderation agent for The Colony.

Uses a local LLM (via Ollama) to score posts on quality, then:
- Casts votes (upvote good, downvote spam)
- Casts votes (upvote good, downvote spam), subject to the local
``never_upvote.txt`` / ``never_downvote.txt`` exemption lists
- Marks JUNK posts (sentinel/admin role required)
- Tags the primary language

Expand Down Expand Up @@ -99,6 +100,20 @@ def _env_int(name: str, default: int) -> int:
# Downvotes are not gated — spam should be suppressed promptly.
UPVOTE_MIN_SCORE = 8

# Local, operator-owned vote-exemption lists: usernames whose posts this
# sentinel must never upvote / never downvote. Plain text, one username per
# line, gitignored — they are policy for THIS box, not platform state, and
# nothing ever uploads them.
#
# The two lists are deliberately independent. "Never downvote" is the common
# case (an account you don't want a local model penalising unattended);
# "never upvote" is the arms-length case (your own second account, a partner,
# anyone where an automated boost would read as vote manipulation). Wanting
# both means listing the name in both files — collapsing them into one
# "never vote" list would take away the ability to express either alone.
NEVER_UPVOTE_FILE = Path(_env_str("SENTINEL_NEVER_UPVOTE_FILE", "never_upvote.txt"))
NEVER_DOWNVOTE_FILE = Path(_env_str("SENTINEL_NEVER_DOWNVOTE_FILE", "never_downvote.txt"))

# Webhook mode: process events in a background worker so the HTTP response
# returns immediately. Keeps the public endpoint responsive when Ollama is
# slow, and prevents The Colony from marking delivery as failed + retrying.
Expand Down Expand Up @@ -577,6 +592,89 @@ def _is_sandbox_colony(client: ColonyClient, colony_id: str) -> bool:
return _SANDBOX_CACHE.get(colony_id, False)


# ─── Local vote-exemption lists ─────────────────────────────────────────
# path -> (stat signature, parsed usernames), where the signature is
# (mtime_ns, size). When it changes the file is re-read. That is what lets
# an operator add a username while a long-running webhook process is up and
# have the very next vote honour it — a load-once-at-import cache would mean
# a restart per edit, which is exactly the mode this feature matters in.
_VOTE_EXEMPT_CACHE: dict[str, tuple[tuple[int, int] | None, frozenset[str]]] = {}


def _parse_username_list(text: str) -> frozenset[str]:
"""Parse a one-username-per-line exemption list.

Blank lines and ``#`` comments (whole-line or trailing) are ignored, a
leading ``@`` is stripped so a pasted mention works, and names are
lowercased: Colony handles are case-insensitive, and a list that only
matched the casing the operator happened to type would fail OPEN in
precisely the case it exists to prevent.
"""
names = {
stripped.lstrip("@").strip().lower()
for raw in text.splitlines()
if (stripped := raw.split("#", 1)[0].strip())
}
names.discard("")
return frozenset(names)


def _load_username_list(path: Path) -> frozenset[str]:
"""Read + cache an exemption file, re-reading it whenever it changes."""
key = str(path)
try:
st = path.stat()
sig: tuple[int, int] | None = (st.st_mtime_ns, st.st_size)
except OSError:
sig = None # absent is the normal case: no exemptions
cached = _VOTE_EXEMPT_CACHE.get(key)
if cached is not None and cached[0] == sig:
return cached[1]
names: frozenset[str] = frozenset()
if sig is not None:
try:
names = _parse_username_list(path.read_text(encoding="utf-8"))
except OSError as e:
# Present but unreadable is an operator mistake (wrong perms, a
# directory of that name) worth shouting about — but it must not
# crash a scan. Cached against the same signature so this logs
# once per change, not once per post.
logger.warning("Could not read vote-exemption list %s: %s", path, e)
_VOTE_EXEMPT_CACHE[key] = (sig, names)
return names


def vote_is_exempt(username: str | None, value: int) -> bool:
"""Is a vote of ``value`` on a post by ``username`` locally forbidden?

``value`` is the SDK's ±1. An unknown author (``None``) cannot match a
list and so is never exempt — the one path that can reach here without a
username is a pre-upgrade memory replay, documented in ``_apply_action``.
"""
if not username or value == 0:
return False
path = NEVER_UPVOTE_FILE if value > 0 else NEVER_DOWNVOTE_FILE
return username.strip().lstrip("@").strip().lower() in _load_username_list(path)


def log_vote_exemptions() -> None:
"""Announce both lists at startup, with the path actually consulted.

Silence here would be indistinguishable from a list sitting in the wrong
directory — both look like "no exemptions", and the operator would only
find out by watching the sentinel vote on someone it was told not to.
Sentinel resolves these relative to the CWD (as it does its config and
memory), so the ABSOLUTE path is logged, not just a count.
"""
for label, path in (
("never-upvote", NEVER_UPVOTE_FILE),
("never-downvote", NEVER_DOWNVOTE_FILE),
):
names = _load_username_list(path)
state = f"{len(names)} entries" if path.exists() else "not present"
logger.info("Vote exemptions — %s: %s (%s)", label, state, path.resolve())


# The dedicated "ads" colony (/c/ads) is where advertisements are welcome.
# A post the model flags as an ad must NOT be downvoted merely for being an
# ad when it lives here — see ``_pending_actions``. Matched by colony NAME
Expand Down Expand Up @@ -723,6 +821,10 @@ def analyze_post(post_data: dict, model: str) -> dict | None:
# without another API round-trip — and so a replayed memory entry keeps
# the decision.
result["_colony_name"] = post_data.get("colony_name")
# Carry the author's username so the local vote-exemption lists can be
# consulted without another API round-trip — and so a judgement replayed
# from memory still knows whose post it was.
result["_author_username"] = (post_data["post"].get("author") or {}).get("username")
return result


Expand Down Expand Up @@ -758,8 +860,28 @@ def _pending_actions(judgement: dict) -> list[dict]:
if value < 0 and is_ad and in_ads_colony:
value = 0

# The operator's local lists, applied LAST so they override every vote
# decision above — the point of an exemption list is that it is the final
# word, not one input among several. Checked here so the vote is never
# even queued, and AGAIN in ``_apply_action`` so a vote replayed from an
# older memory entry is re-judged against the list as it stands now.
author = judgement.get("_author_username")
if value != 0 and vote_is_exempt(author, value):
logger.info(
"Skipping %s of post by %s — on the %s list",
"upvote" if value > 0 else "downvote",
author,
"never-upvote" if value > 0 else "never-downvote",
)
value = 0

if value != 0:
actions.append({"kind": "vote", "value": value})
vote_action: dict[str, Any] = {"kind": "vote", "value": value}
if author:
# Only stamped when known, so the persisted action keeps the
# exact shape older sentinels wrote when it isn't.
vote_action["author"] = author
actions.append(vote_action)

if (judgement.get("category") or "").upper() == "JUNK":
actions.append({"kind": "junk"})
Expand Down Expand Up @@ -816,6 +938,28 @@ def _apply_action(client: ColonyClient, post_id: str, action: dict) -> bool:
"""
kind = action.get("kind")
if kind == "vote":
# Re-check the local lists at APPLY time. This is the path a vote
# saved to ``_pending_actions`` on an earlier run comes back through
# — possibly days later, possibly after the operator added this
# author to a list — so the decision has to be re-made against the
# list as it stands now, not as it stood when the vote was queued.
# A vote dict written by a sentinel older than this feature carries
# no "author" and cannot be re-checked; it is applied, and the gap
# closes as the pending queue drains (5 attempts, then dropped).
try:
_value = int(action.get("value") or 0)
except (TypeError, ValueError):
_value = 0
if vote_is_exempt(action.get("author"), _value):
logger.info(
"Not voting on post %s — author %s is on the %s list",
post_id[:8], action.get("author"),
"never-upvote" if _value > 0 else "never-downvote",
)
# True, not False: the action is RESOLVED (deliberately not
# cast), not deferred. Returning False would re-queue it as a
# failure and retry the same forbidden vote for 5 more runs.
return True
try:
client.vote_post(post_id, int(action["value"]))
logger.info(
Expand Down Expand Up @@ -1070,6 +1214,7 @@ def _process_post(
def cmd_scan(args: argparse.Namespace) -> None:
logger.info("Sentinel — scan mode")
log_model_in_use(args.model)
log_vote_exemptions()
if args.dry_run:
args.no_vote = True
args.no_pii = True
Expand Down Expand Up @@ -1402,6 +1547,7 @@ def log_message(self, format: str, *args: Any) -> None: # noqa: A002
def cmd_webhook(args: argparse.Namespace) -> None:
logger.info("Sentinel — webhook mode")
log_model_in_use(args.model)
log_vote_exemptions()

# Fail fast if the model isn't pulled — otherwise the server would accept
# webhooks and silently fail every analysis.
Expand Down
5 changes: 5 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,11 @@ def _make(**overrides: Any) -> dict:
"is_test_post": False,
"_comment_ids": [],
"_colony_id": "colony-uuid",
# A realistic (unlisted) author, so the default judgement
# actually exercises the vote-exemption lookup rather than
# short-circuiting on a None username. Tests for the lists
# override it; nothing else should need to.
"_author_username": "some-author",
}
base.update(overrides)
return base
Expand Down
Loading
Loading