Skip to content

feat(view): surface classifier reasoning instead of an 'AI slop' verdict#133

Open
realitymatrix wants to merge 1 commit into
Hacksore:mainfrom
realitymatrix:feat/surface-ai-reasoning
Open

feat(view): surface classifier reasoning instead of an 'AI slop' verdict#133
realitymatrix wants to merge 1 commit into
Hacksore:mainfrom
realitymatrix:feat/surface-ai-reasoning

Conversation

@realitymatrix

Copy link
Copy Markdown

Fixes #132.

Problem

The /view page renders a quality verdict — "This product is some form of AI slop and you should probably not use it" — whenever hasAi is true. But the classifier (prompt.txt) only ever sets isAiRelated on a keyword match; it never assesses quality. The reasoning it produces (which terms it matched) was logged at ingest and then discarded. So the UI asserted a judgment the model never made — a false positive on legitimate AI products.

Change

Surface the model's actual reasoning instead of the verdict (option 1 from the issue; no reports/appeal flow, per your comment):

  • schema.prisma — add reasoning String? to Post (nullable; existing rows unaffected).
  • ai-analyzer.tsanalyzePosts now returns { isAiRelated, reasoning }; the API-error and parse-error fallbacks write an honest "Analysis unavailable…" reason instead of silently implying slop.
  • ingest-posts/route.ts — persist both hasAi and reasoning.
  • view/[slug]/page.client.tsx — heading becomes "Flagged as AI-related" followed by the model's reasoning, with a neutral fallback for rows analyzed before this field existed.

Deploy note

Needs pnpm db:push (nullable column, non-destructive). Old rows show the fallback copy until re-ingested; new launches get the reasoning immediately.

Verification

  • tsc --noEmit: clean
  • biome check src: clean

(next build not run here — it evaluates @/app/env at build and needs DATABASE_URL/GEMINI_API_KEY; types and lint are covered above.)

…rdict

The /view page asserted a quality judgment ("some form of AI slop ...
you should probably not use it"), but the classifier only sets
isAiRelated on a keyword match and never evaluates quality. Its
reasoning string was computed at ingest and then discarded.

- persist `reasoning` (the model's own explanation) on Post
- analyzePosts now returns { isAiRelated, reasoning }; ingest stores both
- /view shows "Flagged as AI-related" + the model's reasoning, with an
  honest fallback for rows analyzed before the field existed

No reporting/appeal flow, per maintainer. `reasoning` is nullable, so
existing rows are unaffected; run `pnpm db:push` to apply the schema.

Closes Hacksore#132

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 16, 2026

Copy link
Copy Markdown

@realitymatrix is attempting to deploy a commit to the Hacksore's Team Team on Vercel.

A member of the Team first needs to authorize it.

@realitymatrix

Copy link
Copy Markdown
Author

Tested this locally end-to-end before handing it over, in case it saves you a review pass. Reproduced the deploy against a throwaway Postgres (prisma db push → seed → next buildnext start) and exercised both render paths.

The migration is a safe additive change — the generated DDL is just:

ALTER TABLE "Post" ADD COLUMN "reasoning" TEXT;

Nullable, no default → metadata-only on Postgres (no table rewrite, no blocking lock); existing rows get NULL.

check result
prisma db push on real Postgres column created, schema in sync ✓
next build (production) compiled successfully, 10/10 pages ✓
/view/:id for a row with reasoning 200 · "Flagged as AI-related" + the model's reasoning ✓
/view/:id for a row with NULL reasoning 200 · neutral fallback, no crash ✓
old "some form of AI slop" string gone — 0 occurrences ✓
tsc --noEmit / biome check src clean ✓

One path I couldn't exercise live: the ingest cron writing a reasoning string from an actual Gemini call (needs your GEMINI_API_KEY + the PH pipeline). The write is type-checked and the column provably accepts it, so it's low-risk — existing rows just render the fallback until the next cron run repopulates reasoning.

Deploy note: needs pnpm db:push for the nullable column above (non-destructive).

Happy to tweak the warning-block copy if you'd word it differently — trivial change.

@vercel

vercel Bot commented Jun 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
oghunt Ready Ready Preview, Comment Jun 17, 2026 12:00am

Request Review

@Hacksore

Hacksore commented Jun 17, 2026

Copy link
Copy Markdown
Owner

I was looking at prod logs and saw it's unable to pull in all the changes so I'd say let me get #136 shipped and prod data loading again.

Then we can get this deployed, thanks for the help here!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

False positive: classifier only detects "AI-related," but the view page labels it "AI slop you should probably not use" (e.g. OpenStrate)

2 participants