Skip to content

CLAUDE.md

Latest commit

 

History

History
148 lines (113 loc) · 6.07 KB

CLAUDE.md

File metadata and controls

148 lines (113 loc) · 6.07 KB
Raw

entropiaR — AI Knowledge Base

R package providing a tidyverse-style read-only interface to EntropIA SQLite databases.

Build / Test / Check

# Load package in session
Rscript -e "devtools::load_all()"

# Run tests
Rscript -e "devtools::test()"

# Full check (must be 0 errors / 0 warnings)
Rscript -e "devtools::check()"

# Lint
Rscript -e "lintr::lint_package()"

# Style check (verification only, no auto-commit)
Rscript -e "styler::style_pkg(dry = 'fail')"

# Spelling
Rscript -e "spelling::spell_check_package(vignettes = TRUE)"

# Coverage
Rscript -e "covr::package_coverage(type = 'tests')"

# Build site
Rscript -e "pkgdown::build_site()"

Windows: R is not on PATH. Use full path: "C:\Program Files\R\R-4.5.2\bin\Rscript.exe" -e "..."

Architecture

Layering

entropia_connect() → tbl_sql (lazy, SQL on SQLite)
    ↓
entropia_collect() → tibble (datetime/JSON/BLOB contract applied)
    ↓
Analysis helpers (on tibbles) → ggplot2 viz
    ↓
entropia_export() → csv/tsv/json/rds/parquet/arrow

Every accessor returns a lazy tbl_sql — no collect() inside the package except in explicit collect/export/analysis surfaces.

Module layout (one file per concern, ~14 source files)

File Concern
R/connect.R entropia_connect, S4 entropia_conn subclass, entropia_copy, disconnect
R/schema.R Manifest loader, version detection, compat policy, schema_info/schema_compat
R/validate.R entropia_validate, entropia_status
R/tables.R Entity accessors (one short fn per table, 18 total)
R/corpus.R entropia_corpus, entropia_text, entropia_metadata
R/quality.R OCR/metadata coverage, corpus_quality, entropia_orphans
R/research.R entropia_search, entity_relations, reconstruct_analysis, conversations
R/collect.R entropia_collect, datetime helpers, JSON/BLOB typing
R/analysis.R Temporal/entity/topic/collection analysis + analysis_dataset
R/plot.R ggplot2 helpers (entropia_plot_*, Suggests-guarded)
R/export.R entropia_export (chunked streaming), provenance I/O
R/write.R v2 write stubs (entropia_error_write_disabled)
R/sql.R FTS5 MATCH builder, recursive-CTE marker stripping
R/sync.R entropia_sync_info, sync_versions, conflicts
R/utils.R ent_* internals: error helpers, content hash, sanitize
R/zzz.R .onLoad: package options defaults

Object model

  • S4 entropia_conn subclass of SQLiteConnection (via methods package). Prepending a plain S3 class would break S4 dispatch on DBI generics, so setClass("entropia_conn", contains = "SQLiteConnection") is used.
  • Accessor results are plain tbl_sql (no subclass — every dplyr verb just works).
  • Analysis datasets get entropia_dataset class + entropia_prov attribute.
  • No tibble subclasses, no vctrs types, no R6.

Naming conventions

  • Public: entropia_* (60 exports)
  • Internal: ent_* (e.g. ent_abort, ent_manifest, ent_tbl)
  • Test helpers: expect_entropia_* (inline in tests, not a separate file)
  • Error classes: entropia_error_<kind> (stable, cli-formatted via ent_abort)

Error model

All errors go through ent_abort() (R/utils.R) which wraps rlang::abort with stable condition classes and cli formatting. No bare stop() or warning() in R/. Known classes: entropia_error_not_found, entropia_error_not_sqlite, entropia_error_schema_incompatible, entropia_error_locked, entropia_error_table_missing, entropia_error_column_missing, entropia_error_write_disabled, entropia_error_invalid_argument, entropia_error_missing_dependency, entropia_warn_malformed_json, entropia_error_invalid_connection, entropia_error_unsupported, entropia_error_copy_failed, entropia_error_dest_exists, entropia_error_manifest_missing, entropia_warn_schema, entropia_error_sql_fragment_missing.

Schema compatibility

  • On connect: read _migrations, version = MAX(name).
  • inst/schemas/manifest.json: column contract per table, min_version tagged.
  • Policy: options(entropiaR.schema_policy = "warn"|"error"|"allow") (default "warn").
  • Live PRAGMA table_xinfo re-check on every table (never trust the manifest alone).
  • ent_manifest_required_gated() for tables whose columns span migrations (entities, assets, llm_results, vec_assets, rag_chunks).

Package options (set in .onLoad, R/zzz.R)

  • entropiaR.schema_policy — "warn" (default), "error", "allow"
  • entropiaR.write_dry_run — TRUE (v2 guard, not yet active in v1)

Fixtures

  • Generated by data-raw/make_fixtures.R into tests/testthat/fixtures/
  • Gitignored — regenerated in CI before tests run (see R-CMD-check.yaml)
  • Six variants: mini, full, legacy-pre0019, legacy-seconds, unknown-version, corrupt
  • Each test receives a temp copy (via ent_fixture() in helper-fixtures.R) — tests never mutate the originals
  • Never use data-test/entropia.sqlite as a test fixture (reference only)

Key domain facts

  • Mixed timestamp units: most columns are epoch ms (13 digits); _migrations.applied_at is epoch seconds (10 digits); entities.created_at and triples.created_at use datetime_auto (magnitude guard).
  • JSON in TEXT columns: items.metadata, transcriptions.segments, layouts.regions/blocks, rag_messages.sources, llm_results.result.
  • Conceptual FKs with no physical constraint (detected by entropia_orphans).
  • Contentless FTS5: fts_items requires JOIN items ON items.rowid = fts_items.rowid.
  • Embedding BLOBs: raw little-endian f32, excluded from default selects.
  • 81 triggers guard integrity (48 sync + 33 collection_activity).
  • Soft-deleted entities: source = 'manual_deleted', excluded by default.

Dependencies

Imports: DBI, RSQLite, dbplyr, dplyr, tibble, rlang, cli, lifecycle, jsonlite, tidyselect, digest, methods.

Suggests: testthat, ggplot2, tidyr, lubridate, forcats, arrow, bit64, duckdb, knitr, rmarkdown, pkgdown, covr, lintr, styler, spelling, withr, vdiffr.

Minimum R ≥ 4.1. CI tests release/oldrel/devel × 3 OS.