Skip to content

belarusian/anonize-ts

Repository files navigation

anonize-ts

TypeScript version of anonize using Effect and @opencode-harness/llm.

Overview

This is a document anonymization tool that identifies and replaces PII (Personally Identifiable Information) using an LLM with deterministic pseudonyms.

Features

  • Two anonymization modes:

    • --fake (default): Generates realistic fake PII that looks genuine
    • --mask: Uses [TYPE:hash] placeholders (robotic, deterministic)
  • PII types detected: name, email, phone, url, location, company, age, address, other

  • Architecture:

    • Uses Effect library for type-safe async operations
    • Uses @opencode-harness/llm for LLM integration
    • Direct JSON output with schema validation
    • Deterministic pseudonym generation

Installation

cd anonize-ts
npm install
npm run build

Usage

# Anonymize a single file with fake PII (default)
anonize email.txt

# Anonymize with mask mode (robotic)
anonize email.txt --mask

# Anonymize multiple files
anonize email1.txt email2.txt email3.txt

# Anonymize directory
anonize ./emails/

# Custom output directory
anonize ./emails/ -o ./anonimized/

# Dry run (preview without writing)
anonize email.txt --dry-run

# Custom LLM endpoint
LLM_URL=http://localhost:8080/v1 LLM_MODEL=gpt-oss anonize email.txt

Configuration

Env var Default Description
LLM_URL http://10.106.1.89:8080/v1 OpenAI-compatible endpoint
LLM_MODEL gpt-oss Model name
LLM_API_KEY "" API key (empty for local)
LLM_MAX_TOKENS 32768 Max output tokens (auto-scaled on truncation)
LLM_TEMPERATURE 0.1 Generation temperature

Architecture

anonize-ts/
├── src/
│   ├── config.ts        # LLMConfig dataclass + PiiMode enum
│   ├── prompts.ts       # System prompts for fake/mask modes
│   ├── schema.ts        # JSON schema for LLM output structure
│   ├── anonymizer.ts    # Core logic using LLMClient from @opencode-harness/llm
│   ├── cli.ts           # CLI with commander
│   └── main.ts          # Entry point

├── tests/ │ ├── resources/ # Test input files │ └── resources_anon/ # Anonymized versions └── package.json


### Key Design Decisions

1. **Effect Library**: All async operations use Effect for type-safe error handling and composition
2. **@opencode-harness/llm**: Uses `LLMClient` service with `generateObject` for structured output
3. **Service Pattern**: Effect v4 `Context.Service` for dependency injection
4. **Layer Pattern**: `Layer.succeed` for providing LLMClient to effects
5. **Direct JSON Output**: Uses `response_format: { type: "json_object" }` for reliable JSON parsing

## Development

```bash
# Build
pnpm build

# Run CLI
pnpm start <file> [--fake|--mask]

# Dev mode (watch)
pnpm run dev

License

MIT

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors