Hello fellow wanderer! The Smart Family Pantry Manager is an AI-powered backend program that turns grocery receipts into pantry inventory. All you need to do is snap a photo at checkout, and we extract the items, track expiration, and text you before they go bad.

The core loop: Receipt photo → OpenCV + OCR (EasyOCR w/ Tesseract failover) → parse items → PostgreSQL pantry → hourly Celery jobs → Twilio SMS alerts + webhook delivery tracking.

Architecture

app/
├── db/           Database layer
│   ├── models.py          SQLAlchemy schema
│   ├── database.py        Connection pooling
│   └── shelf_life_data.py USDA reference table
├── vision/       Computer vision & OCR
│   ├── preprocessing.py   OpenCV pipeline
│   ├── ocr_engine.py      EasyOCR + Tesseract failover
│   └── receipt_parser.py  Extract items from text
├── services/     Business logic
│   ├── shelf_life.py      Expiration date estimation
│   └── inventory.py       Pantry persistence
├── api/          Flask REST API
│   ├── families.py        Family management
│   ├── receipts.py        Upload & process receipts
│   └── pantry.py          Inventory queries
├── notifications SMS alerts
│   ├── messages.py        Alert logic
│   └── twilio_client.py   Twilio integration
└── tasks/        Celery async jobs
    ├── receipt_tasks.py   Process receipts
    └── alert_tasks.py     Hourly expiration scans

Quick Start (Docker)

cp .env.example .env
docker compose up --build

This launches Postgres, Redis, the API on :8000, a Celery worker, and Celery beat. Schema and shelf-life data seed automatically.

Try it out

# Create a family
curl -s localhost:8000/api/v1/families \
  -H 'Content-Type: application/json' \
  -d '{"name":"Demo Family","timezone":"America/New_York"}'

# Add a member
curl -s localhost:8000/api/v1/families/<FAMILY_ID>/users \
  -H 'Content-Type: application/json' \
  -d '{
    "full_name":"Dana",
    "phone_number":"(555) 111-2222",
    "expiration_alert_lead_days":2,
    "quiet_hours_start":"22:00",
    "quiet_hours_end":"07:00"
  }'

# Upload a receipt
curl -s -F image=@receipt.jpg -F family_id=<FAMILY_ID> \
  localhost:8000/api/v1/receipts

# Check the pantry
curl -s "localhost:8000/api/v1/pantry?family_id=<FAMILY_ID>&expiring_within_days=3"

Local Setup (No Docker)

python -m venv .venv && source .venv/bin/activate
pip install -e ".[nlp]"
python -m spacy download en_core_web_sm  # optional

sudo apt-get install tesseract-ocr

export DATABASE_URL="postgresql+psycopg2://pantry:pantry@localhost:5432/pantry_manager"
python -m app.db.seed_shelf_life
gunicorn -w 2 -b 0.0.0.0:8000 wsgi:app

In separate terminals:

celery -A app.tasks.celery_app.celery_app worker --loglevel=INFO
celery -A app.tasks.celery_app.celery_app beat --loglevel=INFO

Tip: Set PROCESSING_MODE=sync to process receipts inline (no worker needed), useful for development.

API Reference

Method	Path	Purpose
POST	/api/v1/families	Create family
GET	/api/v1/families/{id}	Get family + members
POST	/api/v1/families/{id}/users	Add member
PATCH	/api/v1/families/{id}/users/{uid}	Update member preferences
POST	/api/v1/receipts	Upload receipt image (multipart)
GET	/api/v1/receipts/{id}?include=items	Check processing status & extracted items
GET	/api/v1/pantry	List pantry items (filters: status, expiring_within_days, q, page, per_page)
POST	/api/v1/pantry/{id}/consume	Mark item as consumed
POST	/api/v1/pantry/{id}/discard	Mark item as discarded
PATCH	/api/v1/pantry/{id}	Edit quantity/unit/name or move storage
POST	/webhooks/twilio/status	Twilio delivery receipt callback
GET	/healthz · /readyz	Health checks

All errors return {"error": {"message": "...", "details": "..."}} with appropriate HTTP status codes.

How Alerting Works

Every hour, per user:

• Check local family time
• Skip if in quiet hours (handles windows that wrap midnight)
• Find items expiring in the next N days (configurable per user)
• Filter out already-alerted items (dedup by key)
• Send one SMS covering all due items (≤320 chars, "+N more" if overflow)
• Log per item

Transient Twilio failures retry silently; permanent errors are logged. Without Twilio credentials, everything runs in dry-run mode (logs only).

Tests

python -m unittest discover -s tests -v

Pure logic tests (parser, shelf-life matching, quiet hours, SMS rules, validation) run with the standard library. Integration tests spin up a real Flask app against a temp SQLite database and exercise the full flow.

Known Limitations (v1)

• No authentication. Deploy behind a gateway or add per-family API tokens before public exposure.
• No schema migrations. Schema is created via create_all + idempotent seed; use Alembic before the first breaking change.
• Push notifications. The model exists but I haven't wired it yet.
• EasyOCR size. PyTorch is ~2 GB. Drop it from requirements.txt for Tesseract-only mode on small hosts.

What's Next

• Mobile application implementation (learning as of now)
• Grocery chain partnerships (La casa del pueblo)
• CPG data licensing (unknown at the moment, biggest hurdle unfortunately)