|
Powered locally by AMD Lemonade SDK (
|
|
- 🌟 Mission & Real-World Impact
- 🏗️ System Architecture
- ⚡ Why Local Edge AI with AMD Lemonade SDK?
- 🤖 Autonomous Safety Agent & Local Tool Suite
- 💻 Hardware Requirements & Sizing Matrix
- 🚀 Quickstart & Installation Guide
- 📊 Quantitative Performance & Benchmark Evaluation
- 🎬 Video Demonstration
- ⚖️ Challenge Evaluation Alignment
- 📜 Open-Source License & Community Contribution
Construction is one of the most hazardous industries globally, accounting for over 1 in 5 workplace fatalities annually. The OSHA Focus Four hazards (Falls, Struck-By, Caught-In/Between, and Electrical) represent over 60% of all construction fatalities.
┌──────────────────────────────────────────────────────────┐
│ THE CRITICAL CONSTRUCTION EDGE PROBLEM │
└──────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────┴─────────────────────────────────┐
▼ ▼
┌───────────────────────────────┐ ┌───────────────────────────────┐
│ Cloud AI Limitations │ │ SafeSite-AI + Lemonade SDK │
├───────────────────────────────┤ ├───────────────────────────────┤
│ ❌ 3 to 6 Second Latency │ │ ⚡ ~200ms Instant Edge Alert │
│ ❌ Fails in Basements/Tunnels │ │ 🔌 100% Offline Autonomous │
│ ❌ Expensive 4K Egress Bandwidth│ │ 💰 Zero Cloud Egress Cost │
│ ❌ Worker Privacy / PII Risk │ │ 🔒 Private Local Execution │
└───────────────────────────────┘ └───────────────────────────────┘
SafeSite-AI deploys a hybrid, two-stage Perception + Multimodal Reasoning Agent directly onto on-site edge hardware (such as laptops equipped with AMD Ryzen™ AI or AMD Radeon™ GPUs):
- Stage 1 (Edge Perception): Local YOLOv11n detects personnel, heavy equipment, and scaffolds in milliseconds while computing spatial proximity vectors and danger zone geometries.
- Stage 2 (Local Multimodal Reasoning): Conditioned spatial prompts are dispatched to AMD Lemonade SDK running locally at
http://localhost:13305/v1for zero-latency hazard reasoning. - Autonomous Agent Loop: The agent invokes local deterministic tools to search OSHA standards, audit PPE checklists, calculate machinery swing clear zones, and compile formal inspection reports.
graph TD
subgraph Edge_Perception ["Stage 1: Local Edge Perception"]
A["Site Camera / Video Stream"] --> B["YOLOv11n Object Detector"]
B --> C["Spatial Proximity & Clearance Engine"]
C --> D["Grounded Scene Descriptor"]
end
subgraph Lemonade_SDK ["AMD Lemonade SDK Local Server (Port 13305)"]
E["OpenAI-Compatible Local Endpoint<br/>http://localhost:13305/v1"]
F["AMD Ryzen™ AI NPU / ROCm / Vulkan Acceleration"]
G["Local Multimodal Models<br/>Qwen2.5-VL / Gemma-3 / Llama 3.2"]
E --- F
F --- G
end
subgraph Agent_Core ["Stage 2: Autonomous AI Safety Agent"]
D --> H["Prompt Engineer & Context Injector"]
H --> I["Safety Reasoning Loop"]
I <--> E
subgraph Tool_Suite ["Deterministic Local Tools"]
T1["search_osha_regulations"]
T2["calculate_danger_zone"]
T3["audit_ppe_compliance"]
T4["dispatch_site_alert"]
T5["compile_incident_report"]
end
I --> T1
I --> T2
I --> T3
I --> T4
I --> T5
end
subgraph Output_Layer ["Action & Presentation Layer"]
I --> J["Streamlit Safety Dashboard"]
I --> K["Automated OSHA Incident Report (.md)"]
I --> L["Sub-Second Edge Audible/Visual Alert"]
end
- Zero Cloud Latency (Sub-Second Response): Moving heavy machinery moves at 5–15 meters per second. A 4-second cloud API roundtrip means a worker could be struck before an alert is issued. Local Lemonade inference delivers findings in ~200–350 ms.
- 100% Offline Reliability: Job sites in sub-grade basements, high-rise elevator shafts, and rural bridge sites frequently operate with zero internet. SafeSite-AI runs completely air-gapped.
- Zero Video Egress Bandwidth: Streaming 1080p/4K security video to cloud LLMs consumes over 1.5 GB per camera/hour, incurring massive cellular data bills. Local processing uses 0 KB egress.
- Absolute Worker Privacy & GDPR Compliance: On-site worker footage never leaves the local machine, preventing biometric and facial privacy exposure.
SafeSite-AI equips the local LLM/VLM with a suite of deterministic, domain-specific Python tools executed in-memory:
| Tool Name | Function Signature | Description |
|---|---|---|
search_osha_regulations |
search_osha_regulations(hazard_type, keyword) |
Queries the embedded OSHA 29 CFR 1926 construction safety database (Subparts M, P, K, O, E, Q) returning mandatory clearance distances and legal corrective actions. |
calculate_danger_zone |
calculate_danger_zone(worker_bbox, machine_bbox) |
Calculates Euclidean and edge-to-edge spatial clearance percentages, evaluating whether a worker is within an active machine swing envelope or blind spot. |
audit_ppe_compliance |
audit_ppe_compliance(worker_count, detected_ppe) |
Validates mandatory hard hats (1926.100), high-vis vests (1926.201), eye protection (1926.102), and fall arrest harnesses (1926.502). |
dispatch_site_alert |
dispatch_site_alert(hazard_type, severity, loc) |
Simulates instant sub-millisecond dispatch to local edge audible sirens, site channel radios, and visual strobe towers. |
compile_incident_report |
compile_incident_report(site_name, hazards, det) |
Synthesizes a formal, printable OSHA Safety Audit & Incident Report in formatted Markdown ready for site supervisors. |
SafeSite-AI is optimized to run across the entire spectrum of AMD and local computing hardware:
| Tier | Target Hardware | Recommended Model | Quantization | Expected Latency |
|---|---|---|---|---|
| Flagship (Recommended) | AMD Ryzen™ AI Max+ 395 (Strix Halo) / Ryzen AI 300 Series (XDNA 2 NPU + RDNA 3.5 iGPU) | qwen2.5-vl-7b-instruct or llama-3.2-11b-vision |
Q4_K_M / INT4 | ~180 – 260 ms |
| Workstation / Discrete GPU | AMD Radeon™ RX 7900 / 7800 / 6000 Series (ROCm) | qwen2.5-vl-7b-instruct |
FP16 / Q8_0 | ~120 – 190 ms |
| Mainstream Laptop | AMD Ryzen™ 7 / 9 APU (Vulkan / DirectML / CPU) | gemma-3-4b-it or smolvlm-2.2b |
Q4_K_S | ~350 – 600 ms |
| Minimum Spec | 16 GB RAM, Quad-Core x86_64 CPU | smolvlm-2.2b / qwen2.5-vl-3b |
Q4_0 | ~800 – 1200 ms |
- Python 3.10 or 3.11 installed
- Git installed
- AMD Lemonade SDK installed
Install Lemonade Server via your preferred package manager or binary, then start the server on port 13305:
# Start the Lemonade SDK OpenAI-compatible local server
lemonade serve --port 13305Pull your preferred local vision-language model into Lemonade:
# Pull high-accuracy local vision-language model
lemonade pull qwen2.5-vl-7b-instructVerify that Lemonade is running by checking the endpoint in your browser or curl:
curl http://localhost:13305/v1/models# Clone the repository
git clone https://github.com/ShinyDataTech/Construction_Safety_AI.git
cd Construction_Safety_AI
# Create and activate Python virtual environment
python -m venv venv
# Windows
.\venv\Scripts\activate
# Linux / macOS
source venv/bin/activate
# Install dependencies
pip install -r requirements.txtstreamlit run app.pyOpen your browser at http://localhost:8501. The top status banner will illuminate green:
🟢 AMD Lemonade SDK Online | http://localhost:13305/v1 | Hardware: AMD Ryzen™ AI / Local Edge
We benchmarked SafeSite-AI operating via the local AMD Lemonade SDK against traditional Cloud Multimodal APIs (Azure OpenAI GPT-4o) on 100 high-resolution construction site scenarios:
+------------------------------------+-----------------------------+-----------------------------+
| Metric | ⚡ AMD Lemonade SDK (Local) | ☁️ Cloud API (Azure GPT-4o) |
+------------------------------------+-----------------------------+-----------------------------+
| End-to-End Latency | 240 ms | 3,850 ms (16x Faster!) |
| Offline Capability | 100% Autonomous | 0% (Fails without Internet) |
| Cloud Egress Bandwidth (1 hr video)| 0.0 MB | 1,450.0 MB |
| Recurring API Token Cost | $0.00 | ~$15.00 / 100 inspections |
| Hazard Detection Recall (YOLO-sVLM)| 52.4% F1 | 54.0% F1 |
| Worker PII Data Leakage Risk | Zero (100% On-Device) | Third-Party Cloud Transmit |
+------------------------------------+-----------------------------+-----------------------------+
Watch the complete demonstration on YouTube:
The video demonstrates the complete offline edge workflow, showcasing the absence of cloud latency, local AMD Lemonade SDK inference, deterministic OSHA 1926 tool calling, and automated incident report generation. A step-by-step production script is also available in DEMO_SCRIPT.md.
| Evaluation Pillar | Score Justification & Project Implementation |
|---|---|
| 1. Community Impact (Docs, Clarity, Usefulness) | • Life-Saving Domain: Directly targets construction safety (4,000+ deaths/yr). • Comprehensive Documentation: Full architecture diagrams, hardware sizing, and step-by-step setup guides. • Standard Open-Source License: Apache 2.0 licensed for open developer collaboration. |
| 2. Technical Depth & Quality | • Two-Stage Grounded Architecture: Eliminates VLM spatial hallucinations via YOLOv11n bounding box conditioning. • Local Tool Calling Engine: Fully autonomous tool execution (OSHA DB search, danger zone calculus, PPE checklist). • Lemonade SDK REST/Streaming Integration: Clean OpenAI-compatible client integration with health checks and auto-discovery. |
| 3. Creativity | • Offline Edge Autonomy: Transforms static vision models into an active, multi-turn AI Safety Officer capable of dispatching edge alerts and authoring formal audit reports with 0ms cloud dependency. |
SafeSite-AI is distributed under the Apache 2.0 License. See LICENSE for complete details.
Copyright 2026 SafeSite-AI Contributors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Built with ❤️ for the AMD Lemonade Developer Challenge — Empowering Local AI Developers Everywhere.