Turning existing CCTV infrastructure into proactive behavioural monitoring —
no new cameras, no facial recognition, no biometric storage.
Built for the FAR AWAY 2026 International Hackathon by Team Accelerate
Every year, railway stations worldwide face thousands of incidents — platform falls, suicides, pickpocketing, and security threats. Most CCTV systems are reactive: footage is reviewed after something goes wrong. By then, it's too late.
Installing new smart cameras at scale costs millions and years. But the infrastructure — the cables, the mounting, the feeds — is already there.
RailMind AI makes existing cameras intelligent.
RailMind processes live CCTV video through a computer-vision pipeline, classifies human behaviour in real time using a trained BiLSTM neural network, and dispatches targeted alerts to the right staff member — all in under 3 seconds, with full reasoning transparency and no face data ever stored.
| Threat | How We Detect It |
|---|---|
| 🚨 Suicide Risk | Edge proximity accumulation + pacing cycles + social withdrawal — modelled over a 30-second behavioural window |
| 🎒 Pickpocketing | Sustained close-following distance + repeated crowd contact patterns |
| 🔒 Security Threat | Erratic/aggressive pose classification + loitering + unusual movement |
| 📍 Track Intrusion | Bounding box crossing the platform edge safety zone |
Real-time 2×2 CCTV grid with AI-annotated bounding boxes, track IDs, and risk classifications overlaid live on every feed.
Full incident analytics — KPI cards, 7-day trend chart, risk distribution, spatial platform heatmap, peak-hour histogram, and per-camera summary.
Centralised alert management with filterable tabs, risk scores, one-click acknowledge/resolve, and a detail panel with CCTV clip preview.
Instant per-platform camera filtering across all dashboard views.
┌──────────────────────── CV PIPELINE ─────────────────────────┐
│ Video Source (MP4 / RTSP) │
│ ↓ OpenCV decode │
│ YOLOv8n → Person detection + bounding boxes │
│ ↓ │
│ ByteTrack → Persistent track IDs across frames │
│ ↓ │
│ YOLOv8-Pose → 17 COCO keypoints per person │
│ ↓ │
│ Feature Extraction → 7-dimensional vector / 30s window │
│ ↓ │
│ BiLSTM Classifier → [Normal | Suicide | Pickpocket | Threat]
└─────────────────────────────┬────────────────────────────────┘
│ classification + confidence
┌─────────────────────────────▼────────────────────────────────┐
│ LangGraph Multi-Agent Pipeline │
│ │
│ [Perception] → [Reasoning] → [Intervention] │
│ ↓ ↓ ↓ │
│ Structures CV RiskScorer Dispatches alert │
│ observation + optional Creates incident record │
│ into typed LLM ±10 Starts 60s escalation timer │
│ dict adjustment WebSocket broadcast │
└─────────────────────────────┬────────────────────────────────┘
│ risk score + action
┌─────────────────────────────▼────────────────────────────────┐
│ FastAPI Backend + SQLite / PostgreSQL │
│ WebSocket → React 19 Dashboard │
│ Email (SMTP) / SMS (Twilio) escalation │
└───────────────────────────────────────────────────────────────┘
Each tracked person generates a 7-dimensional feature vector per 30-second window:
feature_vector = [
edge_proximity_seconds, # Cumulative time within 1.5m of platform edge
loitering_time, # Stationary duration in a single spatial zone
pacing_count, # Back-and-forth movement cycles detected
movement_speed, # Average velocity (m/s)
direction_changes, # Heading reversals per minute
following_distance, # Sustained proximity to one other person
crowd_interactions, # Count of unique close-contact individuals
]Input [batch, 30, 7]
↓
BiLSTM Layer 1 (128 units, bidirectional) → Dropout 0.2
↓
BiLSTM Layer 2 (64 units, bidirectional) → Dropout 0.2
↓
Dense (32, ReLU)
↓
Dense (4, Softmax) → [Normal, Suicide Risk, Pickpocketing, Security Threat]
Model specs: 2-layer bidirectional LSTM · 30-frame sequence window · 7 features · 4-class softmax output · saved as behavior_classifier.pt with fitted StandardScaler
Three nodes compiled once into a StateGraph at startup — stateless per-invocation, safe for concurrent calls:
| Agent | Responsibility |
|---|---|
| Perception | Assembles the 30s feature sequence, runs LSTM inference, produces a typed observation dict |
| Reasoning | Runs RiskScorer (weighted sum across 6 dimensions), optionally calls OpenAI or Claude for a bounded ±10 score adjustment and a plain-language reasoning_summary |
| Intervention | Applies score thresholds, dispatches alert, creates incident record, starts 60s escalation timer |
LLM integration is optional. If no API key is configured, the Reasoning Agent runs purely on rule-based scoring and the system functions identically.
| Risk Score | Action |
|---|---|
| 0 – 39 | Silent log only |
| 40 – 69 | Alert nearest available staff via dashboard |
| 70 – 89 | Alert staff + security simultaneously |
| 90 – 100 | Full emergency escalation + SMS |
| Unacknowledged 60s | Auto-escalate to next tier |
Everything below has been verified by running it, not just reading the code:
- Person detection & tracking — YOLOv8n detection + BYTETracker for persistent IDs
- Pose estimation — YOLOv8-Pose extracting 17 COCO keypoints per tracked person
- Behavioural feature extraction — all 7 features computed per 30s window
- Trained LSTM classifier — real saved checkpoint + fitted scaler, loaded and run at inference time
- LangGraph multi-agent pipeline — real
StateGraphwith three compiled nodes - Optional LLM reasoning — OpenAI GPT or Anthropic Claude for bounded score adjustment; clean fallback when no key is set
- React 19 dashboard — live stats pulled from the FastAPI backend
- Video upload & full pipeline — upload an
.mp4, it runs through CV → LSTM → agents; not a demo loop - WebSocket alert delivery — channel-based pub/sub
ConnectionManagerbroadcasts to all dashboard clients in real time - Escalation timers — unacknowledged alerts escalate after 60 seconds
- Email alerts (SMTP) — via
smtplib, configurable via.env - SMS escalation (Twilio) — implemented and configurable
- 77 passing backend tests (1 skipped, 3 errors) — covering agents, CV pipeline, LSTM inference, risk scoring, alerts, incidents, escalation, and heatmaps
- GitHub Actions CI — runs backend tests + frontend build + Playwright E2E on every push
We'd rather you find out from the README than from the code:
| Planned | Actual Status |
|---|---|
| Edge deployment on Jetson Orin Nano / TensorRT | Not implemented — pipeline runs on the host machine (CPU or CUDA GPU) |
| Multi-camera person re-identification | Not implemented — tracking is per-camera only |
| Automated PA system integration | Not implemented — no MQTT/PA code exists |
| Mobile app for staff (React Native) | Does not exist yet |
| True multi-tenant SaaS isolation | Single-deployment software — station_id column exists but no per-tenant access control |
| LSTM trained on real incident data | Trained on synthetic sequences — accuracy figures are against the synthetic test set only |
| Requirement | Version |
|---|---|
| Python | 3.11+ |
| Node.js | 18+ |
| CUDA (optional) | 11.8+ for GPU inference |
git clone https://github.com/your-org/RailMind-AI.git
cd RailMind-AI/backend
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env # Edit values as neededpython -m app.lstm.cli train
# Models saved to backend/lstm/saved_models/python run.py
# API live at http://localhost:8000
# Swagger docs at http://localhost:8000/docscd ../frontend
npm install
cp .env.example .env
npm run dev
# Dashboard at http://localhost:5173cd backend
python scripts/demo_runner.py# GPU
python scripts/run_video_processors.py --device cuda:0
# CPU only
python scripts/run_video_processors.py --device cpuAll settings live in backend/.env:
# Core
DEBUG=True
DATABASE_URL=sqlite:///./data/railmind.db # PostgreSQL in production
# The SQLite database file is created automatically on first run via init_db() in app/main.py.
SECRET_KEY=your-secret-key-change-in-production
RAILMIND_API_KEY=change-this-admin-api-key
LOG_LEVEL=INFO
# Computer Vision
POSE_MODEL_PATH=./yolov8n-pose.pt
POSE_DEVICE=cpu # or cuda:0
LSTM_SEQUENCE_LENGTH=30
# Risk Scoring
LOW_RISK_THRESHOLD=40
MEDIUM_RISK_THRESHOLD=70
HIGH_RISK_THRESHOLD=90
PLATFORM_EDGE_SAFETY_LIMIT_METERS=1.5
# Behaviour Thresholds
BEHAVIOR_HIGH_SCORE_THRESHOLD=0.65
BEHAVIOR_FOLLOWING_DISTANCE_METERS=1.2
# LLM (Optional — system works without these)
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
ANTHROPIC_REASONING_MODEL=claude-haiku-4-5-20251001
# Email Alerts
SMTP_HOST=smtp.example.com
SMTP_PORT=587
SMTP_USER=alert@example.com
SMTP_PASSWORD=your-smtp-password
ALERT_EMAIL_RECIPIENTS=security@example.com
# SMS Escalation (Twilio)
TWILIO_ACCOUNT_SID=your_twilio_account_sid
TWILIO_AUTH_TOKEN=your_twilio_auth_token
TWILIO_FROM_NUMBER=+1234567890
TWILIO_TO_NUMBERS=+19876543210Frontend (frontend/.env):
VITE_API_BASE_URL=http://localhost:8000/api
VITE_RAILMIND_API_KEY=change-this-admin-api-key
VITE_WS_URL=ws://localhost:8000REST API routes under
/api/*require theX-API-Keyheader. In browser builds the key is public — do not treat it as a production identity system.
cd backend
# Train the 4-class behaviour classifier
python -m app.lstm.cli train
# Custom parameters
python -m app.lstm.cli train --epochs 50 --batch-size 16
# Check training status
python -m app.lstm.cli status --latest
python -m app.lstm.cli status --history 10# Trigger training
curl -X POST http://localhost:8000/api/training/trigger \
-H "Content-Type: application/json" \
-d '{"model_type": "all", "epochs": 30, "batch_size": 32}'
# Monitor progress
curl http://localhost:8000/api/training/runs/1
# Check system status
curl http://localhost:8000/api/training/statusThe APScheduler runs every Sunday at 2 AM UTC, retraining the classifier using operator-labelled false positives accumulated during the week. Each training run is logged to the database with full metadata (accuracy, loss, trigger source, model path, production-ready flag).
Full interactive docs at http://localhost:8000/docs (Swagger) and /redoc. All endpoints require X-API-Key.
| Method | Endpoint | Description |
|---|---|---|
GET |
/api/incidents |
List incidents — filter by date, platform, status, category |
GET |
/api/incidents/{id} |
Full incident detail with LSTM output and agent reasoning trace |
POST |
/api/incidents/{id}/acknowledge |
Staff acknowledges an incident |
POST |
/api/incidents/{id}/false-positive |
Flag a false positive for retraining |
GET |
/api/alerts |
List alerts |
GET |
/api/alerts/stats |
Alert statistics |
PATCH |
/api/alerts/{id}/acknowledge |
Acknowledge an alert |
PATCH |
/api/alerts/{id}/resolve |
Resolve an alert |
POST |
/api/alerts/{id}/escalate |
Manually escalate an alert |
GET |
/api/dashboard/stats |
Headline KPI metrics |
GET |
/api/dashboard/trend |
Incident trend over time |
GET |
/api/dashboard/heatmap |
Spatial heatmap data |
GET |
/api/dashboard/cctv-summary |
Per-camera summary table |
GET |
/api/analytics/lstm-performance |
LSTM accuracy, false positive rate, confidence |
POST |
/api/feeds/upload |
Upload a video file for processing |
GET |
/api/staff/available |
Available staff by platform zone |
POST |
/api/training/trigger |
Trigger LSTM retraining |
GET |
/api/health |
Health check |
WS |
/ws/alerts |
Real-time alert event stream |
WS |
/ws/feed/{camera_id} |
Live annotated video feed stream |
{
"alert_id": "uuid-v4",
"timestamp": "2026-01-15T14:32:07Z",
"platform": "Platform 3B",
"camera_id": "CAM_003",
"risk_score": 88,
"risk_category": "Suicide Risk",
"lstm_confidence": 0.91,
"track_id": 142,
"location": { "x": 0.72, "y": 0.41, "zone": "edge" },
"recommended_action": "Approach immediately and offer assistance",
"reasoning_summary": "30s edge proximity + pacing ×4 + social withdrawal",
"reasoning_mode": "llm",
"escalation_level": 1,
"escalate_at": "2026-01-15T14:33:07Z"
}cd backend
# Full test suite — 57 tests, all passing
pytest
# Individual modules
pytest tests/test_agents.py
pytest tests/test_lstm.py
pytest tests/test_risk_scoring.py
pytest tests/test_cv.py
pytest tests/test_alerts.py
pytest tests/test_incident_and_escalation.py| Test File | Covers |
|---|---|
test_agents.py |
Full agent pipeline — Perception → Reasoning → Intervention |
test_lstm.py |
Model loading and inference |
test_risk_scoring.py |
Risk score calculation and classification |
test_cv.py |
CV pipeline behaviour and degradation handling |
test_alerts.py |
Alert lifecycle and CRUD |
test_incident_and_escalation.py |
Incident creation and 60s escalation timers |
test_heatmap.py |
Heatmap analytics |
test_dashboard_trend.py |
Dashboard trend data aggregation |
test_reliability.py |
Failure-mode handling |
test_api_auth.py |
API key authentication |
test_feeds.py |
Feed registration and video upload |
test_rtsp_ingestion.py |
RTSP stream ingestion (manual) |
Frontend E2E (requires both servers running):
cd frontend
npm run e2ePrivacy is a foundational design constraint, not a policy statement. There is no facial recognition or identity-linking code anywhere in this repository.
| Principle | Implementation |
|---|---|
| No Facial Recognition | Face detection models explicitly excluded — analysis uses body movement and posture only |
| No Biometric Storage | Track IDs are numeric, session-scoped integers that expire when the session ends |
| Minimal Data Retention | Raw video retained 72 hours by default; only flagged clips archived up to 30 days |
| Behaviour-Only Analysis | LSTM feature vector contains only anonymised movement metrics |
| Human-in-the-Loop | All alerts require human staff confirmation — AI recommends, humans decide |
| Transparent Reasoning | Every incident record includes full LSTM confidence, risk score, and LangGraph reasoning trace |
| Regulatory Alignment | Designed for GDPR (EU), PDPA (India), and equivalent frameworks |
| Bias Monitoring | Regular audits of alert rates by platform, time, and incident type |
SQLite in development, PostgreSQL in production:
incidents — id, track_id, timestamp, platform_id, risk_score, risk_category,
lstm_confidence, status, reasoning_mode
alerts — id, incident_id, alert_type, sent_at, acknowledged_at, staff_id,
escalation_level, notification_status, reasoning_mode
tracks — id, track_id, camera_id, session_id, feature_sequence_json,
lstm_label, confidence
analytics — id, date, hour, platform_id, incident_count, avg_risk_score,
false_positive_count, camera_id, zone, hotspot_intensity
staff — id, name, platform_zone, contact_email, contact_phone, is_available
platforms — id, station_id, platform_number, camera_ids_json, edge_zone_config_json
feeds — id, name, status, fps, source_url, stream_url
feedback — id, alert_id, staff_id, is_false_positive, notes, submitted_at
training_runs — id, status, model_type, epochs, final_val_accuracy,
is_production_ready, triggered_by, started_at, completed_atRailMind-AI/
├── assets/ # Dashboard screenshots
├── backend/
│ ├── app/
│ │ ├── agents/ # LangGraph nodes: perception / reasoning / intervention
│ │ ├── analytics/ # Dashboard metrics, heatmaps, incident stats
│ │ ├── api/routes/ # FastAPI route handlers (8 sub-routers)
│ │ ├── core/ # Config, database, WebSocket manager, scheduler
│ │ ├── cv/ # Video processor, pose estimator, behaviour analyzer
│ │ ├── features/ # 7 behavioural feature detectors
│ │ ├── lstm/ # Model definition, training, CLI, predictor
│ │ ├── models/ # SQLAlchemy ORM models
│ │ ├── schemas/ # Pydantic request/response schemas
│ │ └── services/ # Risk scoring, alerts, escalation, notifications
│ ├── lstm/saved_models/ # Trained .pt checkpoint + StandardScaler pickle
│ ├── data/mock_feeds/ # Sample video files for demo
│ ├── tests/ # 57-test pytest suite
│ ├── scripts/ # demo_runner.py, run_video_processors.py
│ └── run.py
├── frontend/
│ ├── src/
│ │ ├── components/ # CCTVFeedCard, TopBar, Sidebar, StatCard, RiskBadge
│ │ ├── components/ui/ # shadcn/ui component library (Radix + Tailwind)
│ │ ├── hooks/ # useWebSocket
│ │ ├── lib/api/ # Typed API client (alerts, dashboard, feeds)
│ │ └── routes/ # live.tsx, dashboard.tsx, alerts.tsx
│ └── tests/e2e/ # Playwright tests
├── .github/workflows/ci.yml # GitHub Actions CI (tests + build + E2E)
├── TRAINING_PIPELINE.md
├── SECURITY.md
└── README.md
| Library | Purpose |
|---|---|
| Python 3.11+ | Core language |
| FastAPI + Uvicorn | Async REST API |
| SQLAlchemy + Alembic | ORM + schema migrations |
| LangGraph 0.5 | Multi-agent orchestration |
| PyTorch 2.9 | BiLSTM training & inference |
| Ultralytics YOLOv8 | Detection, pose estimation, ByteTracker |
| OpenCV | Video decode and preprocessing |
| APScheduler | Weekly LSTM retraining |
| Twilio | SMS escalation |
| smtplib | Email alerts |
| Anthropic / OpenAI | Optional LLM-assisted reasoning |
| Library | Purpose |
|---|---|
| React 19 | UI framework |
| TanStack Router + Query | Routing + server state management |
| Tailwind CSS 4 | Styling |
| Recharts | Analytics charts |
| shadcn/ui (Radix UI) | Accessible component library |
| Vite 7 | Build tooling |
| Playwright | E2E testing |
- Recorded video processing pipeline (CV → LSTM → agents)
- YOLOv8 + ByteTrack + Pose estimation
- Bidirectional LSTM behaviour classifier
- LangGraph Perception → Reasoning → Intervention agents
- SQLite storage + FastAPI REST API
- React 19 dashboard with live alerts
- WebSocket real-time delivery
- Email (SMTP) + SMS (Twilio) notifications
- Operator false-positive feedback loop
- 57 passing backend tests + GitHub Actions CI
- LSTM continual learning pipeline (weekly retraining on real labelled data)
- Multi-camera person re-identification
- Edge deployment on Jetson Orin Nano / TensorRT
- Automated PA system integration for critical alerts
- Mobile app for railway staff (React Native)
- Multi-station SaaS dashboard with per-tenant isolation
- Compliance reporting and audit log exports
- Bias monitoring and periodic model audits
- Predictive analytics — forecast high-risk time windows before incidents occur
- Crowd flow optimisation recommendations
- Integration with emergency services dispatch
- Video Swin Transformer for richer spatio-temporal modelling
- Reinforcement learning for dynamic threshold optimisation
- Cross-network anonymised incident pattern sharing
Team Accelerate — FAR AWAY 2026 International Hackathon
RailMind handles safety-critical incident data. REST API routes are protected by X-API-Key. Please report suspected vulnerabilities privately — do not open a public issue. See SECURITY.md for the full policy and reporting instructions.
Built with ❤️ for the FAR AWAY 2026 International Hackathon
Making railways safer, one frame at a time.