Sravz Platform - Detailed Architecture

System Architecture Overview

The Sravz platform is built with a multi-backend, multi-language architecture designed to handle both realtime financial data streaming and historical analytics.

High-Level Data Flow

Realtime Pipeline:

• Interactive Brokers → C++ WebSocket Server → Redis Cache → Angular Dashboard
• Low-latency quote updates via WebSocket connections

Historical Analysis Pipeline:

• Economic Calendar + Historical Data → Airflow Orchestration → Dask Distribution → Python Analytics Engine
• Batch processing for statistical and ML-based analysis

Core Architecture Components

System Layers

Data Storage Layer

Data Flow Architecture

Realtime Path (WebSocket):

Interactive Brokers (Stock/Crypto) → C++ WebSocket Server
    → Redis Cache → Angular Dashboard (live quotes)

Historical Path (Batch):

Economic Calendar / Historical Data → Airflow Orchestration
    → Dask Task Scheduler → Python Analytics Engine
    → MongoDB (results) + S3 (archive)

Deployment Infrastructure

• Container Orchestration: Docker Swarm
• Load Balancing: Traefik (reverse proxy)
• Infrastructure:
  • Contabo (Primary swarm cluster)
  • Hetzner (Backup/HA swarm cluster)
• Images: AWS ECR (private registry)

Technology Stack by Layer

Frontend Layer

• Angular (frontend-ng): Dynamic SPA with real-time updates
• Hugo + WordPress: Static documentation and content management

Backend Services

Detailed Technical Architecture

Microservices & Data Flow - Complete System Diagram

The architecture follows a multi-tier, loosely-coupled microservices pattern with clear separation of concerns.

Architecture Flow: External Data Sources → API Gateway → Processing Services → Data Storage → Frontend

Architecture Layers:

1. External Data Sources

   • Interactive Brokers (REST API + WebSocket for stock/crypto/forex data)
   • Economic Calendar (Macro events and indicators)
   • Crypto Exchanges (Digital asset data)

2. Frontend Tier

   • Angular Dashboard (frontend-ng): Real-time trading interface with Material UI
   • Documentation: Hugo + WordPress static sites

3. API Gateway & Routing

   • Traefik: Reverse proxy with SSL/TLS, load balancing, service routing
   • Directs requests to appropriate backend services

4. Backend Microservices - Realtime Path

   • REST API (backend-go / Gorilla Mux): Core HTTP API, CRUD operations, data retrieval
   • WebSocket Server (backend-cpp / Boost ASIO): Low-latency quote streaming, real-time updates
   • Auth Service (backend-node / Node.js): OAuth, JWT validation, session management

5. Backend Microservices - Processing Path

   • Ticker Plant (backend-rust / Tokio): NSQ message consumer, quote processing, DataFrame analytics
   • Analytics Engine (backend-py / Pandas/SciPy): Statistical analysis, ML models (PCA, forecasting, risk)
   • Time-Series Warehouse (backend-q / kdb+ TorQ): Lightning-fast OHLCV queries, backtesting

6. Message Queue & Orchestration

   • NSQ: Distributed pub/sub message queue for service-to-service async communication
   • Apache Airflow: DAG-based job orchestration, scheduled analytics jobs
   • Dask: Distributed task scheduler for parallel processing

7. Data Storage

   • MongoDB: Document store for user configs, portfolios, analysis results
   • MySQL: Transactional data, ledgers, audit logs
   • Redis: In-memory cache for real-time quotes, session storage
   • AWS S3: Data lake for historical OHLCV, backups, archives
   • DuckDB: Analytical query engine for ad-hoc financial analysis

8. Infrastructure

   • Docker Swarm: Container orchestration on Contabo (primary) and Hetzner (HA)
   • AWS ECR: Private container image registry (public.ecr.aws/b8h3z2a1/sravz/*)

Service-to-Service Communication Matrix

Deployment Architecture

Docker Compose Services (localhost-docker-compose.yaml):

• mongo - MongoDB (port 27017)
• nsqlookupd-1 - NSQ lookup daemon (port 4160-4161)
• nsqd-1 - NSQ daemon (port 4150-4151)
• nsqadmin - NSQ admin UI (port 4171)
• backend-py - Python analytics (workspace mount)
• backend-rust - Rust ticker plant (workspace mount)
• backend-cpp - C++ WebSocket (port 5001, workspace mount)
• backend-node - Node.js auth (port 3030, workspace mount)
• backend-go - Go REST API (port 8081, workspace mount)
• backend-q - kdb+ time series (local build, TorQ setup)
• redis - Redis cache (port 6379)
• mysql - MySQL (port 3306)

Production Deployments:

• Contabo Docker Swarm (Primary)
• Hetzner Docker Swarm (HA/Failover)
• AWS ECR (Image registry: public.ecr.aws/b8h3z2a1/sravz/*) | WebSocket Server | C++ (Boost/Std) | Real-time quote streaming | Low-latency tick updates, live feeds | | Ticker Plant | Rust | Historical ticker management | OHLCV storage, tick history, time-series | | Analytics Engine | Python (Pandas/NumPy) | Data analysis & ML | PCA, Prophet models, rolling stats, correlations |

Realtime Data Flow

1. Data Ingestion: Interactive Brokers → WebSocket Server (C++)
2. Quote Caching: Redis stores latest quotes
3. Live Analysis: Realtime calculations on streaming ticks
4. Storage: Ticker Plant persists quotes to MySQL/S3

Historical Data Flow

1. Orchestration: Airflow schedules batch jobs
2. Data Processing: Dask distributes computation
3. Task Queue: NSQ manages background tasks
4. Analytics: Python engine computes metrics
5. Persistence: Results → MongoDB, S3, DuckDB

Analysis Capabilities

Realtime Analysis

• Quote Processing: Live bid/ask, last price, volume
• Streaming Calculations: Moving averages, momentum
• Alert Generation: Price targets, threshold crossings
• Dashboard Updates: WebSocket pushes to frontend

Historical Analysis

• Statistical Analysis

  • PCA (Principal Component Analysis)
  • Time Series Analysis (ARIMA, SARIMAX)
  • Covariance & Correlation Matrix
  • Rolling Statistics (volatility, Sharpe ratio)
  • Year-to-Date Returns

• ML Models

  • Prophet Forecasting
  • Pyfolio Returns Analysis

• Data Sources

  • Equities (Stocks, ETFs, Mutual Funds)
  • Futures (Commodities, Indices)
  • Forex (Currency Pairs)
  • Crypto (Digital Assets)
  • Economic Calendars (Macro Indicators)

Data Storage Architecture

Deployment Architecture

Production Infrastructure

• Contabo Swarm: Primary production cluster
• Hetzner Swarm: Failover/HA cluster
• Container Orchestration: Docker Swarm
• Load Balancing: Nginx/Traefik
• Monitoring: Prometheus + Grafana (optional)

Local Development

• Docker Compose with volume mounts
• Hugo server on localhost:1313
• Angular dev server with hot reload
• Python/Rust/Go with local execution

Integration Points

External APIs

• Interactive Brokers: Real-time quotes, historical data, order execution
• Economic Calendar: Macro events, releases
• Crypto Exchanges: Digital asset data

Internal APIs

• REST endpoints for CRUD operations
• WebSocket for real-time updates
• Async task queues for background work

Scalability Features

1. Horizontal Scaling: Services run as Docker replicas in Swarm
2. Task Distribution: Dask + Airflow handle computational load
3. Caching Strategy: Redis reduces database hits
4. Data Partitioning: Time-based partitioning in S3/MySQL
5. Async Processing: NSQ decouples request/response cycles

See also: Tech Stack Details for implementation specifics