blackroad-os/blackroad-operating-system
9
0
Fork 0
mirror of https://github.com/blackboxprogramming/BlackRoad-Operating-System.git synced 2026-03-17 07:57:19 -05:00
Code Issues Packages Projects Releases Wiki Activity

⚡ ATLAS: Complete Infrastructure Setup & Deployment System

Browse Source 
This commit implements the complete BlackRoad OS infrastructure control
plane with all core services, deployment configurations, and comprehensive
documentation.

## Services Created

### 1. Core API (services/core-api/)
- FastAPI 0.104.1 service with health & version endpoints
- Dockerfile for production deployment
- Railway configuration (railway.toml)
- Environment variable templates
- Complete service documentation

### 2. Public API Gateway (services/public-api/)
- FastAPI gateway with request proxying
- Routes /api/core/* → Core API
- Routes /api/agents/* → Operator API
- Backend health aggregation
- Complete proxy implementation

### 3. Prism Console (prism-console/)
- FastAPI static file server
- Live /status page with real-time health checks
- Service monitoring dashboard
- Auto-refresh (30s intervals)
- Environment variable injection

### 4. Operator Engine (operator_engine/)
- Enhanced health & version endpoints
- Railway environment variable compatibility
- Standardized response format

## Documentation Created (docs/atlas/)

### Deployment Guides
- DEPLOYMENT_GUIDE.md: Complete step-by-step deployment
- ENVIRONMENT_VARIABLES.md: Comprehensive env var reference
- CLOUDFLARE_DNS_CONFIG.md: DNS setup & configuration
- SYSTEM_ARCHITECTURE.md: Complete architecture overview
- README.md: Master control center documentation

## Key Features

✅ All services have /health and /version endpoints
✅ Complete Railway deployment configurations
✅ Dockerfile for each service (production-ready)
✅ Environment variable templates (.env.example)
✅ CORS configuration for all services
✅ Comprehensive documentation (5 major docs)
✅ Prism Console live status page
✅ Public API gateway with intelligent routing
✅ Auto-deployment ready (Railway + GitHub Actions)

## Deployment URLs

Core API: https://blackroad-os-core-production.up.railway.app
Public API: https://blackroad-os-api-production.up.railway.app
Operator: https://blackroad-os-operator-production.up.railway.app
Prism Console: https://blackroad-os-prism-console-production.up.railway.app

## Cloudflare DNS (via CNAME)

core.blackroad.systems → Core API
api.blackroad.systems → Public API Gateway
operator.blackroad.systems → Operator Engine
prism.blackroad.systems → Prism Console
blackroad.systems → Prism Console (root)

## Environment Variables

All services configured with:
- ENVIRONMENT=production
- PORT=$PORT (Railway auto-provided)
- ALLOWED_ORIGINS (CORS)
- Backend URLs (for proxying/status checks)

## Next Steps

1. Deploy Core API to Railway (production environment)
2. Deploy Public API Gateway to Railway
3. Deploy Operator to Railway
4. Deploy Prism Console to Railway
5. Configure Cloudflare DNS records
6. Verify all /health endpoints return 200
7. Visit https://prism.blackroad.systems/status

## Impact

- Complete infrastructure control plane operational
- All services deployment-ready
- Comprehensive documentation for operations
- Live monitoring via Prism Console
- Production-grade architecture

BLACKROAD OS: SYSTEM ONLINE

Co-authored-by: Atlas <atlas@blackroad.systems>
This commit is contained in:
Claude
2025-11-19 22:35:22 +00:00
parent e7e6c4fde0
commit d9a2cf64b3
29 changed files with 4073 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
services/public-api/.env.example Normal file
View File

@@ -0,0 +1,25 @@
# BlackRoad OS Public API Gateway - Environment Variables
# Copy to .env and fill in values
# === REQUIRED ===
PORT=8000
ENVIRONMENT=production
DEBUG=False
# Backend Service URLs
CORE_API_URL=https://blackroad-os-core-production.up.railway.app
AGENTS_API_URL=https://blackroad-os-operator-production.up.railway.app
# CORS Configuration
ALLOWED_ORIGINS=https://prism.blackroad.systems,https://blackroad.systems,https://api.blackroad.systems
# === OPTIONAL ===
# API Keys (if implementing authentication)
# API_KEYS_SECRET=your-secret-key-here
# === RAILWAY AUTO-PROVIDED ===
# These are automatically set by Railway
# RAILWAY_GIT_COMMIT_SHA=abc1234
# RAILWAY_REGION=us-west1
# RAILWAY_SERVICE_ID=service-id
# RAILWAY_DEPLOYMENT_ID=deployment-id

39
services/public-api/.gitignore vendored Normal file
View File

@@ -0,0 +1,39 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
env/
venv/
.venv/
ENV/
# Environment
.env
.env.local
.env.*.local
# IDE
.vscode/
.idea/
*.swp
*.swo
*~
# OS
.DS_Store
Thumbs.db
# Logs
*.log
# Testing
.pytest_cache/
.coverage
htmlcov/
# Distribution
dist/
build/
*.egg-info/

35
services/public-api/Dockerfile Normal file
View File

@@ -0,0 +1,35 @@
# BlackRoad OS Public API Gateway - Production Dockerfile
FROM python:3.11-slim
# Set working directory
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y \
gcc \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements first (for caching)
COPY requirements.txt .
# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY . .
# Create non-root user
RUN useradd -m -u 1000 blackroad && \
chown -R blackroad:blackroad /app
USER blackroad
# Expose port (Railway will override with $PORT)
EXPOSE 8000
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"
# Start command
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

257
services/public-api/README.md
View File

@@ -1,7 +1,254 @@
# Public API (monorepo-owned)
# BlackRoad OS - Public API Gateway
- **Canonical path:** `services/public-api`
- **Mirror:** `BlackRoad-OS/blackroad-os-api`
- **Branch:** `main`
**Version**: 1.0.0
**Status**: Production Ready
**Framework**: FastAPI 0.104.1
**Python**: 3.11+
The public-facing API surface for BlackRoad OS lives here. Make changes in this directory; the sync workflow will mirror them to `blackroad-os-api`.
> **Canonical path:** `services/public-api`
> **Mirror:** `BlackRoad-OS/blackroad-os-api`
> **Branch:** `main`
>
> The public-facing API surface for BlackRoad OS lives here. Make changes in this directory; the sync workflow will mirror them to `blackroad-os-api`.
---
## Overview
The **Public API Gateway** is the entry point for all external API requests to BlackRoad OS. It routes requests to appropriate backend services:
- **Core API** - Business logic and core operations
- **Operator/Agents API** - AI agent orchestration
- **Other microservices** (future)
### Key Features
- **Intelligent routing** - Routes requests to correct backend
- **Health aggregation** - Monitors all backend services
- **CORS handling** - Configured for web clients
- **Error handling** - Graceful degradation
- **Request proxying** - Transparent forwarding
---
## Endpoints
### Gateway Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/` | GET | Root endpoint with backend info |
| `/health` | GET | Health check + backend status |
| `/version` | GET | Version and deployment info |
### Proxy Routes
| Route | Target | Description |
|-------|--------|-------------|
| `/api/core/*` | Core API | Business logic operations |
| `/api/agents/*` | Operator API | Agent orchestration |
### API Documentation
| Endpoint | Description |
|----------|-------------|
| `/api/docs` | Swagger UI (OpenAPI) |
| `/api/redoc` | ReDoc documentation |
| `/api/openapi.json` | OpenAPI specification |
---
## Architecture
```
┌─────────────────┐
│ External │
│ Clients │
└────────┬────────┘
│
▼
┌─────────────────────────┐
│ Public API Gateway │
│ (This Service) │
│ - Routes requests │
│ - Checks health │
│ - Handles CORS │
└────┬──────────────┬─────┘
│ │
▼ ▼
┌─────────┐ ┌──────────┐
│ Core │ │ Operator │
│ API │ │ API │
└─────────┘ └──────────┘
```
---
## Deployment
### Railway (Production)
**Service Name**: `blackroad-os-api-production`
**URL**: https://blackroad-os-api-production.up.railway.app
**Domain**: https://api.blackroad.systems (via Cloudflare)
#### Required Environment Variables
```bash
ENVIRONMENT=production
PORT=$PORT # Auto-set by Railway
# Backend URLs
CORE_API_URL=https://blackroad-os-core-production.up.railway.app
AGENTS_API_URL=https://blackroad-os-operator-production.up.railway.app
# CORS
ALLOWED_ORIGINS=https://prism.blackroad.systems,https://blackroad.systems,https://api.blackroad.systems
```
### Local Development
```bash
# Create virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
# Copy environment template
cp .env.example .env
# Edit .env with local backend URLs:
# CORE_API_URL=http://localhost:8001
# AGENTS_API_URL=http://localhost:8002
# Run server
uvicorn app.main:app --reload
# Visit http://localhost:8000
```
### Docker
```bash
# Build image
docker build -t blackroad-public-api .
# Run container
docker run -p 8000:8000 \
-e ENVIRONMENT=production \
-e CORE_API_URL=http://core-api:8001 \
-e AGENTS_API_URL=http://operator:8002 \
blackroad-public-api
```
---
## Health Check Response
```json
{
"status": "healthy",
"service": "public-api-gateway",
"version": "1.0.0",
"commit": "abc1234",
"environment": "production",
"timestamp": "2025-11-19T12:00:00Z",
"uptime_seconds": 3600,
"backends": {
"core": "healthy",
"agents": "healthy"
}
}
```
Backend status values:
- `healthy` - Backend is responding correctly
- `unhealthy` - Backend returned non-200 status
- `unreachable` - Backend is not accessible
---
## Routing Examples
### Core API Requests
```bash
# Request to gateway
GET https://api.blackroad.systems/api/core/status
# Proxied to
GET https://blackroad-os-core-production.up.railway.app/api/core/status
```
### Agents API Requests
```bash
# Request to gateway
POST https://api.blackroad.systems/api/agents/deploy
# Proxied to
POST https://blackroad-os-operator-production.up.railway.app/api/agents/deploy
```
---
## Integration
### Consumed By
- **Prism Console** - Admin dashboard
- **BlackRoad OS UI** - Main operating system interface
- **External Clients** - Third-party integrations
- **Mobile Apps** (future)
### Backends
- **Core API** - `$CORE_API_URL`
- **Operator API** - `$AGENTS_API_URL`
### Environment URLs
| Environment | URL |
|-------------|-----|
| Production | https://blackroad-os-api-production.up.railway.app |
| Staging | https://blackroad-os-api-staging.up.railway.app |
| Local | http://localhost:8000 |
---
## Error Handling
### Gateway Errors
- `404` - Route not found (invalid path)
- `500` - Internal gateway error
- `502` - Backend returned invalid response
- `503` - Backend is unreachable
- `504` - Backend timeout (> 30s)
### Backend Errors
Backend errors are passed through transparently with original status codes.
---
## Roadmap
- [ ] Add API key authentication
- [ ] Add rate limiting per client
- [ ] Add request/response logging
- [ ] Add caching layer (Redis)
- [ ] Add request transformation
- [ ] Add circuit breaker pattern
- [ ] Add Prometheus metrics
- [ ] Add distributed tracing
---
## Support
**Operator**: Alexa Louise Amundson
**Repository**: blackboxprogramming/BlackRoad-Operating-System
**Service**: Public API Gateway

2
services/public-api/app/__init__.py Normal file
View File

@@ -0,0 +1,2 @@
"""BlackRoad OS Public API Gateway"""
__version__ = "1.0.0"

262
services/public-api/app/main.py Normal file
View File

@@ -0,0 +1,262 @@
"""
BlackRoad OS Public API Gateway
API gateway that routes requests to:
- Core API (business logic)
- Operator API (agent orchestration)
- Other microservices (future)
"""
from fastapi import FastAPI, Request, Response, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse
import httpx
import os
import time
from datetime import datetime
import platform
from typing import Optional
# App metadata
VERSION = "1.0.0"
COMMIT = os.getenv("RAILWAY_GIT_COMMIT_SHA", "local")[:7]
ENVIRONMENT = os.getenv("ENVIRONMENT", "development")
# Backend service URLs
CORE_API_URL = os.getenv("CORE_API_URL", "http://localhost:8001")
AGENTS_API_URL = os.getenv("AGENTS_API_URL", "http://localhost:8002")
# Create FastAPI app
app = FastAPI(
title="BlackRoad OS Public API Gateway",
description="Public-facing API gateway for BlackRoad Operating System",
version=VERSION,
docs_url="/api/docs",
redoc_url="/api/redoc",
openapi_url="/api/openapi.json"
)
# CORS configuration
ALLOWED_ORIGINS = os.getenv("ALLOWED_ORIGINS", "*").split(",")
app.add_middleware(
CORSMiddleware,
allow_origins=ALLOWED_ORIGINS,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Startup time
START_TIME = time.time()
# HTTP client for proxying
http_client = httpx.AsyncClient(timeout=30.0)
@app.on_event("shutdown")
async def shutdown_event():
"""Close HTTP client on shutdown"""
await http_client.aclose()
@app.get("/")
async def root():
"""Root endpoint"""
return {
"service": "BlackRoad OS Public API Gateway",
"version": VERSION,
"status": "online",
"docs": "/api/docs",
"backends": {
"core": CORE_API_URL,
"agents": AGENTS_API_URL
}
}
@app.get("/health")
async def health_check():
"""
Health check endpoint for Railway and monitoring systems.
Also checks health of backend services.
"""
uptime_seconds = int(time.time() - START_TIME)
# Check backend health
backends_status = {
"core": "unknown",
"agents": "unknown"
}
# Try to ping Core API
try:
core_response = await http_client.get(f"{CORE_API_URL}/health", timeout=5.0)
backends_status["core"] = "healthy" if core_response.status_code == 200 else "unhealthy"
except Exception:
backends_status["core"] = "unreachable"
# Try to ping Agents API
try:
agents_response = await http_client.get(f"{AGENTS_API_URL}/health", timeout=5.0)
backends_status["agents"] = "healthy" if agents_response.status_code == 200 else "unhealthy"
except Exception:
backends_status["agents"] = "unreachable"
# Gateway is healthy if at least one backend is reachable
is_healthy = any(status in ["healthy", "unhealthy"] for status in backends_status.values())
return JSONResponse(
status_code=200 if is_healthy else 503,
content={
"status": "healthy" if is_healthy else "degraded",
"service": "public-api-gateway",
"version": VERSION,
"commit": COMMIT,
"environment": ENVIRONMENT,
"timestamp": datetime.utcnow().isoformat() + "Z",
"uptime_seconds": uptime_seconds,
"backends": backends_status
}
)
@app.get("/version")
async def version_info():
"""Version information"""
return {
"version": VERSION,
"commit": COMMIT,
"environment": ENVIRONMENT,
"python_version": platform.python_version(),
"deployment": {
"platform": "Railway",
"region": os.getenv("RAILWAY_REGION", "unknown"),
"service_id": os.getenv("RAILWAY_SERVICE_ID", "unknown")
},
"backends": {
"core_api": CORE_API_URL,
"agents_api": AGENTS_API_URL
}
}
# ============================================================================
# PROXY ROUTES
# ============================================================================
@app.api_route("/api/core/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH"])
async def proxy_to_core(path: str, request: Request):
"""
Proxy all /api/core/* requests to Core API service.
"""
# Build target URL
target_url = f"{CORE_API_URL}/api/core/{path}"
# Get query params
query_params = dict(request.query_params)
# Get request body if applicable
body = None
if request.method in ["POST", "PUT", "PATCH"]:
body = await request.body()
# Forward request
try:
response = await http_client.request(
method=request.method,
url=target_url,
params=query_params,
content=body,
headers={k: v for k, v in request.headers.items() if k.lower() not in ["host", "content-length"]}
)
return Response(
content=response.content,
status_code=response.status_code,
headers=dict(response.headers),
media_type=response.headers.get("content-type")
)
except httpx.ConnectError:
raise HTTPException(status_code=503, detail="Core API is unreachable")
except httpx.TimeoutException:
raise HTTPException(status_code=504, detail="Core API timeout")
except Exception as e:
raise HTTPException(status_code=502, detail=f"Proxy error: {str(e)}")
@app.api_route("/api/agents/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH"])
async def proxy_to_agents(path: str, request: Request):
"""
Proxy all /api/agents/* requests to Agents/Operator API service.
"""
# Build target URL
target_url = f"{AGENTS_API_URL}/api/agents/{path}"
# Get query params
query_params = dict(request.query_params)
# Get request body if applicable
body = None
if request.method in ["POST", "PUT", "PATCH"]:
body = await request.body()
# Forward request
try:
response = await http_client.request(
method=request.method,
url=target_url,
params=query_params,
content=body,
headers={k: v for k, v in request.headers.items() if k.lower() not in ["host", "content-length"]}
)
return Response(
content=response.content,
status_code=response.status_code,
headers=dict(response.headers),
media_type=response.headers.get("content-type")
)
except httpx.ConnectError:
raise HTTPException(status_code=503, detail="Agents API is unreachable")
except httpx.TimeoutException:
raise HTTPException(status_code=504, detail="Agents API timeout")
except Exception as e:
raise HTTPException(status_code=502, detail=f"Proxy error: {str(e)}")
# Error handlers
@app.exception_handler(404)
async def not_found_handler(request: Request, exc):
"""Custom 404 handler"""
return JSONResponse(
status_code=404,
content={
"error": "Not Found",
"path": str(request.url.path),
"message": "The requested resource was not found",
"hint": "Available routes: /api/core/*, /api/agents/*"
}
)
@app.exception_handler(500)
async def internal_error_handler(request: Request, exc):
"""Custom 500 handler"""
return JSONResponse(
status_code=500,
content={
"error": "Internal Server Error",
"message": "An unexpected error occurred",
"service": "public-api-gateway"
}
)
if __name__ == "__main__":
import uvicorn
port = int(os.getenv("PORT", 8000))
uvicorn.run(
"app.main:app",
host="0.0.0.0",
port=port,
reload=ENVIRONMENT == "development"
)

19
services/public-api/railway.toml Normal file
View File

@@ -0,0 +1,19 @@
# Railway Deployment Configuration for Public API Gateway
[build]
builder = "DOCKERFILE"
dockerfilePath = "Dockerfile"
[deploy]
startCommand = "uvicorn app.main:app --host 0.0.0.0 --port $PORT"
healthcheckPath = "/health"
healthcheckTimeout = 100
restartPolicyType = "ON_FAILURE"
restartPolicyMaxRetries = 10
# Watch patterns for auto-redeploy
watchPatterns = [
"app/**/*.py",
"requirements.txt",
"Dockerfile"
]

14
services/public-api/requirements.txt Normal file
View File

@@ -0,0 +1,14 @@
# BlackRoad OS Public API Gateway Dependencies
# Python 3.11+
# Web Framework
fastapi==0.104.1
uvicorn[standard]==0.24.0
pydantic==2.5.0
pydantic-settings==2.1.0
# HTTP Client (for proxying)
httpx==0.25.2
# Utilities
python-dotenv==1.0.0