mirror of
https://github.com/blackboxprogramming/BlackRoad-Operating-System.git
synced 2026-03-17 05:57:21 -05:00
⚡ Add BlackRoad OS Kernel, DNS Infrastructure, and Complete Service Registry
This commit introduces a comprehensive infrastructure overhaul that transforms
BlackRoad OS into a true distributed operating system with unified kernel,
DNS-aware service discovery, and standardized syscall APIs.
## New Infrastructure Components
### 1. Kernel Module (kernel/typescript/)
- Complete TypeScript kernel implementation for all services
- Service registry with production and dev DNS mappings
- RPC client for inter-service communication
- Event bus, job queue, state management
- Structured logging with log levels
- Full type safety with TypeScript
Modules:
- types.ts: Complete type definitions
- serviceRegistry.ts: DNS-aware service discovery
- identity.ts: Service identity and metadata
- config.ts: Environment-aware configuration
- logger.ts: Structured logging
- rpc.ts: Inter-service RPC client
- events.ts: Event bus (pub/sub)
- jobs.ts: Background job queue
- state.ts: Key-value state management
- index.ts: Main exports
### 2. DNS Infrastructure Documentation (infra/DNS.md)
- Complete Cloudflare DNS mapping
- Railway production and dev endpoints
- Email configuration (MX, SPF, DKIM, DMARC)
- SSL/TLS, security, and monitoring settings
- Service-to-domain mapping
- Health check configuration
Production Services:
- operator.blackroad.systems
- core.blackroad.systems
- api.blackroad.systems
- console.blackroad.systems
- docs.blackroad.systems
- web.blackroad.systems
- os.blackroad.systems
- app.blackroad.systems
### 3. Service Registry & Architecture (INFRASTRUCTURE.md)
- Canonical service registry with all endpoints
- Monorepo-to-satellite deployment model
- Service-as-process architecture
- DNS-as-filesystem model
- Inter-service communication patterns
- Service lifecycle management
- Complete environment variable documentation
### 4. Syscall API Specification (SYSCALL_API.md)
- Standard kernel API for all services
- Required syscalls: health, version, identity, RPC
- Optional syscalls: logging, metrics, events, jobs, state
- Complete API documentation with examples
- Express.js implementation guide
Core Endpoints:
- GET /health
- GET /version
- GET /v1/sys/identity
- GET /v1/sys/health
- POST /v1/sys/rpc
- POST /v1/sys/event
- POST /v1/sys/job
- GET/PUT /v1/sys/state
### 5. Railway Deployment Guide (docs/RAILWAY_DEPLOYMENT.md)
- Step-by-step deployment instructions
- Environment variable configuration
- Monitoring and health checks
- Troubleshooting guide
- Best practices for Railway deployment
### 6. Atlas Kernel Scaffold Prompt (prompts/atlas/ATLAS_KERNEL_SCAFFOLD.md)
- Complete prompt for generating new services
- Auto-generates full kernel implementation
- Includes all DNS and Railway mappings
- Production-ready output with zero TODOs
### 7. GitHub Workflow Templates (templates/github-workflows/)
- deploy.yml: Railway auto-deployment
- test.yml: Test suite with coverage
- validate-kernel.yml: Kernel validation
- README.md: Template documentation
## Updated Files
### CLAUDE.md
- Added "Kernel Architecture & DNS Infrastructure" section
- Updated Table of Contents
- Added service architecture diagram
- Documented all new infrastructure files
- Updated repository structure with new directories
- Added kernel and infrastructure to critical path files
## Architecture Impact
This update establishes BlackRoad OS as a distributed operating system where:
- Each Railway service = OS process
- Each Cloudflare domain = mount point
- All services communicate via syscalls
- Unified kernel ensures interoperability
- DNS-aware service discovery
- Production and development environments
## Service Discovery
Services can now discover and call each other:
```typescript
import { rpc } from './kernel';
const user = await rpc.call('core', 'getUserById', { id: 123 });
```
## DNS Mappings
Production:
- operator.blackroad.systems → blackroad-os-operator-production-3983.up.railway.app
- core.blackroad.systems → 9gw4d0h2.up.railway.app
- api.blackroad.systems → ac7bx15h.up.railway.app
Internal (Railway):
- blackroad-os-operator.railway.internal:8001
- blackroad-os-core.railway.internal:8000
- blackroad-os-api.railway.internal:8000
## Next Steps
1. Sync kernel to satellite repos
2. Implement syscall endpoints in all services
3. Update services to use RPC for inter-service calls
4. Configure Cloudflare health checks
5. Deploy updated services to Railway
---
Files Added:
- INFRASTRUCTURE.md
- SYSCALL_API.md
- infra/DNS.md
- docs/RAILWAY_DEPLOYMENT.md
- kernel/typescript/* (9 modules + README)
- prompts/atlas/ATLAS_KERNEL_SCAFFOLD.md
- templates/github-workflows/* (4 files)
Files Modified:
- CLAUDE.md
Total: 22 new files, 1 updated file
This commit is contained in:
182
kernel/typescript/jobs.ts
Normal file
182
kernel/typescript/jobs.ts
Normal file
@@ -0,0 +1,182 @@
|
||||
/**
|
||||
* BlackRoad OS - Job Queue
|
||||
*
|
||||
* In-memory job queue for background tasks.
|
||||
* Supports job creation, status tracking, and cancellation.
|
||||
*
|
||||
* @version 2.0
|
||||
* @author Atlas (Infrastructure Architect)
|
||||
*/
|
||||
|
||||
import { Job, JobStatus } from './types';
|
||||
import { logger } from './logger';
|
||||
|
||||
type JobHandler = (params: any) => Promise<any>;
|
||||
|
||||
/**
|
||||
* Job Queue Manager
|
||||
*/
|
||||
export class JobQueue {
|
||||
private jobs: Map<string, Job>;
|
||||
private handlers: Map<string, JobHandler>;
|
||||
private running: Map<string, AbortController>;
|
||||
|
||||
constructor() {
|
||||
this.jobs = new Map();
|
||||
this.handlers = new Map();
|
||||
this.running = new Map();
|
||||
}
|
||||
|
||||
/**
|
||||
* Register a job handler
|
||||
*/
|
||||
registerHandler(name: string, handler: JobHandler): void {
|
||||
this.handlers.set(name, handler);
|
||||
logger.debug(`[Jobs] Registered handler: ${name}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Create and queue a job
|
||||
*/
|
||||
async createJob(
|
||||
name: string,
|
||||
params?: Record<string, any>,
|
||||
schedule?: string
|
||||
): Promise<Job> {
|
||||
const handler = this.handlers.get(name);
|
||||
if (!handler) {
|
||||
throw new Error(`No handler registered for job: ${name}`);
|
||||
}
|
||||
|
||||
const job: Job = {
|
||||
id: this.generateJobId(),
|
||||
name,
|
||||
params,
|
||||
schedule,
|
||||
status: 'queued',
|
||||
createdAt: new Date().toISOString(),
|
||||
};
|
||||
|
||||
this.jobs.set(job.id, job);
|
||||
logger.info(`[Jobs] Job created: ${name} (${job.id})`);
|
||||
|
||||
// Execute immediately (no schedule support yet)
|
||||
if (!schedule) {
|
||||
await this.executeJob(job.id);
|
||||
}
|
||||
|
||||
return job;
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a job
|
||||
*/
|
||||
private async executeJob(jobId: string): Promise<void> {
|
||||
const job = this.jobs.get(jobId);
|
||||
if (!job) {
|
||||
throw new Error(`Job not found: ${jobId}`);
|
||||
}
|
||||
|
||||
const handler = this.handlers.get(job.name);
|
||||
if (!handler) {
|
||||
throw new Error(`No handler for job: ${job.name}`);
|
||||
}
|
||||
|
||||
job.status = 'running';
|
||||
job.startedAt = new Date().toISOString();
|
||||
logger.info(`[Jobs] Job started: ${job.name} (${job.id})`);
|
||||
|
||||
const abortController = new AbortController();
|
||||
this.running.set(jobId, abortController);
|
||||
|
||||
try {
|
||||
const result = await handler(job.params);
|
||||
|
||||
job.status = 'completed';
|
||||
job.completedAt = new Date().toISOString();
|
||||
job.result = result;
|
||||
logger.info(`[Jobs] Job completed: ${job.name} (${job.id})`);
|
||||
} catch (error) {
|
||||
job.status = 'failed';
|
||||
job.completedAt = new Date().toISOString();
|
||||
job.error = {
|
||||
message: error instanceof Error ? error.message : String(error),
|
||||
stack: error instanceof Error ? error.stack : undefined,
|
||||
};
|
||||
logger.error(`[Jobs] Job failed: ${job.name} (${job.id})`, {
|
||||
error: job.error.message,
|
||||
});
|
||||
} finally {
|
||||
this.running.delete(jobId);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get job status
|
||||
*/
|
||||
getJob(jobId: string): Job | undefined {
|
||||
return this.jobs.get(jobId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all jobs
|
||||
*/
|
||||
getAllJobs(): Job[] {
|
||||
return Array.from(this.jobs.values());
|
||||
}
|
||||
|
||||
/**
|
||||
* Get jobs by status
|
||||
*/
|
||||
getJobsByStatus(status: JobStatus): Job[] {
|
||||
return Array.from(this.jobs.values()).filter((job) => job.status === status);
|
||||
}
|
||||
|
||||
/**
|
||||
* Cancel a job
|
||||
*/
|
||||
async cancelJob(jobId: string): Promise<void> {
|
||||
const job = this.jobs.get(jobId);
|
||||
if (!job) {
|
||||
throw new Error(`Job not found: ${jobId}`);
|
||||
}
|
||||
|
||||
if (job.status === 'completed' || job.status === 'failed' || job.status === 'cancelled') {
|
||||
throw new Error(`Cannot cancel job in status: ${job.status}`);
|
||||
}
|
||||
|
||||
const abortController = this.running.get(jobId);
|
||||
if (abortController) {
|
||||
abortController.abort();
|
||||
}
|
||||
|
||||
job.status = 'cancelled';
|
||||
job.completedAt = new Date().toISOString();
|
||||
logger.info(`[Jobs] Job cancelled: ${job.name} (${job.id})`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear completed jobs
|
||||
*/
|
||||
clearCompleted(): void {
|
||||
for (const [id, job] of this.jobs.entries()) {
|
||||
if (job.status === 'completed' || job.status === 'failed' || job.status === 'cancelled') {
|
||||
this.jobs.delete(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate unique job ID
|
||||
*/
|
||||
private generateJobId(): string {
|
||||
return `job-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Global job queue instance
|
||||
*/
|
||||
export const jobQueue = new JobQueue();
|
||||
|
||||
export default jobQueue;
|
||||
Reference in New Issue
Block a user