blackroad-operating-system/.templates/test-orchestrator/PROMPTS.md

AI Assistant Prompts for Test Orchestrator Pattern

Ready-to-use prompts for Claude, Copilot, ChatGPT, and other AI assistants

These prompts help AI assistants implement the Test Orchestrator pattern in any repository.

---

Quick Prompt: Implement Test Orchestrator

Copy and paste this into your AI assistant:

You are an AI engineer helping me implement the BlackRoad Test Orchestrator pattern in my repository.

## Context
I want to create a unified test orchestrator script (test_all.sh) and matching CI workflow for my monorepo.

Repository structure:
[PASTE YOUR DIRECTORY STRUCTURE HERE - use `tree -L 2` or describe it]

Current test commands:
- Component 1: [your test command]
- Component 2: [your test command]
- Component 3: [your test command]

## Task
Using the BlackRoad Test Orchestrator templates located at:
- .templates/test-orchestrator/test_all.sh.template
- .templates/test-orchestrator/test-orchestrator.yml.template
- .templates/test-orchestrator/TESTING.md.template

Please:

1. **Create test_all.sh** adapted to my project structure
   - Replace placeholder suite functions with my actual components
   - Use the correct test commands for each suite
   - Update the suite list in print_summary()
   - Customize the help text

2. **Create .github/workflows/test-orchestrator.yml**
   - Add service containers I need (e.g., Postgres, Redis, etc.)
   - Set up the correct language runtimes (Python, Node, Go, etc.)
   - Configure environment variables
   - Add appropriate caching

3. **Create TESTING.md**
   - Document my specific test suites
   - Include my setup instructions
   - Add troubleshooting for my stack

4. **Make it executable**
   - Ensure test_all.sh has executable permissions

## Constraints
- Keep the same structure and helper functions from the template
- Maintain the two-mode design (best-effort and strict)
- Keep the summary table format
- Don't remove the color-coded output
- Follow the same naming conventions

## Output Format
Provide:
1. Complete test_all.sh file
2. Complete .github/workflows/test-orchestrator.yml file
3. Complete TESTING.md file
4. Brief explanation of what you changed

Ready? Let's implement this!

---

Detailed Prompt: Custom Test Orchestrator

For more complex projects, use this expanded prompt:

I want to implement a comprehensive test orchestrator for my {{LANGUAGE/STACK}} project.

## Project Details

**Name**: {{REPO_NAME}}
**Description**: {{PROJECT_DESCRIPTION}}
**Primary Languages**: {{e.g., Python, TypeScript, Go, Rust}}
**Test Frameworks**: {{e.g., pytest, Jest, Go testing, Cargo}}

## Current Structure

[PASTE TREE OUTPUT OR DESCRIBE STRUCTURE]

## Test Suites

I have the following test suites:

1. **{{Suite Name}}**
   - Location: `{{path}}`
   - Framework: {{framework}}
   - Command: `{{test command}}`
   - Dependencies: {{databases, services, etc.}}

2. **{{Suite Name}}**
   - Location: `{{path}}`
   - Framework: {{framework}}
   - Command: `{{test command}}`
   - Dependencies: {{databases, services, etc.}}

[Add more suites...]

## Service Dependencies

For CI, I need:
- [ ] PostgreSQL (version: {{version}})
- [ ] Redis (version: {{version}})
- [ ] MySQL (version: {{version}})
- [ ] MongoDB (version: {{version}})
- [ ] Other: {{describe}}

## Environment Variables

My tests need these environment variables:
```bash
{{VAR_NAME}}={{description}}
{{VAR_NAME}}={{description}}

Special Requirements

• Need to run database migrations before tests
• Need to seed test data
• Need to build artifacts before testing
• Need to run linters/formatters
• Need to generate coverage reports
• Other: {{describe}}

Task

Create a complete test orchestrator implementation based on the BlackRoad pattern that:

1. test_all.sh that:

   • Runs all my test suites
   • Handles my specific frameworks
   • Sets up my service dependencies (locally or via Docker)
   • Has clear, color-coded output
   • Supports best-effort and strict modes
   • Generates a summary table

2. GitHub Actions workflow that:

   • Sets up all required service containers
   • Installs all language runtimes
   • Caches dependencies appropriately
   • Runs the test orchestrator
   • Uploads artifacts (coverage, reports)
   • Posts results to PRs

3. TESTING.md that:

   • Documents my setup process
   • Explains each test suite
   • Provides troubleshooting for my stack
   • Includes examples for my frameworks

4. Optional enhancements:

   • Matrix testing across versions
   • Coverage reporting (Codecov integration)
   • Performance benchmarking
   • Slack/Discord notifications

Output Format

Provide complete, ready-to-use files with:

• No placeholders (use my actual values)
• Comments explaining key sections
• Examples relevant to my stack
• Clear migration instructions from my current setup

Let's build this!

---

## Minimal Prompt: Quick Setup

For simple projects:

Implement the BlackRoad Test Orchestrator pattern for my project.

My test commands:

• {{component}}: {{command}}
• {{component}}: {{command}}

Create:

1. test_all.sh (based on .templates/test-orchestrator/test_all.sh.template)
2. .github/workflows/test-orchestrator.yml
3. Make it work with my stack: {{language/framework}}

Keep it simple and working. Go!

---

## Migration Prompt: From Existing Setup

If you already have test scripts:

I want to migrate my existing test scripts to the BlackRoad Test Orchestrator pattern.

Current Setup

I currently run tests using: [DESCRIBE CURRENT APPROACH - Makefile, scripts, npm scripts, etc.]

Current test scripts:

• {{path/to/script1}}: {{what it does}}
• {{path/to/script2}}: {{what it does}}

Current CI workflow: [PASTE .github/workflows/*.yml or describe]

Goal

Consolidate everything into:

1. Single test_all.sh orchestrator
2. Unified GitHub Actions workflow
3. Clear documentation

Migration Strategy

I want to:

• Keep my existing scripts temporarily (call them from orchestrator)
• Refactor everything into the orchestrator directly
• Migrate gradually (one suite at a time)

Please:

1. Analyze my current setup
2. Propose a migration plan
3. Implement the orchestrator calling my existing scripts
4. Suggest refactoring opportunities

Let's migrate!

---

## Enhancement Prompt: Add Features

To add features to existing orchestrator:

I already have the BlackRoad Test Orchestrator implemented.

I want to add:

• Matrix testing (test across {{Python 3.9, 3.10, 3.11}} or {{Node 18, 20, 22}})
• Coverage reporting with Codecov
• Performance benchmarking
• Slack/Discord notifications on failure
• Parallel test execution
• Test result caching
• Flaky test detection
• Other: {{describe}}

Current test_all.sh location: ./test_all.sh Current workflow location: .github/workflows/test-orchestrator.yml

Please:

1. Show me how to implement {{feature}}
2. Update both test_all.sh and the workflow
3. Explain the tradeoffs
4. Provide configuration examples

Let's enhance!

---

## Troubleshooting Prompt

When something isn't working:

My Test Orchestrator isn't working correctly.

Problem

{{DESCRIBE THE ISSUE}}

What I've Tried

{{WHAT YOU'VE TRIED}}

Error Output

{{PASTE ERROR OUTPUT}}

Current Configuration

test_all.sh:

{{PASTE RELEVANT SECTION}}

Workflow:

{{PASTE RELEVANT SECTION}}

Environment

• OS: {{Linux/macOS/Windows}}
• CI: {{GitHub Actions/GitLab CI/CircleCI}}
• Languages: {{Python 3.11, Node 20, etc.}}

Please:

1. Diagnose the issue
2. Provide a fix
3. Explain why it happened
4. Suggest prevention strategies

Help!

---

## Framework-Specific Prompts

### Python (pytest) Project

Implement BlackRoad Test Orchestrator for my Python project.

Structure:

• src/myapp/
• tests/
  • unit/
  • integration/
  • e2e/
• pytest.ini
• requirements.txt
• requirements-dev.txt

I want:

• Separate suites for unit, integration, e2e
• Coverage reporting (pytest-cov)
• PostgreSQL for integration tests
• Environment isolation (venv)

Create test orchestrator with these suites.

### Node.js (Jest) Project

Implement BlackRoad Test Orchestrator for my Node.js monorepo.

Structure:

• packages/
  • api/ (Express, Jest)
  • web/ (React, Vitest)
  • sdk/ (TypeScript, Jest)
• package.json (workspace)

I want:

• Suite per package
• Coverage aggregation
• MongoDB for API tests
• Redis for caching tests

Create test orchestrator for this setup.

### Go Project

Implement BlackRoad Test Orchestrator for my Go project.

Structure:

• cmd/myapp/
• internal/
• pkg/
• tests/
  • unit/
  • integration/

Test commands:

• Unit: go test ./...
• Integration: go test -tags=integration ./tests/integration/...
• Benchmarks: go test -bench=. ./...

Create orchestrator that handles these.

### Rust Project

Implement BlackRoad Test Orchestrator for my Rust workspace.

Structure:

• crates/
  • api/
  • core/
  • cli/
• Cargo.toml (workspace)

Test commands:

• All: cargo test --all
• Per crate: cargo test -p
• Doc tests: cargo test --doc

Create orchestrator for Rust testing.

---

## Copilot-Specific Tips

When using **GitHub Copilot Chat**:

1. **Open the template files first** so Copilot has context
2. **Use `/tests` command** to generate suite-specific tests
3. **Reference files explicitly**: "Using .templates/test-orchestrator/test_all.sh.template as a base..."
4. **Iterate in chat**: Ask for changes, then refine

Example Copilot Chat flow:

You: Using .templates/test-orchestrator/test_all.sh.template, create test_all.sh for my Python project

Copilot: [generates script]

You: Add a suite for my FastAPI backend in backend/

Copilot: [adds suite]

You: Make the PostgreSQL connection optional

Copilot: [updates script]

---

## Claude-Specific Tips

When using **Claude**:

1. **Provide full context** in one message
2. **Use code blocks** for templates
3. **Ask for complete files** rather than diffs
4. **Request explanations** for changes

Example Claude prompt:

I'm going to paste three template files. Then I want you to adapt them for my project.

[Template 1: test_all.sh.template] {{paste file}}

[Template 2: test-orchestrator.yml.template] {{paste file}}

[Template 3: TESTING.md.template] {{paste file}}

My project structure: {{describe}}

Please create adapted versions for my project, replacing all placeholders with real values.

---

## ChatGPT-Specific Tips

When using **ChatGPT**:

1. **Break into smaller steps** if context window is limited
2. **Use code interpreter** for testing syntax
3. **Ask for alternatives** to see different approaches
4. **Iterate on sections** rather than whole files

Example ChatGPT flow:

Me: I want to implement a test orchestrator. Here's my project structure: [paste]

ChatGPT: [suggests approach]

Me: Let's start with the test_all.sh script. Here's the template: [paste]

ChatGPT: [creates script]

Me: Now the GitHub Actions workflow...

[Continue]

---

## Tips for All AI Assistants

### 1. Provide Context

Good:

My Python monorepo has:

• Backend (FastAPI, pytest)
• Workers (Celery, pytest)
• SDK (httpx, pytest)

Currently I run: cd backend && pytest, cd workers && pytest, etc.

I want one test_all.sh to run everything.

Bad:

Make me a test script.

### 2. Show, Don't Tell

Good:

My test suite structure: tests/ ├── unit/ │ ├── test_auth.py │ └── test_users.py ├── integration/ │ ├── test_api.py │ └── test_db.py └── conftest.py

I run: pytest tests/unit/ then pytest tests/integration/

Bad:

I have some tests in a tests folder.

### 3. Specify Exactly What You Need

Good:

Create:

1. test_all.sh with suites: backend, frontend, e2e
2. GitHub Actions workflow with Postgres and Redis
3. TESTING.md with setup instructions

Don't include coverage reporting yet.

Bad:

Make it work.

### 4. Iterate

It's okay to refine! After getting initial output:

This is close! Can you:

• Change the timeout from 30 to 60 minutes
• Add a suite for my docs (using mdbook test)
• Remove the Redis service (I don't need it)

---

## Success Checklist

After implementing, verify:

- [ ] `./test_all.sh` runs and shows summary
- [ ] `./test_all.sh --help` shows your suites
- [ ] `./test_all.sh --suite <name>` runs specific suite
- [ ] `./test_all.sh --strict` fails fast on errors
- [ ] GitHub Actions workflow passes
- [ ] CI output is clear and helpful
- [ ] TESTING.md documents your setup
- [ ] New developers can run tests easily

---

## Common Pitfalls to Avoid

❌ Don't:
- Hardcode paths without checking if they exist
- Skip error handling in suite functions
- Forget to update the suite list in `print_summary()`
- Remove the color output (it's helpful!)
- Make the script depend on complex setup

✅ Do:
- Check if directories exist before running tests
- Record results for every suite (PASS/FAIL/SKIP)
- Update all three places when adding a suite (function, main execution, summary)
- Keep the clean, color-coded output
- Make it work out of the box or fail clearly

---

## Next Steps After Implementation

1. **Test it locally**: `./test_all.sh`
2. **Commit the changes**: `git add test_all.sh .github/workflows/ TESTING.md`
3. **Push and verify CI**: Check GitHub Actions
4. **Update team docs**: Link to TESTING.md in README
5. **Share the pattern**: Help other teams adopt it

---

## Support

- **Original Pattern**: BlackRoad Operating System
- **Templates**: `.templates/test-orchestrator/`
- **Issues**: Open an issue in your repository
- **Questions**: Refer to TESTING.md

---

**Happy Orchestrating! 🎼✨**

---

*These prompts are designed to work with Claude, GitHub Copilot, ChatGPT, and other AI assistants.*
*Adapt them to your specific needs and workflow.*