Pre-Launch Checklist
Don’t skip this. The best time to find a problem is before users do.
1. Agent Quality
Agent Quality Checks
- Agent has one clear, specific job — not “be helpful with everything”
- Instructions include: what to do, what tone to use, what to refuse
- Instructions include explicit rules for edge cases you’ve identified
- Agent doesn’t hallucinate answers when tools return no results
- Agent asks a useful clarifying question when the request is vague
- Response format matches the deployment channel (markdown for web, Telegram format for Telegram)
- Responses are under 200 words for simple questions
2. Tools & Integrations
Tool & Integration Checks
- Every connected tool has a clear reason for being connected
- Read-only tools are used where write access isn’t needed
- High-risk write tools (payments, email sends) require explicit confirmation in instructions
- All integration credentials are stored as environment variables — not hardcoded
- OAuth tokens are connected and not expired
- Each tool was tested in the Playground at least once with real data
- Tool failure responses are tested — agent should gracefully explain failures
3. Memory
Memory Checks
- The agent remembers context across at least 3 messages in a test conversation
- Sensitive data (payment cards, one-time codes) is not stored in persistent memory
- Memory keys have descriptive names, not generic ones (
"user_name", not"name") - Session IDs are stable and user-scoped (for API channel)
- Memory TTLs are set appropriately for the use case
4. Deployment
Deployment Checks
- Agent deployed to one channel and tested end-to-end
- Webhook is live and verified (for Telegram and Slack)
- API key is set in environment — not exposed in client code
- Rate limits are configured: runs/user/hour and runs/day
- A rollback plan exists: know which deployment to revert to
- Staging environment tested before production deploy
5. Observability
Observability Checks
- A test run trace is visible in the Runs tab
- The trace shows: user input, context loaded, tools called, response
- Error runs show useful failure information — not just “Error”
- At least one alert is configured (high error rate, slow latency)
- You know how to use HiveMind Debug to diagnose a bad run
6. Security
Security Checks
- No secrets in HiveLang source files or version control
- API channel requires authentication — public endpoint is not exposed
- Agent refuses requests that are clearly outside its scope
- Agent does not expose other users’ data
- High-risk actions (refunds, account changes) require explicit user confirmation
- You have tested a prompt injection attempt — agent should not follow malicious instructions
Post-launch monitoring
1
Watch the first 50 runs
Open the Runs tab and review each run manually for the first 50 real user interactions. Fix anything that looks wrong immediately.
2
Check error rate daily for week 1
A healthy error rate is under 5%. If you see spikes, open the traces immediately — don’t wait.
3
Review user confusion patterns
Look for repeated clarifying questions from users — this signals a gap in your instructions. Fix it the same day.
4
Set a weekly review cadence
Every week, spend 15 minutes reviewing runs and looking for improvement opportunities. The best agents are iterated, not launched and forgotten.