Second Brain Pilot

Second Brain Pilot is a job automation engine that connects personal knowledge systems to external services. The system receives events from connected platforms, processes them through AI-powered workflows, and routes outputs to appropriate destinations.

The core insight behind the system is simple: knowledge management isn't useful if it stays trapped in notes. Real value emerges when captured knowledge flows into action - messages sent, documents created, calendars updated, tasks tracked. Second Brain Pilot bridges that gap.

The Problem It Solves

Personal knowledge systems face a common limitation. You accumulate notes, insights, connections between ideas - but acting on that knowledge requires manual intervention. Copy content from your notes, paste it into an email, format it for the recipient, remember to send it. Every action becomes a friction point, and friction kills follow-through.

The alternative is full automation - but generic automation tools don't understand context. They can move data between services, but they can't reason about what should move, when, or how it should be transformed.

Second Brain Pilot sits between these extremes. It understands your knowledge context (through knowledge graph integration), it connects to your services (through OAuth), and it processes events through AI-powered logic (through job definitions). The result is contextual automation - actions that respect what you know and what matters.

Architecture Overview

The system follows a layered architecture with clear separation between concerns.

Backend Layer

The Python FastAPI backend handles API routes, business logic, and external service communication. SQLAlchemy provides the ORM with Alembic managing database migrations. PostgreSQL stores persistent data including user accounts, OAuth tokens, job definitions, and execution logs.

The backend structure follows a standard pattern:

• /api/v1/ - Versioned API endpoints
• /models/ - SQLAlchemy database models
• /schemas/ - Pydantic validation schemas
• /crud/ - Database operations
• /services/ - Business logic and external service clients

This structure scales predictably as integrations and features expand.

Frontend Layer

A React TypeScript frontend built with Vite provides the user interface. Users configure integrations, define jobs, monitor executions, and review logs through a dashboard. Tailwind CSS handles styling.

The frontend communicates with the backend through a typed API client, catching contract mismatches at compile time rather than runtime.

Event Processing Pipeline

Events flow through a defined pipeline:

1. Webhook Receipt - External services send events to registered webhook endpoints
2. Event Parsing - Handlers extract relevant data from service-specific payloads
3. Job Matching - Events are matched to applicable job definitions based on triggers
4. Context Assembly - Knowledge graph integration provides relevant background context
5. AI Processing - Language models process event data with assembled context
6. Destination Routing - Results route to configured output destinations
7. Execution Logging - All steps are logged for debugging and audit

This pipeline handles both simple data routing and complex multi-step workflows.

OAuth Integration Foundation

External service connections use OAuth 2.0 consistently. This provides several benefits:

Standardized Authentication - Each integration follows the same flow: user authorizes access, system receives tokens, tokens are stored securely and refreshed automatically.

User-Controlled Permissions - Users grant specific scopes rather than sharing credentials. They can revoke access at any time through the original service.

Token Management - Access tokens are encrypted at rest. Refresh tokens handle expiration automatically without user intervention.

Extensible Pattern - New integrations follow the same OAuth flow, reducing implementation time and cognitive overhead.

Active Integrations

The system includes integrations for:

Slack - Send and receive workspace messages. Jobs can trigger on channel activity, direct messages, or slash commands. Outputs can post to channels, DMs, or threads.

Google Drive - Read and create documents. Jobs can access file contents for processing and create new documents from job outputs.

Gmail - Send emails and process inbox events. Jobs can respond to incoming mail or generate outgoing messages based on triggers.

Google Calendar - Read and create calendar events. Jobs can trigger on schedule or create calendar entries as outputs.

Each integration exposes both triggers (events that start jobs) and destinations (endpoints that receive job outputs).

Job Definition Model

Jobs are the core unit of automation. Each job specifies:

Triggers - What events start this job. Can be webhook events from integrations, scheduled times, or manual invocation.

Context Sources - Where to pull background information. Includes knowledge graph queries, previous job results, or static configuration.

Processing Logic - How to transform inputs into outputs. Typically involves AI language model calls with prompt templates.

Destinations - Where results should go. Can route to any connected integration or multiple destinations simultaneously.

Error Handling - What happens when steps fail. Includes retry policies, fallback behaviors, and notification rules.

Jobs compose these elements declaratively. The execution engine handles orchestration, parallelization where appropriate, and error recovery.

Event-Driven Patterns

The system embraces event-driven architecture throughout:

Webhook Handlers - Each integration registers webhook endpoints. When external services send events, handlers validate payloads and dispatch to the job matching system.

Async Processing - Job execution is asynchronous. Webhook handlers acknowledge receipt immediately, then queue jobs for background processing. This prevents timeouts on long-running workflows.

Idempotent Execution - Jobs track execution IDs to handle duplicate webhook deliveries gracefully. The same event won't trigger redundant processing.

Ordered Guarantees - Where order matters, the system maintains event sequence. Most workflows tolerate out-of-order delivery, but the option exists when needed.

How It Fits the Ecosystem

Second Brain Pilot represents a specific philosophy about personal infrastructure: the tools you use daily should work together, and AI should enable that integration without requiring constant manual attention.

The system connects to several related efforts:

Knowledge Graph Architecture - The knowledge graph stores notes, connections, and context that inform job processing. Second Brain Pilot consumes this knowledge; it doesn't replace the capture and connection systems that build it.

Claude Skills Framework - Job processing often uses Claude with skill-based prompting. The same skill definitions that guide interactive sessions can guide automated workflows.

GSD Methodology - The phase-based development approach that built Second Brain Pilot also ran on it. Planning documents, research notes, and context files flowed through the same infrastructure.

This isn't coincidental. Building personal infrastructure with itself creates a feedback loop - the tools improve as you use them, and using them reveals what needs improvement.

Development Approach

Second Brain Pilot development follows a phased implementation pattern:

Phase-Based Progress - Each phase addresses a specific capability area. Phases include research, planning, implementation, and verification steps.

Integration Sequence - OAuth foundation came first, then individual service integrations, then job routing, then cross-integration workflows. Each layer built on the previous.

Incremental Value - Every phase delivers working functionality. You don't wait for the complete system to get value from partial implementation.

Verification Gates - Each phase ends with verification that the delivered functionality actually works in practice, not just in theory.

At time of documentation, the system is in Phase 19, focusing on integration wiring fixes after completing the core integration suite (Slack, Google Drive, Gmail, Google Calendar).

Getting Started

Setting up Second Brain Pilot requires:

1. Environment Configuration - PostgreSQL database, environment variables for OAuth client credentials
2. Integration Setup - Register OAuth applications with each service you want to connect
3. User Authentication - Create an account and authorize desired integrations through OAuth flows
4. Job Definition - Define your first job with trigger, processing logic, and destination
5. Webhook Configuration - Register webhook URLs with external services to receive events

The frontend dashboard guides most of these steps. OAuth flows handle the complex authentication handshakes.

Where This Goes

Second Brain Pilot solves the bridge problem between knowledge and action. Future directions include:

Additional Integrations - The OAuth pattern extends to any service with API access. Common targets include task managers, note systems, communication platforms.

Workflow Composition - Jobs that trigger other jobs enable multi-step workflows. Complex automations compose from simple building blocks.

Knowledge Graph Writeback - Not just reading context from the knowledge graph, but writing back learnings from job executions.

Collaborative Patterns - Extending from personal infrastructure to small-team coordination while preserving individual autonomy.

The foundation exists. Extensions follow the same patterns that built the core.