Hybrid Local/Cloud LLM System: Architecture Guide
The CLAUDE.md that powers a production hybrid routing system - complexity-based scoring from 1-10, automatic model selection across four tiers (local Qwen through cloud Opus), contextual RAG embeddings that improve retrieval by 5-10%, and the architecture that achieves 95-99% cost savings versus all-cloud
The architectural guide for a hybrid local/cloud LLM system - covering the complexity scoring engine (1-10 scale routing across Qwen 2.5-7B, DeepSeek-Coder, Claude Haiku, Sonnet, and Opus), the RAG system with contextual embeddings (LLM-generated context improves code retrieval by 5-10%), production hardening across 8 phases (circuit breaker, rate limiting, structured logging), the FastMCP server exposing 10 tools to Claude Desktop, and the cost tracking system that validates 95-99% savings.
Also connected to
The CLAUDE.md that powers a production hybrid routing system - complexity-based scoring from 1-10, automatic model selection across four tiers (local Qwen through cloud Opus), contextual RAG embeddings that improve retrieval by 5-10%, and the architecture that achieves 95-99% cost savings versus all-cloud
Your LLM can write poetry but can't reliably add two numbers - hybrid architectures solve this by routing each subtask to the system that actually handles it well
Real benchmarks from a three-stage data ingestion pipeline - comparing Llama 3.1 70B, Qwen 2.5 14B, and Llama 3.1 8B on relationship extraction, category assignment, and portfolio curation, with the performance metrics that actually matter for production selection
A production-grade routing system that cuts LLM costs by 95-99% - complexity scoring routes simple queries to free local Ollama models while sending complex reasoning to Claude, with RAG semantic search, real-time monitoring, and 10 MCP tools for Claude Desktop integration
The complete guide to running local LLMs on an RTX 5070 with 12GB VRAM - model recommendations by task type, inference engine comparisons, quantization strategies, Claude Code integration patterns, and the multi-model architecture that handles everything from free coding assistance to privacy-first document processing