Project Overview
Tabitha is an enterprise conversational AI chat assistant designed to automate corporate knowledge retrieval, customer support ticket triage, and compliance workflows without exposing private corporate telemetry to public model training sets.
The Architectural Challenge
Enterprise clients required stringent zero-retention data privacy guarantees, SOC2 compliance, and sub-second token streaming responses. The system needed to orchestrate multiple LLM foundation models dynamically based on query complexity and cost parameters.
Engineering Solution & Tech Stack
Eternitech engineered a modular AI gateway leveraging FastAPI, streaming Server-Sent Events (SSE), and pgvector PostgreSQL embeddings. We designed a client-side encryption layer and automated prompt injection moderation filter that sanitizes all incoming queries before dispatching to model endpoints.
- LLM API Orchestration: Dynamic routing across Claude, GPT-4, and self-hosted open-weights models.
- Vector Search & RAG: Chunking and semantic indexing with pgvector and hybrid keyword search.
- React & Tailwind: Fluid, accessible streaming conversational user interface.
- Security & Moderation: Real-time PII redacting and automated prompt injection defense.
Business Outcomes & Impact
Tabitha reduced customer support response times by 74% across enterprise pilot deployments while cutting operational API costs through intelligent prompt caching and model tiering.