REFERENCE ARCHITECTURE · ENTERPRISE AI & LLMOPS
Production AI Requires More Than a Model.
Instrument your AI applications and agents with a dependable observability platform. This reference architecture covers where Langfuse, ClickHouse, PostgreSQL, Redis and MinIO AIStor fit—and where LLM inference, retrieval, permissions and application orchestration remain separate decisions.
For AI engineering leaders · platform engineers · enterprise architects. Stack: Langfuse · ClickHouse · PostgreSQL · Redis · MinIO AIStor
THE CHALLENGE
AI Demos Are Easy. Operating AI Is Not.
Once an LLM application or agent reaches real users, teams need to see what it did, why it did it, what it cost and whether quality is improving or regressing. That visibility depends on an observability platform with its own databases, queues and storage, which must be sized, secured and recovered like any other production system. At the same time, prompts and responses can contain personal or confidential data, so telemetry itself becomes a governed data asset.
- Limited visibility — No consistent trace of prompts, retrieval steps, tool calls, model responses, latency and token cost across applications and agents.
- Quality without measurement — Changes to prompts, models or retrieval are shipped without evaluation scores or regression baselines.
- Hidden runtime dependencies — Self-hosted observability needs transactional, analytical, queue and object storage components that are often under-sized or unowned.
- Sensitive telemetry — Traces may capture personal, financial or proprietary content that must be masked, retained and located according to policy.
ARCHITECTURE OVERVIEW
How Requests and Telemetry Flow
The pattern separates three concerns: the AI application path that serves users, the observability platform that records and evaluates what happened, and the controls that protect both. Langfuse observes the application; it does not run your models, retrieval or permissions.
Figure 1. Enterprise AI & LLMOps reference architecture (conceptual). Colour key: see How to read our diagrams.
Logical Request and Telemetry Flow
- Call the model and retrievalApplication / agent orchestrator calls an LLM or retrieval layer; the model provider or self-hosted inference service is external to Langfuse.
- Emit masked telemetryApplication SDK emits traces, observations, scores and evaluations into Langfuse Web/API; appropriate privacy controls mask or filter sensitive content.
- Queue asynchronous workLangfuse Worker processes asynchronous work. Redis or compatible cache/queue supports event buffering and background jobs.
- Store analytics, state and blobsClickHouse stores observability trace/observation/score analytics; PostgreSQL supports transactional application metadata and configuration; S3-compatible MinIO AIStor stores event blobs and multimodal assets.
- Optional semantic cacheOptional separate Redis semantic cache accelerates AI application retrieval or repeated prompts; it must not be confused with Langfuse’s required Redis queue/cache usage.
- Protect the full systemIdentity, networking, secrets, encryption, retention, backup/restore, alerting and deployment-specific HA/DR protect the full system.
Conceptual pattern. Connectors, licensed capabilities, versions, security policies and topology are validated in a project-specific architecture.
Last reviewed: — · Version scope: —
COMPONENT ROLES
Who Does What in the Stack
Each component has one clear responsibility. Classifications show whether a component is delivered through a strategic technology partner or independent open-source engineering.
| Component | Classification | Role | Qualification |
|---|---|---|---|
| Langfuse (Web/API + Worker) | Strategic partner | Instrument, trace, evaluate and manage prompt-related workflows | Exact features depend on deployed version and edition |
| ClickHouse | Partner-led or ClickHouse OSS route | Langfuse analytical events and query serving | Do not write custom events directly to Langfuse internal tables |
| PostgreSQL | Independent OSS support | Transactional Langfuse application state | Required in supported self-hosted patterns |
| Redis / compatible | Strategic partner (Redis) | Ingestion queue and application cache | Queue durability, eviction mode and sizing require design |
| MinIO AIStor | Strategic partner | S3-compatible blob assets, raw/multimodal data | Validate object-store config and residency requirements |
| AI runtime / vector retrieval | Customer-owned or separately selected | Application/inference-side model and retrieval choices | Not provided automatically by the Langfuse observability platform |
Links: Langfuse · ClickHouse partner · ClickHouse OSS · PostgreSQL · Redis · MinIO AIStor
Already running Langfuse?
We can review sizing, queue configuration and recovery for your current release.
DESIGN TRADEOFFS
Decisions That Shape the Platform
There is no single correct deployment. These are the decisions we work through with your team, with the evidence needed to close each one.
| Decision | Options | Choose based on | Validate with |
|---|---|---|---|
| Langfuse hosting model | Self-hosted in customer infrastructure · Vendor-hosted cloud, where applicable | Data residency, telemetry sensitivity, operating capacity, required enterprise features | Data-flow review, edition/feature check, residency sign-off |
| ClickHouse support route | Partner-led ClickHouse · Independently operated ClickHouse OSS | Required commercial assurances, scale, in-house skills | Version compatibility with the selected Langfuse release, ingest load test |
| Langfuse queue/cache | Redis · Redis-compatible service supported by the Langfuse release | Durability needs, HA topology, existing platform standards | Eviction policy set so queued jobs are never evicted; failover test |
| Application semantic cache | None · Separate Redis instance for prompt/retrieval caching | Repeat-query rate, freshness tolerance, data sensitivity of cached responses | Hit-rate and correctness sampling; kept separate from the Langfuse queue |
| Blob storage | MinIO AIStor in customer-controlled infrastructure · Other S3-compatible store | Residency, multimodal volume, lifecycle and retention policies | Langfuse S3 configuration test, bucket policies, lifecycle rules |
| Telemetry content policy | Full capture · Masked capture · Metadata-only for sensitive flows | Privacy obligations, debugging needs, retention cost | Redaction tests on representative prompts; retention job verification |
- Langfuse is not a model gateway, vector database or access-control layer for your AI application. Those remain separate design decisions.
- Observability supports quality improvement; it does not guarantee correct model output.
- Do not send application business events into Langfuse’s internal ClickHouse tables. Use the supported SDK and API ingestion paths.
SECURITY & RESILIENCE
Treat AI Telemetry as a Governed Data Asset
Controls are designed for your environment and obligations. The items below are the conditions we design and test for, not default guarantees.
Identity & access
Integrate Langfuse sign-in with your identity provider where the deployed edition supports it. Apply least-privilege project roles and separate API keys per application and environment.
Privacy & masking
Mask or filter sensitive fields in the application SDK before telemetry leaves the application. Define which flows send full content, masked content or metadata only.
Encryption & secrets
Encrypt traffic between applications, Langfuse and its data stores, and encrypt data at rest where the selected components support it. Keep credentials in your secrets manager, never in images or code.
Network & egress
Place Langfuse and its stores in a private network segment. Review all outbound connections, including model providers and any product telemetry, against your egress policy.
Backup & recovery
Define RPO/RTO separately for PostgreSQL, ClickHouse, the Redis queue and object storage, then run a restoration drill. Recovery of one store without the others can leave telemetry inconsistent.
Monitoring & HA
Monitor queue depth, worker lag, ClickHouse ingest and disk, PostgreSQL health and storage growth. Apply deployment-specific high availability once the throughput and recovery objectives are agreed.
PROOF & OUTCOMES
What We Validate Before Go-Live
We do not publish generic performance or quality promises. Instead, every engagement measures the pattern against your applications and agreed acceptance criteria.
Engineering Validation Checklist
- Instrumented trace coverage
- Field redaction and retention rules
- High-throughput ingest load test
- Upgrade / version compatibility
- Secrets and IAM
- Resilient PostgreSQL, ClickHouse, Redis and object store
- Rollback and restoration drill
- Data location / egress review
Measurement Plan
| Metric | How it is measured | Target |
|---|---|---|
| Trace completeness | Share of production requests and agent steps that produce a complete trace | Agreed per application in the PoC |
| Evaluation quality | Coverage of evaluation scores and datasets for priority use cases; regression detection on prompt or model changes | Agreed with AI product owners |
| Cost visibility | Share of model calls with token usage and cost attributed to application, team or tenant | Agreed per cost-allocation model |
| Ingest headroom | Sustained events per second at peak with acceptable queue depth and worker lag | Derived from forecast peak plus margin |
| Incident recovery | Time to restore observability service and data from backup in a drill | Agreed RPO/RTO per store |
What You Receive
- Target architecture and component sizing for the selected Langfuse release
- Telemetry data-classification and masking policy
- Load-test and restoration-drill results against agreed criteria
- Runbooks for upgrades, scaling and incident response
- Knowledge transfer to your platform and AI engineering teams
Enterprise AI & LLMOps FAQs
What does a self-hosted Langfuse deployment need?
In supported self-hosted patterns, Langfuse runs a web/API service and an asynchronous worker. It relies on PostgreSQL for transactional application data, ClickHouse for traces, observations and scores, Redis or a compatible service for queuing and caching, and S3-compatible object storage for event payloads and multimodal assets. Exact versions depend on the Langfuse release.
Is the Redis in this architecture a semantic cache?
Not by default. Langfuse requires Redis (or a compatible service) for its own ingestion queue and caching. A semantic cache for repeated prompts or retrieval results is an optional, separate application component. Keep them on separate instances with separate sizing, eviction and security policies.
Does Langfuse host or run our LLMs?
No. Langfuse observes, traces and evaluates AI applications. Model inference, vector retrieval, orchestration and user permissions remain in your application stack or with your chosen model provider.
Can observability prevent hallucinations?
No tool can guarantee correct model output. Tracing, evaluation and monitoring help you detect, measure and reduce quality problems over time, and give reviewers evidence about what an application did.
How do we keep sensitive data out of traces?
Apply masking or filtering in the application SDK before data is sent, decide which flows send full, masked or metadata-only telemetry, and set retention rules for stored traces and blobs. We test redaction with representative prompts before go-live.
Can we use ClickHouse OSS instead of a partner-led ClickHouse deployment?
Yes, where the ClickHouse version is compatible with your Langfuse release. Bassirah can operate ClickHouse OSS as independent engineering, or deliver ClickHouse through the partner route when commercial assurances are required.
Related Pages
Operate AI with Visibility, Control and Engineering Discipline.
Bring your AI applications, data-sensitivity requirements and target environment. We will design the observability platform, its dependencies and a validation plan.