Case Study · Hybrid Agent

SQL + Docs Hybrid Agent

Single assistant that selects SQL tools for structured facts and RAG for policy/process context through explicit function-calling logic.

ProblemOne source cannot answer both numeric and policy questions reliably.

SolutionRouter decides DB query vs. retrieval path per user intent.

ImpactHigher precision on analytics questions and fewer fabricated figures.

Architecture

Tool-Calling Strategy

Orchestrated as a governed agent runtime: intent arbitration, SQL safety controls, retrieval grounding, and deterministic synthesis.

user query -> intent arbitration -> execution plan -> tool dispatch
                  |                    |                  |
                  v                    v                  v
            uncertainty gate      sql safety policy   docs retrieval policy
                  |                    |                  |
                  v                    v                  v
             clarify route      parameterized SQL      citation-backed context
                           \\          |          /
                            \\         v         /
                             answer synthesis + evidence trace

Why this matters: one assistant can handle metrics and policy questions without unsafe SQL behavior or ungrounded text answers.

Intent Arbitration Layer

Routes each query to SQL, docs, hybrid, or clarify path based on confidence and ambiguity policy.

SQL Guarded Execution

Read-only queries only, schema-aware planning, and post-query validation before values are returned to synthesis.

Grounded Response Composer

Merges structured SQL output and retrieved documentation with citation requirements and conflict-resolution rules.

Evidence

Accuracy and Safety

Query Correctness

SQL output validated with deterministic checks on schema and bounds.

Groundedness

Doc-derived answers require attached citations and retrieval confidence.

Route Precision

Intent router monitored for misroute rate across benchmark prompts.

Reliability Patterns

Read-only DB role and SQL execution guardrails.
Fallback to clarification question on low route confidence.
Trace each tool call for debugging and review.

Presentation path: projects/sql-docs-hybrid-agent/presentations/upcoming/

Technical Peek

Deterministic Route + Tool Orchestration

def answer_with_hybrid_router(env: QueryEnvelope) -> AgentResponse:
    intent = router.classify(env.query)
    decision = resolve_route(intent, policy=route_policy)

    if decision.route in {Route.SQL, Route.HYBRID}:
        sql_plan = sql_planner.plan(env.query, schema=sql_schema)
        sql_guard.assert_safe(sql_plan)
        sql_result = sql_tool.execute(sql_plan)
    else:
        sql_result = None

    if decision.route in {Route.DOCS, Route.HYBRID}:
        docs_result = docs_tool.retrieve(env.query, top_k=8, min_score=0.62)
    else:
        docs_result = None

    if decision.route == Route.CLARIFY:
        return build_clarification_response(decision)

    merged = synthesis.merge(sql_result=sql_result, docs_result=docs_result, query=env.query)
    evidence = evidence_linker.attach(merged, sql_result=sql_result, docs_result=docs_result)
    return policy_guard.finalize_or_fallback(merged, evidence, decision)

Why this matters: route-aware orchestration prevents the model from guessing structured facts and enforces safer SQL boundaries.

Advanced Breakdown

Most Important Engineering Decisions

1. Intent Arbitration with Uncertainty States

The router predicts `sql`, `docs`, `hybrid`, or `clarify`, and low-confidence classifications default to clarification rather than forcing a potentially wrong tool path.

Why this matters: uncertain intent is handled explicitly, not hidden by guesswork.

Routing Quality

2. Strict SQL Safety Envelope

Read-only roles, parameterized query construction, and pattern-based blockers prevent dangerous or malformed SQL while keeping analytics answers precise.

Why this matters: data access remains safe even when prompts are unpredictable.

Security

3. Retrieval Evidence Requirements

Non-SQL claims must carry citation payloads from retrieved docs; unsupported claims are rejected or rewritten with explicit uncertainty notes.

Why this matters: policy/process answers stay grounded in actual documentation.

Groundedness

4. Hybrid Merge Conflict Resolution

When SQL and docs outputs disagree, synthesis rules prioritize deterministic database values and annotate policy interpretation separately for user clarity.

Why this matters: mixed-source answers remain coherent and defensible.

Answer Integrity

5. Tool-Trace Evaluation Loop

Each route decision and tool call is traced and replayed in benchmark suites to improve router prompts, thresholds, and fallback behavior over time.

Why this matters: routing performance gets measurably better with each iteration.

Continuous Improvement