Nduzi AI Assistant Architecture

---

1. Executive Summary & Purpose

Nduzi (Igbo for "Guide" or "Direction") is Debelu's AI shopping and campus life assistant. Designed specifically for university students, Nduzi assists users in finding products on their campus, tracking deliveries, comparing prices, finding coupons, and discovering trending deals.

To make high-scale conversational AI economically viable and lightning-fast on mobile devices, Nduzi is engineered with a 7-Layer Optimization Pipeline that reduces LLM token consumption by 60% to 80% while maintaining sub-second streaming latency.

graph TD
    subgraph Inbound Client Request
        USER[Student User Prompt] --> SSE[SSE Streaming Gateway: /api/gemini/stream]
    end

    subgraph The 7-Layer Optimization Engine
        L1[Layer 1: IntentRouter<br/>Short-circuit greetings & FAQs -> 0 Tokens]
        L2[Layer 2: ResponseCache<br/>Redis cached responses -> 0 Tokens]
        L3[Layer 3: ConversationMemory<br/>Sliding window + summarization -> 50-70% savings]
        L4[Layer 4: ToolSelector<br/>Dynamic tool pruning -> 500-1000 token savings]
        L5[Layer 5: ToolCache<br/>Cached DB queries in Redis -> 0 DB latency]
        L6[Layer 6: ToolFormatter<br/>Compacted text format -> 60-80% tool savings]
        L7[Layer 7: InlineTitle<br/>Piggyback session title on first response]
    end

    subgraph LLM & Tools Execution
        GEMINI[Google Gemini 1.5 Flash]
        TOOLS[Debelu Internal Services<br/>Products, Orders, Carts, Reviews, Campus Hubs]
    end

    SSE --> L1
    L1 -->|Generic Greeting / Static Help| FAST_OUT[Direct SSE Stream]
    L1 -->|Complex Query| L2
    L2 -->|Cache Hit| FAST_OUT
    L2 -->|Cache Miss| L3
    L3 --> L4 --> GEMINI
    GEMINI -->|Function Call| L5 --> TOOLS
    TOOLS --> L6 --> GEMINI
    GEMINI --> L7 --> SSE

---

2. The 7-Layer Optimization Pipeline

Grounded in [ChatOrchestrator.ts](file:///c:/Users/frank/OneDrive/Desktop/Chisom/Debelu/New%20Debelu%20Marketplace/debelu-backend/src/services/ChatOrchestrator.ts), the assistant processes every prompt through seven distinct optimization filters:

Layer 1: Intent Routing (IntentRouter.ts)

• Evaluates input using high-speed deterministic regex and semantic heuristics before invoking the LLM.
• 0-Token Short-Circuiting: Casual greetings ("hi", "good morning"), platform policy queries ("how does escrow work?"), and standard support inquiries stream instant canned responses with zero API token spend.

Layer 2: Response Caching (NduziCache.ts)

• Frequently asked questions and generic trending queries are hashed and cached in Redis with a 2-hour TTL.
• Repeated identical queries return instantaneous responses without external API calls.

Layer 3: Conversation Memory & Window Sliding (ConversationMemory.ts)

• Rather than sending entire multi-turn transcripts, Nduzi maintains a sliding window of the last 6 messages.
• Older conversational turns are asynchronously compressed into a compact 2-sentence summary block, saving 50–70% of prompt token context.

Layer 4: Dynamic Tool Selection (ToolSelector.ts)

• Declaring all 17 marketplace tools in every prompt consumes $\sim 1,200$ tokens of overhead per turn.
• ToolSelector analyzes prompt keywords and injects only the relevant subset (e.g., ordering tools for "where is my package?", product tools for "find sneakers on UNILAG").

Layer 5: Tool Result Caching (NduziCache.ts)

• Product searches, campus trending lists, and vendor profiles are cached in Redis (TTL: 5-15 mins).
• When the LLM calls search_products("calculators"), the database query is skipped if an identical search executed recently.

Layer 6: Tool Result Compactor (ToolResultFormatter.ts)

• Raw database JSON payloads contain bulky metadata (created_at, internal IDs, foreign keys).
• ToolResultFormatter converts JSON into compact, natural text representations, reducing downstream LLM token consumption by 60–80%.

Layer 7: Inline Title Generation

• Rather than making a separate, asynchronous LLM call to name the chat session, Nduzi extracts an inline session title directly from the model's initial reasoning output.

---

3. Server-Sent Events (SSE) Real-Time Protocol

Nduzi streams real-time responses to client applications over persistent HTTP Server-Sent Events:

sequenceDiagram
    autonumber
    actor Client as Storefront Web / Mobile
    participant API as ChatOrchestrator (/api/gemini/stream)
    participant Gemini as Gemini 1.5 Flash
    participant Tool as ProductService

    Client->>API: 1. POST /api/gemini/stream (prompt, conversationId)
    API-->>Client: 2. SSE Event: { type: 'tool_start', toolName: 'search_products', displayText: 'Searching campus marketplace...' }
    API->>Tool: 3. Execute ProductService.search(query, campusId)
    Tool-->>API: 4. Return matching product records
    API-->>Client: 5. SSE Event: { type: 'ui_data', uiType: 'product_list', payload: [...] }
    API->>Gemini: 6. Stream formatted tool results into context
    loop Text Chunking
        Gemini-->>API: 7. Yield text delta
        API-->>Client: 8. SSE Event: { type: 'text_delta', content: 'Here are 3 calculators available at New Hall...' }
    end
    API-->>Client: 9. SSE Event: { type: 'done' }

Event Schema Specifications

• tool_start: Signals the UI to show an animated progress chip (e.g. "Searching campus marketplace...").
• tool_result: Confirms tool execution status.
• ui_data: Transmits rich structured JSON payloads directly to the frontend, allowing the React app to render native interactive widgets (carousel cards, checkout buttons, delivery maps) alongside textual prose:
  • product_list: Interactive product grid with add-to-bag buttons.
  • order_status: Real-time order progress timeline.
  • comparison: Side-by-side product feature matrix.
  • cart: Mini-cart summary with coupon redemption.
• text_delta: Incremental markdown text chunks.
• done: Signals the close of the streaming connection.

---

4. Tool Registry & Campus Scoping

Nduzi is equipped with 17 specialized tools declared via the Gemini Function Calling API:

Tool Name	Frontend Display Label	Functionality
search_products	"Searching campus marketplace..."	Full-text product search scoped to user's campus.
search_with_filters	"Applying your filters..."	Category, price range, and in-stock filtering.
get_product_details	"Fetching product details..."	Detailed specs, condition, and seller ratings.
get_trending	"Finding what's hot on campus..."	High-velocity products on the user's specific campus.
add_to_cart	"Adding to your bag..."	Modifies user's active shopping cart.
view_cart	"Loading your bag..."	Inspects current line items and subtotal.
track_order	"Tracking your order..."	Queries order delivery status and campus hub location.
compare_products	"Comparing products side-by-side..."	Synthesizes comparative table for 2–3 products.
get_coupons	"Hunting for deals..."	Discovers active merchant promo codes.
contact_support	"Connecting to support..."	Escalates conversation to a live staff ticket.

Strict User Context Invariant

NDUZI_BEHAVIOUR_RULES:
1. NEVER ask for information that is already present in the USER CONTEXT block.
2. This includes: user campus, full name, account ID, or student status.
3. If a tool requires campus, use the value from USER CONTEXT silently.

---

5. Security, Guardrails & Jailbreak Defenses

1. System Prompt Hardening: Nduzi's system prompt strictly confines the assistant to marketplace commerce, campus navigation, and order assistance. Any attempt to induce code generation, system prompt extraction, or offensive dialogue is deflected with a neutral polite refusal.
2. Escrow Protection: Nduzi explicitly refuses to facilitate off-platform transactions, direct bank account exchanges, or unescrowed payments.
3. Graceful Fallbacks: If the Gemini API experiences network disruption or quota exhaustion, Nduzi emits a standard fallback event:

   { "type": "error", "message": "Nduzi is taking a brief rest. You can still browse and search manually!" }

   The storefront UI handles this without crashing, maintaining standard catalog browsing and checkout availability.