Mastering Tool Calling & RAG in Real-Time Voice Conversations

Bridging LLMs with Real Enterprise Data

An AI agent that can only recite generic knowledge is just a demo. In production, enterprise voice agents need to:

• Check real-time inventory in a warehouse database.
• Schedule calendar appointments via Google or Outlook.
• Query PDF policy handbooks through semantic vector search (RAG).

The core technical challenge in voice AI is execution time: while an API is running in the background, what happens to the phone call?

Step 1: Sub-100ms Vector Search with Milvus

When a customer asks a policy question, our vector retrieval system fetches chunked embeddings from Milvus with strict latency budgets:

// Fast vector similarity search during live call
const chunks = await milvusClient.search({
  collection_name: "knowledge_base",
  vector: embedding,
  limit: 3,
  output_fields: ["content", "source_url"],
});

Step 2: Filler Audio & Natural Conversational Flow

When a tool call takes more than 400ms (e.g., verifying a loan eligibility score in an external banking system), an awkward pause triggers caller anxiety.

Tunk.ai employs adaptive conversational fillers:

• "Let me pull up your account details right now..."
• "One moment while I check our availability for this Friday..."

This mirrors natural human speech patterns and keeps the caller engaged while the asynchronous tool execution returns the payload to the LLM context.

Step 3: Structured Function Calling Schema

By strictly enforcing JSON Schema on all tool definitions, agents never hallucinate parameters:

{
  "name": "book_appointment",
  "description": "Books a service consultation slot",
  "parameters": {
    "type": "object",
    "properties": {
      "slot_time": { "type": "string", "format": "date-time" },
      "customer_email": { "type": "string", "format": "email" },
      "service_type": { "type": "string", "enum": ["sales", "support", "technical"] }
    },
    "required": ["slot_time", "customer_email"]
  }
}

With Tunk.ai's built-in tool orchestrator, you can connect existing REST and GraphQL APIs in minutes.