Build an AI Voice Agent with Vapi Tutorial! (2024 update)

In this video, I show you how to connect Vapi AI with Make.com. How to send and receive data using web hooks and JSON parsers to make the AI understand and use information in conversations. Check out the resource hub I made for more help and follow me on LinkedIn if you have questions or want to stay updated.

Why Build Your Own AI Voice Assistant?

AI voice assistants are transforming how businesses interact with customers. From booking reservations to handling customer inquiries, they help streamline operations and reduce overhead costs. In this guide, I'll walk you through building your very own AI-powered voice assistant using Vapi AI and Make.com.

Key Benefits of AI Voice Assistants

• Automates repetitive tasks like appointment scheduling and customer service.
• Reduces costs: AI can handle tasks at a fraction of the price of human agents.
• Scales effortlessly: Manage a high volume of interactions without additional resources.
• Customizable: Tailor the assistant's behavior and responses to your specific business needs.

In the video tutorial, I'll show you step-by-step how to connect Vapi AI with Make.com to create a voice assistant that can handle real-world tasks, such as making reservations for a restaurant like in our example, Gino's Pizza. Whether you're building it for fun or for a business use case, this guide is perfect for anyone looking to dive into AI-powered automation.

Watch the video introduction here to get an overview of what we’re about to build: [0:14 - Building the Vapi AI System].

Prerequisites & Tools You'll Need

Before we dive into the setup, here are the tools and platforms you’ll need to get started:

• Vapi AI: The platform we’ll use to create and customize the voice assistant. Create a free account here.
• Make.com: The platform we’ll use to automate workflows and connect APIs. Sign up for Make.com here.
• ElevenLabs: Optional but recommended for creating custom voices. Check out their platform for voice generation. Visit ElevenLabs here.

For easy reference, download the resource PDF with all the code snippets and settings you’ll need for the assistant. Visit the Resource Hub.

Once you’ve gathered these tools, you’re ready to follow along with the video tutorial. Head over to [6:53 - Customizing Assistant Responses] to start configuring your assistant’s responses and voice settings.

Building the Vapi AI System: Step-by-Step

Now that you’ve gathered the necessary tools, it's time to build the foundation of your AI assistant. Follow the steps below to create your voice assistant using Vapi AI. We’ll start with creating an account, setting up the assistant, and customizing it for a pizza restaurant, Gino’s Pizza.

Step 1: Creating Your Vapi AI Account

First things first, head over to Vapi AI and create a free account. Once you’ve registered, log in and navigate to the dashboard where we’ll be setting up the assistant. I walk through this process at [6:53 - Logging into Vapi AI] in the video.

Step 2: Creating and Naming the Assistant

After logging in, create a new assistant by selecting the “Create Assistant” option on the dashboard. Name it something relevant to your use case. In the video, I named mine "Gino's Pizza" for a restaurant example, but you can use any name that fits your business. [11:29 - Creating the Assistant]

Step 3: Customizing the Assistant’s Responses

Now that you have an assistant, it’s time to customize its responses. In this example, we’ll configure it to handle table reservations for Gino’s Pizza. You can download the full prompt from the resource hub or copy the sample below:

Sample Model Prompt

First message: "Bon Giorno! This is Gino from Gino's Pizza. How can I assist you today?"
Model prompt: "You are a voice assistant for Gino’s Pizza, located at 456 Pizza Lane, New York. You will help customers by answering questions and booking reservations in a friendly and witty manner. Always confirm the details of the reservation, and use the GetTodaysTime function to check the date and time."

You can configure additional details like the assistant’s voice, language settings, and the tasks it can handle. I show this process in more detail starting at [16:57 - Customizing Assistant Responses] in the video.

Step 4: Configuring the Assistant’s Voice

One of the most fun parts of building the assistant is configuring its voice. In this tutorial, I used ElevenLabs to give the assistant an Italian accent with the voice model “Giovanni.” If you want to use a different voice, you can explore ElevenLabs or other options like OpenAI’s models for voice generation. Here’s how I set up Gino’s voice at [17:45 - Configuring the Assistant’s Voice].

Step 5: Testing the Assistant

Once you've customized the responses and voice, it's time to test your assistant! In the video, I demonstrate how Gino handles a table reservation with real-time conversation. This includes confirming the date, time, and even a special occasion. Watch the test in action at [7:18 - Testing Gino’s Pizza Example].

Test Example: Gino’s Pizza Reservation

Customer: "I would like to make a reservation for tomorrow evening."
Gino: "Sure thing! What’s your first name?"
Customer: "It’s Jasper."
Gino: "Nice to meet you, Jasper. What time would you like the reservation for tomorrow?"
Gino: "Just to confirm, your reservation is for 7 PM tomorrow, July 26th, for a birthday celebration. Sound good?"

Congratulations! You’ve now built and tested your AI voice assistant using Vapi AI. In the next section, we’ll explore how to connect your assistant to automation workflows using Make.com.

Customizing the Assistant's Voice and Responses

One of the most important steps in building your AI voice assistant is making it sound natural and engaging. In this section, we’ll show you how to customize your assistant’s responses and configure the voice to fit your brand or personality. In the example, I’ve used an Italian-themed voice for Gino’s Pizza, but you can choose any style that fits your use case.

Step 1: Customizing the Assistant's Responses

Your assistant’s responses are what make it functional and interactive. For Gino’s Pizza, I’ve set up responses that help the assistant take reservations, answer general questions, and add some humor to the conversation. To start customizing, log in to Vapi AI and navigate to the assistant’s settings. You can view the detailed process at [16:57 - Customizing Assistant Responses].

Sample Model Prompt

First message: "Bon Giorno! This is Gino from Gino's Pizza. How can I assist you today?"
Model prompt: "You are a voice assistant for Gino’s Pizza, located at 456 Pizza Lane, New York. You will help customers by answering questions and booking reservations in a friendly and witty manner. Always confirm the details of the reservation, and use the GetTodaysTime function to check the date and time."

Step 2: Configuring the Assistant's Voice

Next, we’ll configure the assistant’s voice. In this example, I used **ElevenLabs** to give Gino a playful, Italian accent with the voice model “Giovanni.” Depending on your brand’s tone, you can choose any voice that fits. Watch the setup process starting at [17:45 - Configuring the Assistant’s Voice].

How to Configure Voice Settings in ElevenLabs

• Go to ElevenLabs and create an account.
• Explore different voice models and choose one that aligns with your assistant’s personality.
• Once selected, configure the assistant in Vapi AI to use this voice by pasting the voice model’s ID in the settings.

Step 3: Testing the Assistant

Once your assistant's responses and voice are configured, it’s time to test the setup. In this test example, Gino handles a table reservation for a customer, including verifying the date, time, and special occasion. You can watch the live demo at [7:18 - Testing Gino’s Pizza Example].

Test Example: Gino’s Pizza Reservation

Customer: "I would like to make a reservation for tomorrow evening."
Gino: "Sure thing! What’s your first name?"
Customer: "It’s Jasper."
Gino: "Nice to meet you, Jasper. What time would you like the reservation for tomorrow?"
Gino: "Just to confirm, your reservation is for 7 PM tomorrow, July 26th, for a birthday celebration. Sound good?"

Once testing is successful, your assistant is now fully customized and ready to handle conversations in real-time. You can further enhance it by adding more functionalities, like pulling data from external APIs. In the next part of the tutorial, we’ll dive into how to connect your assistant with automation workflows using Make.com.

Connecting to Make.com and Using Webhooks

Now that your assistant is customized and functioning, it's time to enhance its capabilities by connecting it to external services using Make.com. Webhooks allow your assistant to send and receive data dynamically, enabling features like checking current dates and times, managing reservations, or pulling data from external systems like a CRM.

What is a Webhook?

A webhook is a method used by web apps to send real-time data from one app to another. In this case, your AI assistant will use webhooks to fetch specific data, such as the current time for reservations, from an external service.

Step 1: Setting Up Your Make.com Account

To start using webhooks, create a free account on Make.com. Make.com allows you to automate workflows between apps, APIs, and your assistant. After logging in, you'll need to create a new scenario for your webhook. Watch this step in the video at [17:45 - Connecting to Make.com].

Step 2: Creating a Webhook in Make.com

Once you’re logged in to Make.com, follow these steps to create a webhook:

1. From the dashboard, click on "Create a new Scenario".
2. Search for the "Webhook" module and select "Custom Webhook".
3. Give your webhook a name (e.g., "Get Date and Time") and copy the URL generated for the webhook.
4. Save the webhook, and move on to the next step where we will integrate it with Vapi AI.

You can watch the detailed tutorial for setting up the webhook at [21:00 - Adding Webhooks and Testing].

Step 3: Integrating the Webhook with Vapi AI

Now that your webhook is created, the next step is to integrate it with your Vapi AI assistant. This allows the assistant to call the webhook whenever it needs to fetch the current date and time, which is crucial for tasks like booking a table at Gino's Pizza.

Follow these steps to connect the webhook to Vapi AI:

1. In Vapi AI, go to the assistant’s configuration and locate the "Tools" section.
2. Create a new tool and name it (e.g., "GetTodayTime").
3. Paste the webhook URL from Make.com into the "Server URL" field.
4. Configure the tool to call this webhook whenever the assistant needs to fetch the current date and time.
5. Test the integration by calling the assistant and asking for the current time or booking a reservation.

Sample Webhook Configuration Code

{
  "results": [
    {
      "toolCallId": "X",
      "result": "Y"
    }
  ]
}
    

This is the format your assistant will use when calling the webhook and receiving the response. The result will contain the necessary information (in this case, the date and time) to complete the task.

Step 4: Testing the Webhook Connection

After configuring the webhook, it’s time to test the connection. Ask your assistant to make a reservation for tomorrow. If everything is set up correctly, the assistant will call the webhook, fetch the current date and time, and use it to confirm the reservation.

You can watch a live test of this process at [21:00 - Adding Webhooks and Testing].

By connecting your assistant to Make.com via webhooks, you’ve just expanded its capabilities. Whether you need real-time data like dates and times or more advanced integrations with external apps, webhooks allow your assistant to communicate seamlessly with other systems. In the next section, we’ll explore some of the more advanced features you can add to your assistant.

Testing the Assistant

Once you’ve finished setting up and customizing your assistant, it’s time to test it in action. This is a critical step to ensure your assistant works as expected, handling conversations smoothly and providing accurate information. Let’s walk through how to test your AI assistant, using Gino’s Pizza as an example. You can watch this section of the video for a full demonstration: [7:18 - Testing Gino’s Pizza Example].

Step 1: Running a Test Conversation

To start testing your assistant, initiate a mock conversation. In the case of Gino’s Pizza, we’ll test how the assistant handles table reservations. During the test, make sure it performs the following tasks:

• Greets the customer in a friendly manner (e.g., “Bon Giorno! This is Gino from Gino’s Pizza. How can I assist you today?”)
• Collects relevant details from the customer, like name, reservation date, and time.
• Confirms the reservation information, including any special occasions.
• Uses the GetTodaysTime function to verify the date and time in real-time.

Test Example: Gino’s Pizza Reservation

Customer: "I would like to make a reservation for tomorrow evening."
Gino: "Sure thing! What’s your first name?"
Customer: "It’s Jasper."
Gino: "Nice to meet you, Jasper. What time would you like the reservation for tomorrow?"
Customer: "7 PM."
Gino: "Just to confirm, your reservation is for 7 PM tomorrow, July 26th, for a birthday celebration. Sound good?"

Step 2: Analyzing the Conversation Flow

After running the test, review the assistant’s conversation flow. Ask yourself:

• Did the assistant respond promptly and accurately?
• Was the conversation natural, or did it feel robotic?
• Were there any awkward pauses or repeated phrases?
• Did the assistant correctly use the GetTodaysTime function to verify dates?

If the assistant performed well, congratulations! You’ve successfully built and tested your AI assistant. If not, you can revisit the customization settings or adjust the model prompts to improve performance. For reference, watch the full testing process in the video at [18:27 - Testing Results].

Step 3: Troubleshooting Common Issues

If the assistant didn’t perform as expected, here are a few common issues to look out for and how to fix them:

• Issue: The assistant repeats itself or gives incomplete responses.
  Solution: Review the model prompt and ensure that all relevant information is included. You may also want to set token limits to avoid overly long responses.
• Issue: The assistant doesn’t verify the date or time correctly.
  Solution: Ensure the GetTodaysTime function is set up correctly and that the assistant is using the right time format for your region.
• Issue: The assistant sounds too robotic.
  Solution: Adjust the voice settings in ElevenLabs or try another voice model. You can also tweak the language in the model prompt to make it sound more conversational.

Step 4: Real-World Testing

After making necessary adjustments, it’s time for real-world testing. Use your assistant in a live environment to see how it performs with real customers. This phase will help you fine-tune any remaining issues and ensure your assistant is ready for launch. For a deeper look into refining your assistant, check out the next section of the video: [20:40 - Advanced Testing and Tools].

Testing is an ongoing process, and your AI assistant will improve over time as you continue to refine its responses. Once you’ve perfected the setup, your assistant will be ready to handle real customer interactions, automate repetitive tasks, and provide a seamless user experience.

Advanced Features: Integrating Complex Tools

If you're ready to take your assistant to the next level, there are some exciting advanced tools and features you can explore. In this section, we'll briefly touch on how to enhance your assistant by integrating additional services, APIs, and structured conversation flows.

Pulling Data from APIs

One powerful feature of Vapi AI is the ability to connect external APIs to pull in real-time data. For instance, you can integrate services like Google Calendar to manage bookings or fetch customer data from your CRM system to personalize interactions. Using webhooks, you can dynamically update and retrieve information during a conversation.

Using Blocks and Squads for Structured Conversations

To ensure more structured conversations, Vapi AI supports the use of Blocks and Squads. These are like conversation flows that guide the assistant through specific tasks. For instance, if your assistant needs to verify multiple details during a booking, you can create a block that walks the customer through the process step-by-step.

Watch how these advanced tools work and how you can integrate them in your assistant at [35:00 - Advanced Features and Tools].

Conclusion: Next Steps

Now that you’ve built your first AI assistant, the possibilities are endless. Whether it’s for a restaurant, customer service, or personal use, you can further enhance it by integrating tools like Google Calendar or Zapier to automate more tasks.

Want to dive deeper or need some help along the way? Join our Discord community to ask questions, share your progress, and get feedback from other builders. We’re always here to help you fine-tune your assistant or explore new use cases.

Join the Discord Community

For additional resources and downloadable templates, don’t forget to visit our Resource Hub. You can find everything you need to customize and expand your AI assistant, from code snippets to advanced workflows.

Ready to explore more tools and use cases? Head back to the video and watch the final section to learn how to further enhance your AI assistant: [41:00 - Outro].