Pragmatic AI: A virtual assistant guiding you through our services

Virtual assistants are probably one of the most commonly met artificial intelligence applications. According to some sources, 80% of people have interacted with a chatbot at some point. 

To gain more insight into this area, we created Pragmatic AI – the Open AI-powered chatbot that will guide you through our site and try to answer any questions you might have.

Project background

Our aim was to research the possibilities of utilizing ChatGPT (or any other language model) for creating custom chatbots. We defined the chatbot’s requirements to deliver value to anyone visiting our page.

We wanted it to:

1. Answer users’ questions regarding our experience and services.
2. For more specific inquiries, e.g., time and budget estimates, offer to hand the inquiry over to our business consultant, who will get back to the client.

Challenge

During our work, we encountered several issues and limitations of this tool:

• Limited control over the chatbot’s answers (related to the problem of constructing the “knowledge base” – the custom information we “feed” the bot with);
• Limited conversation length;

To better understand the complexities of creating an AI chatbot, we must recognize that multiple elements influence its behavior. These are:

• prompt, 
• knowledge base, 
• parameters, and 
• external providers’ solutions integrated with our virtual assistant (in our case, Open AI and Pinecone).

A prompt is a direct instruction on how a chatbot should behave. The knowledge base is the information a virtual assistant should base on when answering user questions. 

Parameters dictate how AI generates text. For example, the “temperature” parameter controls the randomness or creativity of the text generated. 

The ChatGPT-3.5 Turbo model we use provides the underlying language generation and understanding capabilities that empower our AI chatbot to process user input, comprehend context, and generate meaningful responses. Finally, Pinecone is a vector database that matches users’ inquiries with the most relevant fragments of our knowledge base.

Now, even a minor change to context or prompt will influence how the chatbot works. This turned out to be one of the biggest challenges behind creating Pragmatic AI. 

Limited control over the chatbot’s answers

Hallucinations are the main problem that comes with building AI-powered chatbots. They refer to the model generating responses not aligned with reality or lacking a factual basis. These can include incorrect information, false claims, or entirely fabricated content. 

Hallucinations might result from the limitations of external solutions (for example, ChatGPT is a pre-trained model that we cannot regulate). Still, the main reason behind chatbots “making up things” is the construction of our prompt and knowledge base – that’s why crafting them became our main focus point. 

We strived to make Pragmatic AI answer users’ questions more accurately through:

• Providing precise and structured information in the prompt and knowledge base;
• Prompting the bot to answer, “I’m sorry, I can’t answer this question. 🙁 Please contact us, and we’ll try to help” instead of making up its replies;
• Changing the value of the “temperature” parameter to 0.1 so that the chatbot became more specific.

Limited conversation length

Apart from the hallucinations issue, we encountered a technical challenge resulting in limited conversation length. 

Initially, we employed a GPT-3.5 Turbo model that comes with 4,000 tokens (each token represents a piece of a word, roughly equivalent to 750 words for 1,000 tokens). 

However, we noticed that, during one of the early bot iterations, when responses tended to be quite lengthy, the token limit was exhausted after approximately 9-12 user questions. 

To extend the duration of interactions, we transitioned to a 16k token model. This improvement now allows users to comfortably engage in extended conversations and ask the bot up to about 40 questions.

Solution

Pragmatic AI is a hybrid solution. On the one hand, it is directed to answer questions strictly related to Pragmatic Coders. On the other hand, its AI component enables it to process natural language and context, leading to more human-like and meaningful user interactions than rule-based bots’ rigid and predefined responses.