Building a Non-Streaming Chat App - Instruction

Creating a Fact Checking App

A chat app, like ChatGPT, asks users for an input, responds, and allows users to send more messages in the same thread. This app takes in all the previous messages, from the user and from itself, as context for its next response. You’re going to make a similar app in this lesson.

If you recall from the previous lesson, there was a similar follow-up ability in the demo or in the instructions section. However, the user only had one time to follow up. A chat app, however, has no end. Users can keep adding follow-up messages to the conversation. To achieve this, you would first need to ask the user’s input and add a loop for further inputs.

In JupyterLab, navigate to the notebook in 05-building-a-non-streaming-chat-app/Starter/lesson5.ipynb. Notice that it contains the code from the previous lesson, minus some code. The second call to chat completion was removed. You’ll edit this to transform it into a chat app.

Fetching URL Content

In the previous lesson, you mocked the response of getting the URL. Note that the starter code for this lesson already includes a real implementation of fetching the text from a webpage. Because this is not central to the lesson, you can try understanding how it’s done on your own.

The code that does this is the 4th block of the starter project, shown in the image below:

Asking the User for Inputs

Because you’ll ask the user for inputs, the first thing to do is to remove the user’s initial message from the history of messages. Find the messages array and remove the message that has the role user. It should look like the code below:

SYSTEM_PROMPT = (
  "You are a fact checker. Verify the validity of the sentence provided by the user, given a reference. You must return a response in JSON format:"
  "{'isFactTrue': <true or false>, 'explanation': <explanation> }"
)
messages = [
  {"role": "system", "content": SYSTEM_PROMPT},
]

Next, find the cell with chat completion, that has the following code:

response = client.chat.completions.create(
  model=model,
  messages=messages,
  response_format={"type": "json_object"},
)

and add the following lines of code at the top of the cell, before the code.

user_input = input("Please enter your input: ")
messages.append({"role": "user", "content": user_input})

You’re asking the user for input using input(), a built-in method from Python. Then, you’re adding the user’s input to the message history.

Run all the cells from the beginning. You should be asked for an input like in the image below:

Because this is a fact-checking chat app, type a sentence that you want to fact-check, such as: Flutter is the best framework in the world. Press Enter.

It should respond with something like the following:

{
  "isFactTrue": false,
  "explanation": "While Flutter is a popular open-source UI framework for building natively compiled applications for mobile, web, and desktop from a single codebase, stating it as 'the best' framework is subjective and depends on individual needs and preferences. There are many other frameworks available such as React Native, Xamarin, and Angular, which may be preferred by different developers depending on their project requirements."
}

It seems this fact-checker is quite reasonable. :]

Chat Loop

Now that you can ask the user for inputs, you’ll want to make the chat app ask the user again after it responds. You’ll implement this in a loop.

Replace the last cell that you edited earlier with the following code:

# 1
while True:
  # 2
  user_input = input("Please enter your input: ")

  # 3
  if user_input.lower() == 'exit':
      break

  # 4
  messages.append({"role": "user", "content": user_input})

  # 5
  # Replace the lines below with try-except
  response = client.chat.completions.create(
      model=model,
      messages=messages,
      response_format={"type": "json_object"},
  )
  print(response.choices[0].message.content)

  messages.append({"role": "assistant", "content": response.choices[0].message.content})

With the code above, you:

1. Make an infinite loop.
2. Like earlier, ask the user for inputs.
3. Break out of the loop if the user said exit.
4. Add the user’s input to the messages array.
5. Call chat completion and print the response.
6. Add the response to the messages array with the role assistant.

Run the cell, and you should see the input box again like below:

Enter a similar sentence that you want to fact-check, such as the same sentence given above: Flutter is the best framework in the world. Press Enter.

You should see the response from the fact-checker and another input box like in the image below:

The user can already add infinite follow-ups on this until they input exit.

Try inputting another bit of text, like I want your honest opinion, instead of fact-checking. Press Enter. You should see something like what’s shown below:

In this case, the fact-checker still didn’t want to give an opinion. It keeps quite well to being a fact-checker chat app.

Error Handling

Giving your user the ability to send infinite responses to the app can lead to some currently unhandled problems. You might recall that you added error handling in an earlier lesson. You should remember to add these when performing calls to chat completion. If the user sent responses too quickly, you might receive a rate-limit error.

Modify the lines of code in #5 from the previous step. It also has a comment that says “Replace the lines below with try-except”. Replace it with this:

# 1
try:
  # 2
  response = client.chat.completions.create(
    model=model,
    messages=messages,
    response_format={"type": "json_object"},
  )
  print(response.choices[0].message.content)

  messages.append({"role": "assistant", "content": response.choices[0].message.content})
# 3
except openai.RateLimitError as e:
  print(f"Rate limit exceeded: {e}")

You first started a try-except block. Then, you wrapped the previous chat-completion code, print response, and adding response to the messages array inside the try block. Last, you added the openai.RateLimitError exception handling and showed the user that they have exceeded the rate limit.

Run the cell, and you should be informed if you reach the rate limit.

Tool Calls

The last part of this chat app is its ability to add content from the web to make its fact-checking more accurate.

First, you would need to remove the line:

messages.append({"role": "assistant", "content": response.choices[0].message.content})

This is because it will be part of a bigger function. To process responses with tools, like in the earlier lesson, find the process_response_with_tools function. Replace it with this code.

# 1
import json

# 2
def process_response_with_tools(response):
  # 3
  response_message = response.choices[0].message
  tool_calls = response_message.tool_calls

  # 4
  if tool_calls:
    # 5
    messages.append(response_message)

    # 6
    available_functions = {
      "get_text_from_url": get_text_from_url,
    }
    # 7
    for tool_call in tool_calls:

      # 8
      function_name = tool_call.function.name
      function_to_call = available_functions[function_name]
      function_args = json.loads(tool_call.function.arguments)
      function_response = function_to_call(
        url=function_args.get("url"),
      )
      # 9
      messages.append(
        {
          "tool_call_id": tool_call.id,
          "role": "tool",
          "name": function_name,
          "content": function_response,
        }
      )
      # 10
      tool_response = client.chat.completions.create(
        model=model,
        messages=messages,
        response_format={"type": "json_object"},
      )
      messages.append({"role": "assistant", "content": tool_response.choices[0].message.content})

      return tool_response
  # 11
  else:
    messages.append({"role": "assistant", "content": response.choices[0].message.content})
    return None

Here, you:

1. Import the json package, which is used for parsing JSON data.
2. Define a function named process_response_with_tools that takes a response object as an argument. This function processes the response to check for any tool calls.
3. Extract the message from the response using response.choices[0].message, which contains the assistant’s response, and the tool_calls.
4. Check whether the response message contains any tool calls.
5. Append the response message to the messages list if tool calls exist. This is needed to indicate that this is the tool call being responded to.
6. Create a dictionary called available_functions that maps function names to their corresponding implementations, allowing for dynamic function calls.
7. Iterate over each tool call found in the response message.
8. Perform the needed function call by extracting the arguments. This is where the URL contents are fetched, for example.
9. Append the result to the messages list, including the tool call ID and role tool.
10. Perform chat completion without tools to process the tool output. Add the response to the messages array with the assistant role. Then, return the response from this.
11. Append the assistant’s response content directly to the messages list if no tool calls are present, returning None because there were no tool calls.

That’s a lot of code. Now you need to re-enable the tools calls with the chat completion call. On the last cell, there’s code that says:

# 1
try:
  # 2
  response = client.chat.completions.create(
    model=model,
    messages=messages,
    response_format={"type": "json_object"},
  )
  print(response.choices[0].message.content)

  messages.append({"role": "assistant", "content": response.choices[0].message.content})
# 3
except openai.RateLimitError as e:
  print(f"Rate limit exceeded: {e}")

Replace the contents inside try, so it looks like this:

try: # did not change this
  # 1
  response = client.chat.completions.create(
      model=model,
      tools=tools,
      messages=messages,
      response_format={"type": "json_object"},
    )

  # 2
  tool_response = process_response_with_tools(response)

  # 3
  if tool_response:
    print(tool_response.choices[0].message.content)
  # 4
  else:
    print(response.choices[0].message.content)
# did not change this
except openai.RateLimitError as e:
  print(f"Rate limit exceeded: {e}")

Here, you:

1. Call the chat completion with tools.
2. Process the response for any tool calls and getting a new response called tool_response.
3. Print the tool_response content if there’s a tool response.
4. Otherwise, you print the response content.

Now, you can run all the cells again. And when asked for input, enter:

Flutter is the most popular framework https://www.statista.com/statistics/869224/worldwide-software-developer-working-hours/#:~:text=Flutter%20is%20the%20most%20popular,of%20software%20developers%20used%20Flutter.

Press Enter. You should see an image like the one below.

It seems the app used the URL as a reference, which is what you wanted it to do. Great job! You can continue adding follow-up messages and see how the app behaves. For example, enter Compare it to React Native and others and press Enter. You should see a response like the one below.

It looks like the app compared the popularity percentages. To escape the loop, enter exit and press Enter.

Well done! You now have a fact-checker chat app that you can ask follow-up questions to.