Text Generation with Google Gemini

Transcript

Demo

To start this demo, open the file from the starter project starting with the name 03-building. Open this file in Visual Studio Code. Add your API key to the .env file and execute the first code cell to install the libraries. Then, execute the second code cell to import the libraries and retrieve the API Key from the environment file.

Insert a third code cell below and add the following:

model = genai.GenerativeModel('gemini-pro')

This code selects your generative model as Gemini Pro. Execute this code cell.

By default, the model doesn’t retain conversation history. To understand this default behavior, add the following code:

response = model.generate_content('I have two dogs and three cats.')
print(response.text)

You have started a conversation with the model about how many pets you have in your household. Execute this code cell, the model prints a response like this:

You have a total of five pets.

Now, add:

print("="*50)
response = model.generate_content('How many paws are in my house?')
print(response.text)

This code verifies the model’s default behavior of not retaining the conversation history. After printing some equals characters and a random mathematical calculation, you asked the model a question based on the information it got from the previous request. Execute this code cell. Printing the result reveals there is no recollection of the previous request.

The generate_content can accept a list of strings to represent conversation history. Insert a code cell and then add:

chat_history = [
  "User: I have three cats and two dogs",
  "Model: That's great, I'll remember that",
  "User: How many paws are in my house?"
]

This creates a list of strings to serve as the chat history. Note how each string starts with a role followed by the request or response. You can specify multiple exchanges if you like.

Now create a cell and then add:

response = model.generate_content(chat_history)
print(response.text)

You now use chat_history to generate the content. Execute these two lines of code, and the model responds with some sort of a calculated response.

3 * 4 + 2 * 4 = 20

It’s now clear the conversation remembered the history. The downside to this approach is that the chat_history has to be maintained throughout the conversation. Add another code cell and then add:

chat_history.append("Model: " + response.text)
chat_history.append("User: How many dogs do I have?")
response = model.generate_content(chat_history)
print(response.text)

This appends the response to the chat_history and adds a new question. Execute this cell. The model’s response is printed, however, to retain this conversation, you’ll need to add this response again to the chat_history.

There is a better way to do this, you can use start_chat. Then, the subsequent calls to send_message, maintains the history automatically without having to append each response and new query.

Delete the cells you added back to the model’s line:

model = genai.GenerativeModel('gemini-pro')

Don’t delete this line that selects the model. You’ll now add new cells that use start_chat and send_message instead. Create a new cell and add:

chat_session = model.start_chat()
chat_session.send_message("I have two dogs and three cats.")
response = chat_session.send_message("How many paws are in my house?")
print(response.text)

This creates a chat object by starting a chat on the model. Then, send_message submits the queries to the model, and the response is printed. Execute this cell, it retains the conversation context without having to manually maintain a list. To view the conversation history, add the following to a new cell:

print(chat_session.history)

This prints the chat session. Execute this statement and this displays the history in its raw format.

If you would like to preload the history with start_chat, an easy way to do that is using AI Studio. Create a new prompt with AI Studio:

My name is Pinal and I am a Gemini.

Press Command-Enter to run the prompt and wait for AI’s response.

Then, continue the conversation and give another prompt:

I worked at Kodeco for 25 years.

Run the prompt again and wait for the model to respond. Now, click Get code and copy this entire code segment generated in Python:

chat_session = model.start_chat(
  history=[
    {
      "role": "user",
      "parts": [
        "My name is Pinal and I am a Gemini",
      ],
    },
    {
      "role": "model",
      "parts": [
        "That's great to know, Pinal! Geminis are known for their
        curiosity, adaptability, and communication skills. \n\n
        Do you want to tell me more about yourself? I'd love to
        hear about your interests, hobbies, or anything else you'd
        like to share! 😊 \n",
      ],
    },
    {
      "role": "user",
      "parts": [
        "I worked at Kodeco for 25 years",
      ],
    },
    {
      "role": "model",
      "parts": [
        "Wow, 25 years at Kodeco! That's quite a commitment. I'm
        guessing you've seen a lot of changes over the years. \n\n
        Tell me, what's it like working there? Is it a good place
        to be? 😄 \n",
      ],
    },
  ]
)

This code has roles, requests, and responses. You may copy this code and save it to the clipboard. Go back to Visual Studio Code. Back in Visual Studio Code, delete the cells again, leaving the cell that selects the model. Add a new code cell and paste the code that you got from AI Studio. In this code cell you can delete the code that you don’t need from AI Studio. You can leave the chat_session as it is. Come down to the response part of the code from AI Studio. In send_message, change this to:

response = chat_session.send_message("What's my name again?")
print(response.text)

You continued the chat session by asking your name again. Execute this code, and you can see that preloading the history worked and the response takes the conversation history into consideration.

Add a new code cell below and then add:

print(chat_session.history[0].parts[0])

You’re printing a single part of the history using indexing. Execute this cell, it works because the history is just a list of JSON objects. This code prints the very first user query in the history at index 0, and it prints the text of that inquiry, which is in the parts attribute.

Now, change the index in history to -1. Execute, when using history with -1 index, it prints the last piece of the history. Similarly, you can use indexing to print all the history items in between. To know a particular chat_session history’s role, add the following code:

print(chat_session.history[-1].role.capitalize())

This will print only the indexed role as a user or model. Now, delete the cells back to the cell that selects the model. It’s a bit cumbersome to code one interaction at a time. Fix this by adding the code:

chat = model.start_chat(history=[])
prompt = input('User:')
while prompt != 'quit':
  response = chat.send_message(prompt)
  print(f'{chat.history[-1].role.capitalize()}: {chat.history[-1].parts[0]
    .text}')
  print('\n' + '-' * 100 + '\n')
  prompt = input('User:')

The first line in this code segment will create a new chat object with a blank history. The parameter with a blank list is optional but it’s a good illustration that the history is empty. The next line prompts the user with their role to type in their first query. You then added a while statetement to loop user’s queries. The loop continues the interaction until the user quits.

While the user has not quit, the inquiry is sent to the model and the response is returned and stored. The latest conversation history is then printed, including the role. The program prints a line of dashes after each inquiry just for clarity. Lastly, the user is prompted for the next query.

At the very end, within the same code cell and outside the while loop, add the following:

print('Ending Conversation ...')
time.sleep(2)
print('Talk to you next time!')

This will gracefully exit the conversation.

It’s time to see the code you added in action. Execute the newly added code cell. When the input window pops up at the top, type, What’s up, buttercup?. Press enter and wait for the model to respond.

After the model responds, when the input window pops up again, type, What did I just call you?. Press enter again.

The model recalls that you called it Buttercup. In the next input window, type quit, enter and the program will end.

Note: Please note for your reference, all the deleted code in this demo is available as comments in the final project.

Congratulations, you’ve reached the end of this demo and built your very own chatbot! In the next section, you’ll learn how to customize the chatbot even more using system instructions.