Battle of the LLM Giants: Google PaLM 2 vs OpenAI GPT-3.5 | by Wen Yang | Jun, 2023

3. Agent using instruments and following directions

Reminder: as a way to use google search API (SerpApi), you’ll be able to join an account here. After that, you’ll be able to generate a SerpApi API key. Its Free Plan permits you to name 100 searches per 30 days.

from langchain.brokers import Device
from langchain.brokers import initialize_agent
from langchain.utilities import SerpAPIWrapper

def chat_agent(question, llm):
#======= Step 1: Search device ========
# google search
search = SerpAPIWrapper()

# ===== Step 2: Reminiscence =========
reminiscence = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

# ====== Step 3: Chain =======

# choice 1: RetrievalQA with supply chain
# qa = RetrievalQAWithSourcesChain.from_chain_type(
# llm=llm,
# chain_type="stuff",
# retriever=vectorstore.as_retriever()
# )

# choice 2: Dialog Retrieval chain
qa = ConversationalRetrievalChain.from_llm(

#==== Step 4: Create an inventory of instruments
instruments = [
# Outside Knowledge Base
name='Knowledge Base',
func=qa.__call__, # won't work!!
description='use this tool when answering general knowledge queries '
# Search
description='use this tool when you need to answer questions about weather or current status of the world '

#==== Step 5: Agent ========

agent = initialize_agent(

return agent(question)

The important thing thought is that our chat agent has an LLM to generate responses, a toolbox with an inventory of instruments, and short-term reminiscence for previous interactions. We would like our agent to make use of Pinecone information base to reply questions more often than not, and solely to make use of the search device to reply questions concerning the climate or the present standing of the world.

Our first query is:

“May you propose a two-day journey to Yellowstone nationwide park with each day itineraries?”

Let’s see the responses generated by each brokers.

From Palm Agent:

Response from Palm Agent

The palm agent had points parsing the LLM output. Additionally, Palm went to make use of the Search device instantly as a substitute of following directions on utilizing the information base for basic inquiries.

From gpt-3.5 Agent:

Response from gpt-3.5 agent

The gpt-3.5 agent had no downside parsing output, and it adopted human instruction extra carefully — utilizing a information base to reply the query. The standard can also be fairly good and it offered an in depth each day itinerary.

Now let’s take a look at a follow-up query, which we wish the agent to make use of the search device. The thought is when a consumer makes use of exterior chat for upcoming journey planning, they may wish to know the climate for the vacation spot. Right here we purposefully used “climate there” as a substitute of “climate in Yellowstone” to check if the agent can keep in mind previous conversations.

“What’s going to the climate there be like over the following 7 days?”

Palm Agent searched the climate in Seattle, which isn’t what we wish.

Palm agent search climate

Gpt-3.5 Agent isn’t any higher. It searched Greenville, NC which can also be removed from our vacation spot Yellowstone.

Gpt-3.5 agent search climate

Each brokers made the proper resolution to make use of the search device, nonetheless, they appear to endure a little bit of amnesia — no recollection of the vacation spot we’ve been chatting about! The problem could also be associated to a possible interplay reminiscence downside with the Langchain agent. When you’ve got encountered comparable points or higher but, have insights on how you can repair this, please let me know!

Constructing an AI-Powered Language Studying App: Studying From Two AI Chatting | by Shuai Guo | Jun, 2023

Outline personalized permissions in minutes with Amazon SageMaker Position Supervisor by way of the AWS CDK