Free = relatively bad
Large companies often employ marketing strategies that involve offering services for free to either gain new users or expand their market share. One such example of free services is ChatGPT.
However, since every prompt, especially prompting a whole chat history, incurs significant costs for the service provider, the free version doesn’t offer certain features, such as Plugins.
Without such features,
- prompting activity is weakened which saves OpenAI some money
- makes users switch to the Plus version which makes OpenAI some money.
Now, let’s consider a scenario where the standard ChatGPT version offered the same features and user experience as the Plus version. In this case, it’s likely that very few users would be willing to pay $20 per month for the premium version, resulting in financial losses for OpenAI.
Competitive market for the win!
On a side note, most people will be expecting AI service to be free of charge, similar to browsing the web or using Google Search. That’s probably why Google’s Bard or Bing AI are free of charge. From a consumer perspective, this is a positive development, as it allows for consumer surplus resulting from the innovative offerings of competing corporations such as Alphabet, OpenAI, and Microsoft.
PubChat ain’t free
PubChat is not free. The service also lacks some important user experience and features compared to the ChatGPT (web) app as highlighted in the first article. However, the aim of PubChat was never to directly compete with industry leaders. The project was initiated with the following motives:
- Utilizing the feeless Shimmer network(details can be found in the first article)
- Demonstrating the implementation of pay-per-use through micro-transactions
- Providing an anonymous alternative to OpenAI without requiring sign-up
- Making ChatGPT accessible to individuals who can purchase cryptocurrencies but are unable to sign up at OpenAI(users in certain third-world countries)
Since PubChat is not free to begin with, we can leverage this circumstance and offer features that OpenAI cannot provide in their free version of ChatGPT. One such feature is the use of agents.
LangChain Agents
The core idea of Langchain is that developers can “chain” together different components to create more advanced use cases around Large Language Models(LLMs) like CHATGPT-3.5-turbo.
An LangChain Agent is an entity that drives decision-making in the framework. It has access to a set of tools and can decide which tool to call based on the user’s input. Agents help build complex applications that require adaptive and context-specific responses.
# Set up the base template
template = """Answer the following questions as best you can.
You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin!
Question: {input}
{agent_scratchpad}"""To put it simply, we provide the ChatGPT model, such as CHATGPT-3.5-turbo, with a template and a set of tools. Equipped with these instructions and tools, the model can repeatedly generate responses based on the initial prompt. This enables us to ask questions that involve multiple steps, such as searching the internet, in cases where the required knowledge is not directly provided within the billions of parameters the model has been trained on.
# Define the large language model(llm)
llm = ChatOpenAI(temperature=0.3)
# Load the tools
tools = load_tools(["google-search", "calculator"], llm=llm)
# Initialize the agent
agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
# run agent
agent.run("Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?", callbacks=[handler])While this capability is indeed fascinating and perhaps even intimidating, it’s important to note that the chaining of prompts and tools by an agent can significantly escalate costs. I once had an agent response that costed more than 20.000 OpenAI tokens!
> Entering new AgentExecutor chain...
I need to find out who Leo DiCaprio's girlfriend is and then calculate her age raised to the 0.43 power.
Action: Search
Action Input: "Leo DiCaprio girlfriend"
Observation: Camila Morrone
Thought: I need to find out Camila Morrone's age
Action: Search
Action Input: "Camila Morrone age"
Observation: 25 years
Thought: I need to calculate 25 raised to the 0.43 power
Action: Calculator
Action Input: 25^0.43
Observation: Answer: 3.991298452658078
Thought: I now know the final answer
Final Answer: Camila Morrone is Leo DiCaprio's girlfriend and her current age raised to the 0.43 power is 3.991298452658078.
> Finished chain.The 2 SMR buffer
PubChat’s recently introduced new feature ChatGPT with web search capabilities utilizes the aforementioned agent. The potential costs it may incur should not be a major concern, as we have allocated 2 SMR(~$0.12) upfront, which is sufficient to cover any potential expenses.
Conclusion
The introduction of the web search feature, leveraging LangChain and its agent functionality, signifies a remarkable advancement of PubChat. The substantial costs associated with agent capabilities make it improbable for companies to incorporate them into their free offerings. Our approach demonstrates the ability to provide this feature on a pay-per-use basis, avoiding the need for high upfront subscriptions while still delivering value to users. This ensures accessibility while balancing the financial implications of offering such advanced functionalities.
