【AIGC 学习】PAL - LangChain 使用3

今天介绍一下  PAL，PAL 的全名是 Program-Aided Language Models，文献链接在https://arxiv.org/pdf/2211.10435.pdf。

PAL 介绍

PAL 是一种新的自然语言推理方法，它利用程序作为中间推理步骤。与现有基于语言模型的推理方法不同，它的主要思想是将求解和计算外包给外部的Python解释器，而不是使用语言模型来理解问题并解决。这样可以保证最终答案在正确的前提下是准确的。PAL在BIG-Bench Hard和其他基准测试中展示了语言模型和Python解释器之间的无缝协同，涵盖了13个任务。在所有这些基准测试中，PAL都优于使用“chain-of-thought”方法的更大的语言模型，如PaLM-540B，并在所有基准测试中设置了新的最佳正确率。我们相信这些结果为未来的神经符号人工智能推理者打开了令人兴奋的方向。

这个表体现了CoT （chain-of-thought）和 PaL （本文介绍的方法）的在符号推理和算法推理任务上的准确率对比。

先看一个论文中的的对比：

准备工作

今天来做数学题 Jan has three times the number of pets as Marcia. Marcia has two more pets than Cindy. If Cindy has four pets, how many total pets do the three have?

还是用昨天和前天代码中的 https://colab.research.google.com/drive/1BkpMLfYEofhNK-PCKCSj9_SJqnUK40gR 中 Plain Conditional Generation - First with OpenAI GPT3 的代码

https://colab.research.google.com/drive/1zTTPYk51WvPV8GqFRO18kDe60clKW8VV

中 PAL Math Chain 的代码

运行代码

直接使用llm

from langchain.llms import OpenAI

llm = OpenAI(model_name='text-davinci-003',
            temperature=0.9,
            max_tokens = 256)
            
text = "Jan has three times the number of pets as Marcia. Marcia has two more pets than Cindy. If Cindy has four pets, how many total pets do the three have?"

print(llm(text))

看一下 text-davinci-003 模型给到答案：

Jan has 12 pets, Marcia has six pets, and Cindy has four pets. Together, they have 22 pets.

这个答案是错误的。

使用 Pal Chain 方法

from langchain.llms import OpenAI

llm = OpenAI(model_name='text-davinci-003',
            temperature=0,
            max_tokens=512)

from langchain.chains import PALChain

pal_chain = PALChain.from_math_prompt(llm, verbose=True)

question = "Jan has three times the number of pets as Marcia. Marcia has two more pets than Cindy. If Cindy has four pets, how many total pets do the three have?"

pal_chain.run(question)

Entering new PALChain chain...

def solution(): """Jan has three times the number of pets as Marcia. Marcia has two more pets than Cindy. If Cindy has four pets, how many total pets do the three have?"""

cindy_pets = 4

marcia_pets = cindy_pets + 2

jan_pets = marcia_pets * 3

total_pets = cindy_pets + marcia_pets + jan_pets

result = total_pets

return result

Finished chain.

'28

这次答案就是正确的了。