Learning / Teknik Prompting

PAL (Program-Aided Language Models)

Gao et al., (2022) memperkenalkan metode yang memakai LLM untuk membaca soal dalam bahasa alami dan menghasilkan program sebagai langkah-langkah penalaran perantara. Metode yang dinamai program-aided language models (PAL) ini berbeda dari chain-of-thought prompting: alih-alih memakai teks bebas untuk mendapatkan solusi, PAL menyerahkan langkah penyelesaiannya ke runtime program seperti interpreter Python.

PAL

Sumber gambar: Gao et al., (2022)

Mari kita lihat contohnya menggunakan LangChain dan OpenAI GPT-3. Kita ingin membuat aplikasi sederhana yang bisa memahami pertanyaan yang diajukan lalu memberikan jawaban dengan memanfaatkan interpreter Python.

Lebih spesifiknya, kita ingin membuat fungsi yang memungkinkan LLM menjawab pertanyaan yang membutuhkan pemahaman tanggal. Kita akan memberi LLM sebuah prompt yang berisi beberapa contoh (demonstrasi) yang diambil dari sini.

Berikut import yang kita butuhkan:

import openai
from datetime import datetime
from dateutil.relativedelta import relativedelta
import os
from langchain.llms import OpenAI
from dotenv import load_dotenv

Pertama, kita atur beberapa hal dulu:

load_dotenv()

# API configuration
openai.api_key = os.getenv("OPENAI_API_KEY")

# for LangChain
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")

Siapkan instance model:

llm = OpenAI(model_name='text-davinci-003', temperature=0)

Siapkan prompt + pertanyaan:

question = "Today is 27 February 2023. I was born exactly 25 years ago. What is the date I was born in MM/DD/YYYY?"

DATE_UNDERSTANDING_PROMPT = """
# Q: 2015 is coming in 36 hours. What is the date one week from today in MM/DD/YYYY?
# If 2015 is coming in 36 hours, then today is 36 hours before.
today = datetime(2015, 1, 1) - relativedelta(hours=36)
# One week from today,
one_week_from_today = today + relativedelta(weeks=1)
# The answer formatted with %m/%d/%Y is
one_week_from_today.strftime('%m/%d/%Y')
# Q: The first day of 2019 is a Tuesday, and today is the first Monday of 2019. What is the date today in MM/DD/YYYY?
# If the first day of 2019 is a Tuesday, and today is the first Monday of 2019, then today is 6 days later.
today = datetime(2019, 1, 1) + relativedelta(days=6)
# The answer formatted with %m/%d/%Y is
today.strftime('%m/%d/%Y')
# Q: The concert was scheduled to be on 06/01/1943, but was delayed by one day to today. What is the date 10 days ago in MM/DD/YYYY?
# If the concert was scheduled to be on 06/01/1943, but was delayed by one day to today, then today is one day later.
today = datetime(1943, 6, 1) + relativedelta(days=1)
# 10 days ago,
ten_days_ago = today - relativedelta(days=10)
# The answer formatted with %m/%d/%Y is
ten_days_ago.strftime('%m/%d/%Y')
# Q: It is 4/19/1969 today. What is the date 24 hours later in MM/DD/YYYY?
# It is 4/19/1969 today.
today = datetime(1969, 4, 19)
# 24 hours later,
later = today + relativedelta(hours=24)
# The answer formatted with %m/%d/%Y is
today.strftime('%m/%d/%Y')
# Q: Jane thought today is 3/11/2002, but today is in fact Mar 12, which is 1 day later. What is the date 24 hours later in MM/DD/YYYY?
# If Jane thought today is 3/11/2002, but today is in fact Mar 12, then today is 3/12/2002.
today = datetime(2002, 3, 12)
# 24 hours later,
later = today + relativedelta(hours=24)
# The answer formatted with %m/%d/%Y is
later.strftime('%m/%d/%Y')
# Q: Jane was born on the last day of Feburary in 2001. Today is her 16-year-old birthday. What is the date yesterday in MM/DD/YYYY?
# If Jane was born on the last day of Feburary in 2001 and today is her 16-year-old birthday, then today is 16 years later.
today = datetime(2001, 2, 28) + relativedelta(years=16)
# Yesterday,
yesterday = today - relativedelta(days=1)
# The answer formatted with %m/%d/%Y is
yesterday.strftime('%m/%d/%Y')
# Q: {question}
""".strip() + '\n'

Contoh ini dibiarkan dalam bahasa Inggris karena pertanyaan dan contoh-contoh di dalamnya adalah bagian dari kode program (string dan komentar Python) yang diambil apa adanya dari repo PAL.

llm_out = llm(DATE_UNDERSTANDING_PROMPT.format(question=question))
print(llm_out)

Hasil output-nya seperti ini:

# If today is 27 February 2023 and I was born exactly 25 years ago, then I was born 25 years before.
today = datetime(2023, 2, 27)
# I was born 25 years before,
born = today - relativedelta(years=25)
# The answer formatted with %m/%d/%Y is
born.strftime('%m/%d/%Y')

Isi llm_out adalah potongan kode Python. Di bawah ini, perintah exec dipakai untuk menjalankan potongan kode Python tersebut.

exec(llm_out)
print(born)

Hasil output-nya seperti ini: 02/27/1998

Gabung grup