Generated Knowledge Prompting
Last updated
Last updated
Image Source:
LLMs continue to be improved and one popular technique includes the ability to incorporate knowledge or information to help the model make more accurate predictions.
Let's try a simple prompt:
Prompt:
Output:
This type of mistake reveals the limitations of LLMs to perform tasks that require more knowledge about the world. How do we improve this with knowledge generation?
First, we generate a few "knowledges":
Prompt:
Knowledge 1:
Knowledge 2:
The next step is to integrate the knowledge and get a prediction. I reformatted the question into QA format to guide the answer format.
Prompt:
Answer 1 (confidence very high):
Answer 2 (confidence is a lot lower):
Some really interesting things happened with this example. In the first answer, the model was very confident but in the second not so much. I simplify the process for demonstration purposes but there are a few more details to consider when arriving at the final answer. Check out the paper for more.
Using a similar idea, can the model also be used to generate knowledge before making a prediction? That's what is attempted in the paper by -- generate knowledge to be used as part of the prompt. In particular, how helpful is this for tasks such as commonsense reasoning?
We are using the prompt provided in the paper by .
