I recently decided to test ChatGPT’s reading comprehension w…
I recently decided to test ChatGPT’s reading comprehension with the following prompt: User: “Sally gave John the book. Who has the book now?” ChatGPT’s response was: ChatGPT: “Based on the given statement, Sally gave the book to John. Therefore John has the book now.” This is correct. But then I asked: User: “Who had the book before Sally gave the book away?”. ChatGPT’s response was: ChatGPT: “Based on the given statement, it does not specify who had the book before Sally gave it away. The statement only mentions that Sally gave the book to John.” The correct answer should have been that Sally had the book before giving it away (I made this example a while ago and the model may have changed since then, or may stochastically answer the question differently maybe even getting the question right. However, for the purposes of this question can shall assume that there are questions such as this that ChatGPT fails to answer.) Suppose you are building a machine-reading based question answering system. On the front end, a user enters a statement and a question into your system about someone doing something, similar to the example above. On the back end a request is sent to ChatGPT (or any other equivalent large language model). Describe how VerbNet v3.2 can be used to help the large language model answer questions such as these. Hint: You cannot change the pre-trained language model but you can do processing between the user’s question and the request to the LLM, or between when a response is received from the LLM and when the final answer is returned to the user.
Read Details