Answer insights provide valuable information about the use of the Answer feature by the users. It provides you with a comprehensive view of the user’s interaction with the application. You can see both the successful or unsuccessful answers presented to the users, along with any unanswered queries by the application. You can use this information to understand user behavior and queries, perform response analysis, identify gaps, and accordingly optimize the application response.

Accessing Answer Insights

You can view the user queries and answers generated by the application on the Answer Insights page under the Analytics tab.

Filtering Options

You can filter the analytics data based on the following parameters:

• Index Configuration: You can choose the index configuration corresponding to which you want to view the data. 
• Type of Answer Model: You can choose to see the analytics corresponding to Extractive or Generative Models of Answer Generation. You can also select ‘Both’ to view all the answers at once. 
• Date Range: You can choose to view the data from the last 24 hrs, or the last 7 days or also provide a custom date range for the data to be displayed. However, note that you can only view the data for a maximum of 30 days at a time. 

Answer Analytics

Based on the filters set for the data, the application displays the following data in two different tabs:

1. Queries with Answers: This section lists all the queries directed to the application for which the application was able to find and display answers. If the same query is posted more than once, it is displayed only once in this list but the count is increased accordingly. 

• • Query – User query posted to the application
  • Answer – The corresponding answer provided to the user by the application
  • Count – The number of times the given query was posted and the corresponding answer was provided as a response during the selected period. Note that the count is increased only when the same answer is presented to the same question. If the answer varies, the query is listed as a new one. 
  • Thumbs up – Count of the number of times the answer was acknowledged positively. 
  • Thumbs down – Count of the number of times the answer was acknowledged negatively.
    
    Note that the feedbacks are displayed(Thumbs up and Thumbs down) only if the Feedback option is enabled for Answers in the Search Interface.

    

• 2. Queries without Answers: This section lists all the queries directed to the application for which the application was unable to find any answers. 
  • Query – User query posted to the application
  • Count – Number of times the query was posted in the selected time duration

Viewing Answer Details

Click on any of the records to see the details.

The details of an answer are presented in three different tabs:

Summary tab gives an overview of the answer generated. The following details are displayed on this tab. 

Qualified Chunks tab lists the chunks used for answer generation. It shows the qualified chunks, top chunks sent to the LLM and chunks used for answering. 

LLM Request and Response tab shows the configuration used for Generative answers, the prompt sent as a request to the LLM, and the response sent back by the LLM.  

Click on the JSON view to see all the details in JSON format.