How to perform sentiment analysis on Qualtrics and push sentiment data back to a service - Qualtrics Technical Part 3

Qualtrics is a great tool for building and distributing surveys related to different operations. In the previous blog post we have seen how to calculate score of a survey response and push it back to a service. Now, lets discuss a new use case. What if there are textual questions in the survey and we want to get the text sentiment of answers to those questions. In this blog post we will go through the Text IQ tool of Qualtrics focusing mainly on Sentiment Analysis. This blog post is a part of my Qualtrics technical series.

Requirements

1. Qualtrics Production account. Trial account doesn’t provide actions and triggers which will be used in this case.

1. A simple back end service to test the integration. I have deployed a Python flask service that just prints the data received from Qualtrics.

1. Basic Knowledge of Qualtrics like creating surveys, survey flows etc.

Process

So, with all that defined, lets get started with the actual thing. There are four main steps that we have to perform to calculate the sentiment score of a question in the survey and push the data. But before that lets check the demo service controller which is as follows.

from flask import Flask, request, jsonify
from waitress import serve
import json
import os

app = Flask(__name__)
port = int(os.getenv("VCAP_APP_PORT", 9000))

@app.route('/')
def check_health():
    return jsonify(
        status=200,
        message="Up"
    )

@app.route('/showSurveyData', methods=['POST'])
def show_survey_data():
    data = json.loads(request.get_data())
    print(data)
    return jsonify(
	    status = 200,
	    results = data
    )

serve(app, port=port)

As you can see, we just have two end points. One for health check and the other one for displaying the survey data pushed back to the service. You can use your own service here to do more with the data. One thing to mention, the Qualtrics instance we are using is on public cloud. So it can only access services that are deployed on public cloud. That’s why I have deployed this service on hanatrial account for now.

Survey

We will work with the same survey used in the last two blog posts. I have just added another question here which accepts text answer.

Now, just to explore the sentiment analysis capabilities, publish the survey and then click on the the “Preview” button on the Survey tab. Answer all the questions (mainly the text question). Do the preview 4-5 times and accumulate that many responses.

Data & Analysis

Now, we can go to the Data & Analysis tab. Here under the “Data” sub tab we can see all the recorded responses. If we go to the “Text” sub tab then we will be able to use the Text IQ tool for different textual analysis, out of which we will see only the sentiment analysis in this post. Record level sentiment analysis generally takes some time to generate. Because based on the previous survey responses the existing RNN based model for sentiment prediction is retrained via transfer learning.

In the meanwhile, I have created a topic and assigned it to all the 4 responses. To create a topic we just type it in the search bar and click on create topic button. Topics are generally different aspects that are covered in the responses. In this case, I have created a topic with the word “training“. So, which ever sentence has the word “training” will belong to this training topic.

Similarly, you can add many other topics based on complicated queries as shown here. Once this is done, we will be able to see both record level as well as individual topic level sentiments for the different responses. But still sentiment scores will take sometime to get generated. So, this is a good time for a cup of coffee :P.

For me it took 10 minutes for the sentiment scores to get generated. Once the first few scores are generated. Next ones won’t take this much time as the model will only predict. Sentiment scores range from -10 to +10 and the scores are as follows.

If we click on the “Edit” icon in the right top corner of each response we would see a detailed sentiment analysis and for each topic you have assigned to a response, it will also show topic wise sentiment as follows.

With this we will close the data analysis tab as I believe now you have some idea regarding Qualtrics sentiment analysis feature. For more information regarding the sentiment analysis done by Text IQ, you can go here.

Text Sentiment Block

Now, that we have understood the sentiment analysis part, lets see how we can store these sentiments, so that they can used later on. In the previous blog post we have added one “Embedded Data” block in the survey flow where we have defined an embedded variable which stores the survey score. But this will not work with sentiment data. Qualtrics provides a separate block just for assigning sentiment related data (Sentiment Label, Sentiment Polarity, Sentiment Score) of a question to embedded variables. Now go to “Survey Flow” and add one text sentiment block as follows.

Now, select a question for which the sentiment has to be predicted. If we want the sentiment to be topic based, we can also add it once the question is added. Now, we can define some embedded variables here and assign Sentiment Score, Sentiment Polarity and Sentiment Label as follows.

With this we can save the survey flow and head back to the “Survey” tab.

Actions

Now that we have embedded variables that save the sentiment score of the question, we can access it via piped text embedded field. So, open the already defined task in the Actions tab as shown in the previous blog posts and create a new object called “sentiment” and assign the “SentimentScore” embedded variable to this object as follows.

Now, lets save it and also publish the survey.

Testing

To test this application, we can simply open the preview mode and complete the survey. Now, if we check the logs of our service we should be able to see the sentiment score along with rest of the survey data as follows.

In this case, I have only sent back the sentiment score, you can also try sending the sentiment label or polarity. With this I conclude this topic. If you really liked it make sure to leave your feedback or suggestions down below.