Articles & Blogs

How Satisfied Are People? Answering That Is More Complex Than You Think.

Published by Quirk’s Media on August 19th, 2019

Height.  Weight.  Volume.

These concrete variables have clear definitions. Height relates to how tall something is; it’s vertical distance.  Weight pertains to how heavy something is regardless of its size.  Volume is the amount of space something occupies; it incorporates height, but not weight.

Although we understand the denotative differences between height, weight, and volume, measuring them as variables requires an extra step.  We need to establish how we are measuring it.  Are we measuring weight at sea level in pounds or on Mt. Everest in stones?  In research it is utterly useless to say one participant weighs 150 and the other weighs 10.7 without establishing our unit of measurement.  Each participant needs to be measured on the same scale – 150 pounds and 10.7 stones are equal weights, but we only recognize the two participants are of equal weight when we compare them on the same scale.

Satisfaction.  Loyalty.  Preference.

These are abstract variables that do not have clear definitions nor standard units of measurement.  Despite this, market research often pretends they are as straightforward to measure as height.  We use a single question – “How satisfied are you with this?” – the same way we might ask “How tall are you?”, and only concern ourselves with how many available answers there are.  Asking people to select one of four responses or one of thirteen (i.e. a 4-point or a 13-point scale) is akin to deciding whether people should indicate their height among options of:

<=4’       5’           6’            >6

or

<=4’    4’3”    4’6”    4’9”    5’    5’3”    5’6”    5’9”    6’    6’3”    6’6”    6’9”    >=7”

Satisfaction and height, however, are not similar concepts.  There is universal understanding of what 6’ represents.  Respondents know how to obtain that information and we can expect that two respondents who give the same answer are of the same height.  Although the 13-point scale above may result in some error – a 5’11” person and a 6’1” person would both select 6” and they are not of identical height – the error variance may be acceptable for the level of accuracy required by the research.  With satisfaction, if two respondents both indicate a 7 on an 11-point scale, what does this mean?  We do not know the process by which the respondent selected that number, nor do we know if who people who selected the same number used the same process.

Beyond the issue of having no standard unit of measurement, by their very nature abstract variables are complex.  Height is simple.  I just need one measurement: the distance from the top to the bottom.  Rather than height, satisfaction may be more akin to volume.  Volume requires multiple measurements that are combined in a predetermined way for the type of object under investigation.  The process of determining height does not change from a triangle to a cylinder.  For volume, however, the individual pieces of information you need changes based on the object being considered.  To get the volume of a cube, you collect one piece of information (the length of one side) and perform a specific operation (multiple the length by itself three times).  For a pyramid, you need more pieces of information (the length, the width, and the height) and a different operation (multiple these together, then divide by three).  What pieces of information are needed and how are they combined to measure satisfaction with a car?  Are they the same pieces and process for determining satisfaction with a medication?

Latent and Constructed Concepts

Before you can determine the pieces and the process needed to measure your abstract variable, you need to look at your theoretical assumptions about the concept itself.  In psychometrics, there are two common ways to consider the nature of abstract psychological variables: latent or constructed.

Latent refers to a state of existing but being hidden.  As psychological constructs, latent concepts exist within the mind but cannot be directly accessed or observed.  Loyalty, knowledge, and desire, for example, occur within the mind and are thought to exert influence on our thoughts, feelings, and behaviors.  These outcomes are then manifestations of the latent construct.  We answer the exam questions correctly because we have knowledge; we purchase from the same manufacturer because we have loyalty; we obsess over and feel affection for our partner because we have desire.  Consequently, because we cannot directly measure the latent concept, we must rely on inferring our understanding of it by observing its manifestations.

Figure 1. A latent variable causes the observable thoughts, feelings, and behaviors.

Constructed concepts, on the other hand, are theoretically different from latent variables.  Rather than a concept that exists independently in the mind, it is an aggregate concept that summarizes related thoughts, feelings, and behaviors.  Satisfaction, preference, and interest, for example, may represent an overall assessment of a topic that is made up of different direct assessments.  When we ask about one’s preference for a product, the response is often assumed to be informed by numerous smaller considerations: how attractive the product is, how useful the product is, how costly the product is, etc.  The overall preference may drive future behavior (i.e. purchasing the product), but it does not drive the smaller considerations (i.e. I like the product overall because I like its parts, not the other way around).

Figure 2. construct variable is made up of observable thoughts, feelings, and behaviors.

How do you determine if the concept of interest is latent or constructed?

There is no simple answer for this.  Psychology does not have a definitive catalogue of whether specific mental concepts are latent or constructed.  Even in the subfield of personality, theorists will debate if you are gregarious and talkative because you are extraverted or if you are extraverted because you are gregarious and talkative. You must rely on your theoretical assumptions for the nature of your variable: is it an active agent influencing thoughts feelings and behaviors, or is it a summative representation of more distinct experiences?  The pieces of information and process required to measure the abstract variable become more identifiable once you have selected your theoretical framework.

For a latent variable, you collect information on the assumed manifestations; select concrete examples of what the abstract variable looks like.  These pieces should all share some common theme but appear as variations on this theme.  What does loyalty look like?  Loyalty can inform exclusivity (I only shop from this store) or frequency (I shop here more than other places); it can inform our reviews (I gave it 5 stars on Google) and recommendations (I told my friends to shop here).  Loyalty could even inform our risk taking (I wouldn’t normally buy this product but because it was at this store I gave it a try).  Each of these items are a variation on the theme of loyalty to the store and would likely correlate with one another.  For a latent variable, the process is to use structural equation modeling for estimation (which is beyond the scope of this article to describe).

For a constructed variable, you collect information on the defining characteristics of the concept, those variables which form it.  Unlike latent variable inputs, constructed inputs do not need to correlate (but can).  You may want to define satisfaction with an entrée by averaging respondents’ ratings of taste, appearance, and smell.  In this instance, taste need not be correlated with appearance for them to be useful predictors of satisfaction.  The process here is simple: combine the scores of individual items arithmetically (e.g. sums, averages) for an overall constructed metric.  On the other hand, you may acknowledge you do not know all the inputs for satisfaction.  Customer service satisfaction can be informed by several inputs – politeness, helpfulness, knowledge – but you recognize there may be those you are not collecting data on (e.g. being listened to, language of customer rep., etc.).  In this situation, simple arithmetic is insufficient and structural equation modeling or other advanced statistical operations should be used to include random error into the equation.  This error term would be the theoretical equivalent of saying “there is something else influencing the responses in customer service satisfaction, but it was not captured with this data”.

As you can see, psychological concepts like satisfaction and loyalty are far more complex than physical concepts like height.  The lack of a standard unit of measurement is only half the battle because there is no universal agreement on how these concepts function in the first place (are they causal or summative?), the necessary inputs to determine them, nor the most appropriate computational methods to use.  Furthermore, these concerns are all separate from, and more important than, the number of alternative options from which respondents select their response.  What we can do is appreciate these concepts for their complexity and take the time to think about how we propose they function.  Only then can we begin to appropriately identify the pieces of information to collect and the processes to use to measure them.