Insights from Psychology: Simple Symbols Can Influence Evaluations

When it comes to capturing honest, representative information from respondents about their attitudes, their behaviors, and their feelings, even the smallest of things can risk skewing the results.  We know to avoid leading questions.  We have an awareness that the order of questions matters.  But what about seemingly innocuous things, like whether we ask respondents to use a check mark or an X to indicate agreement?

Yoon and Vargas (2018) found that something as subtle as symbols with positive or negative associations are enough to prime a more positive or negative response when making evaluative judgements. Priming is the phenomenon wherein concepts in the mind are made more accessible but not are immediately activated.  When subsequently asked to respond, think, or behave, those highly accessible concepts will be more quickly activated than other, less accessible concepts.  In this case, the symbol of a check-mark primes the concept of “good.”  The idea of “good” is not made conscious; you’re not actively thinking “good.”  When subsequently asked how you feel about something, though, “good” is the more immediately accessible response.  The symbol of an X operates the same way, except with a negative connotation and priming effect.

Study 1 of Yoon and Vargas, of their four-study investigation, confirmed the positive associations of the check mark and the negative associations of the X-mark using the Implicit Association Task, a psychological tool developed to measure unconscious attitudes.  In Study 2, participants were asked to indicate their opinions on sociopolitical items from the GALLUP Social Poll (e.g. “banning handgun ownership”) by indicating agreement or disagreement dichotomously.  Participants were randomly assigned to either using the check-mark, the x-mark, or a neutral O-mark as their indicator.  Results showed there was a statistically significant effect of mark type on the tendency to agree with statements; those who used a check-mark agreed with more of the items (M = 11.03, SD = 2.30) than those who used an X-mark (M = 9.35, SD = 2.13).  Whereas Study 2 was conducted as a paper-and-pencil task, Study 3 replicated the findings with an online survey where a check-mark or x-mark appeared when the clicking a box.  This result addressed the question if manually drawing the symbol was necessary to elicit the priming effects; it was not.

Study 4 investigated if it was possible to prevent these effects.  Priming effects are often explained by people misattributing where their feelings are coming from.  In this case, the priming of a “positive” or “negative” affect results from the positive/negative symbols, but this is not on a conscious level – respondents aren’t aware that the symbol itself is influencing them.  Thus, when they evaluate the statement and a “positive” or “negative” feeling is at the ready, they attribute the readiness as indicating that’s how they feel about the statement.  Study 4 demonstrated that by enabling people to properly attribute the positive/negative feelings to the symbols, the priming effect goes away.  This was achieved with a “forewarning” condition, where half of participants were given explicit information and warning about possible associations between the marks and the concepts and their effects.  As a result, participants in the “forewarning” condition did not demonstrate the previously established priming effect: there was no significant difference in the number of statements respondents agree with when they used the check- or x-mark if they were first giving this explicit warning, t(384) = -0.27, p = .79.

The use of neutral symbols, then, is important to consider when designing surveys.  Online surveys that use neutrally-valanced radio buttons, such as those developed by KJT Group, are already taking the steps to protect against unintended priming effects. But don’t feel overwhelmed with all the possible innocuous symbols that could be biasing your survey results!  By developing protocols and using templates that already account for these concerns, the work of constructing a good questionnaire can be focused on the larger, study-specific concerns, rather than if there’s a smiley hidden somewhere on the page. ????

Yoon, G. & Vargas, P. (2018). The subtle influence of check and x marks: How symbolic markings influence judgement. Journal of Consumer Psychology, 28(4), 682-688. doi:10.1002/jcpy.1048