Quantitative Research for Small Markets and Samples
During our 2017 webinar “Research for Small Markets and Samples” I made the point that small market business objectives addressed via central tendency measures are less helpful – I would like to revisit and challenge this. The point I would like to challenge is that not only are central tendency measures less helpful, but that they can also be entirely misleading and hide important patient characteristics because of how variance and outliers are traditionally handled. In the two years since the webinar we’ve continued conducting market research in various rare disease spaces and these two components have presented as a consistent thematic undertone. Of course, these themes are present in most market research studies, but offer unique challenges when the market, and therefore the research sample, is relatively small.
There is a well-known example in statistics called “Anscombe’s Quartet” that consists of four unique, two variable, data sets all with the same mean (central tendency), variance, and correlation. These summary statistics might lead us to conclude that the data are the same; however, when graphing each set of data, we see entirely different relationships between the two variables. In two of the data sets there is an outlier impacting the results and across the four sets there are three unique functional forms that lead to entirely different insights. In large markets these issues can be argued as inconsequential – outliers can be removed and a common understanding of the variation in the data will still apply to a sufficiently large number of individuals. In any rare disease space this is no longer true. That outlier may be representative of an important patient sub-population that we were unable to adequately sample. A simplified understanding of the variation will miss a sufficiently large proportion of the market.
These issues highlight the need to look beyond central tendency and, instead, towards a deeper understanding of the causes of sample variation – especially outliers. This starts with research design and the idea that we need to maximize the utility of the smaller sample by gathering repeat observations with as much variation as possible. In patient samples this could mean a designed experiment (e.g. conjoint), while in physician samples it could mean collecting as much unique information about their patient pool, treatment algorithms, etc. as possible (e.g. a chart audit). For both patient and physician samples repeat observations over time (e.g. an ATU) can also be leveraged along-side a quantitative meta-analysis to pool data across studies. In all scenarios this ends with producing a more holistic picture of the sample instead of the average picture.
We know that patients with rare diseases have relatively more unique challenges and needs. Successes in these markets rely on effectively addressing and managing this diversity throughout product design and marketing cycles. Consequently, when weighing commercial trade-offs and decisions that will impact these populations it is critical to gain an understanding of this diversity. Leveraging substantive market research techniques concerned primarily with the characterization of patient heterogeneity is a principal starting point down this path. Contact KJT Group today to learn more about our capabilities and offerings in this space.
