When User Research Is Wrong, Is It Fair to Say Someone Lied?

When User Research Is Wrong-I don’t really eat much fast food, but even I’ve noticed that most of the major fast food chains are introducing new menu items that veer from the traditional hamburgers and fries. According to a new Business Week article, this shift is an attempt by fast food companies to woo a large market segment, millennials, based on their stated food preferences.

Unfortunately, what the purchase data reveals is that millennials, like the generations before them, go to McDonald’s and the like to purchase the comforting standards. Sriracha drizzles at Pizza Hut may appeal in a survey, but the pizza that’s actually getting ordered is covered in pepperoni and mushroom. As a result, many fast food places are scaling back their experimental efforts and refocusing on the basics.

What really caught my attention in the story was the author’s use of the verb “lying” to describe millennials’ purported interest in evolving fast food offerings. (As a side note, the same author previously published a story titled “Millennials are telling a big lie about McDonald’s.” Perhaps she is prone to hyperbole?)

When I talk about user research, I sometimes joke that people are liars. By this I mean that people offering feedback to a prototype or a product idea don’t necessarily tell you how they would really use the eventual product. I think there are two basic reasons for it:

  • People are trying to be nice. The more the prototype you share reflects your effort and passion, the harder it is for people in a survey or usability study to knock it down. This is one reason why it’s important to work with unbiased moderators who can help set expectations that criticism is ok.
  • People are poor predictors of what their future self wants. I’ve explored this one before: We are really optimistic when it comes to our future selves. Our future selves are fitter, more motivated, and will have no problem waking up at 6 am on a Sunday to take on that race.

I don’t know how the fast food companies in question gathered their data about what millennials are interested in buying and eating, but I imagine both of the factors above come into play. In one case, someone is presented with a food product idea–maybe even a free sample–and asked to comment on what he thinks of it. Chances are, the food is reasonably palatable, and if you don’t want to offend the person who just gave you something free to eat, you might say you like it more than you actually do. Or maybe the idea of the food is appealing; sure, I would absolutely order fresh spinach on my pizza because that sounds healthy and easy. However, when the siren song of food delivery calls at 2 am, do I go for the fresh spinach, or the sausage? You know how this story ends.

As much as I’ve said people are liars in the context of product research, I don’t believe people are willfully misleading researchers. Rather, we aim to please, and we strive to become better versions of ourselves that would use that product or order that food item. Unfortunately, what users want to be true isn’t always so, and that’s why observational research (and in the case of fast food, test market launches) will continue to be so essential to successful product development.