by Judy Langer, President, Langer Qualitative, LLC.
As published on TheModerator.Net, Winter 2007

I’ve been a qualitative researcher since the early 1970s and it’s to be expected that the field has changed during that time. The rate and degree of change in just the last few years, however, has been faster and more dramatic, I believe, than at any other point over those 30+ years. Some of the shifts I applaud, some I approve, some I accept — and others I find appalling.

The changes can be looked at from three perspectives:

• Research quality
• Business considerations for independent qualitative consultants
• Personal satisfaction

Here’s a list of the six biggest changes I'm seeing today, along with my own views on what they mean for research and researchers.

1. ABFG “Anything But Focus Groups.” It used to be that focus groups were totally identified with qualitative research (QLR). Clients would talk about or ask for “focus groups” when they really meant that wanted something qualitative, and it was the qualitative research consultant’s (QRC’s) job to point out that when another open-ended, exploratory approach was better suited to their study’s purpose. Now, thanks to competition in the field, and bashing in the media (including, of course, Malcolm Gladwell’s book Blink and his highly publicized speeches), focus groups have gotten a bad name.

• Research quality: Focus groups remain a productive and efficient technique, I believe. The interactivity of respondents can work for the research, pulling out more from participants than individual interviews, building new ideas, helping the researcher and observers see their target consumers/customers firsthand.
  
  One-on-one interviews are also useful in many cases, especially advertising communication and decision-making studies. However, I don't agree that they are the best way to encourage people to open up about sensitive subjects. Sometimes people are more comfortable relating to their peers who experience the same problems than just talking with an interviewer. Rather than the ABFG approach that condemns focus groups wholesale, we need a more selective research-based view of methodology.

• Business: Focus groups remain the mainstay of qualitative research and business is still strong among a large segment of clients. Most QRCs, of course, have a variety of other methods in their “toolbox,” so they shift gears to other methods as appropriate. (Some, of course, get on the ABFG bandwagon themselves.)

• Personal: I continue to enjoy the close contact with people that takes place in the in-person focus groups – hearing what they really feel and experience, observing them open up to one another, feeling the sparks when ideas build on one another. While I use – and also enjoy – a variety of other qualitative methods, I would miss focus groups.

2. “My Method’s Better Than Yours — Believe Me.” When I started doing qualitative research, the field was relentlessly criticized by quantitative researchers as “unscientific.” Now much of the criticism of qualitative methods, especially the so-called “traditional” focus group, comes from QRCs claiming superiority for their methods. Typical they assert that their “new” technique offers “unique” benefits and uncovers insights that would never have been possible any other way.

• Research quality: Most of these claims are presented with assertions but no supporting evidence or side-by-side comparisons of methods. It’s great to open up the possibilities in QLR to more diverse methods, but as a research community we need to understand better what really does and does not work.

• Business: For practitioners with new “products,” it can, of course, be good business to sell something different. Practitioners resisting the pressure to devise new twists can come across as uninteresting.

• Personal: Frankly, I find this trend to knock other QLR methodologies disturbing. Much of is hype and overselling.

3. Incredible Shrinking Samples. QLR has never been about large or projectable samples; just about every qualitative research report states this upfront in the caveat. The actual number of respondents is almost always relatively small by quantitative standards, even in a relatively sizable study – say, 10 focus groups, or 50 in-depth interviews.

Lately, though, clients are requesting near to non-existent sample sizes in order to keep costs down. Some recent examples: A client initially requested one focus group on customers’ acceptance of a new product. Another client wanted a bid on five IDIs to be split between the U.S. and Canada, to explore how patients with several different medical conditions experience their problems.

• Research quality: We all know that QLR is intended to generate hypotheses, but with such tiny studies we lack confidence in our own hypotheses.

• Business: Set-up work (screener, topic guide, briefing, etc.) on such “micro studies” is the same as for much larger studies. Charging a flat fee for this time, rather than the same price-per-interview as a larger study, can make these tiny studies prohibitively expensive for a client. Alternatively, not charging means the QRCA is either underpaid or does not get the job (which may be all right with some researchers, of course).

• Personal: I have very mixed feelings on this. These tiny studies hardly seem like “real” research. The good part, though, is that there is no chance for the research fatigue entailed in larger qualitative studies.

4. DIY Moderating. More and more clients are using in-house resources for qualitative research. A number of marketers have hired researchers with QLR training, and others send employees (some with no prior research background) to be trained. Some conduct only a few projects a year or, with no training at all, gather “respondents” (sometimes company employees at the company site!) to ask a few questions.

• Research quality: Few people can do QLR well without doing it regularly. While in-house researchers know more about their company/product than outsiders do, they can be too close to the subject to study it objectively. It may also be difficult for in-house researchers to withstand internal political pressures. Most independent consultants have multiple clients and can risk losing one who occasionally “shoots the messenger” because of a report saying respondents rejected the ad/product/concept. In-house researchers by definition have one client, their employer, and may find it harder (consciously or unconsciously) to speak frankly.

• Business: Needless to say, this trend shifts work away from independent consultants.

• Personal: Admittedly, we consultants have a vested interest in this issue, but that does not mitigate the obvious pitfalls, in my view.

5. Ethnography’s Hot. Observational research has gained popularity in recent years. To some extent, this is part of the ABFG trend. The underlying premise: people are often unaware of or lie about what they really do, so it’s best to see how they behave in their natural habitats versus having them blather on in front of one-way mirror at a focus group facility.

• Research quality: Seeing people’s behavior and interviewing them “in context” (at their home, office, a store, a car, a bar, etc.) is a very valuable technique. However, the term “ethnography” is now being misused to describe any kind of in-person interviews that take place outside a formal research facility. Doing a “girlfriend group” in a woman’s home is not ethnography — it’s a focus group in a home with the usual pluses and minuses. Hanging out with drinkers at a bar and pretending to be just another patron may be fun but it’s not ethnography either.

• Business: The buzz about ethnography is creating a bandwagon effect. Clients now request that technique, or what passes for it, on many studies. True ethnography is justifiably expensive (often double or triple the price of regular in-home or in-facility depth interviews); clients sometimes have sticker shock when they receive a cost estimate and decide against using the technique. I am not an ethnographer so I don't claim to do ethnography (although I certainly interview people outside a facility when appropriate), which means that this business goes elsewhere.

• Personal: The desire for methods to reveal true behavior and feelings is one that motivates all good researchers. I think ethnography is an excellent approach when done right but, again, just one of several methods for quality qualitative research.

6. Online vs. Offline QLR. Remember the dot-com boom when just about everything online was considered the next big thing and the “offline” world was considered hopelessly old-fashioned? Online qualitative methods have indeed grown but in no way are they fully replacing face-to-face and phone interviewing.

• Research quality: The online bulletin board (extended discussion for several days or longer) is a terrific technique, I think. It provides rich conversation with a geographically dispersed group of people, including even some hard-to-recruit executives. The live focus group/chat, on the other hand, is fairly limited, I believe — yielding rushed, superficial answers without the depth needed in good QLR.

• Business: Mixed reactions on my part here. Bulletin boards account for a larger share of the business I've conducted over the last few years and, even now, a number of clients still view them as new and exciting. The technique is highly labor-intensive, though, so typically I do two and no more than three boards per study. In addition to the moderating time each day, the analysis time can seem overwhelming, far more than for most focus group or IDI studies.

• Personal: Also mixed reactions. It’s great not having to get on a plane to talk to people. I also love that respondents have no idea of my personal characteristics (age, gender, hipness, etc.), unless I make a point of telling them. If I only did online research, however, I would greatly miss the more personal contact of in-person interviews and even phone interviews.

Maybe what we need to remember is that it’s great that our methodology expands to incorporate new technology and new/renewed techniques. My hope is that we – qualitative practitioners and clients – will focus more on what works best in uncovering the real answers and not just on what’s trendy. Let’s view research methods as researchers, trying to figure out what works best.