The Effects of Data Costs on Wireless Surveys

Monday, June 14, 2010 by Sean Conry

If you haven’t heard yet, AT&T is dropping unlimited data! By the time I post this, it probably will be unavailable altogether. I'm sure it had nothing whatsoever to do with the timing of the Apple WWDC development conference where the new iPhone is expected to be announced - you know, the one that will probably make it even easier to consume hoards of wireless data. AT&T is letting existing customers keep their plans, but isn’t signing up new customers.

Now there will be two basic plans, one for ‘regular’ users and one for heavy users. If you believe AT&T, then this move will actually save most consumers money. But the problem is that people have no idea how much data they are using. Sure you can get apps to monitor your data use, and in reality, it’s only heavy video conference and mobile TV users who will surpass the new limits and who are causing problems, but it’s an easy decision for them – it ‘s the rest of us I worry about.

Is this a sign of things to come? Sprint still offers unlimited data, as I’m sure do many others. As the iPhone is released on more networks (if those rumours come true), then we will likely see the other carriers experience the same bandwidth problems that AT&T is experiencing.

p.s. for a good read, check out the previous post on this blog post “Are we in for a wireless traffic jam?”.

Furthermore, there’s some talk that 4G will deliver the ability for metered billing.

So what does this mean for research? Well, in one sense it bodes well for the use of Apps over the Mobile Web because Apps can be so much more efficient than the web. Furthermore, survey data is measured in kilobytes, let alone megabytes (until you start incorporating heavy multimedia transfer for display or capture). These plans allow for gigabytes – so we’re probably safe on that front… Apps also don't suffer from Latency like mobile websites do, so the experience is better (a little more on that here).

So will it truly matter to research? Users could become more hesitant to use their precious data allotment on surveys, no matter how compact our data is. Our primary challenge could become one of education in our screening and recruitment activities when we go to launch a customer satisfaction questionnaire, or any other mobile research program. Of course, we must also incent adequately to cover those costs AND the respondent's time.

Personally, I think the mobile Internet will eventually reach the state of the “regular” Internet, where most broadband companies stratify their offerings based on bandwidth (How fast can I get what I’m downloading), not total use (How much will it cost me if I download this)?

We have had good success getting people who are on a wide variety of networks to participate in a cellular survey, so frankly I’m not that worried. But, we continue to keep an eye on the evolution of the wireless marketplace, optimize our products, and work with our partners and clients to educate participants.

Google includes Apps in Search
Meanwhile, the rest of the mobile world continues to trudge along the App path – Google now includes Apps in search results.

It’s all part of the fascinating evolution of mobility and research!


New Dynamic Data Tools for Market Research and Digital Ethnography

Monday, June 7, 2010 by Sean Conry

In a recent blog post I wondered about who was going to tackle the issue of new multi-faceted datasets that incorporate rich media like photos alongside traditional survey answers. Aside from the sheer overwhelming nature of these datasets, the industry is crying out for non-static dashboards and reports as opposed to the tried and true PowerPoint file & stacks of cross-tabs (this was driven home as a major theme at the recent CASRO technology conference).

At that conference I showed how we used Pivot by Microsoft to dig into quali-quant ethnographic data from a recent diary research pilot project. It’s very cool – I recommend checking it out. Microsoft is releasing a way to integrate Pivot “collections” as they’re calling them into Silverlight Websites! It’s a way to share meaningful data in a cool and informative way like I’ve never seen before. (p.s. a big thanks goes out to Candice at Luminosity for pointing me towards this…)

… And if you like that, check out SeaDragon, the underlying technology. This 6 minute overview from the TED site will blow your mind.


Lastly, it looks like there are some new players in this space in MR. I learned about Cognicient, a UK company that specializes in dynamic dashboards for longitudinal data.

Connect these tools to your survey system, and you may just tackle that data yet!
 

How to do Mobile Research... and Why!

Tuesday, June 1, 2010 by Sean Conry

We're presenting this week at the CASRO conference with Ipsos. We worked together to field a mobile, self-initiated digital ethnography with users who have Blackberry and Nokia phones. We think it's the first study of its kind, and I wanted to put down my thoughts on why we did it.

With all of the press that Apple creates, I think most people can intuitively guess about the 'how' of mobile research. All modes have their use. It ranges from quick polls via SMS, to mobile web through to Apps.

How to do Mobile Research

We use SMS for notifications and invitations because that's what method of communication works best in many cases. But we don't collect data that way. Readers of this Blog know that we are strong believers in Apps. And here's the reason why... In general, businesses and academics alike look to Mobile Research to accomplish one of three things.

Why do Mobile Resarch?

Researchers A) want to access populations that aren't responding to other modes, B) know some of their respondent's are going to attempt take their online survey on their mobile if they darn well please, or C) want to accomplish something new.

We're trying to get at C - with the rich capabilities being built into phones, we have a tremendous opportunity to get at the moment of experience, understand a day in the life, and much more through photos, GPS, etc.

Apps are the only mode that allows this today. Yes, you have less reach, but you can't believe the richness of what comes back. 

Just like every other mode, Mobile is just one more option in the Researcher's toolbox. For those who know when, how and why to use it, the results can be pretty amazing... As things unfold, I'll look to share some of those over the coming weeks.

Quant Scale Qual

Monday, April 26, 2010 by Sean Conry

Quant Scale Qual... I thought that sounded more elegant than "Qualnt - the new Quali-Quant". 

With the ability to engage with people via an App on their mobile phone, researchers can do much more than just an 8-question mobile survey or a "traditional" digital ethnography. We can ask people to take pictures, record audio, tag a GPS coordinate, and generally produce mountains of rich, but unwieldy data. 

So we may be trudging towards a new research opportunity and problem: Quant-scale Qual. Maybe a lot of you are already dealing with this. I've seen a lot of quali-quant, but it's usually 200-300 intercepts with some photos and recorded verbatims.

What happens when your diary studies routinely generate 2000 completes? Or maybe they're not "completes", but life blurbs, media micro blogs, or any other new research thing you can invent a name for. Our clients are starting to work through these issues, and companies like Language Logic provide a great starting point with tools that broach predictive/automatic coding and easily handle multimedia data formats.

But multimedia data is a complex beast. It may take Google or Apple to bring us the tools we need to get over the automation hump. For example, check out Google’s face detection technology and iPhoto’s Faces technology.

How this could be licensed, bent, twisted or re-purposed for research is yet to be seen… but the possibilities are exciting and I can't wait to see who solves it first.

Is Co-Creation the Newest Research Fad?

Monday, April 12, 2010 by Sean Conry
At the recent Mobile Research Conference in London, Mick Couper, methodologist extraordinaire and mode effects expert from the University of Michigan, reminded us to think about the "what's in it for them" when it comes to our respondents. The message I took was that if we don't, then we can expect mobile research participation to erode in very short order.

There may just be a groundswell afoot about making research more meaningful for respondents. Well I hope so anyway - for example, take this blog on Co-Creation of surveys.  This post in particular focuses on cooperating with respondents to ensure the translations in your interview survey make sense.

That's a really great idea, but there's also a new future emerging where wireless survey participants will also help create and shape our research instruments over time - to actually influence the crux of the insight we're trying to gather. Some of the next innovations in research thinking will be to engage people over the long term, with functions of real value to them. Check out how the American Legacy Foundation is engaging people who are trying to quit smoking.

Sure they're doing diaries to collect valuable research data which will help society (maybe even their neighborhood) over time, but they are also providing real value to the participants by establishing a new social and support network of peers, and by providing access to their own trends and data. This could even provide compelling inputs to motivate changes to the research instrument over time.

I do not envy the analyst who's job it will be to figure out how the evolving changes in the research, and respondent knowledge of their habits, might affect the data! Some companies already offer respondent-generated response lists that grow over the life of an Internet survey - For example, the first 10 answers are basically a 'specify other' which gets turned into a multi response option.

But WOW! What an exciting thought that people will actually drive the insight, rather than just be put in to pre-defined check boxes. I can't wait.

Choosing the Right Hardware for Survey Research

Monday, March 29, 2010 by Sean Conry

A client of ours is giving a presentation to a group of colleagues on why they chose the hardware they did for a self-completed study. We help clients through this all the time, but it got me thinking... What are rules when it comes to picking the right hardware for MCAPI, diary studies or other mobile data collection.

  1. Consider hardware and software in Tandem.

    In many cases, the software provides the capability you are looking for, and the hardware is secondary.
     
  2.  (well, maybe this one's really 1.b.) Don't be swayed by what's sexy, what's inside counts too

    You'd be surprised how often we hear from someone who has bought a fleet of iPhones, or sourced a no-name device from "a guy who could get them a really good deal". They soon learn that it's not a good deal if it wasn't the right hardware for the job.  
  3. Don’t buy too far into the future.

    Technology is going to change, so it’s likely not even possible to try and pick something you think will still work for you 6 years from now.
     
  4. Ask your technology partners. They will often have
    1. Blogs: http://blog.techneos.com 
    2. Whitepapers : http://www.techneos.com/resources/white_papers
    3. Staff who are up to speed on the latest offerings
       
  5. Ask your colleagues. They will undoubtedly have valuable experiences to share.

Mobile Research Concepts in Practice

Monday, March 15, 2010 by Sean Conry
The basics might seem obvious... The inherent benefits of mobile diaries & digital ethnographies, as well as other mobile research techniques delivered via mobile applications include:


1. Augment traditional survey questionnaires with photos and locations to enrich data and drive new insights

2. Greater mobility means a higher likelihood that respondents will have the ability to take a survey anytime, anywhere
3. Time & location stamps give higher confidence that data was entered where and when it was supposed to/reported to be
4. Real-time data allows for compliance monitoring
5. Eliminate the problem of recall


But a lot of research practices, even those departments which are "ad-hoc" by definition, are built around creating an operational expertise which is replicated over and over again to collect market research data for various clients and scenarios. So sometimes we like to share specific examples to help people understand where a new technique might be useful...

Scenarios which can benefit significantly from a mobile survey system include:

·         Detailed Category Interaction (diary studies)– EG. Snacking/Eating:

o   Where and when did you eat? Why did you choose what you did? What options were available? How did you feel? Show us what you bought/made?

·         Consumption/Usage that happens anywhere, or where location affects choice:

o   FMCG, or disposable consumer goods

o   Tobacco & Alcohol (typically done in academic or social research settings)

o   Diapers

·         Understanding a day/week/month in the life of a respondent segment:

o   GPS capture for significant insight in to patterns

·         Shopping/Retail:

o   Exercises to create outfits, or understand how segments choose where to buy different articles of clothes

o   Pseudo-mystery shopping (audit retail vendor knowledge or Sales Rep attention to customer)

·         Medical:

o   Physician Drug rep encounters

o   Prescription/consumption and results of regulated products

·         Exploratory & Innovation. Perhaps most valuable for dynamic, important or changing segments:

o   New parents, Brand Mavens, Primary Grocery Shoppers, Baby Boomers

·         Advertising and Media Exposure:

o   Where & when did you see advertising? Show us the ad. How did it affect you?

·         Mobile populations, or users of a product or service that by definition is mobile:

o   Frequent fliers / Airline V.I.P.’s

o   Commuters /Heavy Transit Users

o   Sales/Service Reps


Mobile Research - How many participants is "enough"?

Tuesday, February 23, 2010 by Sean Conry
A lot of our clients immediately jump to the concept of "reach" when they first begin to think about mobile research and wireless surveys. They worry that only a small percentage of their panel might be willing to take a survey on their phone, and they think that hard-to-reach groups (like teens) are the perfect audience for mobile research. 

Sure, using consumer engagement techniques that make your interactions more personal and portable should make them inherently more relevant, thereby increasing response and reducing churn.

But that's not what's really exciting about cellular surveys. My favourite conversation with clients is the one when the light goes on the realization sets in that mobile research is about so much more than putting traditional online surveys on a small screen.

What's really exciting is the new reality that as researchers we can take advantage of the billions of dollars that device manufacturers pour into R&D. Built-in functions such as taking photos and capturing GPS coordinates are just the beginning - even so, these basic capabilities provide us with some pretty astounding options for gaining insights from targeted mobile groups and communities!

So how many people do I need in my mobile panel? I by no means decry the important science of sampling, but check out this article that explores how big the ideal online research community should be, and hopefully you too will start to become a believer in quality over quantity.



A New Type of Professional Respondent

Monday, February 15, 2010 by Sean Conry
We talk a lot about trying to get deeper insights by using new mobile research tools that are personal, portable and relevant. But maybe we're missing the boat. There's a lot of money to be made in being a professional respondent, right? 

"It's hard to keep up with all of those online polls on my busy schedule..."

Check this out for a fun view of our industry that brings the concept of "professional respondent" to a whole new level:



New Restrictions on Research Incentives

Thursday, January 21, 2010 by Sean Conry
The rules of research are about to change in Spring 2010. The MRS (UK's professional association for researchers) will begin enforcement of a new ban on use of client goods or services as incentives in a research project.

So many corporate communities, especially branded panels, rely heavily on this very technique for survey conducting.

Who can deny the reason behind it? I remember answering a poll in a teen magazine (was that really decades ago?) where I purposefully skewed my answers to highlight my desire to see more of the content that they were providing in that very issue. I thought it would increase my chances of winning the cash prize! But in fact I couldn't have cared less about Menudo.

With DIY research making a stronger push than ever, how does this affect the position of the professional researcher who is already challenged when trying to coach clients about the "right" way of doing things?

Maybe the promise of mobile research where true engagement through 2-way communication occurs will emerge one technique that can gain answers without bribes. But we'll never escape it will we? Time is money, especially when it comes to hard-to-reach populations.

Maybe it's no big deal. Cash always been king when it comes to the incentive that works for everyone. But where does that leave financial research? :)

The Frustration of Mobility and Reach

Wednesday, October 28, 2009 by Sean Conry
In my last post, I wondered about device market share and it's affect on our ability to reach the right people for survey conducting.

I focused on the manufacturer / operating system, but another element also has a significant affect on reach. I guess you could call it Mode. We normally talk about mode in terms of whether we're collecting data over the web, over the phone, face-to-face, etc. Mobile is it's own mode, but there are choices within that mode.  

Specifically, the primary mobile mode possibilities for a mobile survey system are text message, mobile web, and applications.

Check out this interview/podcast from a recent interview at a CASRO event. Heidi Dickert has some good points about how a different type of technology is required when looking at mobile surveys. The message? You can't just replicate the same old approach when it comes to mobile research.





Text messages give you ubiquitous reach, but the commonly held experience among those I speak with is that SMS surveys are good only for very, very quick polls. The burden on the respondent is high, and you can only squeeze in 3-4 questions before participation drops off - and if you haven't seen it yourself, believe me, it doesn't just stumble off the curb, it careens off the grand canyon.

Mobile web feels like it should be familiar since CAWI has taken over as the primary mode of choice in so many countries. However, the problem is that the mobile web isn't great yet for the vast majority of users. You still need a lot of patience. Think about a news page that requires 10 seconds to load, but then you spend 5 minutes reading. That's much more palatable when compared to refreshing screen after screen after screen of a mobile web survey. To reinforce this, check out Mobile Marketer's recent article about slow load time is the #1 performance issue on the mobile web.

Off course we believe in Applications (particularly Java where possible, native where needed). With iPhone App store just reaching it's 2 Billionth download in September, and Facebook's install base leaping from 2 million to 10.5 million in two months with the release of targeted Java versions, the path seems pretty clear to us...

Cheap Netbook for Mobile Data Collection

Tuesday, October 27, 2009 by Sean Conry
Hardware markets have never been so attractive. 


Check out the hot deal on the ASUS Eee PC with Windows at $285.

MCAPI used to only apply to PDA Survey Software, but with the introduction and proliferation of Netbooks, and powerful mobile survey packages like the ones offered by my company,  now you can easily deploy complex surveys to small and large screens alike, depending on your project need.

The latest on mobile market share - coverage & reach

Monday, October 19, 2009 by Sean Conry
The two main considerations in picking a mobile research methodology are capability (what kind of work am I enabled to do?) and coverage (who can I reach with that capability?). Those of us in research care so much about market share of mobile devices because it profoundly affects both elements.

We need to reach people. But it's not just any people we need - we need to reach the right people. A colleague of mine recently held a focus group and noted - "hey - there are an awful lot of artists and musicians in this group..." Did his recruiter get the mix of participants right? Maybe, but probably not (you'd think so too, if I told you the category ).

To a marketer, the more people you can reach, the more mindshare you can gain with your ads to promote revenue growth. Marketers talk about how to reach a target market with their message because they need to know they are spending their money wisely.  However, the reason researchers care about the number of people who see our "message" (say, a survey invitation), is because of our sampling frame. We need the right people to provide us with feedback in order to help solve the business problem at hand.

In short, researchers have to worry about it even more because the science of our analytics depends on it. Without reach, my colleague wouldn't even have had artists and musicians to talk to. 

So, it's very interesting to see what Gartner is predicting for the future of Mobile OS market share. They say Android will overtake Research In Motion’s BlackBerry OS, the iPhone OS and Windows Mobile to capture 14.5% of the smartphone market within three years.

"Symbian’s market share will fall from around half of the global market to just 39%, according to Gartner. Apple will maintain its third place and Windows Mobile will stay in fourth. However, RIM’s BlackBerry OS will fall from second place to fifth,"

If you're thinking about partnering with a survey system supplier to provide cellphone survey capabilities, then make sure they have an expertise and plan in place to meet all of the different flavours of these platforms as they explode in the next two years, otherwise you may never overcome the problem of reach.

Cheap Interview Devices Still Exist

Thursday, October 8, 2009 by Sean Conry

If you're a real cheapskate like me, and you've been biding your time before making a new hardware investment, then look no longer. The next release of inexpensive smartphone technology for MCAPI interviews and diary studies has arrived.


Treo 500 - inexpensive interview deviceCheck out the unlocked Palm Treo 500 for $119.  Incredible.


Just make sure you're willing to live with a QWERTZ keyboard - (in Germany, they switch the Z with the Y.. go figure).

Some shameless promotion here, but with Techneos' new SaaS pay-per-complete options, there's never been a more affordable time to equip your field force or diary respondents with a world-class mobile interview system.

Market Research makes it in Hollywood

Wednesday, September 30, 2009 by Sean Conry

For many outside of our industry, Market Research has always been a fuzzy concept, and let's face it, a weird choice of career.

But no longer! A major Hollywood movie (Last Chance Harvey) features a market researcher as a main character.

Last chance Harvey Interviewer Screen capture

But what is she doing with pen and paper? What an antiquated approach! We would expect her to be using a computer assisted personal interviewing system or some kind of PDA survey software, of course. Where are these movie script writers doing their research?

Legal Note: This post in no way confirms that my wife convinced me to watch a movie which could be described as a Romance Drama.

Social Media and Research - More Thoughts...

Tuesday, September 22, 2009 by Sean Conry

Can anyone deny that the conventional understanding of what constitutes "real" research is being challenged? This Research Live article nails it:

"The rise of the Internet and social media has caused the received wisdom to be questioned on issues including sampling, anonymity, the relationship between researchers and participants, and whether there aren’t better ways to understand what people think, feel and do than just asking them."

With online research being at the forefront of so many researchers' mentality, do we move to an approach where we simply monitor and observe? Of course there's the problem of access as it relates to your sampling frame. But at least thanks to Anderson Analytics', we know 'Who's using Social Media?':

Who uses social media?


Personally, I think a moderate, measured approach is what's needed. If you read one article on this, check out the Zinc research blog entry on Social Media and so-called "legitimate research":

"...the traditional model for marketing research needs to be overhauled, and social media will be one of the impetuses for changing the role of research in the marketing toolkit. That said we should not throw out the existing research model too quickly"

And finally, if you want to read a sound, measured debate on both sides, then I'd suggest checking out this article: Is Social Media Measurement Meaningless?

The Affect of Social Networking on Research

Thursday, August 27, 2009 by Sean Conry
As it gets harder for businesses (and researchers!) to reach people, the buzz about the affect of social networking applications is starting to get louder in our ears...

Apparently some people are wondering if the rise of direct forms of customer interaction like voice of the customer programs will make the use of Mystery Shoppers for mobile data collection a thing of the past.

Judi Hess, president of MSPA calls mystery shopping "a means to measure customer service, product knowledge, and sales ability." More important, she says, it offers subjective, targeted feedback that companies just can't get from less structured kinds of feedback, like social networking tools or surveys.

Ok, I can buy that. Read more here (you'll need to sign up for a free account).


Tom Anderson in particular is pushing us to think about the role of Social Networks and what he's calling "next gen marketing research". He recently interviewed representatives from Facebook and LinkedIn. It's just a start, but you can see that the social networks are starting to think about the convergence of these worlds, too.

LinkedIn & FB InterviewClick here for the Podcast


All three of them will be a highlight of an upcoming ESOMAR panel session. This will be one you don't want to miss...



These are great articles to get us thinking about engagement, the future of survey conducting and the way we deliver insight from the market... But one of my favourite approaches comes from a seasoned researcher right here in Canada.

More on that next week...

Do-Not-Contact Trumps Opt In

Friday, July 24, 2009 by Sean Conry
I recently read that in this economy "opt out is the new opt in". 

Yeah right.

This was literature on a site (which I won't mention) that sells marketing lists, so I had to chalk it up to wishful thinking rather than benevolent distribution of informed market information.

For those of us that take this stuff seriously and know that blatant Spamming erodes respondent cooperation, we need to know that even when we think we're following the rules, local laws might derail our efforts. As as we move too add SMS survey invitations to our toolkit, the landscape is getting tricky to navigate.

Now, marketers who advertise products and services such as alcohol, tobacco and gambling cannot market to any phone number or address which is registered on a do-not-contact registry within Utah or Michigan. It's all about protecting families, and you can check it out here.....

"But I don't advertise tobacco! I'm in research!"

Maybe... But my point is that we need to be careful. Extend this to our world of mobile data collection for a moment, and is it really so hard to believe that similar legislation could be introduced to protect certain groups.... like children for example, a segment for whom we already need explicit parental consent before we can engage.

"One odd aspect of the laws enforcing these programs is that even if a consumer double-opts-in for an SMS campaign and verifies that he or she is 21 or older, marketers would still be in violation if that person previously submitted their mobile phone number to either state’s do-not-contact list."

Could you be unwittingly inviting minors to participate in your mobile marketing survey? Or are you treating SMS invitations with less ethical scrutiny than email? Might be time to re-scrub that list...


Summing up the cell-only problem for market research

Thursday, July 16, 2009 by Sean Conry

It seems as though reseach online has recently reinvented themselves. They were always a great source for information, but they seem to be exploding with news and original content lately.

A recent article titled "Survey Geek vs. The Cord-Cutters" offers an audio interview with self-confessed 'survey geek' and blogger, Reg Baker.  There are also great links to relevant articles on the cell-only population and the affects on research. You can listen to it here:



This interview is great in my opinion, particularly in that he begins by describing the "cell-only" problem in terms of the sampling frame. He also eloquently yet concisely descsribes the issue of bias due to undercoverage, and how the target group of the research and the topic may affect results. Not to mention cost and data implications of doing a survey on mobile phone. Reg also identifies the "wireless mostly" problem. I won't replicate every point here in text, but here's a researcher who knows his stuff.

Brilliantly, he surmises (and hopes) that researchers will adapt to the problem by matching the method to the problem - execute "fit-for-purpose methodology decision", to paraphrase slightly.

But one thing stuck out to me... there's no mention of three significant and growing ways to conduct a mobile device survey, cellphone survey, mobile marketing survey (mobile research by whatever name you choose): WAP, SMS and Survey Applications.

Check out the latest numbers from CASRO. You might find it's worth getting ahead of the curve by adding other mobile techniques to your methodological tool box.




Wireless is changing the way we live - for better or worse?

Wednesday, July 1, 2009 by Sean Conry
Mobile is mandatory. 

You can't get away from it and the evidence is everywhere. Even the University of Missouri has made smartphones mandatory for journalism freshmen!

And everyone now agrees that "Mobile" is more than 'my cell phone'. Mobile is changing the way we live. Business and society is seeing the benefit, but we are also seeing the pitfalls.

Rapid growth of the use of the wireless Internet is causing wireless broadband services to slow down and interrupt from data overload, and according to one report, the result is that mobile broadband sales are slowing. Don't get me wrong - there is still impressive growth overall, it's just that the growth is slowing as consumer frustration sets in.

There's more than an inconvenience factor... now we have to worry about 'cellphone elbow'. Doctors are indeed seeing more instances of this.

As people get more engaged with their devices, I'm betting we'll see more opportunities for doing deep and personal research with respondents, as can be done when people are recruited for a targeted, self-completed digital ethnography. 

Regardless of the specific research techniques to be employed, we will undoubtedly begin to see advances in the sophistication of mobile market research methods arising from this cultural shift.

In the meantime, watch this space for more updates on what fabulous new ailments arise as a result of increased mobility of the population!