Diary Studies

Qualitative Research on a Quantitative Scale?

Friday, March 11, 2011 by Joeline Cross

@jtimed asked us on Twitter the following question:

jtimed twitter post scaleability multimedia market research

It’s a thought that has been echoed by market research professionals we met at NetGain 5.0 and the CASRO Online Conference in Vegas. It’s definitely a challenge, and we are only scratching the surface, but we’ve started to find some success by looking outside of MR for solutions (necessity being the mother of invention, and all...).

Firstly, photo and video data is qualitative in nature so you can’t get around coding it in some fashion to get the most out of what you’ve collected.

Even in a very open unstructured research design, we often include a few quant type questions which helps respondents to code the content of the photo or video for us according to pre-defined options. Or you can quantify a qualitative preference like below.

Of course, this approach doesn’t capture everything, so you then might go to a coding specialist software and service like ‘Ascribe’ by Language Logic. They innovated in text coding, but have applied efficient time saving code book techniques to multimedia coding, too. Still, this is a time-intensive process...

So back to the quantifying of your qualitative data... Microsoft provides MS Pivot, which we’ve used to pull in metadata (i.e. survey answers) associated with photographic data – then you can start to do some dynamic sorting and get a feel for the quant story behind the mountains of qual data you’ve collected. We first explored this with Ipsos last May on our Great British Weekend (MCASI) project.

We asked respondents what they were doing, who they were with, and to take a photo and GPS capture of where they were over the course of a day. Pivot allowed us to filter the images by response data such as:

• Location

• Family

• Friends

• Food

• Media

And other activities that people were engaging in.

It’s a dynamic sort and looks like this (sorry for the lack of sound - we didn't have time to make it look very polished - we just wanted to respond to you!):

When you look around to other tools like Google Maps, you can start to create other insightful views like what follows:

From here you can see stories evolve from the visual trends such as people watching media, feeling relaxed, eating with family, being excited about being out with friends etc..

It’s getting easier and easier to collect photos, video and GPS (heck – even barcode capture will be mainstream by 2012) as consumers become more savvy with smartphone technology and handset sales continue to increase. This will only continue to rise, as will the adoption of tablet devices built on Google’s Android and Apple’s iPad platform.

We don’t have all the answers – but we want to find them with you.

Industry trend watchers continue to talk about what to do with the large volume of qualitative data that is being captured (not to mention the space it requires to store them all) and it’s an issue that industry leaders and innovative market researchers are continuing to advance. The key point is that the information provided in photos, video, GPS and barcodes (and other forthcoming advances) in real time is valuable in offering richer insights into the consumer’s experience which is valuable to clients and researchers alike.

We can offer market researchers the opportunity to provide a holistic view of the consumer experience validated in real time as they take their smartphone from home to work, to the supermarket, cinema, restaurant and back home again. If your clients knew this was available for diary studies, mobile mystery shopping, CAMI and mobile panel, wouldn’t they ask for it?

With apps-based research that is appropriately designed to answer the research question, for the audience, for the device and for how the respondents use it, it is possible to conduct qualitative research on a quantitative scale. We want to work with you to make this happen.

@jtimed and others in our community – we’d love to hear your thoughts on this industry challenge.

Quant Scale Qual

Monday, April 26, 2010 by Sean Conry

Quant Scale Qual... I thought that sounded more elegant than "Qualnt - the new Quali-Quant".

With the ability to engage with people via an App on their mobile phone, researchers can do much more than just an 8-question mobile survey or a "traditional" digital ethnography. We can ask people to take pictures, record audio, tag a GPS coordinate, and generally produce mountains of rich, but unwieldy data.

So we may be trudging towards a new research opportunity and problem: Quant-scale Qual. Maybe a lot of you are already dealing with this. I've seen a lot of quali-quant, but it's usually 200-300 intercepts with some photos and recorded verbatims.

What happens when your diary studies routinely generate 2000 completes? Or maybe they're not "completes", but life blurbs, media micro blogs, or any other new research thing you can invent a name for. Our clients are starting to work through these issues, and companies like Language Logic provide a great starting point with tools that broach predictive/automatic coding and easily handle multimedia data formats.

But multimedia data is a complex beast. It may take Google or Apple to bring us the tools we need to get over the automation hump. For example, check out Google’s face detection technology and iPhoto’s Faces technology.

How this could be licensed, bent, twisted or re-purposed for research is yet to be seen… but the possibilities are exciting and I can't wait to see who solves it first.

Choosing the Right Hardware for Survey Research

Monday, March 29, 2010 by Sean Conry

A client of ours is giving a presentation to a group of colleagues on why they chose the hardware they did for a self-completed study. We help clients through this all the time, but it got me thinking... What are rules when it comes to picking the right hardware for MCAPI, diary studies or other mobile data collection.

Consider hardware and software in Tandem.

In many cases, the software provides the capability you are looking for, and the hardware is secondary.
(well, maybe this one's really 1.b.) Don't be swayed by what's sexy, what's inside counts too

You'd be surprised how often we hear from someone who has bought a fleet of iPhones, or sourced a no-name device from "a guy who could get them a really good deal". They soon learn that it's not a good deal if it wasn't the right hardware for the job.
Don’t buy too far into the future.

Technology is going to change, so it’s likely not even possible to try and pick something you think will still work for you 6 years from now.
Ask your technology partners. They will often have
1. Blogs: http://blog.techneos.com
2. Whitepapers : http://www.techneos.com/resources/white_papers
3. Staff who are up to speed on the latest offerings
Ask your colleagues. They will undoubtedly have valuable experiences to share.

Mobile Research Concepts in Practice

Monday, March 15, 2010 by Sean Conry

The basics might seem obvious... The inherent benefits of mobile diaries & digital ethnographies, as well as other mobile research techniques delivered via mobile applications include:

1. Augment traditional survey questionnaires with photos and locations to enrich data and drive new insights
2. Greater mobility means a higher likelihood that respondents will have the ability to take a survey anytime, anywhere
3. Time & location stamps give higher confidence that data was entered where and when it was supposed to/reported to be
4. Real-time data allows for compliance monitoring
5. Eliminate the problem of recall

But a lot of research practices, even those departments which are "ad-hoc" by definition, are built around creating an operational expertise which is replicated over and over again to collect market research data for various clients and scenarios. So sometimes we like to share specific examples to help people understand where a new technique might be useful...

Scenarios which can benefit significantly from a mobile survey system include:

· Detailed Category Interaction (diary studies)– EG. Snacking/Eating:

o Where and when did you eat? Why did you choose what you did? What options were available? How did you feel? Show us what you bought/made?

· Consumption/Usage that happens anywhere, or where location affects choice:

o FMCG, or disposable consumer goods

o Tobacco & Alcohol (typically done in academic or social research settings)

o Diapers

· Understanding a day/week/month in the life of a respondent segment:

o GPS capture for significant insight in to patterns

· Shopping/Retail:

o Exercises to create outfits, or understand how segments choose where to buy different articles of clothes

o Pseudo-mystery shopping (audit retail vendor knowledge or Sales Rep attention to customer)

· Medical:

o Physician Drug rep encounters

o Prescription/consumption and results of regulated products

· Exploratory & Innovation. Perhaps most valuable for dynamic, important or changing segments:

o New parents, Brand Mavens, Primary Grocery Shoppers, Baby Boomers

· Advertising and Media Exposure:

o Where & when did you see advertising? Show us the ad. How did it affect you?

· Mobile populations, or users of a product or service that by definition is mobile:

o Frequent fliers / Airline V.I.P.’s

o Commuters /Heavy Transit Users

o Sales/Service Reps

Making sense of the mobile platform jungle

Monday, March 1, 2010 by Mark Cameron

I've been getting very close to most of the mobile platforms on the market today, and wanting to summarize my thoughts about their potential impact on mobile research. Seeing Samantha's February 25th post about smartphone market share, a brain-dump on the subject should dovetail nicely...

Looking back on key announcements over the past year, Google, Apple and more recently Microsoft have made the most significant splashes in terms of mobile innovation. However, when the dust settles we still see Nokia and Research in Motion (RIM) leading the charge with 47% and 20%, respectively, of the mobile OS market in 2009. Why is this the case, and what can we expect to see moving forward?

I believe the current mobile market is a race between 5 horses, but I will not count others out over the long term due to the still-early nature of this arena. The key players today are:

Apple: since its release, the iPhone has set the bar for smartphone usability; its slightly thinner sibling, the iPod touch, is the definition of a sleek, modern Personal Digital Assistant (PDA); and the forthcoming iPad is one of the most anticipated products of all time. Simply put, Apple has a lot of swagger and momentum in the mobile space. But Apple runs a very closed environment that people either love or hate, and this creates opportunities for other more open platforms to shine. Also, while Apple has done very well in the consumer space, it has made fewer inroads into the enterprise, where RIM and Microsoft thrive.

Research in Motion (RIM): the BlackBerry is one of the most impressive brands of our time. While RIM is often considered less marketing savvy than Apple, I would suggest that their marketing tactics have been every bit as effective as Apple's -- just different. Despite a barrage of criticism about the sexiness and usability of their products, RIM continues to grow and profit at an astounding rate. I personally questioned the usability of BlackBerry products compared to more elegant competitors like iPhone and Android, but I have come to appreciate them as solid, enterprise-worthy devices, and to see the company as a very savvy player in the mobile space. RIM has developed very deep roots with both wireless carriers and enterprise IT departments, as well as a powerful brand that is almost synonymous with thumb-typing on a mobile phone.

Nokia: the Nokia/Symbian world is complicated. Having personally handled dozens of Nokia phones, I would summarize that their strength is in their diversity of offerings to multiple levels of the marketplace. This is also their weakness. With countless products, three current operating systems, and a solid-but-aging feel on much of their hardware, it is hard to believe that Nokia still outsells its nearest smartphone competitor at more than a 2-to-1 ratio. But while they are not as strong in North America, Nokia is a major player in Europe and downright dominant in many other regions of the world. It is hard to discount a company that produces over 1 million phones per day (yes, you read that correctly). With the incredible depth of carrier relationships and distribution channels which they have developed, Nokia's challenge now is to fill those channels with products that compete with their ever-growing range of competitors.

Google: Google has garnered a lot of attention since announcing the open-source (and freely available) Android platform in 2007. By providing a smartphone operating system that is free and extensible, Android has garnered support from dozens of handset manufacturers including major players like Motorola and HTC. In contrast to Apple, Google's greatest strength (and weakness) is its openness. I am personally very impressed with most of the Android devices I have used, and as a consumer I have great optimism that Android will be a force to be reckoned with in mobile technology. But I also recognize that openness can lead to fragmentation, and I've heard a lot of grumbling from developers about the lack of standards on Android devices. As Microsoft learned when it allowed device manufacturers and wireless carriers to customize experiences based on its Windows Mobile platforms, I believe that the many emerging flavours of Android devices will make it difficult for developers to target. That said, I believe that Google's ability to integrate the mobile experience with all of their other web-based services will make it a formidable player in the mobile space, and I believe the fragmentation issue can be overcome as Google and other Android licensees learn to coexist.

Windows Mobile: I recently spoke about the forthcoming Windows 7 Phone Series, so I won't repeat myself on the details. Suffice to say that I think Microsoft has re-entered the mobile race, and demonstrated that they are not planning to turn away from this increasingly important battlefield. Microsoft has learned a lot of lessons over more than a decade in mobile computing, and I believe their enterprise roots will serve them well as they re-assert themselves with a brand new mobile platform. What remains to be seen is whether Microsoft can garner enough consumer interest to unseat competitors in the mass market, or whether it will continue to play a more niche role as an enterprise solution.

I have not even mentioned the likes of Palm (webOS), Samsung (bada), Linux Mobile (LiMo), or expanded on the now open-source Symbian OS (the Nokia-bred OS that was recently spun off as a freely available platform). And there are others... but the rabbit hole is simply too deep to cover here, so I will get back to the purpose of my post: to discuss the impact of mobile platform trends on market research.

There are fundamentally two ways to engage people on their mobile devices in a data-intensive way: (1) via their web browser; and (2) using a downloadable application.

Mobile web browsers are improving in capability and usability, and are finally emerging as a lowest-common-denominator approach to mobile engagement. If you need to reach a lot of people in a relatively shallow way, e.g. to conduct a brief mobile survey about a product or experience, then a mobile browser may well be the way to go. Although mobile browsers are still fragmented, the emergence of mobile web technologies such as HTML 5, JavaScript, CSS and Adobe Flash are making it more viable to reach a wide mobile audience.

Downloadable applications provide a richer user experience for applications such as diary studies and mobile panels, but they are typically harder to deploy to a broad range of users. Also dovetailing with Sean's recent post about quality over quantity, I would argue that in many cases a more select audience that is highly engaged is more valuable than a broad audience that is minimally engaged. It is these cases -- for example digital ethnography with alarms to trigger highly contextual questions at random times -- where mobile apps really shine.

Bear with me as I attempt to tie all of this information together...

The mobile jungle is in some ways becoming more wild every day, but I am also seeing signs that a handful of gorillas may be starting to establish some turf. While I don't believe that all five gorillas outlined here will win over the long term, each one represents sufficient market share and resources to be considered as key players for the foreseeable future. With that in mind, I think we will begin to see some stabilization of mobile platforms, resulting in more reliable ways to reach the masses via both web browsers and downloadable applications.

For the next while solutions focused on mobile research will have to choose between "wide and shallow" or "narrow and deep" -- i.e. either focus on reaching a broad range of people with a more basic level of engagement, or on providing a high level of engagement within a more narrow scope of users. I don't feel that one is inherently better than the other, and both represent significant opportunities within the burgeoning mobile research space.

Over the long term the two paths that I have identified will converge. In the meantime, having invested a lot of time and energy developing methods to engage people in a deep and meaningful way, I am a big believer in the power of rich mobile applications. With over 3 Billion application downloads in less than 18 months, Apple has more than proven the viability of downloadable apps, and all other major platforms have since poured significant resources into their own mobile app stores.

Over the next while it is prudent to focus on the five key players I have identified here: Apple, RIM, Nokia, Google and Microsoft. But peripheral vision is often what sets the great apart from the good, so I allow my eyes to wander a bit in search of innovative smaller players that might just have a thing or two to teach the gorillas.

Cheap Interview Devices Still Exist

Thursday, October 8, 2009 by Sean Conry

If you're a real cheapskate like me, and you've been biding your time before making a new hardware investment, then look no longer. The next release of inexpensive smartphone technology for MCAPI interviews and diary studies has arrived.

Check out the unlocked Palm Treo 500 for $119. Incredible.

Just make sure you're willing to live with a QWERTZ keyboard - (in Germany, they switch the Z with the Y.. go figure).

Some shameless promotion here, but with Techneos' new SaaS pay-per-complete options, there's never been a more affordable time to equip your field force or diary respondents with a world-class mobile interview system.

Google and WPP offer $4.6m in research grants

Monday, September 28, 2009 by Susan Bilczo

If you are an academic researcher, you may want to take advantage of Google's and WPP's offering of $4.6 million in funding for media research grants on a variety of topics, including online and offline media interaction, relevance and effectiveness measurement, audience types and engagement, and verticals and new media.

Our company specializes in diary research tools that have been used extensively to study media exposure, linking it to environment, mood and other consumer behavior. So, using our mobile data collection tools may be a perfect solution for research you conduct with this grant money.

Is the future of research directly related to the past?

Sunday, August 2, 2009 by Mark Cameron

I've found myself pondering the effect of mobile technology on the future of research, and figured it warranted a blog post. I've been involved with both handheld surveys and wireless technology for most of my career, and throughout that time I've been amazed at both how quickly and how slowly things change.

Mobile technology itself moves incredibly quickly, but adoption of technology can range from lightning fast to incredibly slow. Market research professionals are analytical and pragmatic by nature, so it is understandable that researchers have been slower to embrace new technology than many other industries. Ironically, market research is often a key driver for decisions which drive technological advances, yet it can take years for those advances to be reflected back into the research process itself.

Methodology is central to any decision relating to survey research methods. When new techniques are introduced to "improve" tried-and-true processes, it is important for researchers to understand all of the implications of the potential change before implementing it en masse. So research about research, or more specifically about new techniques for conducting research, is very important. But this is easier said than done, as most researchers are so busy generating revenue through existing methods that they lack time to explore new ones. The academic community breaks a lot of ground in this regard, but it takes time for academic research to reach--and to be embraced by--the commercial market research sector.

Recently I have seen mobile research being embraced as a reasonably mainstream approach for survey data collection. It is still "leading edge", but it is no longer "bleeding edge" -- at least for face-to-face interviews and diary studies. While the application of mobile survey software is still a niche play today, we are about to see it extend beyond its traditional application to touch every other aspect of survey research -- including web surveys, phone surveys, mystery shopping and other methods.

What intrigues me more than anything is where this bottom-up thinking will really lead the research industry. While we are busy planning for the evolution of survey research to involve mobile technology, I believe it is equally important to see things from the top-down: i.e. to realize that researchers are losing control of people's attention, and consumers are increasingly recognizing the value of their opinions. Will the methodologies of today be effective in the future, or do we need to reshape our thinking to embrace emerging realities?

I believe the future of mobile survey research looks a lot different than the past. We will not simply see old methods enhanced by new technology; entirely new methods will emerge around the cultural phenomena that shape our societies around the globe. There is no limit to the opportunities that will be enabled by social networking and location-aware technology, which will be bundled into mobile technology that will make today's most impressive devices seem as archaic as early PCs appear today.

Those of us who wrap our arms around the cultural changes that emerge as a result of new technology, rather than just trying to shape new technology to meet old and tired methods, will realize amazing new opportunities for mining insight from consumer opinions. There will be many false starts and a lot of experimentation, but in a few years we will look back at the way we did things in 2009 and be amazed by how dramatically human communication -- and in turn market research -- have changed within a very short time.

I'll share some of my predictions on this blog over the coming months. Today I just wanted to get the thread started with some background thoughts...

More options for mobile surveys: PDAs, Smartphones, Netbooks and more...

Thursday, June 25, 2009 by Mark Cameron

Techneos clients have been using Entryware survey software on Windows Mobile devices for many years. The release of Entryware 6.4 on July 8th will take our Windows Mobile support to a whole new level. We have streamlined the licensing and installation process for Windows Mobile devices, improved the look-and-feel of the entire Entryware Mobile application, and added key functionality to enable diary studies on a wide range of Windows Mobile PDAs and Smartphones (including photo diaries on supported devices).

Entryware software now has four distinct mobile "engines", all of which run questionnaires from a common authoring tool, Entryware Designer, and produce data in a common structure that can be exported to SPSS, ASCII, Excel, and other formats. Today, Entryware Mobile can be deployed seamlessly to the following mobile platforms:

Smartphone and PDA surveys:

Palm OS / Garnet OS
Windows Mobile
webOS (Palm Pre)

Tablet PC, Ultra-Mobile PC and Netbook surveys:

Windows 98/XP/Vista

It is also possible to run Entryware software using emulators or "virtual machines" (available from StyleTap or Access) on Symbian devices, including many Nokia Smartphones and Internet Tablets. We have done limited testing on those platforms to date, so we do not consider them "officially supported" yet. Let us know if there is a particular Symbian device that you want to use for mobile surveys, and we'll work with you to see what is possible in this next frontier...

If you haven't checked out Entryware software recently, drop us a line to get a fresh look at the platform. You might be amazed to see the level of survey research that can be conducted using a mobile device!

Entryware on Palm Pre: mobile surveys never looked so sexy!

Thursday, April 2, 2009 by Mark Cameron

In January, Palm took the world by storm by announcing both its new webOS and the Pre smartphone, which have been touted by many as setting a new bar for mobile phone functionality. Planned for release sometime this quarter, the Pre has leapfrogged Palm from a once-prized but aging provider of PDAs and smartphones, to an innovative leader with the coolest mobile device to hit the market since the iPhone. With a new team of superstars at the helm (many of whom came from Apple), Palm's recent announcements have impressed techies and investors alike.

But the question we have all been asking for the past couple of months is... what is the future of Palm OS applications like Entryware?

Today we heard the news we've been waiting for... that Palm OS applications will be supported on Pre devices. We had reason to believe that some form of Palm OS support would be available on webOS devices, but we've been in a bit of a holding pattern awaiting confirmation.

Palm devices have always been ideal for survey research, because Palm OS offers a simple, elegant interface that is consistent and easy to use. Although Entryware software also runs very well on Windows devices, as long-time Palm fans we are thrilled to see that a whole new generation of Palm devices will continue to be turned into best-of-breed wireless survey systems.

Techneos is committed to ensuring customers a smooth transition from existing Palm devices to the next generation of webOS phones. In the meantime, existing Palm devices like the Centro, Treo and TX -- all of which are excellent tools for mobile research applications like face-to-face interviews, diary studies and mystery shopping -- should be available at bargain prices while Palm makes room for new inventory.

Good going Palm! We knew you had it in you...