Ask better questions

By Mike Kuniavsky

I’ve done a lot of usability testing over many years; so much that I wrote a book about it. And usability testing is an absolutely essential part of evaluation. Assuming you know what people want and have iteratively developed a product with these things in mind, usability testing makes sure your assumptions are correct. It’s the final check.

But what if you want to do the exact opposite and evaluate a product before you make it? That’s what we do with our probe method.

Innovations succeed when they’re designed around users. In this 3-part miniseries, we’ll look at Xerox’s approach to user-centric design by examining the three elements of the process – Discovery, Ideation and Evaluation – and introduce you to some of the researchers in our innovation team.

In the final post of the three, Mike Kuniavsky – an ethnographer at PARC, a Xerox company — talks about how probe methodologies can help innovators decide if their great ideas can become great products – without having to build them first.

Where usability testing is about evaluating known technology on a known user group with a known outcome in mind, we test unknown technology on unknown users performing unknown tasks with little or no idea what the end product or service looks like.

Why?

Well, it’s to counter an old dilemma: How can you know if anyone wants what you’ve made until you’ve made it?

It’s incredibly expensive and time-consuming to go through the process of developing a new hardware product or service. But it’s unlikely someone in an interview or a focus group is going to tell you they’d like a Wi-Fi-connected speaker that listens to them all the time and tries to predict what kind of music they’d like.

So that makes innovation – big, exciting innovation – hard. You’re either building an idea and hoping for the best (innovation from ignorance) or you’re playing it very safe with tiny, incremental changes to products and services you already know will sell (innovation from fear).

Neither of these are good.

And, if you do make a big exciting innovation, but you can’t replicate it because you don’t know what you did right or wrong — well, that’s the worst. (Big consumer electronics companies do this all the time.)

So, we don’t start with an actual prototype of what we think the design will be and test that, we start with no idea of what tech and what design people want. Our probes may be completely unrelated to the final experience, but what they do is find out whether people value that experience.

Xerox researchers create the future today. Learn more about innovation at Xerox.

Quick and dirty risk reduction

Right now, the most common way to reduce the chances of failure in hardware design is traditional, customer-centered, new product development derived from industrial design – a million miles from the world of software development.

So, how can you get away from the old linear waterfall of Pre-Alpha –> Alpha –> Beta > etcetera, and quickly find out whether an idea is even interesting to people?

That’s where our quick and dirty probe methodology comes in. Let me give you an example of what that looks like.

We recently did a project in China for a client who wanted to make a mobile recommender system. A user would take photos using the app and then the tech would make suggestions on things they might like to buy based on this.

To actually build the service would have taken ages and cost a lot – and the client didn’t even know if anyone would like the idea.

So we faked it.

We found 10 people who matched the target audience and asked them to take a photo any time they wanted more information about a product. Instead of the app, we got them to upload their pictures to WeChat, a popular Chinese mobile social network. And instead of a clever algorithm, we had real people responding with typed out suggestions.

After the trial week, we interviewed our test users and found out what they thought – not about the tech; the tech didn’t exist – but about how they’d use it, where they’d use it and why.

Better questions get better answers

The “tech” we use for probes has to be cheap and disposable. It may not even do anything. The artifacts that stand in for the technology solution are designed to be thrown away at the end of the probe experiments. What we want at this stage are insights into the value the fully-developed solution might have.

So, on another project, we wanted to test the value of “sensing data” in people’s houses. We gave people a box with a “sensor” on it but didn’t tell them where to put it. We asked them to place it somewhere in their house – in the hall, in the kitchen, behind the sofa – and then sent them “data” about what we were “sensing” in that location. From where they placed it and how they changed their behavior based on the fake data we sent them, we learned the value placed on certain types of data.

Ultimately, we’re looking for signals around value – what is a valuable experience that technology could enable? What parts of a tech innovation are valuable?

For sure, our probes don’t get you all the way there – they’re much more about telling you where not to go than coming up with a final answer.

But if you want to find out if your innovative new idea has value before you start making it, quick and dirty probes can get you critical information about whether people will actually want it before you’ve invested very much time or resources into development.