"What do you mean?" - Strange interview questions for Japanese participants.

Testing for international clients in Japan often needs a lot of localisation work.

The recruiting screener needs to be localised to capture the correct participants; prototypes and test materials need to be checked to ensure they make sense for testers.

However, unexpectedly for international clients, one area which always requires extra work and attention is the discussion guide.

International clients testing in Japan often assume the discussion guide will be the item which requires the least changing and localisation. Clients will expect it to be translated directly and assume it will provide the same intended purpose and results as their original English version. This couldn't be further from the truth and in reality will provide the exact opposite - confused Japanese participants giving unclear answers.

When testing abroad, i.e. in the UK or US, guides tend to have many ice-breakers or "throw away questions", intended to relax the participant or get them in the mood for talking. However, such questions do not always translate well into Japanese. Japanese participants may take questions too literally or seriously, take too much time thinking about the answer, or take too much time trying to understand the reasoning behind the question (even when there is none).

As an example, here is a question which would have caused delay in testing. It was intended as a very quick throw away question, with the client expecting some quick keywords for answers.

The question was: "In general, what is the coolest thing that your phone can do for you?"

One might think it would be easy and quick to answer -e.g. "I can easily take and send photos" or "video chat with my family".

However, for Japanese users, the question is not specific enough. "What do you mean by 'cool'?", "How does a phone do anything for me?" and so forth.

There may also be confusion from the participant mindful about what the interviewer might want to hear by asking such a question. The participant might want to know exactly what the questions mean and in turn ask the interviewer for more details. By this point, ascertaining clarity over the questions will likely have taken up precious interview time and may have made the participant feel more nervous and tense rather than relaxed and talkative.

What would have been a quick warm-up chat usually takes much longer than the initial time allocated by the client.

Therefore, before testing in Japan, we always need to localise and amend questions to avoid confusion for participants and to help move the test along. Yet at the same time, it is important to stay true to the guide, the intended purpose behind the questions, and what answers the clients are trying to gather.

So, the previous question of "In general, what is the coolest thing that your phone can do for you?" may turn into "携帯電話の中で、一番好きな機能は何ですか？" (Keitaidenwa no naka de, ichiban suki na kinou wa nandesuka?). This would translate into "On your phone, what is the feature you like the best?"

To save time and confusion, we always try to explain to international clients the cultural differences, and always try to understand what the client actually wants to ask. We always work together with the client to come up with a guide that is helpful for the client and especially the user; otherwise the interviewer will not able to get any good answers out of them.

To conclude, even for "throw away questions ", always try to write the questions as simply and clearly as possible; so even when localised, it will be close and true to the original version. Otherwise, you will have questions which may even confuse the translators. I'll leave you with this closing example, which we have come across at Mitsue-Links:

Client's Discussion Guide: "If you had a magic wand, what would you change on this website?"
Translator: "What's a magic wand?"