Generating Hypotheses Through Observational Studies

A Situation Observed in a Home Electronics Store

The other day, when I visited a home electronics store in Shinjuku, Tokyo, I saw something of great interest. A male shopper in his early twenties was holding a small tablet which he proceeded to show the sales assistant, the male was then guided to a section of the store selling microwave ovens. Standing in front of the microwave ovens, the man observed and tested the target product. He then withdrew his tablet again and spent around ten minutes using the device. Then he looked at several alternative microwave ovens and after he continued using his tablet. He then repeated the process for a second time. In the end, he withdrew a smartphone from his pocket, took several photographs of the ovens on offer and exited the shop without buying anything.

What could the male customer be investigating on his tablet while standing in front of the microwave ovens? What site or application was he using? I was not sure because, of course, I could not clearly see his screen. I then began to speculate and below I surmise what his actions may have been.

1. Firstly, he browsed the internet to search for a microwave oven that he was interested in purchasing. Then he went to an electronics retailer and showed the photo of his preferred choice to the sales assistant who confirmed the model number and whether the product was available in the store.
2. While inspecting the target microwave oven, the young customer visited the maker`s website to determine product specifications and confirm other detailed information.
3. The shopper felt inclined to research several of the other ovens on display. He proceeded to check the respective makers` sites and gathered information by reading reviews posted online or otherwise perhaps used social networks to consult friends or family.
4. He decided not to buy on that particular day because he required more time to carefully consider his options. Therefore, he photographed some of the ovens in the store as a point of reference to assist in his decision making. Furthermore, he elected to use his smartphone for the in-store photography as it is much less conspicuous than using a tablet which would likely draw the attention of the sales assistants.

Whether this conjecture is correct it cannot be said, as he may also have been doing something that I cannot comprehend. I am unsure whether the actions undertaken by the man in his early twenties are usual or unique, however for me it was something that I had not previously witnessed and was of great interest. I felt that this was a new behavioral style.

In the Era of Multi-Device Use, it is Difficult to Ascertain the Reality of User Preferences

As with the example of the young male in the electronics store, in recent years, with both the trend toward mobile computing and device diversification, both real space and digital space has become complexly jumbled together. With these changing circumstances, there are increasing opportunities to perform research questionnaires and interviews in an attempt to clarify the reality of user website engagement in a multi-device environment.

However, we are confronted with the problem that the research results are not comprehensive enough to fully grasp the reality of user behavior. Perhaps, as the context of web usage in a mobile environment is very complex and in a state of flux, a survey approach that only relies on users` memories of earlier actions produces a fragmentary segment of past conditions making it difficult to understand the context behind the actions. Furthermore, in the multi-device era, as compared to the pc era, it is now more difficult to ascertain patterns concerning utilization. Questions relating to which device, when used, where used, why used, and how used, now demonstrate much more fluid and diverse responses.

Considering User Experience Through the Technique of Conversation Analysis

Based on an awareness of the issues, last autumn I had the opportunity to co-facilitate a workshop titled “Mobile UX Analysis (page in Japanese)”. The workshop was run in collaboration with Professor Takahito Kamitaira, of Senshu University, and Ayami Joh, of Japan`s National Institute of Informatics. In this workshop, we undertook experiments in order to analyze user behavior and investigate new solutions for mobile environments. Specifically, two separate couples, both who were meeting their partner for the first time, were tasked with seeking an eatery in the restaurant district of Shibuya, Tokyo. One couple was provided with two mobile devices, the other with no access to a device; we both observed and recorded their actions. Then, in the workshop, we undertook group sessions where, to gain insight, careful attention was paid to the couples` subtle exchanges.

In this entry I will avoid a discussion on the detailed particulars and effectiveness of conversation analysis; however, I will focus on the approach of “gaining insights through the observation of interaction” of which there were extremely significant results. For example, as mentioned, we set one couple the challenge of finding a restaurant in the Shibuya area without the use of a mobile device. The process of locating a restaurant yielded changes in communication between the couple, as they made eye contact etc. and we were able to observe their varied interactions at a level of detail that is usually obscured to behavior analysts. Conversely, concerning the couple provided with mobile devices to locate a restaurant, it was noticeable that the two had little interaction as both used their own mobile to search for a restaurant.

Importance of Generating Hypotheses through Observation

I believe that in the future an extremely interesting topic will be the use of conversation analysis to improve web user experience in the multi-device era. However, in this new age, where innovative and disruptive information technology appears in the market changing users' patterns of action, I feel that the following three elements are very important.

1. “Observational Studies” at the Place of Use

With regard to mobile, the context of use is extremely complex and fluid. Therefore a survey undertaken away from the place of actual mobile use, such as group interviews and depth interviews are not sufficient as they only provide fragmentary insights. Therefore, the first step should be going to the actual place where the action to be analyzed occurs. Then observing what truly happens in order to understand the reality of complex user actions. It is often said that our five senses and cognitive function process a lot more information than we realize, so through employment of the “observational studies” technique we can understand the complex and interesting actions of users.

2. Rather than Statistical Authenticity, Importance is applied to Insights Guided by Hypotheses

To gain an understanding of the user and verify usability, it has always been said that survey data should have statistical reliability. However, to collect a significant level of data when undertaking an observation study while on location at the actual place of use, a considerable budget must be secured and complex schedule drawn-up. The next concern is the tendency for discussions over "whether the sample data is sufficient enough". However, there is a likelihood that user behavior patterns may change over time in ways that are unexpected and that by utilizing the existing framework as a base there is a risk there are errors in the research methods themselves. With such fluidity, first a qualitative approach is required to gain a profound understanding and insight of users, and then the generation of a new hypothesis should equal a new framework.

3. Involvement of Diverse Expertise in “Observational Studies”

Up to the present day, mainly specialists, such as researchers and analysts, have been involved in observation research, however I believe that in the future that all those with a career related to website development should contribute to “observational studies”. As previously mentioned, we, humans, have greater sensory and cognitive functions than we are aware. People with diverse expertise and backgrounds can observe users from different perspectives. Therefore, in discussions over the provision of solutions, a wealth of innovative ideas are generated and put forward.

This year, Google Glass and other similar wearable devices will usher in an increase in the use of augmented reality and with this there are expectations that user behavioral patterns will further diversify. In order to gain a foothold in the thinking around next generation web user experience it is important to observe the actions of the people around us, in the spaces of daily life, for example when commuting on the train, when walking the streets, when in shops etc. We can then begin to build hypotheses about the reality of people and society. Perhaps, with regard to the technology of the future, this action will inspire innovative ideas relating to future online services.