Thursday, August 26, 2010

On opinions vs data for UX Design

This is one of the reasons why I don’t put too much trust into opinion-based usability reviews: There’s a lot of guesswork involved, and guessing how humans behave is an endeavor fraught with peril.2 Expert reviews can be helpful, but they are no substitute for actual testing.

(Source: http://ignorethecode.net/blog/2010/08/13/opinions_vs__data/)

Friday, April 25, 2008

My Experience with NM4210-User Experience Design

Today is the day. God has given me this day to use it as I will. And I have chosen to use it to wrap up my final assignment for this module that is to be NM4210: User Experience Design.

I am typing this post on the infamous bus service 151 on my way to school from the other end of the service line, to submit my final report and peer-evaluation form. And I must say that I am largely satisfied with the journey thus far; the bus captain was all smiley and chirpy when I boarded the bus and he even assisted an old lady to her seat. It’s a positive experience. I just hope that the traffic will be smooth flowing. And I digressed.

As my first level 4000 module, NM4210 is more than just a module to fulfill my modular credit requirements. I took it with the intention of testing to see if honors year may be a possibility or no-go for me. I foresee that it will be tough and hectic if I am taking more of such modules.

Despite the busy schedules and weekly deadlines for presentation, I must say that my experience with the module is highly positive overall, albeit peppered with some negativity along the way.

In the beginning, the vagueness of the module kind of threw me away as it differs greatly from the clear knowledge structure of all the previous modules that I have taken; one week we were taught a method to design for user experience, the next week we were told that the method will not work. However, as the lessons unfold and the assignments kicked in one-by-one, I begin to appreciate the lack of structure as it allows me a great deal of possibilities and creativity to explore the methodologies taught and twist and change them to render them more appropriate and useful to our assignments.

And then the final assignment started.

The weekly deliverables for the final assignment meant that we have to consistently work on it while coping with the requirements of other modules. We ended up working on most all the Monday nights until three-plus four a.m. in the morning (our class is on every Tuesday, 9am to 12pm). And since I stayed rather far from school, it meant that I can only catch approximately 3 hours of sleep before I make my way to school. It didn’t help that Tuesdays are long days for me with my last lesson ending at 8pm.

However, this common experience actually bonded my group members together, and I believe it is one that we look back fondly at as we counted down to the last of the Monday late-nights.

The module got even more confusing (and the traffic halts to a jam now along Bukit Timah Road).

More and more user research methodologies were taught to us and we have to decide how we want to adapt them to our final assignment in order to find out what we need to find out about our target users. At times, I felt like we were on the right track only to be thrown off-track by the comment/ feedback of the class, especially those by the tutor.

Often time, I was awed-inspired by the kind of insights the tutor, Mr Reddy, has although he was not directly involved in the doing of our assignment. It strengthened my belief that good UX designs do not only evolve from excellent user research but also with the accumulation of experience by the designer.

And like the bus ride that was affected by external factors such as the traffic condition, the experience with the module was affected by the increasing work commitments demanded by other modules. I believe this is also why UX design is difficult to the point of being impossible; there are too many external factors that the designers cannot control/ manage in the designs.

We approached the final lap.

As the deadlines for all module projects loom, the workload of our final assignment intensifies. At some point, I felt that we will never be able to complete the assignment. This may be true in reality though, as there will never be an end to UX design and user research; it is a pure wicked problem. Fortunately (or unfortunately) for us, the assignment has to end with the user evaluation, going a full cycle. I guess deadlines for design rollout in the corporate world will be based more on the profit and cost timeline.

Although there are still ways to improve our designs by leaps and bounds, we have to call it a wrap and submit our final report by 5pm today.

In conclusion, I believe I have learned a lot from this module. More than the hard facts commonly taught in most modules, I’ve learned about the endless possibilities that UX design entails, and the creation and application of knowledge. While this experience will not be sufficient to make me a designer, it is definitely a positive start! The end is only the beginning of greater things to come.

With that, I shall end my post. (Actually it’s because my bus has reached its destination and I’m in school now; it was a positive experience travelling as well despite the minor hiccups). =)

P/s: Weiwei printed the report and submitted it. The pages turned out to be a little messy with some on single-sided, and some double due to the school's printer setting and the printing of the coloured pages. Personally I feel that this presentation isn't good when it comes to the reader's experience. But we decided to save the Earth (and save money) and not reprint everything in single side. :)

Thursday, April 24, 2008

Final Project; My Reflections

I've just finished compiling our final report.

Mentally going through the process of Discovering needs, Defining our target users, Designing our prototypes, to User Evaluating our prototypes as I read the report, I suddenly realized that amount of effort that has been put into "trying" to create successful UX designs; I say "trying" because you can never tell if your design will work until the users really use it. And even if it works for now, it may not work forever.

Going through all the researches and designing, researches and redesigning, I feel the effects of design being a wicked problem; it is an endless cycle of designing and redesigning to improve it further. And having to take the users' perspectives into consideration makes it even more wicked to the point of being impossible. It means that UX designers needs to be acutely aware of their users (through means of research or otherwise) in order to perform their job well.

As a novice designer, sometimes I got lost in the process of designing and failed to recognize that the design is actually what I wanted and not what the users wanted. This is dangerous as it will mean that the final design will not be accepted by the users.

Besides taking the users into consideration while designing the product, good UX designers I realized should also be able to take the users into consideration when designing their user researches. For example, in the case of the cultural probe we carried out in our project, the respondents will not be able to fully utilize a probe pack that consists of a camera as the camera is not allowed in their military camps.

From doing this project, I feel that it is impossible to ensure positive user experience throughout the whole usage of the product. The thing about experience is that it is too subjective. What a designer can attempt to do is perhaps to micro-manage the whole usage process by breaking down each stages of the usage process into tiny parts and then design to meet the requirement of these parts. While this may sound easy, the difficulty comes in when the designer has to piece this parts together to form the whole design; the parts has to be well connected to ensure a smooth process of usage. At this point, I am suddenly reminded of the analogy about enlightenment that Mr Reddy told us.

He said that there are 3 stages in enlightenment. The first stage is to recognize that there is a mountain. The second stage is to recognize the elements e.g. the rocks, the trees, etc., on the mountain. The last stage is to recognize how these elements come together to form the whole mountain.

Using the analogy, I believe good UX designers are those who are able to see how every small part relates to the whole experience of using the product. It's a tough job indeed.

Overall, the whole project is an excellent experience for novice designers like me as it provides me with real life experience that will aid me if I do decide to persist on in this field.

Right now, I will just appreciate the good designs in my life and be thankful to the designers.

Wednesday, April 16, 2008

User Research Smoke & Mirrors

The article (in 5 parts) by Christopher Fahey can be found here:
1 2 3 4 5

My Reflections
Part 1: Design versus Science

When we talk about User Experience Design, the main focus is undoubtedly on the user i.e. it is human-centric. Given this focus, I believe that the field of science that we are talking about here is the Social Sciences as opposed to Natural Sciences.

The key difference between the two fields is that social sciences study subjective, inter-subjective and objective or structural aspects of societies, whereas natural sciences focus on objective aspects of nature; there are no definitive answers in social sciences and depend largely on the context of the study.

Unfortunately, the problem with many UX designers (as identified by Fahey) is that they confused user research with the hard (natural) sciences research, and take for granted that their social-scientific research will provide definitive solutions to their design problems.

THEY DO NOT!

However, from an academic viewpoint, I still strongly believe that the values of these researches should not be discounted even in the field of user design. Designers, like academicians, should review each research data with a pinch of salt and gather as many varied perspectives as possible. The understanding of such diverse perspectives will aid in laying the foundation of the designers’ experience i.e. designers can gain experience from these researches. And experience, as Fahey pointed out, is highly necessary to be expert designers.

Part 2: Research as a Design Tool

The main problem I analyzed from reading this part of the article is that user researchers sometimes over inferred from their research data.

I believe this problem isn’t exclusive to user research as it is present in all other types of researches. For example, a direct positive statistical correlation between two factors (X and Y) simply tells us that when X increases, Y increases as well. It does not tell us that the increase in X causes the increase in Y or vice versa, or that a third factor Z causes both to increase at the same time.

So the issue in question here is the researchers’ ability to interpret the data as what the data says (or does not say), and NOT with the research itself. The data only tells you what it does, NOTHING MORE, NOTHING LESS.

With regards to the question on whether we need such expensive and complicated research methodology to inform us something that is ‘commonsensical’, I will think that it depends. From a commercial viewpoint where decisions are often made based on the dollar-sign, it may be rather wasteful and extravagant (both in terms of money and time) to conduct such research. But from an academic viewpoint, I will say ‘why not?”.

My point is that what is ‘common sense’ may not be so common after all. Commonality is perhaps just a social construct that is dependent on the society in question. For example, it may seem quite common sense that people will not put anything that stinks in their mouth, but in reality we have people who swear by the delicacy that is to be the smelly bean curd, which is touted to taste better if it smells worse (not to me though, the smell is enough to put a few meters between me and the dish). Another example somewhat related to eyetracking is that theatre practitioners often preached that the movement from upstage right (audience’s left) to downstage left (audience’s right) is the strongest, most attention-grabbing. This knowledge seems quite commonsensical but it does not quite apply to the Chinese audience; the thing is that English texts read from left to right whereas Chinese texts read from right to left. Thus, what is common sense may in fact be common only to a particular person, group, or society.

Hence, the reasoning in part 2 of the article does not discount the value of research at all.

The factor that determines the value of research to me is whether or not new knowledge can be garnered from it, and that the value of this new knowledge outweighs the cost of gaining it. There’s no point trying to prove something that is long proven again right? Isn’t that why we should always look through secondary research data first?

Part 3: Research as a Political Tool

This is the part whereby I fully agree with Fahey. My only gripe about this is that most often than not, designers are forced to justify every single aspect of their designs. And the effort needed to do so will mean that less effort can go to ‘perfecting’ the designs. It’s a lose-lose situation.

Part 4: Research as Bullshit

By now, I guess it is quite obvious that I place more significance on research as compared to the author (I may be wrong on this though; I suspect that the author places as much emphasis on well-planned and well-executed research that generates thoughtful insights). Hence, when I saw the sub-heading, I thought that this part of the article has potential of being flame bait. However, as I read further, I cannot help but agree with his observations.

As mentioned above, the value of research to me is whether or not new knowledge can be garnered from it, and that the value of this new knowledge outweighs the cost of gaining it. In the examples cited by Fahey, it is rather obvious that the cost outweighs the benefits. Seriously, a persona room?! Like, huh?

With regards to the passing off of subjective analysis as objective findings, I believe the issues here are on the integrity of the actual research methodologies and how the researchers present their findings. Unfortunately, unlike academic researches whereby researchers have to provide details of how they carry out their researches, or even the raw data collected (usually with a small administrative fee), and discuss the limitations, commercial researches are not bounded by such practices (due to the competitive nature of business). Hence, it makes it harder to verify the accuracy, biases, reliability and validity, etc., of the (commercial) research. Thus, the onus is on the readers to take the results with a pinch of salt and not over rely on a singular source; multiple perspectives from varied sources should be sought to gain a clearer understanding.

Part 5: Non-Scientific User Research isn’t a Bad Thing

Indeed. All types of research should be awarded with the same level of scrutiny and analysis.

Knowledge of user, like any other knowledge, can be gained from empirical research or critical/ cultural research, empiricism or rationalism. As such, our understanding of the users can only progress with the accumulation of knowledge from all varied perspectives. This is not unlike the accumulation of experience, which (I repeat) is a requirement of expert designers.

“If I have seen further it is by standing on the shoulders of giants.”

~ Sir Isaac Newton

My Conclusion
User Experience research (both scientific and non-scientific) is crucial in aiding designers’ understanding of the users especially in unfamiliar contexts. However, designers should not be over reliant on them as the interpretation of the research data may be biased and flawed; knowledge of the research background, limitations, etc., is necessary to sieve out such errors but such knowledge is usually unavailable in commercial researches.

It is true indeed, that research alone does not guarantee good user experience design; the experience of a good designer is also crucial. But I feel that the two are kind of intertwined, in that experience results in the ability to plan good research that will aid in the design, and good research will add to the experience of the designer.

Understanding the users is like a never-ending cycle; thorough verifications with the users should be carried out to ensure that the designers’ understandings (derived from their own experience or from research results) are reflective of the actual users. Ultimately, it all boils down to the users when it comes to User Experience Design.

Tuesday, April 15, 2008

Final Project; Phase 4 - Evaluation

Having developed our product prototypes, it's time we have the users evaluate it. What we did was to ask users to navigate around our website prototype, and have them fill up a survey questionnaire that consisted of questions in Likert Scale measuring their emotions.

Using the Circumplex of Emotions, we realized that the general feeling about our website prototype is that it is pleasant but calm/ boring. It lacks the feeling of excitement that we wanted to create in our users.

However, we also noted that the Circumplex of Emotions does not tell us why the users feel so; it will be over inferring from the data to do so. Hence, we carried out laddering to further ascertain the 'why' behind their feelings.

And here's our presentation for Phase 4:

Class feedbacks
The result is kind of expected as our website prototype is rather bare. Besides expecting the results, Mr Reddy also commented that it appears to be just another website selling mobile phone. And this is a crucial point that we have to work on in order to sell our product, which is a customizable mobile phone. He also suggested that we need to carry out user evaluation of our product concept i.e. customizable mobile phone. This is the part we missed because we only evaluated the users’ emotions about the website i.e. the look and feel of the website.

My reflection
I agree with the class and Mr Reddy on their feedbacks. The fact that the result is kind of expected also pointed to Fahey's point that some design problems are easily identified without having to go through research. I am glad that we went though the process though, as we managed to gain experience from carrying out the researching planning, measurement, and analysis. I believe that the experience of planning and executing researches will also enable a designer to hone his/ her research skills. This will ensure that the research carried out will allow the designer to know what he/ she wants to know.

I also learned that one research method can be employed in diverse settings to find out different things. As in the case of the circumplex of emotions, we can use it not only to measure the look and feel of our website, we can also use it to measure our product concept. One group also experimented with flashing their website along with competitors' websites, and then have the users evaluate his/ her emotions for all websites. I feel that this will allow designers to learn about their website as compared to the competitors. The next step to this, I guess, is to ascertain why users feel a certain way about the various websites, and incorporate the findings in the updated design.

Wednesday, April 9, 2008

Final Project; Phase 3 - Design

Phase 3a
For Phase 3a, we refined our 3 user personas - the advance user, the novice user, and the anti-user based on last week's feedbacks. We decided to target the military users and come out with a military phone. We then did some user research on participants who fit our updated advance user persona before attempting to design our product.

Our product basically can be divided into 2 sub products: the mobile phone, and the website that allows user to customize the mobile phone and purchase it.

For the mobile phone, we carried out a simple cultural probe using just a small notebook and pen, like in Assignment 3. This is not only due to our budget constraints, but also because it will be impossible for our participants to bring in visual/ audio recording devices into their military camps.

Because military personnels in Singapore will all carry a little black notebook and pen with them during training, we decided to give them the same notebook to record down their feelings and happenings during the search, the usage, and the disposal of their mobile phone. This is in accordance to the Product Life Cycle, the Value Creation and Delivery Sequence Model, and the Simplified UX Model, whereby the user experience of the product can be enhanced by tracking each stage of their usage, from the product manufacturing raw material (think Body Shop), to the sourcing/ purchase, to the actual use, and finally to the disposal of the product.

We also carried out survey to find out about the needs and excitement features of a mobile phone as perceived by our target users. Likewise, we also seek to find information online through the online forums that may be frequent by our target user.

For our website, we analyzed numerous mobile phone companies' websites (e.g. Nokia, Samsung, etc.), service providers' website (e.g. Singtel, M1, and Starhub), and also Dell's website to come out with a common-to-all information-categorization list. Card sorting was then carried out to determine the most user-friendly information structure for our target users.

Here's our presentation for Phase 3a:

Phase 3b
There is no presentation for Phase 3b. What we are required to do, was to develop low-fidelity prototype based on our user research findings. Hence, we sketched out potential mobile phone designs and listed the specifications of our mobile phones, and we also came out with paper prototype of our website.

Mobile Phone - Low Fidelity Prototypes

Slide Phone

Clamshell Phone

Bar Phone

Showing our mobile phone designs to Mr Reddy, he feedbacked that we should probably do away with the clamshell phone and the slide phone, as the nature of these two designs is that they are less durable compared to a solid bar phone with the slides and hinge mechanisms etc., which are dust magnets. He also asked about the use of our mobile phone by soldiers during the night whereby there is a need to be tactical.

We also decided on 4 main tasks that we want our users to undertake to test the paper prototype of our website, and see if the users are able to derive positive user experience or face difficulties trying to carry out the tasks.

The 4 tasks are as follow:
1.Sign up for a user account
2.Purchase a mobile phone
3.Customize and purchase a mobile
4.Trade-in a mobile phone

And here's our low-fidelity prototype:


Phase 3c
For our mobile phone, we incorporated the ideas we got from our research and came out with this medium-fidelity prototype.

Unfortunately, we were unable to do a high-fidelity prototype like the dummy phones we often see on display. Here's our mobile phone prototype.


For the website, we managed to create a high fidelity prototype, which can be found HERE! and incorporated the findings from our previous researches as well.

Wednesday, March 19, 2008

Final project; Phase 2 - Define

We refined our Needs Analysis from Phase 1 through ethnography (observation of people in our daily life). Having reaffirmed our belief that customized mobile phone is the way to go, we went on to do In-depth interviews with 15 randomly selected respondents about their mobile phone usage.

We found out that different users have distinct usage and needs for mobile phone. What strikes us the most is that often times, social circumstances restricts/ affords only certain usage patterns e.g. National Servicemen in our conscripted army are not allow to bring in camera phone into military compounds. Hence, we decided that a customizable mobile phone for this user group is the way to go.

Based on this user group, we came out with our user personas, as well as the three levels of emotional appeals, and four-pleasure analysis of our advance user.

Here's our presentation for Phase 2:

Class feedbacks
Feedbacks we got says that our target users are too varied and diverse. It's not a niche-enough group and that will increase difficulty in our product design to satisfy them; this despite our argument that our product is customizable and hence, able to cater to a larger group of users with different needs.

My reflection
In retrospect, I have to agree with the class that it is better if we are able to zoom in with a very clear focus on who we want to target, and then seek to maximize the user experience of this target group.