Usability report - Tina Hansen Portfolio

B.J.Trading
Erhvervserkademiet Lillebælt, Odense
E-Concept Development 2013
www.bj-trading.dk
Afleveret den 12. 04. 2013
Word Count: 24.768
Made by:
Tina Hansen, Thor Stentebjerg Frederiksen,
Adnan Halilovic & Maibrit Christiansen
Counselors:
Lars Bojen ([email protected]), Erik Østergaard ([email protected])
Usability report
Page 1
Indholdsfortegnelse
1. Project description
1.1 Short description of BJ-trading.dk
Page 4
Page 4
Page 4
2. Scientific Theory Method
2.1 Positivism
Page 7
Page 7
Page 7
Page 7
3. Why test and the importance of a test plan
3.1 Why test
Page 10
4. Our test methods and technology
4.1 Why we chose ”Think-Aloud” Test
Page 15
1.2 The Reason why we choose bj-trading.dk as our test site
2.2 Hermeneutics
2.3 Persona
3.2 The importance of a test plan
3.3 The different test methods
4.2 During the “Think-Aloud” Test
5. The result of the “Think-Aloud” Test and conclusion
6. Bibliography
Page 10
Page 11
Page 12
Page 15
Page 16
Page 18
Page 19
Page 2
Short Introduction to the project
Report content
The project description
WHY we chose this
specific website
1. Project description
Page 3
1. Project description
In this project, we have selected a site that we are going to run a usability test on. We have selected the
website www.bj-trading.dk, which we believe, provides clear usability problems. We are testing on online
sale and has chosen a site using card payment. The usability testing will be documented in this report and
on video.
In this report, we are looking at the methods that exists within philosophy of science when we are
talking usability testing. The test is a think aloud test, which put us in a position to hear out each test person’s
experiences with the site while performing the test.
During the test, we will use the science theories positivism and hermeneutics to analyze and evaluate each
test person’s experiences of the site. By using positivism, we can extract useful data on the subjects’ interaction with the website. The hermeneutic part of the test is to understand the individual test
person’s motives, when he is faced with a usability test on 1www.bj-trading.dk.
To create an accurate picture of the website’s performance in a usability test, it is important that the website’s typical audience is conducting the test and therefore we have a persona, based on our target group,
with in our report.
When we have finished our method description, we will look at why companies choose to use a test plan
and in doing so come up with a description of our own test plan. We will also describe the various types of
existing usability tests, while clarifying the advantages and disadvantages of each test. Including the reasons
for the choice of our own test method and the technology we use in the test.
After the test plan is completed and analyzed, we will create a client report that shortly describes usability
findings and problems plus our solutions.
The report closes and sums up with a conclusion.
1.1 Short description of BJ-trading.dk
The website stems from B.J. Trading’s physical store in Thorsø, which was founded back in May 1997 by John
Jessen. The company’s revenue grew quickly and today the annually 2sale is over 200 million kroner. Nowadays; B.J. Trading has been converted into an A/S, but continues to be run daily by founder John Jessen.
The company strives, by its own admission, to ‘3Sell the product at a reasonable price and to provide a short
delivery time as possible, and to provide customers with the services they need.’
Bj-trading’s web design seems old and a little bit outdated, measured from a modern point of view. The
landing page displays several different offers and loads of options in the navigation menu. The web shop offers the sale of equipment to both individuals and companies. It sells a wide range of computers, TVs, photo
equipment, household appliances and other electronic items. You can, as a customer chose to pay with credit
card or to pay by invoice. All orders in Denmark are shipping free and is send trough 4GLS. You can also choose
to have your items send through Post Denmark5, if you do not mind paying the shipping cost of the item.
1 www.bj-trading.dk (tilgået den 10.04.2013)
2 http://www.bj-trading.dk/bjshop/default.asp?side=profil.asp (tilgået den 10.04.2013)
3 http://www.bj-trading.dk/bjshop/default.asp?side=profil.asp (tilgået den 10.04.2013)
4 http://www.gls-group.eu/276-I-PORTAL-WEB/content/GLS/DK01/DA/1.htm (tilgået den 10.04.2013)
5 http://postdanmark.dk/da/Sider/Post-Danmark.aspx (tilgået den 10.04.2013)
Page 4
1.2 The Reason why we choose bj-trading.dk as our test site
The reason why we choose this website is that we heard some complaints about the site and wanted to see
if these accusations were true. If we look at the reviews of the website 6www.trutspilot.dk, we can see that
www.bj-trading.dk has a score of 5.9 out of 10, which is not bad, but there are some improvements to be
made, that really could benefit the website. Based on the 700+ reviews on www.trustpilot.dk, we can conclude that the target group consist mostly of men, in the age between 20 to 40 years old. They are the typical
EDB buyers online and will be our persona in the usability testing.
B.J. trading is a big firm and does provide their customers with a wide range of great products and good prices. Therefore, their website should be as good as the rest of the company and a usability test is a good start
to get to know your target groups feelings about the web shop. We do not think that bj-trading.dk is aware
of how they can use their target groups’ point of view to create a better online solution and that is why we
want to do the usability testing on their particular website.
6 http://www.trustpilot.dk/review/www.bj-trading.dk (tilgået den 10.04.2013)
Page 5
Positivism
Scientific
Persona
Method
Hermeneutics
2. Scientific Theory Method
Page 6
2. Scientific Theory Method
The scientific method is a process by which scientists, collectively over time, try to construct an accurate representation of the world. Recognizing that personal and cultural beliefs influence both our perceptions and
our interpretations of natural phenomena.
For our project we have chosen to use two theories from the philosophy of science. We have chosen to use
the positivism theory which only analysis the human behaviour without taking their preferences or thoughts
into consideration. As a supplement we will also use the hermeneutics theory so we better can understand
and interpret the participant’s behaviour and thoughts and get a better insight why they do the things they
do. This is done through a think a loud test. For our usability test we will use 5 participants because a study
has shown that after the fifth user you are you are wasting your time by observing the same findings repeatedly but not learning much new7.
2.1 Positivism
Positivism8 is a philosophy of science and is very effective for gathering empiric data and taking an inductive
approach to a usability test. We want to test the websites usability and we chosen to use multiple participants so we can get a more accurate scientific result of which problems the website has. The cons of using
positivism is that the theory doesn’t take the human thoughts and feelings into consideration, it only looks
at their behaviour, that is the reason we have also chosen the hermeneutics theory.
2.2 Hermeneutics
Hermeneutics9 theory tries to understand and interpret the human nature. It is not about being neutral and
without prejudice because you are never neutral. According to Hans-George Gadamer people always have
some prejudice because we don’t start from scratch.
This is the reason we have chosen to use a think a loud test so we can get a insight what the participants think
and feel while they are testing the site. This we also help us understand better why the participants have
problems with completing some of the tasks or it takes them to much time for completing a task.
2.3 Persona
A persona is a description of a specific person who is a target user of a system being designed, providing demographic information, needs, preferences, biographical information, and a photo or illustration. The point
of developing personas is to avoid the mistake of designing for the “average” user that doesn’t actually exist,
and instead to make sure that the design will work for somebody specific rather than no one in particular. A
persona will help us better understand what they expect from www.bj-trading.dk and helps us find the right
participants who fits in our target group.
7 http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users (tilgået den 11.04.2013)
8 Andreas Beck Holm: Philosophy of Science p.24-43
9 Andreas Beck Holm: Philosophy of Science p.83-100
Page 7
Thomas is 29 years old he lives with his girlfriend in an apartment in centrum Odense, he studies computer
science at the university. He has experience across a wide variety of platforms and follows tons of different
tech news sites. He is an indoor person who in his spare time loves to play videogames and read sci-fi novels.
Page 8
Why test?
Test plan
Met
hod
s
Importance
3. Test plan & Methods
Page 9
3. Why test and the importance of a test plan
3.1 Why test
In the latest year internet business has been on a rise in Denmark10. The reason why is often that the prices
is lower on the internet than in a physical store. A good example of this in the music industry. With iTunes,
TDC play and other sites, customers can download a whole album or just a single number for a low cost. The
music industry is now forced to think in new ways and L.O.C. offered last year to download one of his albums
for free for 6 weeks11. and the album does only exist as download after the free download period.
So when customers do shop more and more on the internet and can buy products from all over the world,
usability, and testing, become really important.
If customers can not find what they are looking for in a few clicks, it can have the consequence that customers find another and more user friendly website to buy the product12.
So testing is an important tool for companies to find the problem areas of their website and eliminate those,
ensure profit and essentially to get ”good karma” in that sense that, the social media and mouth to mouth
recommendations is a powerful thing today13. If an company establish a good relationship with the customer,
the customer tends to return to the company.
When a company wants to test their product, or in our case, a web shop there are some things to take into
considerations.
There are different ways to conduct a test. One is the controlled experiment method where the company
formulates a specific hypothesis of what the company expects will happen during the test. The test persons
a chosen randomly and every test person should have the same experience before the test and during the
test to ensure correct results. The controlled experiment method also requires a control group to validate the
results of the test and the test group must be of a large enough size to measure the difference.
The controlled experiment is good for basic research but the method has its flaws when we talk about usability testing.
It can be difficult to conduct a usability test because of constraints or political reasons. Besides when we talk
about usability it is not always a very specific hypotheses you cant to confirm, but instead information about
design and the test persons experience witch can vary. Another reason for not using the controlled experiment method is sample size. The lack of information about the target group can lead to a large sample size
to ensure correct results.
But the most important reason not to use the controlled experiment method is the fact that the classical
experiment method is used to obtain quantitative results instead of obtaining qualitative information on the
design and the problems with the design and user experience.
The other approach is a less formal, well-designed test which is better suited for usability testing.
10
11
12
13
http://www.kfst.dk/?id=15470 (tilgået 10.04.2013)
http://www.soundvenue.com/nyheder/2012/03/09/locs-kommende-album-bliver-gratis (tilgået 10.04.2013)
http://interface.dk/da/hjemmeside/design/hjemmesidedesign-og-brugervenlighed (tilgået 10.04.2013)
http://www.business.dk/innovation/mund-til-mund-metoden-paa-steroider (tilgået 10.04.2013)
Page 10
The basic is
* Development of research questions or test objectives rather than hypotheses.
* Use of a representative sample of end users which may or may not be randomly chosen
* Representation of the actual work environment
* Observation of the end users who either use or review a representation of the product
* Controlled and sometimes extensive interviewing and probing of the participants by the test
moderator
* Collection of quantitative and qualitative performance and preference measures
* Recommendation of improvements to the design of the product.
But also this test method has its limitation.
The test situation is always an artificial situation and can effect the test results. A test is not a guarantee that
the product works and depends on the way the test is conducted.
It can be hard to identify the end user and the test persons are just representatives of the target group you
have classified. And finally the fact that testing not always is the best method to use.
But bottom line remains : it is better to test than not to test!
3.2 The importance of a test plan
When you conduct a test, a test plan is essential. The test plan contains the how, when, where, who, why and
what you want to test. A test plan keeps you on track and ensures that all test are done in a similar way, in
the same environment, on a specific target group, keeps deadlines and that the test persons are asked the
same questions and tasks.
The test plan is the blueprint for the test. It is important to have a description on what, how, and who you are
testing on. The test plan also ensures that all involved in the test, knows what are going to happen, on who
and when. With a test plan you will also know how many resources and what resources you will need to do
the test and finally a test plan keeps you focused and serves as milestones. However it its okay to develop the
test plan in stages due to new information or changes in the test.
The parts of the test plan
* Purpose, goals and objectives of the test
* Research questions
* Participant characteristics
* Method (test design)
* Task list
* Test environment, equipment, and logistics
* Test moderator role
* Data to be collected and evaluation measures
* Report contents and presentation
Page 11
3.3 The different test methods
When using user centered design there are some different test methods that can be used. We will list the
pros and cons with these methods.
Ethnographic Research
In ethnographic research you observe the test person in their natural environment using the product or
website. This method is a qualitative research method from where you can gather profiles about your users,
develop personas and task descriptions.
If we have had more time, we could have used this method to gather information about our test persons and
their internet habits.
Focus group Research
Focus groups are most useful in the early stages and the method used on the focus group is paper and pencil
drawings, storyboards, plastic models or screen based prototypes.
The object is to get a few people opinion on concepts, are the concepts good or bad, how can the concepts
be improved. The good thing about focus groups are the ability to explore a small groups judgement and
feelings in depth and learning how the users think and feel.
The focus group is a qualitative method and is not a substitute for usability test and differs from this method.
The negative about this method is, that what they are telling you not necessarily is the truth.
We have not used this method because www.bj-trading.dk is an established website and the method is not
so suitable for usability testing.
Surveys
With surveys you can gather information from a large group of people. and get a general opinion. With surveys the same problem as with focus groups occurs; the answers are not necessarily what people actually
feels or think. The problem with surveys is that you can not control who answers your survey and might get
answers outside your test target group.
We could have used surveys if we had more time to gather a general opinion about www.bj.trading.dk and
the potential problems with the webshop.
Card sorting
In cardsorting the test person are given a set of cards with terms already written on them (closed card
sorting) or a set of blank cards where the test person can write their own terms on them. This method is
very inexpensive and easy to conduct. The test person then place the cards with terms where they think it
makes ost sense on the website.
For further testing we could also have used the A/B test.14 A/B testing is a simple way to test changes to your
page by creating a “fake” site and putting it against the current design and measure which ones produces
better results. It is a method to validate that any new design or change to an element on your webpage is
improving your conversion rate before you make that change to your real site. The challenges this method
14 https://www.optimizely.com/ab-testing (tilgået den 10.04.2013)
Page 12
has is that designing the changes takes time and collecting quantitative data takes time to collect.
If we where to test further on www.bj-trading.dk we could have used card sorting and/or A/B test to test
where the test person would put categories according to where it would make most sense.
Walk Throughs
A Walk Through is used in the initial fase of the design or it mayor changes is made. A Walk Trough is where
a test person walks through the website to measure the flow of the website. The test person should idealy
be a outsider or a client.
This test can be time consuming.
This test could be relevant for us if we were to remodel www.bj-trading.dk´s website.
Paper prototyping
In paper prototyping you mock up the website in paper. The test can be conducted where questions is asked
or just to get an idea of the design choices. This test is inexpensive and be conducted on a single site, just on
navigation or on several sites. Information can be obtains very quickly. Another benifit is that you can eliminate a weak navigation or problems before you code anything.
Just like card sorting this test could be used if we where to remodel www.bj-trading.dk´s existing website.
Think a loud test
In a think aloud test you observe, and possibly record, the test person while they are performing specifik
tasks. In this test the test person the test person think out loud and tells where they would look for information or their opinion about the site, design or other things.
This is the method we have choosen for our test and there is more information about this test in the next
chapter.
Page 13
Technology
Think Aloud
Problems?
4. Test
Page 14
4. Our test methods and technology
In our project we have chosen to use a “Think-Aloud” Test to make our usability test. For that we needed
some “easy to use” tool, so we didn’t need to use a lot of time to figure out the technology.
In the usability class we were introduced to different usability user experience testing tools15. All with focus
on different things. For example “Simple Mouse Tracking”, who is a tool where you can see how the participants interacting with the website. We needed a tool that could do a bit more than that.
Silverback was one of them. Silverback is a great tool to use on people who test websites. Silverback can
capture screen activity, record video of the participants faces and record their voices. We saved a lot of time
using Silverback, because we didn’t used a lot of time on knowing the other tools. We knew Silverback from
the beginning and it was a very simple tool who did what we was expected it to do.
We did have one problem with silverback on one computer. For some reason on of our Mac wouldn’t open
Silverback. Instead he had to use the webcam on the computer. Even though we didn’t get the screen recording we still got his facial expression and his thoughts. Overall it was a good tool to use, and we made some
pros and cons for using this tool and there was definitely more pros:
Pros
* Simple
* Knew the program from the beginning
* Capture screen activity
* Record the participants face and voice.
* Easy to export the video
* Has almost 300 gb space
Cons
* Works only on a Mac
* Only 30 days free trial
That was some of our thoughts about the tool we have used. Now we want to describe more about the
“Think-Aloud” Test and why we have chosen this test methods.
4.1 Why we chose ”Think-Aloud” Test
We have chose to use “Think-Aloud” Test from the book “Handbook of Usability Testing” by Jeffrey Rubin and
Dana Chisnell.
In a “Think-Aloud” test can we hear the participants’ thoughts about the different tasks while they are doing
the test. It can often be an advantage for us, because the expression on the participants’ face says more than
words. If they have some problems during the tasks, they will seem frustrated or maybe surprised, that they
have some problems to solve a “simple” task. Then afterwards some of them will maybe say, “oh, that was
easy” because they don’t want to seem stupid. But in a Think-Aloud test we can compare what the participants says and what they actually do.
In our usability project our focus had been about the usability on the website www.bj-trading.dk, can the
users use the website without getting frustrated?
Time on how long it takes to complete a task is not important in this assignment. It is more important how
15 http://uxdesign.smashingmagazine.com/2011/10/20/comprehensive-review-usability-user-experience-testingtools/ (tilgået den 10.04.2013)
Page 15
the participants are doing the different tasks and how they react on the website. That is also the reason for
we have chose to make a “Think-Aloud” test. If it had been important to see how long time it takes for the
participant to make the test we couldn’t have used the “Think-Aloud” test because it slows down the process
to have to think aloud all the time.
The best way to gather data and capture how the participants react is to record it. For this we have chosen
to use the usability tool Silverback16. This allows us to capture the participants behavior and their body language while they move across the website. This combined with a think aloud test gives the moderator and
the observers a great opportunity to hear the participants thoughts and opinions while they systematically
perform different tasks. By observing and conducting a think a loud test we can gain more solid results and
the answer to questions, how and why. If we only did one of them we would only have gotten either the
why or how. If the time had allowed it we could have supplemented the think a loud test with a card-sorting
test17. For some studies have shown that the think a loud test has some flaws18 for instance, it’s a bad idea to
ask a participants to think a loud while you are measuring time on tasks. Thinking aloud slows performance
significantly. And its impossible to know everything what is going through a participants mind.
4.2 During the “Think-Aloud” Test
During the test, all five participants had problems completing the first task, which was finding information
about the company. The first thing they all did was to scroll down to the bottom of the page to look for a
“about us” button. When they didn’t find the button they started to look to the right side menu. This is an obvious usability problem, since all five participants searched for the same information the same place. Some
of the participants didn’t even find the information and had to move to the next task, which was finding a
specific and item and completing the purchase.
None of the participants had problems finding and using the search field at the top of the page, but when
the search results showed up on the screen you can clearly see the surprise on their faces, like for instance
participant Martin here.
Picture 1: Surprised
16 http://silverbackapp.com/ (tilgået den 10.04.2013)
17
http://www.usability.gov/methods/design_site/cardsort.html (tilgået den 10.04.2013)
18 Jeffrey Rubin & Dana Chisnell Handbook of Usability Testing, Ch. 4 p. 53
Page 16
Before the test we all had an assumption that one of the problems would be that you couldn’t use the logo
to get back to the homepage, since it wasn’t clickable. But as it turned out, it was only two of the five participants who actually tried to use the logo to get back to the homepage. The rest used the “startside” button
in the top menu.
The task opening hours turned out to be rather challenging all five participants searched for the opening
hours under “forhandler” in the top menu of the page or the right sidebar menu. Only two participants found
the information, one used the search field the other one found it by accident under the left side menu.
In the last task the participants had to complete the purchase and it turned out there were a lot of challenges.
One of the problems is the way the buttons are placed in such an atypical way, especially during the checkout
procedure. One problem we found during our “Think-Aloud” Test was the credit card fee. No matter what
card you used to pay with, you have to pay a fee between 2,45 to 188 Dkr.
Picture 2 Prices
Page 17
5. The result and the conclusion
The website www.bj-trading.dk has definitely some usability issues. We have tested what we assumed
were the biggest problems but during the test we encountered problems that we probably wouldn’t have
discovered by our self if we didn’t do the test. During the Think-Aloud test, we observed the participants
and thereby obtained a better insight into their thoughts. This refers to Positivism and Hermeneutics, as
we have written about in Scientific Theory Method.
By making a few small changes, bj-trading could improve their website significant and make their website
more “up to date.”
Page 18
6. Bibliography
Books;
Beck Holm, Andreas: ”Philosophy of Science”. PDF på Fronter.com/eal
Rubin, Jeffrey & Chisnell, Dana. (2008) ”Handbook of Usability Testing”. Wiley Publishing, Indiana
Web:
www.bj-trading.dk (tilgået den 10.04.2013)
http://www.bj-trading.dk/bjshop/default.asp?side=profil.asp (tilgået den 10.04.2013)
http://www.bj-trading.dk/bjshop/default.asp?side=profil.asp (tilgået den 10.04.2013)
http://www.gls-group.eu/276-I-PORTAL-WEB/content/GLS/DK01/DA/1.htm (tilgået den 10.04.2013)
http://postdanmark.dk/da/Sider/Post-Danmark.aspx (tilgået den 10.04.2013)
http://www.trustpilot.dk/review/www.bj-trading.dk (tilgået den 10.04.2013)
http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users (tilgået den 11.04.2013)
http://www.kfst.dk/?id=15470 (tilgået 10.04.2013)
http://www.soundvenue.com/nyheder/2012/03/09/locs-kommende-album-bliver-gratis (tilgået
10.04.2013)
http://interface.dk/da/hjemmeside/design/hjemmesidedesign-og-brugervenlighed (tilgået
10.04.2013)
http://www.business.dk/innovation/mund-til-mund-metoden-paa-steroider (tilgået 10.04.2013)
https://www.optimizely.com/ab-testing (tilgået den 10.04.2013)
http://uxdesign.smashingmagazine.com/2011/10/20/comprehensive-review-usability-user-experience-testing-tools/ (tilgået den 10.04.2013)
http://silverbackapp.com/ (tilgået den 10.04.2013)
http://www.usability.gov/methods/design_site/cardsort.html (tilgået den 10.04.2013)
Page 19