the entire thesis - Southern Cross Business School

Transcription

THE EFFECT OF WEB SITE STRUCTURE ON
USERS’ PERCEPTIONS OF NAVIGABILITY
WILLIAM J. SMART
A dissertation submitted as partial fulfillment of the requirements of the degree of
Bachelor of Applied Science (Computing) with Honours at
Southern Cross University
November, 1997
ABSTRACT
__________________________________________________________________________________________
Abstract
The popularity of the personal computer and the Internet continues to grow in the
global community. Estimates suggest that by the end of the year 2000, 480 million people will
be connected to the Internet. With the advent of HyperText and the World Wide Web, a
global pool of information has been created.
This study investigates the effect of Web site structure on users’ perceptions of
navigability. To accomplish this task, two Web sites were constructed on the World Wide
Web. The two Web sites labeled “red” and “blue” use different structures. The red Web site is
a strictly hierarchical site containing a total of 72 links. The blue Web site uses compromised
and overlapping hierarchies as well as sequential structures. The blue site contains a total of
736 links.
The 113 subjects who undertook this experiment were drawn from the staff and
student body of Southern Cross University. Each subject was given two tasks and a
questionnaire to complete for each of the two Web sites. The tasks required the subjects to
navigate each site. The questions posed in the questionnaire are designed to capture from the
subjects their perceptions of the structure of the two Web sites. Additional data were also
taken for each subject from the server log file.
Conclusions that can be drawn from this work:
•
•
•
•
•
•
•
The majority of users perceive a structure that allows multiple access (heavily crosslinked structure) to the content to be more navigable than a strict hierarchy.
Both types of Web sites tested -- strict hierarchy and heavily cross-linked structure -- can
cause a user to experience the “lost in hyperspace” phenomenon.
Users with low levels of experience with computers and the WWW feel more lost in both
structures tested.
Inexperienced users of the WWW take longer to complete the task in a strict hierarchy
than in a heavily cross-linked structure.
Of the facilities offered by modern browsers, users most frequently use the Back facility
for navigation within a Web site. Users utilize this facility even if there are dedicated
Hyperlinks provided to perform this function.
The Favorites (Bookmarks) facility of modern browsers is not utilized by users as a
navigation aid within a Web site.
Metrics developed by Botafogo et al. (1992) were applied to the WWW and found to
accurately identify structures within Web sites. However, an anomaly in the design of the
Converted Distance Table for a strictly hierarchical Web site was identified and a
modification was developed to correct it.
Page i
ACKNOWLEDGEMENTS
__________________________________________________________________________________________
Acknowledgements
I would like to thank my supervisor, Barry Wilks for all his advice and assistance
throughout this year. The long, laborious process that led to the completion of this thesis
would have been impossible without his valuable guidance.
Thanks also to Lyndon Brooks for providing me with the statistical information and
expertise to properly analyse all of the data.
To Mark Orman, for all the discussions, assistance, loan of books, articles and your
valuable comments, thanks pal.
To the staff and students of Southern Cross University that allowed me to subject them
to my experiment, a heart felt thank you.
To my friends, who offered suggestions, read drafts of chapters and dragged me away
from the all-consuming task of this thesis, thanks for all your support.
Finally, to my wife, Christine and son, Zachariah, thank you for putting up with my
odd working times and behaviour during this last year. Your love and understanding are the
most valuable part my life.
William J. Smart
Page ii
TABLE OF CONTENTS
__________________________________________________________________________________________
Table of Contents
Abstract …………………………………………………………………………………
i
Acknowledgments …..……………………………………………………………….…..
ii
Table of Contents …..……………………………………………………………….…...
iii
List of Tables ……………………………………………………………………………
viii
List of Diagrams .………………………………………………………………………..
ix
Chapter 1: Introduction …………………………………………………………… 1
1.1 Background ...………………………………………………………………………… 1
1.2 Research Question and Hypothesis ……………………………………………..…… 2
1.3 Justification of the Research ...………………………………………………………. 3
1.4 Definitions …………………………………………………………………….……… 4
1.5 Methodology …………………………………………………………………………. 5
1.5.1 Statistical Processes ………………………………………………………... 5
1.6 Limitations and Key Assumptions …………………………………………………… 7
1.7 Outline of Thesis ……………………………………………………………………... 8
1.8 Summary ……………………………………………………………………………… 10
Chapter 2: Literature Review…………………………………………………….. 11
2.1 Introduction …………………………………………………………………………... 11
2.2 Rise of the World Wide Web (W3) ………………………………………………….. 12
2.2.1 Building Blocks of the WWW .……………………………………………. 15
2.2.2 Browsers and Navigation …………………………………………………... 18
2.2.3 Navigation ………………………………………………………………….. 22
2.3 HyperText ……………………………………………………………………………. 23
2.3.1 HyperText History …………………………………………………………. 23
2.3.2 HyperText and Hyperlinks ………………………………………………… 24
2.3.3 HyperText Structure ……………………………………………………….. 25
2.3.3.1 Exploration (Hierarchical) Structure …………………………….. 27
2.3.3.2 Sequential (Next) Structure ……………………………………… 29
Page iii
TABLE OF CONTENTS
__________________________________________________________________________________________
2.3.4 HyperText Metrics ………………………………………………………… 30
2.3.4.1 Converted Distance Matrix (CDM) ……………………………... 31
2.3.4.2 Converted Out Distance (COD) …………………………………. 32
2.3.4.3 Converted Distance (CD) ………………………………………... 32
2.3.4.4 Relative Out Centrality (ROC) …………………………………... 33
2.3.4.5 Compactness (Cp) ………………………………………………... 33
2.3.4.6 Stratum (St) ………………………………………………………. 33
2.3.4.7 Summary of Structure Metrics …………………………………… 35
2.3.5 Lost in Hyperspace – Metrics ……………………………………………… 35
2.4 Web Sites …………………………………………………………………………….. 38
2.4.1 Web Site Definition ………………………………………………………... 39
2.4.2 Web Site Structure …………………………………………………………. 39
2.4.2.1 Web Site Structure - Style Guidelines …………………………… 40
2.4.3 Web Site Metrics …………….……………………………………………... 41
2.5 Summary ……………………………………………………………………………… 42
Chapter 3: Methodology…………………………………………………………… 45
3.1 Introduction …………………………………………………………………………... 45
3.2 Experiment Construction: Web sites ………………………………………………… 47
3.2.1 Quick Response Time ……………………………………………………… 48
3.2.2 Content of Sites Interesting to Subjects ……………………………………. 49
3.2.3 Quantifiably Different Red and Blue Web Site Structures ………………… 49
3.2.3.1 Verifying Red and Blue Web Site Structures are
Quantifiably Different ……………………………………………. 53
3.2.4 Minimise Variables Between Pages of Each Web Site …………………….. 55
3.2.5 Accessible HTTP Log ……………………………………………………… 56
3.3 Experiment Construction: Browser …………………………………………………... 56
3.4 Experiment Construction: Subjects …………………………………………………... 57
Page iv
TABLE OF CONTENTS
__________________________________________________________________________________________
3.5 Experimental Task …………………………………………………………………… 58
3.5.1 Questionnaire General Information ………………………………………... 58
3.5.2 Questionnaire Site Red …………………………………………………….. 59
3.5.3 Questionnaire Site Blue ……………………………………………………. 60
3.5.4 Questionnaire Conclusion …………………………………………………. 61
3.5.5 Questionnaire Summary …………………………………………………… 61
3.6 Data Collected ……………………………………………………………………….. 62
3.7 Experiment Summary ………………………………………………………………... 63
Chapter 4: Analysis………………………………………………………….………. 64
4.1 Introduction …………………………………………………………………………... 65
4.2 Statistical Analysis Introduction ……………………………………………………... 66
4.2.1 Data Excluded ……………………………………………………………… 66
4.2.2 Data Transformed ………………………………………………………….. 66
4.3 General Linear Model (GLM) - Repeated Measures ………………………………… 68
4.3.1 Hypotheses …………………………………………………………………. 69
4.3.1.1 Within Subject Null Hypotheses …………………………………. 70
4.3.1.2 Between Subject Null Hypotheses ……………………………….. 71
4.3.2 GLM - Repeated Measures Interpretive Strategy ………………………….. 72
4.3.3 Results of Tests of Multivariate Significance ……………………………… 73
4.3.4 Results of Tests of Univariate Significance Involving
the Within-Subjects Effects ………………………………………………… 74
4.3.4.1 Red - Blue by Experience with WWW, Measure Task time …….. 75
4.3.4.2 Red - Blue by Age, Measure Task Time …………………………. 75
4.3.4.3 Red - Blue by Order, Measure Task Time ……………………….. 76
4.3.4.4 Red - Blue, Measure Task Time …………………………………. 76
4.3.4.5 Red - Blue, Measures the use of Back Button and History List …. 77
4.3.4.6 Summary of Within-Subjects Effects …………………………… 78
Page v
TABLE OF CONTENTS
__________________________________________________________________________________________
4.3.5 Results of Tests of Univariate Significance Involving
the Between-Subjects Effects ……………………………………………… 79
4.3.5.1 Experience with Computers, Measure Lostness ………………… 80
4.3.5.2 Experience with Computers, Measure Task Time ………………. 80
4.3.5.3 Experience with WWW, Measure Lostness Rating ……………... 81
4.3.5.4 Gender by Age, Measure Lostness Rating ………………………. 81
4.3.5.5 Gender by Age, Measure Comfort Rating ………………………. 82
4.3.5.6 Gender by Age, Measure Number of Nodes Visited …………… 82
4.3.5.7 Summary of Between-Subjects Effects ………………………….
83
4.3.6 Summary of General Linear Model - Repeated Measures ………………… 84
4.3.6.1 Supported Within-Subject Null Hypotheses …………………….. 84
4.3.6.2 Supported Between-Subject Null Hypotheses …………………... 84
4.3.6.3 Rejected Within-Subject Null Hypotheses ……………………… 85
4.3.6.4 Rejected Between-Subject Null Hypotheses …………………….
87
4.4 Logistic Regression ………………………………………………………………….
88
4.5 Modification of Hypertext Structure Metrics for the Web ………………………….. 90
4.5.1 Improved Converted Distance Matrix for a Web site
Constructed as a Strict Hierarchy …………………………………………..
93
4.5.2 Application of Modification to the Experimental Red Web Site ………….
96
4.5.3 How the CDMh Effects the Methodology of this Thesis ………………….
99
4.5.4 Multiple and Compromised Hierarchies and the CDMh ………………….. 99
4.5.5 Where to Use the CDMh Modification ……………………………………
100
4.6 Examination of Lostness & Efficiency Indicators and the Web …………………….
101
4.7 Examination of Questions raised in Chapter 2 ………………………………………
105
4.8 Summary …………………………………………………………………………….. 110
Chapter 5: Conclusion………………………………………………………….….. 111
5.1 Introduction ………………………………………………………………………….. 111
5.2 Summary of Methodology …………………………………………………………… 112
5.3 Summary of Findings ………………………………………………………………... 113
5.4 Conclusions About the Research Problem …………………………………………... 117
Page vi
TABLE OF CONTENTS
__________________________________________________________________________________________
5.5 Limitations …………………………………………………………………………… 118
5.6 Implications of Research for Web Site Design .……………………………………... 119
5.7 Further Research …………………………………………………………………….. 121
5.8Conclusion ……….…………………………………………………………………... 122
Bibliography …………………………………………………………………………... 121
Appendix A: “A Brief History of the Internet” ………………………………………….. 126
Appendix B: “Web Site Maps” and “Nodes and Hyperlinks of Red and Blue Web sites”. 133
Appendix C: “Converted Distance Matrix – Red and Blue” …………………………….. 139
Appendix D: “Sample Web Pages Red and Blue Sites” …………………………………. 141
Appendix E: “Questionnaire” ……………………………………………………………. 147
Appendix F: “Sample Data From access_log” …………………………………………... 152
Appendix G: “Univariate Tests” ……………………………………………………….… 153
Appendix H: “CDM and CDMh Red Web Site” …………………………………………. 155
Appendix I: “Table of Results Lostness/ Efficiency Indicator” ………………………….. 157
Appendix J: “Experimental Data Set” ……………………………………………………. 159
Page vii
LIST OF TABLES
__________________________________________________________________________________________
List of Tables
Table 2.1: Domain .com growth 1993 - 1997 ……………………………………………. 14
Table 2.2: WebTrends (tm) Complete Summary Report Web Browsers,
Report (Nov. 1996- Feb. 1997) ………………………………………………. 18
Table 2.3: The path measures from which indicators of
lostness and efficiency are derived …………………………………………... 36
Table 3.1: Comparison of red and blue Web site Metrics ……………………………….. 53
Table 4.1: Multivariate Tests (Wilks' Lambda) ………………………………………….. 73
Table 4.2: Excerpt -Univariate Tests of Within-Subjects Contrasts ……………………... 74
Table 4.3: Excerpt -Univariate Tests of Between Subjects Effects ……………………… 79
Table 4.4: Predictors of Preference ……………………………………………………… 89
Table 4.5: Comparison of red and blue Web site Metrics ………………………………. 90
Table 4.6: Original and Modified red Web site metrics
compared to blue Web site metrics ………………………………………….. 96
Table 4.7: A perfect search - Results from Access_log …………………………………
102
Table 4.8: Excerpt - Table of Results Lostness/ Efficiency Indicator …………………..
103
Page viii
LIST OF DIAGRAMS
__________________________________________________________________________________________
List of Diagrams
Diagram 2.1: Growth of Internet Browsing Services …………………………………… 13
Diagram 2.2: HTTP Process …………………………………………………………….. 16
Diagram 2.3: Netscape Navigator 3.01 ………………………………………………….. 20
Diagram 2.4: Microsoft Internet Explorer 3.02 …………………………………………. 21
Diagram 2.5 Exploration (Hierarchical) Structure …………………………………….… 27
Diagram 2.6 Strict Hierarchy ……………………………………………………………. 27
Diagram 2.7 Compromised Hierarchy …………………………………………………... 28
Diagram 2.8 Overlapping Hierarchies …………………………………………………... 28
Diagram 2.9 Sequential (Next) Structure ………………………………………………... 29
Diagram 2.10: Distance Matrix ………………………………………………………….. 30
Diagram 2.11: Converted Distance Matrix (CDM) ……………………………………... 31
Diagram 2.12: Converted Out Distance (COD) …………………………………………. 31
Diagram 2.13: Relative Out Centrality (ROC) ………………………………………….. 32
Diagram 2.14: Worked Example of Stratum ……………………………………………. 34
Diagram 4.1: Male and Female Frequencies by Age ……………………………………. 67
Diagram 4.2: Transformed Male and Female Frequencies by Age ……………………... 67
Diagram 4.3: Red - Blue by Experience with WWW, Measure Task Time …………….. 75
Diagram 4.4: Red - Blue by Age, Measure Task Time ………………………………….. 75
Diagram 4.5: Red - Blue by Order, Measure Task Time ………………………………… 76
Diagram 4.6: Red - Blue, Measure Task Time …………………………………………... 76
Diagram 4.7: Red - Blue, Measure the use of Back Button and History List …………… 77
Diagram 4.8: Scatter Plot - Exp. with Computers, Measure Lostness …………………... 80
Diagram 4.9: Scatter Plot - Exp. with Computers, Measure Task Time ………………… 80
Diagram 4.10: Scatter Plot - Exp. with WWW and Lostness Rating ……………………. 81
Diagram 4.11: Gender by Age, Measure Lostness Rating ………………………………. 81
Diagram 4.12: Gender by Age, Measure Comfort Rating ………………………………. 82
Diagram 4.13: Gender by Age, Measure Number of Nodes Visited ……………………. 82
Diagram 4.14: Converted Distance Matrix (CDM) ……………………………………... 94
Diagram 5.1: Combined Hierarchy ……………………………………………………… 120
Page ix
LIST OF DIAGRAMS
__________________________________________________________________________________________
Diagram 4.15: Strict Hierarchical Web Site Converted Distance Matrix (CDMh) ……... 95
Diagram 4.16: Estimated Marginal Means of Task Time per Site ……………………… 97
Diagram 4.17: Overlapping Hierarchy …………………………………………………..
99
Diagram 4.18: Compromised Hierarchy …………………………………………….…... 100
Diagram 4.19: Estimated Marginal Means of Lostness and Comfort …………………...
108
Diagram 4.20: Estimated Marginal Means of Visited Nodes and Task Time …………... 108
Page x
INTRODUCTION
__________________________________________________________________________________________
Chapter 1
INTRODUCTION
1.1 Background
The popularity of the personal computer and the Internet continues to grow in the
global community. Currently the Internet is growing in popularity by an “order of magnitude”
yearly (Bowman et al., 1994). Estimates predict that by the end of the year 2000 AD, 480
million people will be connected to the Internet.
With the advent of HyperText, first proposed by Vannevar Bush (1945) and the World
Wide Web (Berners-Lee & Calliau, 1990), a global pool of information has been created. The
World Wide Web is being used for education, marketing, on-line publishing, business and by
many other special interest groups and individuals.
Personal Web sites are becoming increasingly popular, where the Web site is a
carefully constructed portrayal of the person themselves (Erickson, 1996). Just as individuals
are increasing their presence on the Web, so are corporations. The World Wide Web has now
become part of the commercial world. Web sites are now advertised on radio and television as
part of overall marketing strategies.
Users locate Web sites on the World Wide Web by a variety of methods. The most
common are (GVU, 1996); search engines (index and meta-index), advertising (television,
radio and print), opportunistic (following links from page to page, as encountered) and
bookmarked/favorites (revisiting pages added to the user’s bookmark/favorite file).
Regardless of what method a user utilizes to locate a Web site, once it is located the user must
navigate the structure of the Web site to find the information sought.
Users browsing a Web site may not be able to remember how they arrived at a
particular piece of information and then become disorientated. This is known as “lost in
hyperspace” (Nelson & Kearsley, 1989; Rivlin et al., 1994; Botafogo et al., 1992). The
structure of the Web site should be designed to reduce this feeling of lostness (defined in
Section 2.3.5).
Page 1
INTRODUCTION
__________________________________________________________________________________________
Much work has been done with HyperText relating to the subject of structure,
including the development of metrics. These metrics were developed to specifically measure
and compare the structure of HyperText documents (Botafogo, Rivlin & Scheiderman, 1992;
1994). Since the World Wide Web is a modified form of HyperText, the body of research on
this subject (HyperText) can be directly related to the World Wide Web.
However, with the improvements to the browsers (Section 2.2.2) utilized by users
accessing the World Wide Web, the previous research on HyperText may require
modification.
The researcher of this thesis conducted a search of the World Wide Web, scientific
journals and academic papers for up-to-date work on user perceptions of Web site structures.
None was found. As the World Wide Web continues to grow, the structure of Web sites
becomes increasingly important. Web site developers, whether personal or corporate, desire
users to be able to quickly locate the information they seek.
1.2 Research Question and Hypothesis
The structure of a Web site is the way the pages of the site are linked together by
Hyperlinks. Previous research into HyperText defines two main types of structure -sequential and exploration (Thuring et al., 1991; Section 2.3.3). Exploration (hierarchical)
structures are classified as strict, compromised and overlapping hierarchies (Woodhead, 1990;
Section 2.3.3.1). These structures and the user’s perception of these structures are the focus
of this thesis. The research question is:
“What is the effect of Web site structure on users’ perception of navigability?”
The overall hypothesis of this thesis is:
Ho:
Web site structure does not affect the user perception of navigability.
HA:
Web site structure does affect the user perception of navigability.
Page 2
INTRODUCTION
__________________________________________________________________________________________
In endeavouring to answer the research question, four other related questions were
raised:
1. Do users utilize the features of modern browsers to aid navigation within a Web site?
2. Which structure would users prefer -- heavily cross-linked or a strict hierarchy?
3. Which type of Web site -- heavily crossed-linked or strictly hierarchical -- increases the
users’ likelihood of experiencing the “Lost in Hyperspace” phenomenon?
4. Is the previous research in HyperText metrics still valid, when applied to Web sites
accessed by modern browsers?
1.3 Justification of the Research
The research question is a justifiable research task for a number of reasons:
•
understanding a user’s perceptions of the structure of Web sites, will lead Web developers
to design sites that reduce the user’s likelihood of experiencing the “lost in hyperspace”
phenomenon;
•
understanding a user’s perceptions of the structure of Web sites, should allow Web site
designers to construct sites that the user can easily understand and navigate (user friendly
design). This should lead to an increase in speed of access to desired information;
•
as there is no (to the researcher’s knowledge) up-to-date research on users perceptions of
Web site structures, an investigation into this area may uncover influences on users based
on age, gender and experience;
•
the previous research into HyperText structures and metrics may require updating when
applied to the World Wide Web. The exploration of structures and metrics in this thesis
will provide empirical evidence of the validity of the previously proposed metrics, in
reference to the World Wide Web.
Page 3
INTRODUCTION
__________________________________________________________________________________________
1.4 Definitions
The researcher has attempted throughout this thesis to provide the expansion of each
acronym used.
Examples: National Science Foundation Network (NSFNET)
National Center for Supercomputing Applications (NCSA)
CERN (Conseil Europeen pour la Recherche Nucleaire)
However, throughout this thesis there are several terms that are interchangeable, these
are listed below:
•
Favorite (Microsoft) and Bookmark (Netscape): a collection of user saved URLs
(Universal Resource Locators) which are addresses of Web sites on the World Wide Web.
•
Hierarchical and exploration structure: Structure that allows user access to nodes in an
undefined order. Hierarchical structures are classified as strict, compromised or
overlapping, defined in Section 2.3.3.1
•
Homepage and root node: the initial page (node) on entry to a World Wide Web site or
HyperText document. Also can be the apex of a hierarchical structure.
•
Hyperlink and link: each Hyperlink has a source, which the user may activate, and a
target (destination). The procedure of moving electronically from the source to the target
is known as traversing the link.
•
Node and page: node refers to a document in HyperText and page refers to a document on
the World Wide Web. For the purposes of this thesis node and page are interchangeable.
•
Sequential and next structure: Structure that defines the order of user access to nodes,
defined in Section 2.3.3.2
•
World Wide Web, WWW, W3 and the Web. A massive, distributed hypermedia
environment that is part of the Internet, consisting of documents, sites and indexes
(Taylor, 1995).
Page 4
INTRODUCTION
__________________________________________________________________________________________
1.5 Methodology
The focus of this thesis is the effect of Web site structure on user perception of
navigability. To research this question the author constructed two experimental Web sites
(labeled “red” and “blue”), using different structures, to examine the users’ perceptions of
navigability.
The underlying structure of both sites is hierarchical, but different types of hierarchical
structure were utilized. The red site is a strict hierarchy (Section 2.3.3.1) which contains a
total of 72 Hyperlinks. The blue site uses compromised and overlapping hierarchies (Section
2.3.3.1) as well as sequential structures (Section 2.3.3.2). The blue site contains a total of 736
Hyperlinks. The sites were specifically designed to be at the extremes of Botafogo et al’s
(1992) metrics.
student body (undergraduate and postgraduate) of Southern Cross University. Each subject
was given two tasks and a questionnaire to complete for each of the two Web sites. Fifty
percent of subjects were directed to complete the red site first, while the remainder were
directed to complete the blue site first.
The questions posed in the questionnaire are designed to capture from the subjects
their perceptions of the structure of the two Web sites. Also captured in the questionnaire are
the user’s utilization of facilities intrinsic to the Web browser and general information about
the subject including age, gender and experience both with computers and the World Wide
Web.
The experimental data collected consisted of completed questionnaires and the server
log file for each subject in the experiment.
Page 5
INTRODUCTION
__________________________________________________________________________________________
1.5.1 Statistical Processes
Two types of statistical analyses were performed on the data collected in the
experiment.
The first analysis performed is a doubly multivariate analysis of variance - repeated
measures (Section 4.3). This analysis tests the significance of differences between the red and
blue Web sites on lost rating, usage of Back button, usage of History list, comfort rating,
number of nodes visited and time to complete the task (within subject factors). It also tests the
significance of the influence of the between subjects factors:
1. Experience with Computers.
5. Order (red or blue completed first).
2. Experience with the WWW.
6. Gender by Age.
3. Gender.
7. Gender by Order.
4. Age.
8. Age by Order.
Where the previously mentioned factors do not influence red – blue differences, the
effect of factors will be examined for overall (both Web sites) influence.
The second statistical analysis performed on the experimental data was a logistic
regression (Section 4.4). This was done to analyze the results of the experiment to find which
component(s) of the data predict Web site (red or blue) preference. The components analyzed
are:
1
Experience with Computers
7
Usage of History list
2
Experience with WWW
8
Lost
3
Gender
9
Comfortably
4
Age
10 Time to complete (task time)
st
5
Order (red or blue Web site 1 )
6
Usage of Back button
11 Visited nodes
Page 6
INTRODUCTION
__________________________________________________________________________________________
1.6 Limitations and Key Assumptions
Limitations:
1 The experiment was restricted to the campus of Southern Cross University as the
equipment (server and workstations) required to conduct the experiment were located on
campus grounds. The off campus delay in response time from the server would have
distorted the measure of time to complete the task for each Web site.
2 The experiment used Microsoft’s Internet Explorer 3.02 for browsing the experimental
Web sites. The most popular browser in use at time of writing this thesis was Netscape’s
Navigator (Webtrends, 1997; Section 2.2.2). Policy at the School of Multimedia and
Information Technology of Southern Cross University prevented usage of any version of
Netscape Navigator after 2.02. As Internet Explorer 3.02 was a much more recent release
it was chosen as the browser.
3 Each subject’s exact traversal path could not be tracked. This information would have
allowed the metrics of lostness and efficiency developed by Smith (1996), as described in
section 2.3.5, to be applied to each subject of the experiment. This would have tested
these metrics in relation to the Web with empirical evidence. Internet Explorer 3.02
caches each Web page as it downloads. The next time a cached Web page is accessed,
whether via the back, history list or another Hyperlink, Internet Explorer accesses the
client’s cache, not the server (host). As no GET command is issued to the server by the
browser for accessing a previously visited node these data could not be collected.
Assumptions:
1 It is assumed that the university students (undergraduate and postgraduate) and staff
represent all WWW users in their ability to navigate Web sites.
2 It is assumed that any person who knows more than 20 comic book characters represented
in the content (Section 3.2.2) of the Web sites will know one or more of the answers to
the tasks set (Section 3.5.3 and Section 3.5.4). Therefore this subject should be removed
from the sample.
Page 7
INTRODUCTION
__________________________________________________________________________________________
3. It is assumed that a user’s perception of navigability is a function of perceptions lost,
comfort and preference. Users that experience a low level of lost, high level of comfort
and prefer a particular structure are considered to perceive that structure as more
navigable. The inverse of this condition, users whom experience high level of lost, low
level of comfort and do not prefer a particular structure are considered to have
experienced the “lost in hyperspace” phenomenon within this structure.
1.7 Outline of Thesis
This thesis has five chapters. Chapter 1 is designed to establish the reasons for,
framework and design of the intended research work.
Chapter 2 is a literature review that will explore the following areas:
•
History of the W3.
•
The protocols and standards associated with the W3.
•
The common browsers used to access the W3.
•
Strategies for navigating the W3.
•
History of HyperText.
•
Hyperlinks.
•
Previous research into HyperText structures.
•
The metrics associated with HyperText.
The previous research into HyperText structure and HyperText metrics will then be
related to Web sites.
Chapter 3: Methodology, provides a complete description of the experimental design
in relation to the Web sites, browser and subjects. Also examined, is the design of the
questionnaire and additional data collected.
Page 8
INTRODUCTION
__________________________________________________________________________________________
Chapter 4: Analysis of Data, examines in detail the two analyses performed and
reports the results of these analyses (Section 4.2 to Section 4.4). Chapter 4 also proposes a
modification (Section 4.5) to metrics developed by Botafogo, Rivlin and Scheiderman (1992;
1994), for strictly hierarchical Web sites. In addition to this, Chapter 4 compares experimental
evidence with the HyperText lostness and efficiency indicators (Section 4.6) developed by
Smith (1996) in relation to Web sites.
Finally Chapter 4 answers the four questions that were raised in Chapter 2;
•
Do users utilize the features of modern browsers to aid navigation within a Web site?
•
Which structure would users prefer - heavily cross-linked or a strict hierarchy?
•
Which type of Web site - heavily crossed-linked or strictly hierarchical - increases the
user’s likelihood of experiencing the “Lost in Hyperspace” phenomenon?
•
Is the previous research in HyperText metrics still valid, when applied to Web sites
Chapter 5: Summary and Conclusions, reports rejection of the null hypothesis (HO)
and supports the alternative hypothesis (HA) of this thesis. Chapter 5 also examines the four
additional questions raised in researching the thesis’ overall hypothesis. Finally, Chapter 5
recommends areas of further research arising from the findings of this thesis.
Page 9
INTRODUCTION
__________________________________________________________________________________________
1.8 Summary
This chapter has presented the background of the research to be undertaken in this
dissertation. It has introduced the research question and hypothesis. The purpose of the
research has been established, as well as the benefits to be obtained from the completion of
the research.
The method of obtaining the required data has been briefly outlined, as well as the
statistical processes the data will be subjected to. The limitations and assumptions of the
research have been stated.
An overview of the thesis was also presented describing what will be accomplished in
each chapter of the thesis. The next chapter of this thesis will review the relevant literature
associated with the overall research question.
Page 10
LITERATURE REVIEW
_________________________________________________________________________________________________________
Chapter 2
LITERATURE REVIEW
2.1 Introduction
The popularity of the Internet continues to grow in the global community. Once the
exclusive domain of academia and scientists the Internet has also become, with the advent of
HyperText and the World Wide Web (W3), a global pool of information for education,
marketing, hobbyists, on-line publishing, business and many other special interest groups.
As the W3 continues to grow the structure of Web sites (defined in section 2.4.1)
becomes increasingly important as Web site developers desire users to be able to quickly
locate the information they seek. Users can quickly be overwhelmed with the vast array of
links available on each Web site. This in turn can cause the user to experience the sense of
“lost in hyperspace" (Shneiderman & Kearsley, 1989, p.49). This condition is a result of the
user experiencing disorientation from traversing a complex structure.
The focus of this thesis is the effect of Web site structure on the users’ perceptions of
navigability. In order to facilitate this focus, the following areas will be explored:
•
History of the WWW.
•
The protocols and standards associated with the W3.
•
The common browsers used to access the W3.
•
Strategies for navigating the W3.
•
History of HyperText.
•
Previous research into HyperText structures.
•
The metrics associated with HyperText.
The previous research into HyperText structure and HyperText metrics will then be
related to Web sites.
Page 11
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.2 Rise of the World Wide Web (W3)
The origin of the Internet can be traced back to the early 1960s (Appendix A, A Brief
History of the Internet). The WWW is perhaps, the Internet’s most widely known application.
This section of the thesis will explore the history of the W3.
The W3 was first proposed in the paper “Information Management: A Proposal” by
Tim Berners-Lee for CERN (Conseil Europeen pour la Recherche Nucleaire). This document
was followed in November of 1990 with the proposal “WorldWideWeb: Proposal for a
HyperText Project” co-authored by Tim Berners-Lee and Robert Calliau. The proposal
outlined a project to use HyperText to link and access information from reports, notes,
databases and on-line help systems. A user would be able to browse this information at will.
The project required two main building blocks, a browser and a server (Berners-Lee &
Calliau 1990).
The browser would be a native application running on the client machine. The
browser would display and allow navigation of the HyperText nodes (documents or pages).
Also, the browser would remember what links were traversed and how to go back to previous
visited nodes. The server would have a native application program that manages nodes on the
machine and negotiates presentation format with the browser.
In 1992 there were twenty-six (26) W3 servers in operation, browsers included
Erwise, Viola and Ver. 1.2 of the original Line mode browser (Calliau, 1995). In January
1993 there were 50 known W3 servers in operation. This number increases to 200 in October.
In February the National Center for Supercomputing Applications (NCSA) releases “Mosaic”.
In March W3 traffic measures 0.1% of National Science Foundation Network’s (NSFNET)
backbone this figure increases to 1% by September. (Calliau, 1995)
Page 12
LITERATURE REVIEW
_________________________________________________________________________________________________________
In 1994 Mosaic Communications Corporation (Now Netscape) was formed. The first
International WWW Conference is held at CERN, Geneva, Switzerland. The second
International WWW Conference is held in Chicago. Growth of W3 traffic on the NSFNET
moves to second place (11.7%) behind File Transfer Protocol (19.6%). By April of 1995 W3
traffic reaches first place (21.4%) well ahead of Telnet (7.5%) and FTP (14%). (NSFNET
Statistics cited in Berghel, 1996) The growth of WWW vs. Gopher traffic is shown
graphically below.
Diagram 2.1: Growth of Internet Browsing Services
Today the W3 has become a pool of knowledge and allows the collaboration between
remote sites, its creator Berners-Lee, intended. (Berners-Lee et al. 1994, p.76) However, the
World Wide Web has become much, much more.
Page 13
LITERATURE REVIEW
_________________________________________________________________________________________________________
“Personal pages and the web are not being used to ‘publish information’; they are
being used to construct identity—useful information is just a side effect. A personal page is a
carefully constructed portrayal of a person. This insight leads me to characterize the web as a
social hyper-text. The nodes—at least some of them—are becoming representations of
people.” (Erickson 1996, p.15)
Just as individuals are increasing their presence on the Web, so are corporations. The
W3 has now become a part of the commercial world. Web sites are now advertised on both
television and radio as part of overall marketing strategies. Examples of this are:
*
*
*
*
*
*
*
http://www.harveynorman.com.au (Harvey Norman)
http://www.toyota.com (Toyota)
http://www.cocacola.com (Coca-Cola)
http://www.microsoft.com (Microsoft)
http://www.coles.com.au (Coles Supermarkets)
http://www.ford.com (Ford)
http://www.paramount.com (Paramount Studios)
A large part of the future of the Web is tied to business interests as on-line shopping
continues to grow and the amount of business taking place on the web increases. The
following table shows the growth of the .com domain on the W3. A .com site is a commercial
site.
Table 2.1:
Domain .com growth 1993 - 1997
(Gray, 1996)
Month/Year
Number of Web sites
% of .com sites/all sites
06/93
130
1.5
12/93
623
4.6
06/94
2,738
13.5
12/94
10,022
18.3
06/95
23,500
31.3
01/96
100,000
50.0
06/96
230,000 (est.)
62.6
Page 14
LITERATURE REVIEW
_________________________________________________________________________________________________________
The W3 in its short history has grown into a gigantic, multifaceted tool for humanity.
However, to understand what the WWW is, it is necessary to understand the building blocks
that allow the Web to work.
2.2.1 Building Blocks of the WWW
The three main concepts that allow the W3 to work are:
URL (Uniform Resource Locator, a.k.a. Universal Resource Locator)
The address system of the W3, the URL contains the protocol to be used, address &
type of server, type of organization, location (geographically), and address within server of
web site. An example follows;
http://www.scu.edu.au/research/grc
1
2
3
4
5
1 (http://) Indicates http protocol is to be used on the string (see over page for a
description of this protocol)
2 (www.scu) Name of the server in this case the World Wide Web server at Southern
Cross University
3 (.edu) Type of organization, in this case an educational institution. Other common
types are: .com, (commercial) .org (organization) .gov (government)
and .net (network). (see Note below)
4 (.au) Geographic location, in this case Australia. Other examples are: .ca, (Canada)
.uk (United Kingdom) etc. The absence of geographic location indicates the
site is located in the United States of America.
5 (/research/grc) Address within the server of the W3 site. The “/” defines a
sub-directory within the server.
Note: On the 4th of February 1997 the International Ad Hoc Committee (IAHC) announced
the introduction of 7 new generic names. When the plan is implemented, the additions to the
existing domains will be .firm, .store, .web, .arts, .rec, .info and .nom .
Page 15
LITERATURE REVIEW
_________________________________________________________________________________________________________
HTTP (HyperText Transfer Protocol)
“Perhaps misnamed, rather than being a protocol for transferring hypertext, HTTP is
a protocol for transferring information with the efficiency necessary for making hypertext
jumps. The data transferred may be plain text, hypertext, images, or anything else.” (BernersLee et al. 1994, p.78)
Both FTP (File Transfer Protocol) and NNTP (Network News Transfer Protocol) have
been in use for years on the Internet. What makes HTTP different is that it is stateless. This
means that it runs over a TCP (Transmission Control Protocol) connection and is held for the
duration of a single operation. This stateless model is efficient when a link can refer to an
address not only within the same server, but could be contained within any other server as
well.
A client’s (browser’s) HTTP request starts with a method (GET, PUT, POST etc.) and
the URL of the object the client is addressing. The HTTP server (host) then returns the result
of the HTTP request to the client (browser) that requested it. This process is shown in figure
2.3.
Client (browser)
Web page
.
Link1
Request for Object
Server (host)
Object Located
Response
Time
Object
Object Returned
Object 2
displayed
1
2
Link can be graphic or text
Object being returned could be graphic, text,
multimedia or another Web page
Diagram 2.2: HTTP Process
Page 16
LITERATURE REVIEW
_________________________________________________________________________________________________________
HTML (HyperText Markup Language)
“Despite the ability of HTTP to negotiate formats, W3 needed a common basic
language of interchange for hypertext. HTML is that language, and much of the fabric of the
Web is constructed out of it. It was designed to be sufficiently simple so as to be easily
produced by both people and programs, but also to adhere to the SGML standard…”
(Berners-Lee et al. 1994, pp.78-79)
HTML 3.2 is the current standard for HTML. HTML 3.2 is an SGML application
conforming to International Standard ISO 8879 -- Standard Generalized Markup Language
(SGML). As an SGML application, the syntax of a HTML 3.2 document is defined by the
combination of the SGML declaration and the document type definition (DTD). This
specification defines the intended interpretation of HTML 3.2 elements, and places further
constraints on the permitted syntax which are otherwise inexpressible in the DTD (Ragett,
1997).
HTML is a language for formatting documents. HTML files are stored in ASCII
(American Standard Code for Information Interchange) format and contain tags that can be
read (parsed) by a browser. These tags refer to the formatting of the document. Some
examples of the tags are;
•
•
•
•
•
•
•
•
•
•
•
<HTML> … </HTML> encloses the entire HTML document.
<HEAD> … </HEAD> encloses head of HTML document.
<BODY> … </BODY> encloses body (text and tags) of HTML document.
<TITLE> … </TITLE> title of HTML document.
 … a plain paragraph.
 … boldface text
 … italic text
<A> … </A> creates link or anchor uses HREF = “…” for URL of object linked.
 … change size of font for enclosed text uses SIZE = “1 to 7”.
<HR> horizontal rulers line.
 a line break.
Having briefly looked at the three main concepts that define the W3, the next section
of this thesis will explore the browsers commonly used to access the W3.
Page 17
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.2.2 Browsers and Navigation
There are many different browsers available to “surf” (browse) the W3. However,
only the four most common will be explored. Particular emphasis is placed on Netscape’s
“Navigator” and Microsoft’s “Internet Explorer” as at the time of writing these two browsers
account for over 96% of all browsing on the Web. This is demonstrated in the partial
reproduction of the statistics from WebTrends in Table 4.
Table 2.2:
WebTrends (tm) Complete Summary Report
Web Browsers Report (Nov. 1996- Feb. 1997)
(Webtrends, 1997)
Ranking
Browser
1
Netscape Navigator
2
Microsoft Internet Explorer
Hits
% of Total
User Sessions
1462377
65.89%
46883
693435
31.24%
24856
………………………………………………………………………………………….
13
Lynx
1005
0.04%
420
………………………………………………………………………………………….
32
NCSA Mosaic
239
0.01%
33
Statistics based on browser access to WebTrends World Wide Web (HTTP) server.
In the examination of each of the four browsers, the features that support navigation of
the Web will be given specific attention.
Lynx
Lynx was one of the first W3 browsers available, it was developed by Lou Montulli,
Charles Rezac and Michael Grobe of Academic Computing Services at The University of
Kansas. The latest version of Lynx (at time of writing this thesis) is Version 2.3. (Blythe et
al., 1996)
Page 18
LITERATURE REVIEW
_________________________________________________________________________________________________________
Lynx is a text-based browser: no graphics, no sound, and not even varied fonts. The
result of not supporting multimedia is that Lynx is very fast. Lynx, even though it does not
support real-time multimedia, allows links to multimedia to be followed and downloaded.
Typically, Lynx is an application for UNIX or VMS platforms. Remote users dial up host,
using a VT100 terminal emulator to access the Lynx browser. Lynx uses only text input for
control of the system.
Navigation aids in Lynx:
Display - Current URL shown in text displayed to screen.
Back - Return to previous Web page. Accessed by the left cursor key.
History List - Displays the title of each Web page, in order of access, visited in
current browsing session. Accessed by the backspace key.
Bookmarks - User can add or visit a bookmark. The bookmark file contains the URLs
of the W3 locations. Accessed by the “A” key to add a bookmark. The “V” key is used
to display both the bookmarks and the selection menu.
Mosaic
The National Center for Supercomputing Applications (NCSA) released Mosaic in
1989, free of charge to the Internet community. The current version of Mosaic in use (at time
of writing this thesis) is 2.6. Mosaic has full multimedia capability and uses a mouse as the
primary navigation device (NCSA, 1997). The Mosaic browser has versions available for
Windows, Macintosh and X-Windows (UNIX).
Navigation aids in Mosaic:
Display - Document title and URL are displayed.
History List - Contains the URLs of the current session. Both Back and Forward
facilities use the information contained within this list to perform their function.
Back - Loads the last Web page visited in history list.
Forward - Loads next Web page in history list.
Home - Returns to default Web page (set by user).
Hotlist – User’s list of Web pages.
Page 19
LITERATURE REVIEW
_________________________________________________________________________________________________________
Netscape Navigator (a.k.a. Mozilla)
Jim Clark and Marc Andreessen
founded Mosaic Communications
Corporation (later Netscape
Communications Corporation) in April
1994. Marc Andreessen developed the
idea for the NCSA Mosaic browser for
the Internet in the fall of 1992 while he
was an undergraduate student at the
University of Illinois and a staff member
at the university's National Center for
Supercomputing Applications in
Champaign, Illinois. (NCC, 1997)
Diagram 2.3: Netscape Navigator 3.01
At the time of writing this thesis, the current version of Navigator is 3.01. Navigator
has full multimedia capability and allows further development through the use of third party
plug-ins. Navigator uses a point and click environment, with the mouse as the primary
navigation device. The Netscape browsers have versions for Windows, UNIX (9 different
versions), OS/2 Warp, Open VMS and Macintosh.
Navigation aids in Navigator include:
Bookmarks – User’s list of saved Web pages.
Link / Visited Link - All HyperText links are shown in coloured text (typically blue),
if user has previously accessed this URL a different coloured text is displayed
(typically purple).
Page 20
LITERATURE REVIEW
_________________________________________________________________________________________________________
Microsoft Internet Explorer
Internet Explorer appeared in
August 1995 with the introduction of
Windows 95. Originally based on Mosaic
technology, Internet Explorer has rapidly
evolved into a full-featured browser.
Originally developed only for Windows
95, Internet Explorer has been modified
to operate with Windows 3.11,
Macintosh and Windows NT.
At the time of writing this thesis,
the current version of Internet Explorer is
3.02. Microsoft’s Internet Explorer uses
Diagram 2.4: Microsoft Internet Explorer 3.02
a point and click environment, with the mouse as the primary navigation device.
Navigation aids in Internet Explorer include:
Favorites – User’s list of saved Web pages.
Link / Visited Link - - All HyperText links are shown in coloured text (typically
blue), if user has previously accessed this URL a different coloured text is displayed
(typically purple).
Having explored the four most common browsers used to navigate the W3, with a
particular emphasis on their navigational aids, the next section of this thesis will examine
navigation of the World Wide Web.
Page 21
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.2.3 Navigation
Current navigation on the Web depends on the strategy adopted by the user. The 6th
Graphic, Visualization & Usability Center (GVU) WWW user survey asked respondents:
“People browse the Web in many different ways. Please put a check against all of the
following that accurately describe how you like to browse the Web.
Go to links from my Favorites/Bookmark
Type in the URL I want to go to
Go from one link to another, page to page - a happy-go-lucky browser
Locate the URL through search pages - like Alta Vista, Lycos, etc.
Locate the URL thought Meta-Indexes - like Yahoo, McKinley, etc.”
(GVU, 1996)
Strategies and percentage of usage that have been adopted by users were:
•
Favorite/Bookmark (users revisiting pages they have added to their favorite/bookmark
file). 82.69%
•
Index (using search engines such as Lycos). 78.1%
•
Meta-index (using large indices such as Yahoo). 59.45%
•
Opportunistic (following links from page to page as they are encountered). 64.16%
•
URL (typing in known URLs). 69.44%
What is of particular interest to the researcher is the fact that regardless of what
strategy the user adopted, once the Web site is located, the user must interact with the
structure of the site. Also of interest, is whether users utilize the navigational aids inherent to
their browser to assist in searching a Web site structure.
Navigation on a Web site consists of the traversal of HyperText style links between
documents (pages). The structure of a Web site is similar to HyperText document structure.
There is a considerable amount of literature on HyperText structures. The next section of this
thesis will review the research on HyperText.
Page 22
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.3 HyperText
This section of the thesis will examine the history, links, structure types and metrics
associated with HyperText. Since the WWW is a modified form of HyperText, the structures
and metrics examined in this section will then be related to Web sites in section 2.4.
2.3.1 HyperText History
Vannevar Bush provided the first detailed account of a HyperText system in 1945,
while Director of the U.S. Office of Scientific Research and Development, and a trusted
science advisor to the President. Bush described a device he called “Memex” (Bush, 1945).
In 1965 Ted Nelson was writing about hypermedia (he is credited with coining the
terms HyperText and Hypermedia), with an emphasis on the literary point of view. Nelson is
also famous for his description of Xanadu - a worldwide network of computers with access to
a database of all of the world's knowledge. Nelson spent much of his time in the 70s & 80s
attempting to bring Xanadu to market as a commercial system. Xanadu was very much a
forerunner to today's WWW (Phelps, 1994; Nelson, 1992).
Some other early and classical HyperText systems were:
(Calliau, 1995; Brown, 1991; Shneiderman, 1992; Rada, 1991)
• NLS system (demonstrated by Doug Engelbart in 1968)
• HyperText Editing System (Andries van Dam at Brown University in 1968)
• ZOG, or the Knowledge Management System - KMS (Allen Newell and others at
Carnegie-Mellon University in 1972)
• Dataland (Nicholas Negroponte and Richard A. Bolt at MIT in 1976)
• OWL (On With Learning) Guide (Peter Brown at the University of Kent in 1986, and later
OWL International as a commercial product)
• Hypercard (Bill Atkinson at Apple Computer in 1987)
Page 23
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.3.2 HyperText and Hyperlinks
Information in a HyperText document is threaded together with links. These links are
referred to as Hyperlinks. The term Hyperlink can also be used as a verb, as in, one part of a
document is hyperlinked to another. The user of a HyperText document need not progress
sequentially through a document but instead, may make electronic leaps from one part to
another (Martin, 1990; Nelson, 1992).
Each Hyperlink has a source, which the user may activate, and a target (destination).
The procedure of moving electronically from the source to the target is known as traversing
the Hyperlink (Martin, 1990; Nelson 1992).
A source of a Hyperlink is usually one of the following:
*
*
*
*
*
A word in text.
A contiguous group of words in text.
A marked area of a diagram.
A label on part of a diagram.
A graphic.
The target (destination) of a Hyperlink is usually one of the following:
•
•
•
•
•
•
A line of text.
A segment of a HyperText document.
Another document.
A graphic.
A video, audio or animation sequence.
A program.
When a Hyperlink points to another document, the user can traverse the link to the
destination document. These types of links are sometimes known as “semi links” (Pam,
1995). They are univisible (seen from the originating end) and unifollowable (followed in one
direction).
Ted Nelson’s Xanadu incorporated bivisible and bifollowable links. A bivisible link is
a link that is visible from the source document and the destination document. A bifollowable
link can be traversed from the source document to the destination document and vice versa
(Deemer, 1994; Nelson, 1992).
Page 24
LITERATURE REVIEW
_________________________________________________________________________________________________________
A document containing Hyperlinks highlights the Hyperlinks either with reverse video
or colour. For a user to traverse these links, a pointing device (keyboard, mouse etc.) guide a
cursor displayed on the screen over the highlighted Hyperlink, the user then activates (clicks)
the pointing device. As a user makes choices presented to them in a HyperText system, they
are traversing between the source and the destination of each link followed. This process is
commonly referred to as navigating the HyperText structure.
2.3.3 HyperText Structure
Hierarchical or tree hypermedia can be particularly useful at a user’s conceptual level
by providing manageable structure in large documents or applications; at progressively
higher levels in the hierarchy increasing abstraction occurs, thus decreasing complexity. …
This is a common default structure in commercial hypermedia systems.”
(Woodhead, 1990, p. 121)
The comprehension and navigation of a HyperText document depends on the reader's
ability to construct a coherent mental representation. It is the HyperText author’s
responsibility to ensure the construction of the HyperText document as a coherent entity. The
construction of a coherent HyperText document can be considered to be a design problem.
Guidelines have been developed, for example, by Thuring et al., for the construction of a
coherent HyperText document. Such a document should consist of the following three
components - the content part, the organizational part, and the presentation part (Thuring et
al., 1991).
The content part contains design objects that carry information. They are content nodes
and content links. Content nodes are nodes that contain the Hyperdocument’s actual
information (content).
Content links are links that join content nodes based on a semantic relationship between
the nodes. The content link (a.k.a. cross-reference link) is part of the structure of a HyperText
document and is also part of the content of the document. This type of link is easy to
understand in context, but can if used excessively damage the user’s understanding of the
structure of the HyperText document (Woodhead, 1990).
Page 25
LITERATURE REVIEW
_________________________________________________________________________________________________________
Design objects of the organizational part increase coherence by structuring the
Hyperdocument in a reader-oriented perspective. Structures organize content nodes and links
in a specific manner. There are two main structures that can be utilized: sequential (next) and
exploration (hierarchical).
The presentation part is concerned with the actual display of structure and content and
provides an aid to navigation. Authors can adopt one of three styles:
*
Textual Style: Presentation is limited to the textual display of the content of nodes.
*
Graphical Style: There is a graphical display, such as a map of the structure.
*
Combined Style: Both text and graphics are used to provide a map of the site.
The organizational and context parts are the structure of the HyperText. The
presentation part is a map (textual or graphical) provided to increase the user’s understanding
of the structure of the HyperText document.
Thuring et al. have suggested only two types of HyperText structures: exploration and
sequence. While sequencing constrains the reader's navigation through the document,
exploration nodes and links allow unconstrained access to content in a hierarchical manner
(Thuring et al., 1991).
Woodhead previously had suggested that the hierarchical model has three subversions, being strict hierarchy, compromised hierarchy and overlapping hierarchies.
As the structure of Web sites is the focus of this thesis, the concepts behind
exploration (hierarchical) and sequential structures will be explored further in the following
sections of this thesis (Sections 2.3.3.1 and 2.3.3.2).
Page 26
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.3.3.1 Exploration (Hierarchical) Structure
Exploration structure allows the reader to explore the content of a document. The
reader can simply follow the content links to explore the content of nodes. Exploration links
provide access to exploration nodes. An exploration link can be embedded into a sequencing
node or an exploration node, and points to the beginning of an exploration node. Diagram 2.6
shows an exploration structure. On entry to node 1 the user has access, in any order, to the
content in nodes 2 to 8.
1
2
3
4
5
6
7
8
Diagram 2.5 Exploration (Hierarchical) Structure
Woodhead previously elaborated on hierarchical structure by introducing strict,
compromised and overlapping hierarchies (Woodhead, 1990 p. 122). Diagram 2.7 below
displays a strict hierarchy, all links in the diagram are the type of link Thuring et al. calls a
exploration link.
Diagram 2.6 Strict Hierarchy
Page 27
LITERATURE REVIEW
_________________________________________________________________________________________________________
A compromised hierarchy according to Woodhead, is a hierarchy that contains loops
and cross-referential links. Diagram 2.8 demonstrates this concept visually. Note that in
diagram 2.8 “A” is a cross-reference link and “B” is a loop link.
“A”
“B”
Diagram 2.7 Compromised Hierarchy
The final structure suggested by Woodhead (1990) is overlapping hierarchies. This
type of structure, contains several hierarchical structures that are cross-referenced diagram
2.9 displays this concept.
Diagram 2.8 Overlapping Hierarchies
Page 28
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.3.3.2 Sequential (Next) Structure
Sequential nodes and links allow the author to define the reading sequence through
the content. Readers can read only those content nodes that are determined by the sequencing
structure. Sequencing links associate the content of each sequencing node with a presentation
sequence. Sequencing links can be used to define ordering such as linear sequence through
the content. Diagram 2.10 shows a sequential structure. On entry to node 1 the user is forced
to follow the author’s path to node 8, through the content in a linear manner.
1
2
3
4
5
6
7
8
Diagram 2.9 Sequential (Next) Structure
The sequential structure can be used to provide an author’s guided tour of a
HyperText document.
Now that an understanding of the structures in HyperText has been achieved several
questions were raised in the researcher’s mind:
•
Which structure would users prefer -- heavily cross-linked or a strict hierarchy?
•
Which type of structure -- heavily cross-linked or strictly hierarchical -- increases
the users’ likelihood of experiencing the “Lost in Hyperspace” phenomenon?
The next section of the thesis will examine the metrics available to measure the
structure of a HyperText document.
Page 29
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.3.4 HyperText Metrics
HyperText metrics were developed by Botafogo et al. (1992) to provide authors with
useful measures of the properties of nodes and the entire HyperText structure. The metrics
developed by Botafogo et al. (1992) include:
•
•
•
•
•
•
•
CDM
COD
CD
ROC
Cp
St
LAP
Converted Distance Matrix
Converted Out Distance
Converted Distance
Relative Out Centrality
Compactness
Stratum
Linear Absolute Prestige
In this section of the thesis each of the metrics developed by Botafogo et al. (1992)
and Rivlin et al. (1994) will be examined.
2.3.4.1 Converted Distance Matrix (CDM)
A distance matrix contains within its entries the distance of each node to every other
node (shortest path). An example of a graph with its distance matrix is represented below by
Figure 2.7.
a
b
c
a
b
c
d
a
0
1
1
2
b
∞
0
1
2
c
∞
∞
0
1
d
∞
∞
∞
0
∞ = infinity
d
Diagram 2.10: Distance Matrix
The shortest path from node a to node b is one. When a node can not reach another
node, then an infinity value is entered within the distance matrix. For example in Figure 2.7,
node b can not reach node a so an infinity entry is inserted.
Page 30
LITERATURE REVIEW
_________________________________________________________________________________________________________
Due to the inconvenience of working with infinite values Botafogo et al. (1992)
modified the distance matrix. They introduced a conversion constant (K) to replace the
infinite values. Botafogo et al. (1992) assigns the value for the conversion constant to be at
least the number of nodes within the HyperText structure. Diagram 2.8 displays the same
graph used in diagram 2.7 however, infinite values have been replaced with the conversion
constant.
a
b
c
a
b
c
d
a
0
1
1
2
b
4
0
1
2
c
4
4
0
1
d
4
4
4
0
K = 4 (4 nodes in graph)
d
Diagram 2.11: Converted Distance Matrix (CDM)
2.3.4.2 Converted Out Distance (COD)
The COD for a node is the sum of all of the entries in a row of the converted distance
matrix. For example, the COD value for node a would be 0 + 1 + 1 + 2 = 4. The node with
the lowest COD value is regarded as the central node, because there are fewer traversals to
reach the majority of nodes within the HyperText structure. COD is used to identify
hierarchical structures within a HyperText document. Figure 2.9 shows the COD table.
a
b
c
a
b
c
d
COD
a
0
1
1
2
4
b
4
0
1
2
7
c
4
4
0
1
9
Note: Node a is the
d
4
4
4
0
12
most central.
K =4
d
Diagram 2.12: Converted Out Distance (COD)
Page 31
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.3.4.3 Converted Distance (CD)
The CD value of Hyperdocument is required to calculate ROC (Relative Out
Centrality) and Cp (Compactness) values for a HyperText document (Sections 2.3.4.4 and
2.3.4.5). The CD of a HyperText structure is calculated by taking the sum of all COD values.
For example, the CD for the converted distance matrix in figure 2.13 is calculated by:
CD = COD (node a) + COD (node b) + COD (node c) + COD (node d)
4 + 7 + 9 + 12 = 32
2.3.4.4 Relative Out Centrality (ROC)
“It is natural to define a central node as one whose distance to all the other nodes in
the hypertext is small. As that distance grows, nodes become less central. Consequently, the
smaller the COD the more central the node. For a single hypertext, the COD is a good
indication of the node centrality as compared with another node, but this number indicates
little when two different hypertexts are compared.”
(Botafogo et al., 1992, p.147)
For comparisons of centrality between HyperText structures Botafogo et al. (1992)
developed the ROC metric. Relative Out Centrality of a node is calculated by using the
formula: ROC i = CD/COD. This allows direct comparisons between different HyperText
structures. The highest ROC is the most central node in the structure. For example, Diagram
2.10 shows all calculated ROC values.
a
b
c
a
b
c
d
COD
ROC
a
0
1
1
2
4
8
b
4
0
1
2
7
4.6
c
4
4
0
1
9
3.6
d
4
4
4
0
12
2.7
CD
32
d
Diagram 2.13: Relative Out Centrality (ROC)
Page 32
K=4
Note: Node a
has the highest
ROC value and
is the most
central node.
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.3.4.5 Compactness (Cp)
This metric indicates whether each node can easily be reached from another node. The
compactness metric has a value between zero and one regardless of the size of the HyperText
structure. As the Compactness value moves closer to one the more connected the HyperText
structure is. If the Cp value is equal to one (1) the structure is completely cross-connected. If
the value of Cp is equal to zero (0) the structure is completely disconnected. Compactness is
calculated using the following formula:
Cp = (MAX - CD) / (MAX - MIN)
Where
MAX = (n2 - n)K and MIN = (n2 - n)
n = number of nodes
K = conversion constant (Section 2.3.4.1)
For example, Diagram 2.11 shows CD = 32, n = 4 and K = 4
Cp = ((( 42 - 4 )4) - 32) / (((42 - 4)4) - (42 - 4))
Cp = ((12 * 4) - 32) / ((12*4) - 12)
Cp = 16 / 36
Cp = .44
2.3.4.6 Stratum (St)
The stratum metric was designed to reflect the linear ordering of a HyperText
structure. This indicates whether it is important to begin reading the HyperText from a
particular node. A completely linear structure has a stratum value of one (1), a completely
cross-connected structure will have a stratum value of zero (0).
The stratum of a HyperText structure is calculated using the formula:
Stratum = absolute prestige / LAP
Page 33
LITERATURE REVIEW
_________________________________________________________________________________________________________
Status for a node is the sum of the node’s row in the distance matrix. Infinite
values are ignored (see diagram 2.14).
Contrastatus for a node is the sum of the node’s column in the distance matrix.
Infinite values are ignored (see diagram 2.14). Prestige of node is calculated by subtracting
the contrastatus of the node from the status of the node. Absolute prestige is the sum
(regardless of sign) of the prestige of all nodes in the HyperText structure.
Linear Absolute Prestige is given by the following formula:
LAP = n3 / 4
if n is even
LAP = n3 - n / 4
if n is odd
n = number of nodes
Figure 2.12 shows a worked example of calculating the stratum of a HyperText
structure.
a
b
a
b
c
a
0
1
1
2
4
4
b
∞
0
1
2
3
2
c
∞
∞
0
1
1
-1
d
∞
∞
∞
0
0
-4
Contrastatus
0
1
2
5
c
d
d
Status Prestige
11 = Absolute prestige
LAP = 43 / 4 , as, even number of nodes.
Stratum (St) = absolute prestige / LAP
LAP = 64 / 4
St = 11 / 16
LAP = 16
St = .69
Diagram 2.14: Worked Example of Stratum
Page 34
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.3.4.7 Summary of Structure Metrics
When this set of metrics was developed the authors Botafogo, Rivlin and
Shneiderman (1992 and 1994) had no exposure to modern browsers. Due to this fact, the
metrics fail to take into account the increased freedom of browsing that the navigational aids,
inherent to the browser, give the user.
Particularly, the design of the Converted Distance Matrix (CDM) neglects the Back
and History facilities of all modern browsers. When the Back facility is applied to the CDM
all Hyperlinks present within a Web site become two-way links, thereby removing the need
for the infinity value or the conversion constant (Section 2.3.4.1) in the CDM. Similarly the
History facility allows the user to return to a previous node in a Web site. This change would
affect the results of all Botafogo et al. (1992) metrics when applied to a Web site. So the
question raised in the researcher’s mind is; Are these metrics valid, when applied to Web
sites accessed by modern browsers? A modification to the CDM will be proposed in Section
4.5.
2.3.5 Lost in Hyperspace - Metrics
“Hypertext systems are used in many applications because of their flexible structure
and the great browsing freedom they give to users. However, this same flexibility and freedom
is the cause of a major concern: the "lost in hyperspace" problem.” (Botafogo et al., 1992,
p.142)
HyperText systems are designed both to facilitate the finding of relevant information
when it is required and to encourage exploration of the information space. Nevertheless, a
system, which is designed to support browsing, can fail if the users become lost when they
use it. As the focus of this thesis is user perception of structure of Web sites, an understanding
of measuring, how lost the user is within the structure is desirable.
Page 35
LITERATURE REVIEW
_________________________________________________________________________________________________________
It is assumed that users, who cannot locate information that exists in the system, can
be regarded as lost and that users who become lost cannot use the system effectively when
they have a specific information-finding task to perform (Smith 1996).
Clearly, 'lostness' is something that needs to be measured.
(Smith, 1996, p. 366)
Smith (1996) created metrics to measure lostness and efficiency. Both metrics rely on
the path taken by user to reach the information required within a HyperText document. Table
5 contains the path measurements required for Smith’s metrics.
Table 2.3:
The path measurements from which indicators of lostness and efficiency are derived
D
R
S
N
V
T
Number of different nodes accessed
Number of nodes, which need to be visited to complete a task
Total number of nodes visited whilst searching
Number of different nodes visited whilst searching
Total number of nodes visited whilst verifying (checking information)
Total number of nodes accessed (S + V)
Lostness Indicators:
1. Different nodes accessed (D) compared with total number of nodes accessed (T).
lostness = D/T
A value approaching zero shows many repeated visits to the same nodes, which suggests the
user may be lost. Nodes visited whist verifying (V) are removed from total (T).
2. Number of visited nodes to complete required task (R) compared with number of different
nodes visited while searching (N).
lostness = R/N
A value approaching zero or greater than one (task incomplete), indicates the user is lost.
Page 36
LITERATURE REVIEW
_________________________________________________________________________________________________________
Lostness rating: L
Whilst each of the previous measures might be used alone as an indicator of lostness, a
more accurate picture is derived if they are combined to give a rating of lostness compared to
a user performing a “perfect search”.
Assuming that in a perfect search a user visits exactly the number of nodes assessed as
being required to complete a task (no deviation, no verifying visits) then:
T = D = R and S = T
therefore
N/S = 1 and R/N = l
If this is considered as the point (1,1) in a two-dimensional space which has its axis
N/S and R/N then a lostness rating (L) can be calculated as a distance from this point such
that:
L=
((N/S - 1) 2 + (R/N - 1) 2)
Where L increases as lostness increases. For the perfect search L = 0.
Efficiency Indicators:
1. Different nodes accessed (D) compared with total number of nodes accessed (T).
efficiency = D/T
A value of one shows perfect efficiency. Nodes visited whist verifying (V) are included in
total (T).
2. Number of visited nodes to complete required task (R) compared with number of different
nodes visited while searching (N).
efficiency = R/N
A value of one shows perfect efficiency. A value greater than one indicates task was not
completed.
Page 37
LITERATURE REVIEW
_________________________________________________________________________________________________________
3. When considering a user’s efficiency, successful completion of the task is a good indicator.
A value is assigned for task accuracy (A), 0 for incorrect answer, .5 for a partially correct
answer, and 1 for a correct answer.
Efficiency rating: E
Whilst each of the measures above might be used alone as an indicator of efficiency,
when combined they give an overall rating of a user’s efficiency compared to a perfectly
efficient search. Assuming that in a perfect search a user visits exactly the number of nodes
assessed as being required to complete a task successfully then:
T = D = R and A = 1
If this is considered as the point (1,1,1) in a three-dimensional space then the
efficiency rating which has its axis D/T, R/D and A then a efficiency rating (E) can be
calculated as a distance from this point such that:
E =
((D/T - 1) 2 + (R/D - 1) 2 + (A-1) 2)
Where E increases as efficiency decreases, and for a perfectly efficient search E = 0.
Section 4.6 of this thesis will examine the indicators of lostness and efficiency
proposed by Smith (1996) in relation to Web sites rather than HyperText.
Now that an understanding of the research into HyperText structure and the metrics
associated with HyperText has been achieved, the next section of the thesis will relate the
previous research on HyperText structure and metrics to the structure of Web sites.
2.4 Web Sites
How well do the theories of HyperText structure and metrics examined relate to Web
sites? First a Web site will be defined, then a comparison will made between the design of a
Web site and the design of a HyperText document. Particular emphasis will be placed on the
structure and metrics examined in the previous sections of this chapter.
Page 38
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.4.1 Web Site Definition
What is a Web site? This thesis adopts the definition proposed by Abrams in his
masters thesis (1997) that a Web site is a thematic organization. That is, all pages on the site
are organized by a theme with a clear subject domain. Thematic organizations contain
structural clues and explicit semantic relations.
Structural Clues: all Web pages provide graphical design, layout, or rhetorical clues,
which reflect the structure and content of the entire Web site.
Explicit Semantic Relations: HyperText links between Web pages within the site
represent semantic relationships from source pages to target pages.
For example, a person’s home page and its supporting pages are a Web site. The
Southern Cross University World Wide Web site can be classified as a single Web site, as its
pages have been organized thematically, have structural cues, and represent semantic
relationships through HyperText links.
2.4.2 Web Site Structure
The types of structures examined in HyperText are the same as the structures used by
Web site developers. Most Web sites have a hierarchical or sequential structure, or a
combination of the two. Sequential structures in some cases have been overlaid on a
hierarchical Web site to provide a guided tour through the structure of the site for first time
users.
Some Web sites also contain keyword searches, indexes, multimedia and database
access. These additions to the Web sites can largely be fitted into the structures examined in
the previous sections of this chapter. Each can be considered to be just another node within
the site. All of the aforementioned additions could arguably be called exploration nodes
(Thuring et al., 1991). Indexes and keyword searches are perhaps the ultimate extension of
the exploration node, as they contain Hyperlinks to all or most of the pages in the Web site.
Page 39
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.4.2.1 Web Site Structure - Style Guidelines
The author of this thesis searched the World Wide Web, scientific journals and
academic papers for up-to-date work on users’ perceptions of Web site structures. None was
found. However, information was found at many WWW locations in the form of “Web Site
Style Guides” following are a few examples:
“If you have in mind a body of information to put across to your reader, you probably
have a mental organization for it. Normally this is a sort of hierarchical tree, like the chapters
of a book if you were to write a book. Keep this structure. It helps readers to have a tree
structure as a basis for the book: it gives them a feeling of knowing where they are. You can
also use this structure for organizing your files in directories.”
(Berners-Lee, T., 1995)
“If yours is an essentially hierarchical document having chapters or other predictable
sections, consider adding a heading on each page that links back to beginning of the
document. Also, for pages within each chapter or section, add a secondary header that takes
readers back to the beginning of that chapter.”
(Sun Microsystems, 1996)
“Information hierarchies are one of the best ways to organize complex bodies of
information. Hierarchical organization schemes are particularly well-suited to Web sites,
because Web sites should always be organized as off-shoots of a single home page.” …
“Except in sites that rigorously enforce a sequence of pages, your users are likely to use any
Web site in a free-form "web-like" manner, just as most non-fiction or reference books are
used. But the nonlinear usage patterns typical of Web surfers do not absolve you of the need
to organize your thinking and present it within a clear, consistent structure that complements
your design goals for the site”
(Lynch, P.L.& Horton, S., 1997)
While these and other style guides provide detailed guidance and experiential reports
on the structure of Web sites. The style guides do not address the central issue of this thesis,
the effect of Web site structure on the users’ perceptions of navigability.
Page 40
LITERATURE REVIEW
_________________________________________________________________________________________________________
2.4.3 Web Site Metrics
The metrics developed by Botafogo et al. (1992) for analysis of HyperText
documents, when applied to the researcher’s experimental Web sites performed well (Chapter
3).
The Converted Out Distance (COD) and Relative Out Centrality (ROC) can clearly
identify the root nodes of hierarchies within the researcher’s experimental Web sites.
The Compactness (Cp) metric when applied to the experimental Web sites correctly
indicates the level of cross-referencing the Web site contains.
The Stratum (St) metric indicates how important it is to access a Web site in a
particular order. The experimental Web sites, to which this metric was applied, were not
designed to be accessed in a linear manner. The result clearly shows this.
As previously stated (Section 2.3.4.7) when this set of metrics was developed the
authors had no exposure to the modern browsers used to access the World Wide Web. Due to
this fact, the metrics fail to take into account the increased freedom of browsing that the
navigational aids, inherent to the browser, give the user. Netscape’s “Navigator” and
Microsoft’s “Internet Explorer” both include several navigation aids.
•
The “Back” facility of both browsers provides a link from one node to the
previous node in both hierarchical and sequential structures. This facility makes
any Hyperlink in a Web site a two-way Hyperlink.
•
The “History list” in both browsers can act as a Hyperlink backwards through
several levels of a hierarchy or across several sites.
•
The “Bookmark” (Netscape) or “Favorites” (Microsoft) facilities allow users to
save a Web page’s URL, and at any time return to this Web page.
Page 41
LITERATURE REVIEW
_________________________________________________________________________________________________________
Particularly, the design of the Converted Distance Matrix (CDM) neglects the Back
and History facilities of all modern browsers. When the Back facility is applied to the CDM
of a strict hierarchy (Section 2.3.3.1) all Hyperlinks present within this Web site become twoway links, thereby, removing the need for the infinity value or the conversion constant
(Section 2.3.4.1) in the CDM. This change affects the results of all Botafogo et al. (1992)
metrics calculated for this type of Web site. A modification to the CDM will be proposed in
Chapter 4.
Regardless of these facts, the metrics when applied to the researcher’s experimental
Web sites do reflect an accurate indication of the different aspects of the structure (Section
3.2.3).
2.5 Summary
In this chapter the author briefly examined the rise of the World Wide Web (W3), the
protocols and standards associated with the W3 and the browsers used to access the W3.
User strategies for exploring the World Wide Web were also examined in this chapter.
Regardless of what strategy a user adopts in searching the W3 he/she still has to contend with
the structure of a Web site when it has been accessed (located).
Since the structure of a Web site is almost identical to HyperText structures, previous
research into HyperText structures was reviewed (Section 2.3.3). This review showed that
there are several parts to the design of a HyperText document. These include the content,
organizational and presentation parts (Thuring et al., 1991). The organizational part can be
considered the actual structure of the HyperText document. The organizational part was
further expanded to show the different types of structures available.
Page 42
LITERATURE REVIEW
_________________________________________________________________________________________________________
Thuring et al (1991) expand the organizational part into two types of structures,
sequential and exploration (hierarchical). Woodhead (1990) shows hierarchical structures can
be classified into three sub-types strict, compromised and overlapping hierarchies. The
previous research into HyperText structure was related to Web sites and was found to be
accurate. (Section 2.4.2) These structures can be identified in HyperText by the use of metrics
developed by Botafogo et al. (1992). These metrics were examined in detail.
The research into HyperText structural metrics (Section 2.3.4) was conducted prior to
the creation of modern browsers. The back, visited link, history and bookmark (favorites)
facilities in modern browsers have changed the way users traverse HyperText on Web sites
(Section 2.4.3).
Also, explored were Smith’s (1996) metrics designed to identify the degree of lostness
and efficiency of a user’s interaction with HyperText.
The author of this thesis searched the World Wide Web, scientific journals and
academic papers for up-to-date work on user perceptions of Web site structures. None was
found.
Several questions were raised in this chapter:
•
•
•
Which type of Web site -- heavily cross-linked or strictly hierarchical -- increases the
users’ likelihood of experiencing the “Lost in Hyperspace” phenomenon?
•
Page 43
LITERATURE REVIEW
_________________________________________________________________________________________________________
Two experimental Web sites (red and blue) were designed and subjected to Botafogo
et al.’s (1992) metrics (Chapter 3). The red Web site will then be subjected to a modified
version of Converted Distance Matrix for strict hierarchies (Section 4.5.2). The modification
recognizes the Back facility of a browser, by treating all Hyperlinks in a Web site as two-way
links.
An experiment was conducted using these two Web sites. A questionnaire was used to
measure user perceptions of the Web sites. Additional data relating to timing and Web pages
visited will be collected from the Web server hosting the experimental sites for each user.
The data collected in this experiment will be subjected to a doubly multivariate analysis of
variance (Section 4.2) to look for influences on subjects’ preferences (heavily cross-linked
hierarchy versus strict hierarchy) not attributable to Web site structure. The same data will
then be subjected to a logistic regression (Section 4.3) to build a model to predict user
preference.
Page 44
METHODOLOGY
__________________________________________________________________________________________
Chapter 3
METHODOLOGY
3.1 Introduction
Null Hypothesis:
Ho: “Web site structure makes no difference to the users’ perceptions of navigability”
The focus of this thesis is the effect of Web site structure on users’ perceptions of
navigability. To research this question two experimental Web sites (red and blue) were
constructed using different structures. In Chapter 2 several questions were raised, two of
them were;
“Which structure would users prefer -- heavily cross-linked or a strict hierarchy?”
“Which type of Web site -- heavily crossed-linked or strictly hierarchical -- increases the
users’ likelihood of experiencing the “Lost in Hyperspace” phenomenon?”
The two sites were constructed to explore these questions. The underlying structure of
both sites is hierarchical, but different types of hierarchical structure were designed. The red
site is a strict hierarchy, as described in Section 2.3.3.1. The red site contains a total of 72
links. The blue site uses compromised and overlapping hierarchies (Section 2.3.3.1), as well
as sequential structures (Section 2.3.3.2). The blue site contains a total of 736 links. The
metrics proposed by Botafogo et al. (1992) were applied to the two Web sites. The application
of these metrics to the two Web sites demonstrated the sites to be at the extremes of Botafogo
et al.’s (1992) metrics.
student body (undergraduate and postgraduate) of Southern Cross University.
Page 45
METHODOLOGY
__________________________________________________________________________________________
Each subject was given two tasks and a questionnaire to complete for each of the two
Web sites. The tasks required the subjects to navigate each site. Fifty percent of subjects were
directed to complete the red site first, while the remainder, were directed to complete the blue
site first. The questions posed in the questionnaire are designed to capture from the subject
their perceptions of the structure of the two Web sites. Data also captured by the
questionnaire will answer the question raised in Chapter 2;
“Do users utilize the features of modern browsers to aid navigation within a Web site?”
The sources of the experimental data collected consisted of completed questionnaires
and the server log file for each subject in the experiment.
This chapter examines the experiment construction. The first area of the experiment
explored is the construction of the Web sites (red and blue). This will investigate the criteria
used to design the Web sites. Following this is an examination of the browser, subjects and
task set in the experiment. Also, examined is what data will be collected from the experiment.
Page 46
METHODOLOGY
__________________________________________________________________________________________
3.2 Experiment Construction: Web sites
The researcher for comparison purposes designed two experimental Web sites (red
and blue). The two Web sites were specifically designed to explore user perceptions about
their structures, by having the users compare a strict hierarchy and a heavily cross-linked
multiple hierarchy. The content in both sites remained identical, with the exception of the
additional Hyperlinks in the blue Web site. The sites were designed to be at the extremes of
Botafogo et al.’s (1992) metrics.
Microsoft FrontPage 97 was the development environment used to develop the two
experimental Web sites.
The following list shows the criteria used by the researcher in developing both sites.
Design Criteria for Web sites.
Quick response time.
Content of sites interesting to subjects.
Quantifiably different red and blue Web site structures.
Minimise variables between pages of each site.
Accessible site log.
The above criteria will be examined in detail in the following sub-sections.
Page 47
METHODOLOGY
__________________________________________________________________________________________
3.2.1 Quick Response Time
The two Web sites were constructed on a UNIX server, on the Southern Cross
University campus. The server is a Sun SPARC station 20. The operating system on the
server is Solaris 2.4, and the web server in use was NCSA. The server (spike.scu.edu.au) was
chosen for several reasons. Firstly, as the researcher could use information taken from the
server and its HTTP access log file (Section 3.2.5) to examine subjects’ time to complete the
task set and visits to pages of each Web site. Secondly as the server is a local server, response
time was minimized. Response time for each node of the Web site was indistinguishable from
other work station activities.
Each node of both the red and blue Web sites contained several graphics. This gave
the sites the “look and feel” of professional Web sites. Three types of graphics were used; gif
(Graphical Interchange Format), gif89a (Graphical Interchange Format 1989 Animated) and
jpg (Joint Photographic Experts Group).
The size of the graphics relates directly to the response time of the Web page loading
in the browser environment. Graphics stored as a gif are larger in size (measured in bytes)
than the same graphic converted to jpg. Therefore, all graphics were either resized (physical
size of displayed image) or changed in format (gif to jpg), so that the size (measured in bytes)
of the graphics would not adversely affect download time for each Web page.
The largest graphic used was 37k (kilo) bytes in size the average size of graphic was
less than 18k bytes.
Page 48
METHODOLOGY
__________________________________________________________________________________________
3.2.2 Content of Sites Interesting to Subjects.
The content of both Web sites with the exception of Hyperlinks on each page was
identical for each node in the hierarchy. Each page contained graphics, textual Hyperlinks or
a textual description. (Appendix D for example pages). The subject matter of both Web sites
was comic book heroes and villains. The graphics used were all found on the Internet, at
various World Wide Web Comic book fan sites or scanned from the researcher’s collection.
The comic characters on all pages of both Web sites are copyrighted to Marvel Comics Inc.
and DC Comics Inc.
This content was chosen, as:
•
It is non-threatening.
•
It is non-technical and easily understood by all subjects.
•
Entertaining to the subjects.
3.2.3 Quantifiably Different Red and Blue Web Site Structures.
The two Web sites were specifically designed to explore user perceptions about their
structures, by having the users compare a strict hierarchy and a heavily cross-linked
overlapping hierarchy. The content in both sites remained identical, with the exception of the
additional Hyperlinks in the blue Web site. The sites were designed to be at the extremes of
Botafogo et al’s (1992) metrics.
The metrics for HyperText structures proposed by Botafogo et al. (1992) and Rivlin et
al. (1994) were applied to the two Web sites. This was done to demonstrate the differences
between the two site structures. The application of these metrics is as follows:
Page 49
METHODOLOGY
__________________________________________________________________________________________
The structure of the red Web site is a strict hierarchy (Section 2.3.3.1). The red site
consists of 73 nodes and 72 Hyperlinks between the nodes. The structure of the blue Web site
is a combination of compromised and overlapping hierarchies (Section 2.3.3.1) as well as
sequential structures (Section 2.3.3.2). The blue site consists of 73 nodes and 736 Hyperlinks
between the nodes.
Appendix B contains maps of both Web sites and a description of the Hyperlinks
contained in each node of both sites. The metrics in section 2.3.4 were applied to both sites.
Appendix C contains the distance matrix of the both the red and blue Web sites. The value of
each metric for both Web sites follows:
Note: for all following calculations (as described in Section 2.3.4.1) the conversion
constant (K) = 73. Botafogo et al. (1992) recommend the value of the conversion constant be
the same as the number of nodes within the site.
Converted Out Distance (COD)
The COD value of a node, is the sum of the node’s row in the HyperText structure
distance matrix (described in Section 2.3.4.2). The node with the lowest COD value is
regarded as the central node within the structure.]
Red site -- Node 1, COD = 266
In the red site node 1 is the most central with a COD value of 266. Other COD values
ranged from 2798 (node 2) to 5256 (node 73).
Blue site – Node 9, COD = 125
In the blue site node 9 is the most central with a COD value of 125. Other COD values
ranged from 126 (node 12) to 266 (node 1). When looking at the map of the structure
(Appendix B) each of the 4th level nodes (8-15) show a high degree of centrality.
The distance matrix for both Web sites, in Appendix C, displays all the COD values.
Page 50
METHODOLOGY
__________________________________________________________________________________________
Relative Out Centrality (ROC)
ROC = CD (Converted Distance) / COD
The ROC value for individual nodes allows direct comparisons between different
HyperText structures. The node with the highest ROC value is the most central node within
the structure.
Red site – Node 1, ROC = 1,371.85
Within the red site node 1 has the highest ROC value. No other node in the red site
approached the ROC value of node 1 (1,371.85). Other ROC values in the red Web site
ranged from 130.42 (node 2 and 3) to 69.43 (node 16 to 73).
Blue site – Node 9, ROC = 99.89
In the blue site node 9 with a ROC value of 99.89 is the most central node. All the 4th
level nodes (see Appendix B for site map) were very close to this value ranging from 96.05 to
99.10.
The distance matrix for both Web sites, in Appendix C, displays all the ROC values.
Compactness (Cp)
This metric indicates whether each node can easily be reached from another node. The
more connected the HyperText structure is, the closer the Compactness value is to 1.
Compactness is calculated using the following formula:
Cp = (MAX - CD) / (MAX - MIN)
Where:
MAX = (n2 - n)K and MIN = (n2 - n)
n = number of nodes = 73
K = conversion constant = 73
Red site – Cp = .04962
Blue site – Cp. = .9809
Page 51
METHODOLOGY
__________________________________________________________________________________________
Stratum (St)
This metric was designed to reflect the linear ordering of a HyperText structure. This
indicates whether it is important to begin traversing the HyperText from a particular node and
in sequential order (Section 2.3.4.6 for description). The stratum of a HyperText structure is
calculated using the formula:
Stratum = absolute prestige / LAP (Linear Absolute Prestige)
Where:
Absolute prestige = Sum of (status – contrastatus)
Status = the sum of a node’s row in the distance matrix
Contrastatus = the sum of a node’s column in the distance matrix
Note: status and contrastatus both ignore infinite values.
LAP = (n3 - n)/ 4 as, n is odd
n = number of nodes
Red site – St = .01203
Blue site – St = .02471
The distance matrix for both Web sites, in Appendix C, displays all the values of
Status, Contrastatus and Prestige.
Page 52
METHODOLOGY
__________________________________________________________________________________________
3.2.3.1 Verifying Red and Blue Web Site Structures are Quantifiably Different
The table below displays the results of the metrics developed by Botafogo et al.
(1992) and Rivlin et al. (1994) when applied to both experimental Web sites.
Table 3.1:
Comparison of red and blue Web site metrics.
Attribute
Red Web Site
Blue Web Site
Most central node:
Node 1
Node 9
ROC value:
1,371.85
99.88
Next highest ROC
values:
Node 2 = 130.42
Node 12 = 99.10
Node 3 = 130.42
Node 14 = 98.31
Least central node:
Nodes 16-73
Node 1
ROC value:
69.43
46.94
Cp value:
.0496
.9809
St value:
.01203
.02471
Number of Nodes:
73
73
Number of Hyperlinks:
72
736
ROC (Relative Out Centrality)
Cp (Compactness)
St (Stratum)
The red Web site is a strict hierarchy with 72 Hyperlinks, this is demonstrated by the
ROC and Cp calculations for this site. The Homepage of the red Web site, node 1, has the
highest ROC value (1,371.85) followed by nodes 2 and 3 (130.42) this reflects the fact that
within this structure there is only one hierarchy. This is further evidenced by the fact that all
the level 5 nodes (Appendix B for site map) have the lowest centrality value on the red site
(69.43).
Page 53
METHODOLOGY
__________________________________________________________________________________________
The Cp value of .0496 demonstrates the red Web site is poorly connected, that is, the
ability to traverse from any node to any other node requires multiple steps. This low value
for Compactness is further demonstrated by the fact that the only possible entrance point in
the red Web site with Hyperlink access to all pages, would be node 1 (Homepage). If a user
entered the site at any other node it would be impossible to view the entire contents of the
site.
The blue Web site is a heavily cross-linked overlapping hierarchy (736 Hyperlinks).
This is reflected by the ROC and Cp calculations. The ROC values for all level 4 nodes
(nodes 8-15) fall in the range of 96.05 to 99.89 with node 9 having the highest value. These
values for ROC demonstrate that the blue Web site contains overlapping and compromised
hierarchies (Appendix B for site map). The lowest value of ROC in the blue Web site, is that
of the Homepage (node 1). This clearly demonstrates that the two Web sites have different
structures.
The blue Web site has a high compactness value of .9809, this reflects the fact that
each node can quickly and easily be reached from another node. This is further demonstrated
by the fact that any node within the blue Web site, could be used as a entrance point and still
have access to all nodes on the site.
The Stratum values for the two Web sites reflect their linear ordering, that is, whether
the site should be traversed in a particular sequence. The red Web sites value is .01203, this
value indicates that there is little linear ordering to this site. The value for the blue Web site
is .02471, this indicates that the site also has little linear ordering within it.
The results of the application of the metrics developed by Botafogo, Rivlin and
Shneiderman (1992; 1994) demonstrate that the red and blue sites are different in structure.
Page 54
METHODOLOGY
__________________________________________________________________________________________
3.2.4 Minimise Variables Between Pages of Each Web Site
Considerable attention to detail was put into the design of each page of both Web
sites. This effort was directed at reducing extraneous variables that could affect user
perceptions.
•
Background colour (browser default) was held constant in all pages (red and blue sites).
No graphic was placed on the background, as quick response time was a design criterion.
•
An animated GIF (Graphical Interface Format 89a) was used to divide each page of both
sites into three sections. The image used was 5k bytes in size. Each page contained a
maximum of three sections:
1. Graphic(s) and title (both sites).
2. Hierarchical Hyperlinks or Content.
3. Additional Hyperlinks (blue site only).
As the graphics displayed were of different physical sizes and orientations (vertical /
horizontal) care was taken in the design of each page that the Hyperlinks appeared at the
same screen location on each of the pages in the Web sites. This was particularly
important in the design of the blue site as the additional textual Hyperlinks were placed
into HTML 3.2 tables. The 5th level nodes of the blue Web site contained (Appendix B
for site map), within their HTML tables a sequential (next) Hyperlink. In order to reduce
the movement of the mouse on screen these Hyperlinks had to appear at exactly the same
location on screen.
•
Each page was designed to be accessed by a browser using 800 X 600 resolution. This
was done so that a scroll bar would not appear. This resulted in reduced number of
mouse clicks as no area of the page was hidden from view. Subjects viewed the Web
sites at this resolution.
Appendix D contains samples of the Web pages in both the red and blue Web sites.
Page 55
METHODOLOGY
__________________________________________________________________________________________
3.2.5 Accessible HTTP Log
A client’s (browser’s) HTTP request starts with a method (GET, PUT, POST etc.)
and the URL of the object the client is addressing. The HTTP server (host) then returns the
result of the HTTP request to the client (browser) that requested it. The server also logs all
HTTP requests. In the server used this log is in a file named “access_log”. Access to the data
contained in this server log file was arranged. Section 3.6, Data Collected, details the
information extracted from this file for each user of the experimental Web sites.
3.3 Experiment Construction: Browser
A computer lab at Southern Cross University was chosen as the venue for the
experiment. Each subject in the experiment used a Microsoft NT 4.0 Pentium 166 with 32
megabytes of RAM and a 17-inch monitor to access the experimental sites. The IP address of
each machine was recorded to correlate with the access log.
The browser chosen was Microsoft Internet Explorer 3.02 (IE 3.02)
Procedures in preparing venue for experiment:
1.
2.
3.
4.
5.
6.
7.
Resolution of each computer set to 800 X 600 at 24bit true colour.
IE 3.02 opened and set to full screen resolution.
IE 3.02 history list deleted.
IE 3.02 favorites deleted.
IE 3.02 RAM and disk caches deleted.
Computer IP address noted on Questionnaire.
Start time of experiment noted on Questionnaire.
The favorites, history list and caches are deleted so that each subject starts each Web
site without the previous user’s accesses to the Web sites reflected by the browser’s intrinsic
facilities.
Page 56
METHODOLOGY
__________________________________________________________________________________________
3.4 Experiment Construction: Subjects
A focus group of 11 subjects undertook the experiment on 12/06/97 to verify the
following:
•
The questionnaire (Section 3.5) was understandable.
•
The tasks set in both Web sites was achievable.
•
The structures appeared different from a user perspective.
•
The content of the Web sites was entertaining.
Discussion with the focus group satisfied the researcher that the above criteria had
been met. As no change was required to the design of the experiment, the focus group’s data
has been included in the data set of the experiment.
The 113 subjects who undertook this experiment were drawn from the students and
staff of Southern Cross University. The large number (greater than 100) of subjects was
required as the data generated by the experiment was to be analyzed using a doubly
multivariate analyses of variance to look for interaction effects between cells (i.e.
male/female, age etc.). The reasoning behind examination of these effects is to see if any
factors other than the structure of each Web site influences the user’s preference. The only
criterion for a subject was experience with navigating the World Wide Web. Due to this
criterion the largest proportion of subjects was drawn from the School of Multimedia and
Information Technology.
A maximum of eighteen subjects undertook the experiment at any one time. The
experiment was run during the period 12/06/1997 to 15/08/1997.
Page 57
METHODOLOGY
__________________________________________________________________________________________
3.5 Experimental Task
Each subject was given a questionnaire to complete (Appendix E). No time limit to
complete the task was given to subjects.
The questionnaire had four sections:
•
Questionnaire General Information
•
Questionnaire Site Blue
•
Questionnaire Site Red
•
Questionnaire Conclusion
To facilitate statistical comparison, 50% of questionnaires had sections 2 and 3
reversed. Each of the above sections will be examined in detail, identifying the reasoning
behind the question and the type of data the question returns (categorical or ratio).
3.5.1 Questionnaire General Information
This first section contains five questions.
1. Gender. Provides categorical data. The reason for this question is to examine if there is
any difference between the genders in their perception of Web site navigability.
2. Age. Provides categorical data. This question is to examine whether there is a difference
in perception based on the age of the subject.
3. Experience with WWW. Provides ratio data, using a Likert scale. Is there a difference in
user perception based on WWW experience?
4. Experience with Computers. Provides ratio data, using a Likert scale. Is there a
difference in user perception based on experience with computers?
5. Knowledge of content. Provides categorical data. This question was included to capture a
user who already knew the answers to the tasks set in the blue and red site questionnaires. If
a user identified him/herself to have knowledge of more than 20 comic book characters, the
questionnaire would be removed from the sample population.
Page 58
METHODOLOGY
__________________________________________________________________________________________
3.5.2 Questionnaire Site Red
This section of the questionnaire instructed the subjects to open the browser on the
URL of the red site.
The subject was then given two tasks to perform.
1. What is the secret identity of Guardian?
2. What is the secret identity of Chameleon?
Each task required the subject to search the site to find the target node (page) to
answer the question. Appendix D contains the target nodes. The characters designated as
targets were chosen, as they are not well known characters.
After the subject had completed the task, four questions were put to the subject. All
questions used a Likert scale to supply ratio data for statistical analysis.
1. Lost in Hyperspace. Did the structure of this Web site cause the user to experience this
phenomenon?
2. IE 3.02 Back facility. How often did the subject use the browser’s back facility? The
structure of the Red site forces the user to access this function.
3. IE 3.02 History facility. How often did the subject use this facility? Use of this facility
would reduce the number of traversals required to accomplish the tasks in the red site.
4. Comfortable with structure. Was the user at ease with the structure of the site?
Page 59
METHODOLOGY
__________________________________________________________________________________________
3.5.3 Questionnaire Site Blue
This section of the questionnaire instructed the subjects to open the browser on the
URL of the blue site.
The subject was then given two tasks to perform.
1. What is the secret identity of Clayface?
2. What is the secret identity of Calypso?
Each task required the subject to search the site to find the target node (page) to
answer the question. Appendix D contains the target nodes. The characters designated as
targets were chosen, as they are not well known characters.
After accomplishing the tasks the subject was asked four questions. All questions
used a Likert scale to supply ratio data for statistical analysis.
1. Lost in Hyperspace. Did the structure of this Web site cause the user to experience this
phenomenon?
2. IE 3.02 Back facility. How often did the subject use the browser’s back facility? The
structure of the blue site is designed so that there is no need to use this facility.
3. IE 3.02 History facility. How often did the subject use this facility? Use of this facility
would not help to reduce the number of traversals required to accomplish the tasks in the
blue site.
4. Comfortable with structure. Was the user at ease with the structure of the site?
Page 60
METHODOLOGY
__________________________________________________________________________________________
3.5.4 Questionnaire Conclusion
The conclusion of the questionnaire contains two questions. Both questions provide
categorical data.
1. Favorites. Did the user utilize this facility?
2. Preference. Which Web site did the user prefer, red or blue?
3.5.5 Questionnaire Summary
The data provided by subjects in the general information section of the questionnaire
(Section 3.5.1) is required to look for influences on the subjects’ preference of Web site
attributable to experience with the WWW, gender, age, experience with computers and order
of completion of the Web sites. Also, explored will be the influence of interactions between
gender, age and order of completion of Web sites.
The questions dealing with usage of the Back, History and Favorites facilities, are
designed to acquire data to explore whether users utilize these facilities to explore Web sites.
The questions dealing with feelings (lost and comfort with structure) in the
questionnaire are designed to acquire user perceptions of each Web sites’ structure.
The final question in the questionnaire is whether the user preferred the red or blue
Web site.
Page 61
METHODOLOGY
__________________________________________________________________________________________
3.6 Data Collected
The questionnaires were collected from each subject undertaking the experiment.
Also, details of time taken and all nodes visited to complete the task set for each Web site
were extracted from the server. The above data (Appendix F contains a sample listing of
data) will be statistically analyzed in Chapter 4.
It had been initially hoped that the user’s traversal of the Web site could be tracked.
This information would have allowed the metrics developed by Smith (1996), described in
section 2.3.5, to be applied to each subject of the experiment. However, to accomplish this
tracking of the user’s traversals the browser itself would have to be modified. Internet
Explorer 3.02 caches each Web page (HTML node) as it downloads. The next time a cached
Web page is accessed, whether via the back, history list or another Hyperlink, Internet
Explorer accesses the client’s cache, not the server (host).
Using a modified browser that logged all URLs or sent a GET command to the server
with each access to a Web page could solve this. This modification could not be
accomplished within the scope of this thesis. The details that were extracted from the server
do, however, allow some of the indicators Smith (1996) developed for lostness and
efficiency to be calculated.
Page 62
METHODOLOGY
__________________________________________________________________________________________
3.7 Experiment Summary
In summary, two experimental Web sites were designed to explore the effect of Web
site structure on the users’ perceptions of navigability. The two Web sites were demonstrated
to have different structures by subjecting them to the HyperText metrics developed by
Botafogo, Rivlin and Shneiderman (1992; 1994).
Each of the 113 subjects who participated in this experiment was given two tasks and
a questionnaire to complete for each of the two experimental Web sites (red and blue). The
tasks required the subjects to traverse the structure of each Web site. Fifty percent of subjects
were directed to complete the red site first, while the remainder, were directed to complete
the blue site first.
The results of each questionnaire were collated with the results from the Web server
log, The log provided timing and visited node information for each subject undertaking the
experiment. This data will be statistically analyzed in Chapter 4. The analysis of the data will
answer the following questions raised in Chapter 2;
•
•
•
Which type of Web site -- heavily crossed-linked or strictly hierarchical -- increases the
Also, examined in section 4.5 will be a modification to the Converted Distance
Matrix (CDM) for Web sites with a strict hierarchical structure. This modification affects all
of the HyperText metrics developed by Botafogo, Rivlin and Shneiderman (1992; 1994). The
modification to the CDM is designed to recognize the navigation aids inherent in modern
browsers, particularly the Back facility.
Page 63
ANALYSIS OF DATA
__________________________________________________________________________________________
Chapter 4
ANALYSIS OF DATA
4.1 Introduction
This chapter of the thesis will report the results of two separate analyses of the data
collected from the experiment outlined in Chapter 3. Section 4.2 introduces the two analyses
and examines the exclusions and transformations of experimental data.
The first statistical analysis performed (Section 4.3) is a doubly multivariate analysis
of variance – repeated measures. This analysis will examine the data for influence on both
The second statistical analysis performed (Section 4.4) is a logistic regression. This
analysis examines the data set gleaned from the experiment to find a model to predict a
subject’s preference of the experimental Web sites.
Section 4.5 of this chapter proposes a modification to the Converted Distance Matrix
(CDM) developed by Botafogo, Rivlin and Scheiderman (1992; 1994) for strictly hierarchical
Web sites (CDMh).
Section 4.6 of this chapter examines the indicators of lostness and efficiency
developed by Smith (1996) using the data resulting from the experiment.
Section 4.7 answers the questions raised in Chapter 2 of this thesis;
•
•
•
Which type of Web site -- heavily crossed-linked or strictly hierarchical -- increases the
•
Page 64
ANALYSIS OF DATA
__________________________________________________________________________________________
4.2 Statistical Analysis Introduction
This section and the following two sections of this chapter will report the results the
two analyses of the data collected during the experiment detailed in Chapter 3.
Sections 4.2.1 and 4.2.2 examine and explain the exclusions and transformations of
the data gleaned from the experiment.
Section 4.3 will examine the design and results of a doubly multivariate repeated
measures analysis of variance. This analysis tests the significance of differences between the
red and blue Web sites on (within subject factors) lost rating, usage of Back button, usage of
History list, comfort rating, number of nodes visited and time to complete the task. It also
tests the significance of the influence of the between subjects factors:
1. Experience with Computers.
5. Order (red or blue completed first).
2. Experience with the WWW.
6. Gender by Age.
3. Gender.
7. Gender by Order.
4. Age.
8. Age by Order.
Where the previously mentioned factors do not influence red – blue differences the
effect of factors will be examined for overall (both Web sites) influence.
Section 4.4 will analyze the results of the experiment to find which component(s) of
the data predict Web site (red or blue) preference by using logistic regression. The
components analyzed are:
•
•
Usage of History list
•
Experience with WWW
•
Lost
•
Gender
•
Comfort
•
Age
•
Time to complete (task time)
•
Order (red or blue Web site 1st)
•
Visited nodes
•
Usage of Back button
Page 65
ANALYSIS OF DATA
__________________________________________________________________________________________
4.2.1 Data Excluded
The results of the question dealing with the utilization of the Favorites facility,
explained in section 3.5.4 (Questionnaire Conclusion), was dropped from the statistical
analyses as only 3 subjects in the sample reported using the facility.
Several errors occurred due to a misunderstanding of the instructions or lack of
interest in completing the tasks set out in the instructions. The details of the errors made and
how these errors were treated follows:
•
Five subjects failed to visit both sites and completed all tasks by traversing a
single Web site only. Due to the fact that the statistics are based on a comparison
between the two Web sites, these results were removed from the sample.
•
One additional result was removed from the sample. As stated in section 3.5.1
(Questionnaire General Information) the question dealing with “Knowledge of
content” was included in the questionnaire to capture a user who may already
know the answers to the tasks set in the blue and red sites. One subject identified
himself to have knowledge of over 40 comic book characters, due to this fact this
subject was removed from the sample.
These exceptions resulted in the sample size being reduced from 113 to 107.
4.2.2 Data Transformed
Two subjects completed the traversal of the two Web sites in a reverse order. The
server log confirmed this. Both subjects brought this error to the researchers’ attention and
stated it was due to sticky pages in the questionnaire.
This presented no problem as, fifty percent of subjects were to complete the red site
first, while the remainder, were directed to complete the blue site first. These two results
were entered into the sample with the order variable changed to reflect the order in which the
subjects undertook the tasks.
Page 66
ANALYSIS OF DATA
__________________________________________________________________________________________
The categorical data supplied by question 2 (Age) of the General Information portion of the
questionnaire (Appendix E) had to be modified. Data collected placed each subject in one of
60
60
50
50
40
40
Number of Male Subjects
Number of Female Subjects
five categories. Below are bar charts of the subjects age categories:
30
20
10
0
18 - 25
26 - 33
33 - 40
41 - 47
30
20
10
0
Over 47
18 -25
Age Categories
26 - 32
33 - 40
41 - 47
Over 47
Age Categories
Diagram 4.1: Male and Female Frequencies by Age
Due to the small size of some of the frequencies within some of the gender by age
cells, the age categories were transformed into 18 – 25 and over 25. The diagram below
shows the new frequencies.
60
60
50
40
Number of Male Subjects
Number of Female Subjects
50
30
20
10
0
18 - 25
40
30
20
10
Over 25
18 - 25
Transformed Age Categories
Over 25
Tranformed Age Categories
Diagram 4.2: Transformed Male and Female Frequencies by Age
Page 67
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3 General Linear Model (GLM) – Repeated Measures
“The GLM repeated measures procedure provides analysis of variance when the
same measurement is made several times on each subject or case. If between-subjects
factors are specified, they divide the population into groups. Using this general linear model
procedure you can test null hypotheses about the effects of both the between-subjects factors
and the within-subjects factors. You can investigate interactions between factors as well as
the effects of individual factors. In addition, the effects of constant covariates and covariate
interactions with the between-subjects factors can be included.
In a doubly multivariate repeated measures design, the dependent variables
represent measurements of more than one variable for the different levels of the withinsubjects factors.”
(SPSS Inc., 1997, p.27)
This particular statistical analysis was chosen, as the data collected from
questionnaires and from the server (Section 3.6, Data Collected) is a comparison between
the two Web sites (red and blue). The model used was a doubly multivariate repeated
measures analysis of variance. In this analysis the dependent variables are:
•
Lost rating.
•
Usage of Back button.
•
Usage of History list.
•
Comfort rating.
•
Number of nodes visited.
•
Task time (time to complete task)
The between subject factors are:
•
Gender.
•
Age.
•
Order (red or blue Web site completed first).
•
Gender by Age.
•
Gender by Order.
•
Age by Order.
Page 68
ANALYSIS OF DATA
__________________________________________________________________________________________
Two covariates used in the model were:
•
Experience with computers.
•
Experience with the WWW.
Effects tested by the model:
•
Within Subject (red – blue) by:
1. Experience with computers.
5. Order (red or blue 1st).
2. Experience with WWW.
6. Gender by age.
3. Gender.
7. Gender by order.
4. Age.
8. Age by order.
•
Within subject (red – blue).
•
Between subject on averaged variables (e.g. red + blue) by:
1. Experience with computers.
5. Order (red or blue 1st).
2. Experience with WWW.
6. Gender by age.
3. Gender.
7. Gender by order.
4. Age.
8. Age by order.
The hierarchical (or sequential; Type I) method of decomposition of sum-of-squares
was used. With this method, each term (factor or covariate) is adjusted for each term
preceding it in the model. The order of terms is as listed above.
4.3.1 Hypotheses
Section 3.5, Experimental task, described the questionnaire completed by all
subjects. Several questions were included in the questionnaire specifically to capture
information used to examine if there were differences due to experience with computers,
experience with the WWW, gender, age and order (red or blue site completed first).
Following are the hypotheses the multivariate analysis will examine.
Page 69
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.1.1 Within Subject Null Hypotheses
Hypotheses involving the within-subjects (red – blue effects):
1. H0: There will be no influence on the difference between the red and blue Web site
measurements (Lost rating, usage of Back button, usage of History list, Comfort rating,
number of nodes visited and Task time), attributable to experience with computers.
measurements, attributable to experience with the WWW.
measurements, attributable to gender.
measurements, attributable to age.
measurements, attributable to order.
measurements, attributable to age by gender interaction.
measurements, attributable to age by order interaction.
measurements, attributable to gender by order interaction.
9. H0: There will be no difference between the red and blue Web sites on the mean
measures of the dependent variables (Lost rating, usage of Back button, usage of History
list, Comfort rating, number of nodes visited and Task time).
Note:
Each hypothesis is adjusted for the effect of the preceding hypotheses.
Page 70
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.1.2 Between Subject Null Hypotheses
Hypotheses involving the between-subjects (red + blue effects):
1. H0: There will be no influence on the averages of measurements (Lost rating, usage of
Back button, usage of History list, Comfort rating, number of nodes visited and Task
time) on the red and blue Web sites, attributable to experience with computers.
2. H0: There will be no influence on the averages of measurements on the red and blue Web
sites, attributable to experience with the WWW.
sites, attributable to gender.
sites, attributable to age.
sites, attributable to order.
sites, attributable to age by gender.
sites, attributable to age by order.
sites, attributable to gender by order.
Note:
Each hypothesis is adjusted for the effect of the preceding hypotheses.
Page 71
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.2 GLM – Repeated Measures Interpretive Strategy
The strategy for interpretation of the results rests on two basic principles:
1. That effects found to be non-significant in the multivariate test will not be interpreted as
significant even if the univariate test of the effect shows p < .05. This principle uses the
multivariate test as a form of control over experiment-wide type 1 errors* arising from
tests on multiple variables for each effect.
2. That interaction effects are examined first, from the higher order effects down to the
lower order interaction and main effects. In particular, significance of a within by
between subject interaction effect implies that the red – blue difference (difference
attributable to the structures of the red & blue Web sites), differs between the levels of
the between subjects factor. As a result the lower order red – blue and between subjects
effects should not be separately examined as if they were independent.
* Type 1 error – Erroneous rejection of a null hypothesis.
Page 72
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.3 Results of Tests of Multivariate Significance
Following is the results of the SPSS GLM – repeated tests model. Wilks’ Lambda is
displayed, as it is the most commonly used multivariate test (SPSS Inc., 1997, p.136).
Table 4.1:
Multivariate Tests (Wilks’ Lambda)
Computed using alpha=.05
Within Subjects Effects
RB * Exp with Computers
RB * Exp with WWW
RB * Gender
RB * Age
RB * Order
RB * Gender * Age
RB * Gender * Order
RB * Age * Order
RB (red-blue)
Value
0.921
0.880
0.949
0.858
0.738
0.908
0.943
0.970
0.340
F
1.327
2.104
0.841
2.570
5.513
1.562
0.929
0.479
30.144
Hypothesis df Error df
6.000
93.000
6.000
93.000
6.000
93.000
6.000
93.000
6.000
93.000
6.000
93.000
6.000
93.000
6.000
93.000
6.000
93.000
Significance
0.253
0.060*
0.541
0.024 *
0.000 *
0.167
0.478
0.822
0.000*
F
Hypothesis df Error df
596.421
6.000
93.000
4.610
6.000
93.000
2.175
6.000
93.000
0.596
6.000
93.000
1.689
6.000
93.000
1.448
6.000
93.000
2.044
6.000
93.000
0.776
6.000
93.000
0.881
6.000
93.000
Significance
0.000
0.000 *
0.052 *
0.733
0.132
0.205
0.067 *
0.591
0.512
Within Subjects Design: RB (red-blue)
Between Subjects Effects
Intercept
Exp. with Computers
Exp. with WWW
Gender
Age
Order (red or blue 1st)
Gender * Age
Gender * Order
Age * Order
Value
0.025
0.771
0.877
0.963
0.902
0.915
0.883
0.952
0.946
Design:
Intercept + Exp with Computers + Exp. With WWW + Gender + Age + Order
+ Gender*Age + Gender*Order + Age*Order *Order
Significant difference marked with *
Univariate tests for these effects showing multivariate significance will now be
explored.
Page 73
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.4 Results of Tests of Univariate Significance Involving the Within-Subjects Effects
The Multivariate tests (Table 4.1) shows significant effects for several factors:
•
•
•
•
Red - Blue by experience with the WWW.
Red - Blue by Age.
Red - Blue by Order (red or blue Web site completed first).
Red – Blue.
The table of tests of univariate effects is reported in full in Appendix G. An excerpt
from this table showing the results of univariate tests of those effects above, manifesting
multivariate significance is shown in table 4.2 below.
Table 4.2:
Excerpt –Univariate Tests of Within-Subjects Contrasts
Computed using alpha = .05
Source
RB * Exp. With WWW.
RB * Age
RB * Order
RB (red-blue)
* Denotes significant difference
Measure
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
F**
0.025
2.228
0.817
0.234
0.271
5.477
1.710
0.220
0.016
1.629
0.009
7.403
0.004
0.234
1.400
0.816
2.273
22.920
0.703
159.72
6.371
0.423
0.194
4.666
Significance
0.876
0.139
0.368
0.630
0.604
0.021 *
0.194
0.640
0.900
0.205
0.925
0.008 *
0.948
0.630
0.240
0.369
0.135
0.000 *
0.404
0.000 *
0.013 *
0.517
0.661
0.033 *
** All tests have 1, 98 degrees of freedom
Page 74
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.4.1 Red - Blue by Experience with WWW, Measure Task time
As displayed in figure 4.3, subjects with more experience with the WWW completed
the task more quickly in the red Web site than those subjects with less experience with the
WWW. Whereas, experience with the WWW is negatively related to completion time for
the red Web site, there is no relationship between experience with the WWW and
completion time on the blue Web site.
Scatter plot Blue site
400
400
300
300
Time to complete task
500
200
100
0
0
1
2
3
4
200
100
0
0
5
1
2
3
4
Experience with the WWW
Experience with the WWW
Diagram 4.3: Red - Blue by Experience with WWW, Measure Task Time
4.3.4.2 Red - Blue by Age, Measure Task Time
As exhibited in diagram 4.4, in the red Web site, subjects in the younger age group
took longer than the older age group to complete the task, this pattern was reversed for the
blue Web site.
200
190
180
Estimated Marginal Means
Time to complete task
Scatter plot Red site
500
170
160
150
140
R ed
B lu e
130
120
18 - 25
Over 25
AG E
Diagram 4.4: Red - Blue by Age, Measure Task
Page 75
5
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.4.3 Red - Blue by Order, Measure Task Time
As displayed in diagram 4.5, subjects regardless of which Web site they completed
first completed the second site faster. This measure seems to reflect experience with the
tasks set on the two Web sites.
200
180
160
140
120
R ed
B lu e
100
red s ite firs t
blue s ite firs t
O rd er
Diagram 4.5: Red - Blue by Order, Measure Task Time
4.3.4.4 Red – Blue, Measure Task Time
As described previously, the relationship between red or blue Web sites, and
completion of task is moderated by experience with WWW, age and order. However, as
shown below, overall, subjects completed the task more quickly in the blue Web site than
the red Web site.
16 6
16 4
16 2
16 0
15 8
15 6
15 4
15 2
15 0
14 8
R e d site
B lu e site
Diagram 4.6: Red – Blue, Measure Task Time
Page 76
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.4.5 Red – Blue, Measures the use of Back Button and History List
As shown in diagram 4.6, the difference between the red and blue Web sites on the
usage of the Back button and History list, was that subjects used these facilities more
frequently on the red Web site. This had been expected as the red Web site design required
usage of Back button or History list to complete traversal.
Unexpected was the level usage of the Back button in the blue Web site. The blue
Web site contained dedicated Hyperlinks that:
•
•
•
performed the function of the Back facility;
completely cross-connected the 2nd, 3rd and 4th levels of the structure;
returned the user from any node in the structure to root node of the
site(Appendix B for site map and Hyperlinks; Appendix D for sample pages).
However, 31.8% of subjects reported high levels of usage of the Back facility
(frequency value >= 2.5 out of 5), with only 12.1% of subjects reporting no usage. This
appears to suggest users utilize this facility even when not required by the structure of a Web
site.
Also, unexpected was the very low frequency of usage of the History list on both
Web sites, suggesting users do not often use this facility while traversing Web sites.
Use ofBack Button
Use of History List
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
.5
0.0
Red site
Blue site
1.00
.95
.90
.85
.80
.75
.70
.65
.60
.55
.50
.45
.40
.35
.30
.25
.20
.15
.10
.05
0.00
Red site
Use of Back Button:
0 = Not at All
5 = Frequently
Use of History List:
0 = Not at All
5 = Frequently
Diagram 4.7: Red – Blue, Measure the use of Back Button and History List
Page 77
Blue site
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.4.6 Summary of Within-Subjects Effects
Task time was the only dependent measure showing significant red – blue by
experience with WWW, red – blue by age and red – blue by order effects:
i)
Subjects with low experience with WWW take longer to complete the task in
the red Web site.
ii)
Subjects in the 18-25 age group complete the task in the blue Web site faster
than those subjects over 25. This pattern is reversed in the red Web site.
iii)
Regardless of which Web site subjects completed first, they completed the
second site faster. This measure seems to reflect experience with the tasks set
on the two Web sites.
Significant red – blue differences not moderated by within-subjects variables were
found on the measures usage of the Back button and usage of the History list:
i)
Subjects used the Back button frequently in the red Web site. This had been
expected. However, unexpected was the usage of the Back button in the blue
Web site.
ii)
Subjects used the History list more frequently in the red Web site than the
blue.
Page 78
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.5 Results of Tests of Univariate Significance Involving the Between-Subjects Effects
Table 4.1, shows significant multivariate difference between-subjects effects for the
factors experience with computers, experience with WWW and gender by age. The table of
tests of univariate effects is reported in full in Appendix G. An excerpt from this table
showing the results of univariate tests of the aforementioned effects, manifesting
multivariate significance is shown in table 4.3 below.
Table 4.3:
Excerpt –Univariate Tests of Between Subjects Effects
Source
Exp. With Comp.
Exp. With WWW
Gender * Age
Measure
F**
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
5.644
0.008
0.129
0.362
0.000
15.937
9.836
0.058
1.526
0.924
0.019
1.085
5.471
1.408
0.257
7.183
4.300
0.796
Significance
0.019 *
0.927
0.720
0.549
0.983
0.000 *
0.002 *
0.810
0.220
0.339
0.892
0.300
0.021 *
0.238
0.613
0.009 *
0.041 *
0.374
Transformed Variable: Average
* Denotes significant difference
** All tests have 1, 98 degrees of freedom
Each of the above measures showing significant difference will be examined to find
the cause(s) of the difference that the factor reflected in the Multivariate Test (Table 4.1).
Page 79
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.5.1 Experience with Computers, Measure Lost
As shown in diagram 4.8, increasing experience with computers is associated with a
decreasing feeling of being lost.
5
Lost Rating:
0 = Not at All
5 = Frequently
4
3
Experience with Computers:
0 = Inexperienced
5 = Very Experienced
Lost Rating
2
1
0
0
1
2
3
4
5
Diagram 4.8: Scatter Plot - Exp. with Computers, Measure Lost
4.3.5.2 Experience with Computers, Measure Task Time
As displayed in diagram 4.9 below, increasing experience with computers is
associated with decreasing Task time.
400
Task time:
Average of time taken to
complete tasks both sites.
300
Experience with Computers:
0 = Inexperienced
Task time
200
100
0
0
1
2
3
4
5
Diagram 4.9: Scatter Plot - Exp. with Computers, Measure Task Time
Page 80
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.5.3 Experience with WWW, Measure Lost Rating
The scatter plot below shows that as a subject’s experience with the World Wide
Web (WWW) increases the perception of being lost across both Web sites decreases. The
line of best fit demonstrates this.
5
4
Lost Rating:
0 = Not at All
5 = Frequently
3
Experience with WWW:
0 = Inexperienced
Lost Rating
2
1
0
0
1
2
3
4
5
Experinence with WWW
Diagram 4.10: Scatter Plot - Exp. with WWW and Lost Rating
4.3.5.4 Gender by Age, Measure Lost Rating
As is displayed in diagram 4.11, men in both age categories have similar mean
ratings of lost. However, women in the older age group felt more lost than their younger
counterparts. Please note that this effect is independent of experience with computers or the
WWW.
2.6
2.4
2.2
Lost Rating:
0 = Not at All
5 = Frequently
2.0
Mean Lostness
1.8
1.6
1.4
Gender
1.2
male
1.0
18 - 25
female
Over 25
AGE
Diagram 4.11: Gender by Age, Measure Lost Rating
Page 81
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.5.5 Gender by Age, Measure Comfort Rating
As displayed in diagram 4.12 below, women over the age of 25 in the sample tend to
feel less comfortable with the structures of both Web sites. Whereas men over 25 feel
significantly more comfortable with the structures of both sites, than do men under 25.
3.8
3.6
Mean Comfort Rating
3.4
Comfort Rating:
0 = Not at All
5 = Very
3.2
Gender
3.0
male
female
2.8
18 - 25
Over 25
AGE
Diagram 4.12: Gender by Age, Measure Comfort Rating
4.3.5.6 Gender by Age, Measure Number of Nodes Visited
As displayed in diagram 4.13 below, women over the age of 25 in the sample tend to
visit more nodes in both Web sites to complete the tasks, whereas men over 25 visit fewer
nodes than their younger counterparts.
16.5
16.0
Mean Number of nodes visited
15.5
Comfort Rating:
0 = Not at All
5 = Very
15.0
14.5
Gender
14.0
male
13.5
18 - 25
female
Over 25
AGE
Diagram 4.13: Gender by Age, Measure Number of Nodes Visited
Page 82
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.5.7 Summary of Between-Subjects Effects
The dependent measure lost showed significant between-subject effects on the
following factors; experience with computers, experience with WWW and gender and age
interaction:
i)
Subjects with low experience with computers feel more lost in both Web
sites.
ii)
Subjects with low experience with WWW feel more lost in both Web sites.
iii)
Women over 25 tend to feel more lost in both Web sites.
The dependent measure task time showed a significant between subject effect on the
factor experience with computers. Subjects with low experience with computers take
longer to complete the task on both Web sites.
The dependent measure comfort displayed a significant between subject effect on
the factor gender by age interaction. Men over 25 in the sample feel more comfortable with
both Web sites than their younger counterparts. This pattern is reversed for women. Women
over 25 in the sample feel less comfortable than their younger counterparts.
The dependent measure nodes visited showed a significant between-subject effect on
the factor gender by age interaction. Men over 25 in the sample visited fewer nodes to
complete the task in both Web sites than their younger counterparts. This pattern is reversed
for women. Women over 25 in the sample visited more nodes to complete the task in both
Web sites than their younger counterparts.
Page 83
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.6 Summary of General Linear Model – Repeated Measures
Upon examination of the results of the sample, the following null hypotheses tested
by the doubly multivariate repeated measures analysis of variance were supported.
4.3.6.1 Supported Within-Subject Null Hypotheses
•
H0: There will be no influence on the difference between the red and blue Web site
measurements (Lost rating, usage of Back button, usage of History list, Comfort rating,
number of nodes visited and Task time), attributable to experience with computers.
•
measurements, attributable to gender.
•
measurements, attributable to age by gender.
•
measurements, attributable to age by order.
•
measurements, attributable to gender by order.
4.3.6.2 Supported Between-Subject Null Hypotheses
•
H0: There will be no influence on the averages of measurements (Lost rating, usage of
Back button, usage of History list, Comfort rating, number of nodes visited and Task
time) on the red and blue Web sites, attributable to gender.
•
H0: There will be no influence on the averages of measurements on the red and blue Web
sites, attributable to age.
•
sites, attributable to order.
•
sites, attributable to age by order.
•
sites, attributable to gender by order.
Page 84
ANALYSIS OF DATA
__________________________________________________________________________________________
Examined next will be the hypotheses rejected by the doubly multivariate analysis of
variance. Each will be briefly discussed.
4.3.6.3 Rejected Within-Subject Null Hypotheses
•
measurements, attributable to experience with the WWW.
This hypothesis was rejected as experience with the WWW seems to affect the
measure of task time. The sample indicates that inexperienced subjects take longer to
complete the task in the red Web site than in the blue Web site.
•
measurements, attributable to age.
Hypothesis rejected due to the fact that when examined it was found that in this
sample those subjects in the 18 - 25 age group completed the task in the blue Web site faster
than the task in the red Web site. Whereas subjects in the over 25 age group completed the
task in the red Web site faster than the task in the blue Web site.
•
measurements, attributable to order.
This hypothesis was rejected as the analysis shows that there is an effect on the
measurement of task time. Regardless of which Web site subjects completed first, they
completed the second site faster. This measure seems to reflect experience with the tasks set
on the two Web sites
Page 85
ANALYSIS OF DATA
__________________________________________________________________________________________
•
H0: There will be no difference between the red and blue Web sites on the mean
measures of the dependent variables (Lost rating, usage of Back button, usage of History
list, Comfort rating, Number of nodes visited and Task time).
This hypothesis was rejected as there are differences in the measures between the red
and blue Web sites. These differences are on the measures of Back button, History list and
task time. The difference on the Back button had been expected, as the red Web site
required the use of the Back button to complete the tasks. However the level of usage of the
Back button in the blue Web site was unexpected. The blue Web site had been designed in
such a fashion that usage of this facility was not required.
Subjects in this sample used the Back button and the History list more frequently in
the red Web site. However usage of the History list was very low in both sites.
The difference highlighted by the task time has been reflected in the preceding
higher order interactions, but was examined to show the overall averages of time to complete
the task set in both Web sites. Subjects completed the task more quickly in the blue Web site
than the red Web site.
Page 86
ANALYSIS OF DATA
__________________________________________________________________________________________
4.3.6.4 Rejected Between-Subject Null Hypotheses
•
sites, attributable to experience with computers.
This hypothesis was rejected due to the fact that experience with computers seemed
to affect the measures lost and task time. The sample shows that inexperienced subjects feel
more lost and take longer to complete the tasks in both Web sites.
•
sites, attributable to experience with the WWW.
The hypothesis was rejected as the sample shows that as subjects’ experience with
the WWW increases the perception of being lost across both Web sites decreases.
•
sites, attributable to age by gender.
This hypothesis was rejected as in the sample women over 25 tend to feel more lost
and less comfortable with both Web sites. Men over 25 feel less lost in and significantly
more comfortable with the structures of both Web sites than men in the 18-25 age group do.
Women in the 18-25 age group and men over 25 visit fewer nodes in both Web sites to
complete the required tasks.
The next section of the thesis, section 4.4, Logistic Regression analysis will examine
all of the measures and factors examined in the multivariate analysis to identify which of the
measurements is a predictor of preference (red or blue Web site).
Page 87
ANALYSIS OF DATA
__________________________________________________________________________________________
4.4 Logistic Regression
“Logistic regression is useful for situations in which you want to be able to predict the
presence or absence of a characteristic or outcome based on values of a set of predictor
variables. It is similar to a linear regression model but is suited to models where the
dependent variable is dichotomous. Logistic regression coefficients can be used to estimate
odds ratios for each of the independent variables in the model. Logistic regression is
applicable to a broader range of research situations than discriminant analysis.”
“Data. The dependent variable should be dichotomous. Independent variables can be
interval level or categorical; if categorical, they should be dummy or indicator coded.”
(Norusis, 1997, p.1)
This section of the thesis will analyse the data collected from the experiment to
identify which component(s) of the data predicts user preference of Web site (red or blue).
The model used to analyze the data is a logistic regression. The dependent variable in this
analysis is preference; this variable is dichotomous (red or blue). The data components and
their characteristics analyzed are:
•
Experience with Computers; interval level data.
•
Experience with WWW; interval level data.
•
Gender; categorical data, dummy values; 0 = male; 1 = female.
•
Age; categorical data, dummy values; 0 = 18-25; 1 = over 25.
•
Order (red or blue Web site 1st), categorical data, dummy values; 0 = red; 1 = blue.
•
Usage of Back button; red Web site value minus blue Web site value; interval level data.
•
Usage of History list; red Web site value minus blue Web site value; interval level data.
•
Lost; red Web site value minus blue Web site value; interval level data.
•
Comfort; red Web site value minus blue Web site value; interval level data.
•
Time to complete (task time); red Web site value minus blue Web site value; interval level
data.
•
Visited nodes; red Web site value minus blue Web site value; interval level data.
Page 88
ANALYSIS OF DATA
__________________________________________________________________________________________
The backward conditional method was employed to estimate a model containing only
significant parameters. The backward conditional method resulted in a model containing 4
predictors, Lost, Comfort, Task time and Visited nodes.
Overall significance of the model is tested by a chi-square statistic: x2 = 77.8275 with
11 df, Probability = .000. Coefficients, univariate Wald statistics and significance values (df =
1) for each of the predictors are displayed in table 4.4 below.
Table 4.4:
Predictors of Preference
Predictor
Lost
Comfort
Task time
Visited nodes
Constant
Coefficient (B)
-.6169
.9618
-.1952
-.0105
-1.2964
Wald Statistic
6.3117
9.2313
4.4294
4.0892
11.7113
Significance
.0120
.0024
.0353
.0432
.0006
Subjects preferred the blue Web site to the red Web site 74:33.
The derived model predicted 4 of the 74 who preferred the blue site to prefer the red
site, and 8 of the 33 who preferred the red site to prefer the blue site. In all, the model
successfully predicts the red or blue Web site preference of 95 of the 107 subjects.
The fact that the model could not remove the perception based data, lost and
comfort is a clear indication that user perception of the structure is a factor in
preference.
Page 89
ANALYSIS OF DATA
__________________________________________________________________________________________
4.5 Modification of Hypertext Structure Metrics for the Web
In Chapter 3 the two experimental Web sites (red and blue) were subjected to the
metrics developed by Botafogo et al. (1992) and Rivlin et al. (1994). Whereas, these metrics
have successfully highlighted the difference between the two Web sites the large amount of
difference between the results bears closer examination. Table 3.1; comparison of red and
blue Web site metrics is reproduced below.
Table 4.5:
Comparison of red and blue Web site Metrics.
Attribute
Red Web Site
Blue Web Site
Most central node:
Node 1
Node 9
ROC value:
1,371.85
99.89
Next highest ROC
values:
Node 2 = 130.42
Node 12 = 99.10
Node 3 = 130.42
Node 14 = 98.31
Least central node:
Nodes 16-73
Node 1
ROC value:
69.43
46.94
Cp value:
.0496
.9809
St value:
.01203
.02471
Number of Nodes:
73
73
Number of Hyperlinks:
72
736
Cp (Compactness)
St (Stratum)
As, ROC values are designed to be used for comparisons between hypertexts the
results above would seem to be reflecting node 1 in the red Web site is 13.79 times (1,371.85 /
99.89) more central than node 9 in the blue Web site. This would appear to be an anomaly, as
both sites have the same number of nodes.
Page 90
ANALYSIS OF DATA
__________________________________________________________________________________________
Botafogo et al. (1992) state:
“For a single hypertext, the COD* is a good indication of node centrality as
compared with another node, but this number indicates little when two different hypertexts
are compared. For instance, a node with COD* 200 in a hypertext with 1000 nodes might be
much more central than a node with COD* 50 in a hypertext with 100 nodes. To try to
compensate for this difference, the Relative Out Centrality metric for node i is defined as:
ROC i = CD**/COD* i
The higher the ROC metric of a node, the more central (the inverse of the COD*).
Observe, also that the ROC is normalized in relation to the size of the hypertext (CD**),
making it more convenient for comparisons between hypertexts.”
* COD (Converted Out Distance)
** CD (Converted Distance)
The Cp results seem to be similarly skewed in that the red site has a value of .0496 in
comparison to the blue sites value of .9809. Yet when the experiment was analyzed the
estimated marginal means of time to complete the task (Diagram 4.7: Red – Blue, measure
Task time) of subjects, shows only a difference of 8.5% between the two Web sites. Overall,
it took subjects on average 8.5% longer to navigate the red Web site than the blue. To once
again quote Botafogo et al. (1992);
“… a compactness metric is developed that varies between 0 and 1, independent of
the hypertext size, and that reflects differences between hypertexts even they have the same
number of nodes and links. The metric is 0 when the hypertext is completely disconnected and
1 when completely connected.”
The value of the metric for the red Web site (.0496) reflects the fact that this site is
very close to being disconnected. This is not the case, this Web site is quite easy to traverse
as, evidenced by the fact that all 107 subjects successfully completed the task for this site.
Page 91
ANALYSIS OF DATA
__________________________________________________________________________________________
The results of both the ROC and Cp metrics, when compared with the experimental
evidence, do not support Botafogo et al.’s statements.
As stated in section 2.3.4.7 (Summary of Structure Metrics), the metrics fail to take
into account the increased freedom of browsing that the navigational aids inherent to the
browser, afford the user. In particular the design of the Converted Distance Matrix (CDM)
neglects the Back and History facilities (Section 2.2.4.1, Browsers) of all modern browsers.
In the design of the CDM, Hyperlinks are considered to be unidirectional. Once the
Hyperlink is traversed, there is no direct way to return to the previous node unless another
Hyperlink is provided to do so. In the case of no return Hyperlink being provided, the value
inserted into the CDM is either the infinity value or as Botafogo et al. recommends, the
conversion constant (Section 2.3.4.1). This also implies that some nodes may be unreachable
from other nodes.
As a user traverses a Web site by selecting which Hyperlinks to follow, the browser
stores the traversals followed by the user in a History list. The History list can be used to
return to a previous point in the path followed by the user. The browser does this
dynamically; that is, the page selected from the History list is loaded from the browser’s
cache rather than a “get” command being issued to the host. The Back facility is a subset of
the History facility, which returns the user dynamically to the previous page within the
History list.
As shown in section 4.3.4.5 (Red – Blue, Measures the use of Back Button and
History List), subjects used the History facility very infrequently.
However, most subjects in the experiment used the Back facility in both experimental
Web sites, suggesting that this facility is utilized whether required or not by users when
traversing Web sites. Therefore, the modification to the Converted Distance Matrix (CDM)
will specifically address the use of the Back facility within a strictly hierarchical Web site.
Page 92
ANALYSIS OF DATA
__________________________________________________________________________________________
4.5.1 Improved Converted Distance Matrix for a Web site
Constructed as a Strict Hierarchy
Assumption 1: User starts traversing Web site at the root node (Homepage).
Assumption 2: Web site is a strict hierarchy.
Definition 1: CDMh – Improved Converted Distance Matrix for a Web site with strict
hierarchy.
Definition 2: Bi-followable Hyperlink – Once a unidirectional Hyperlink has been
traversed in a Web site, and no dedicated return Hyperlink is provided, the link can be
considered to be bi-followable that is, the user has the ability to use the Back facility to return
to the previous node. This description is similar to the bifollowable link proposed in Nelson’s
Xanadu (Section 2.3.2). This type of Hyperlink is not however, bivisible as it cannot be seen
in the destination document (Deemer, 1994; Nelson, 1992).
When considering the Converted Distance Matrix in relation to a strict hierarchical
Web site, where the user enters site at the root node, all Hyperlinks should be considered to be
bi-followable (CDMh). This modification recognizes the Back facility provided by all modern
browsers.
Page 93
ANALYSIS OF DATA
__________________________________________________________________________________________
Following is an example of Botafogo et al.’s converted distance matrix (CDM) using
the conversion constant (Section 2.3.4.1) applied to a small, strict hierarchical Web site.
Example using Botafogo et al. (1992) Conversion Constant
K =5
a
b
d
c
e
a
b
c
d
e
a
0
1
1
2
2
b
5
0
5
1
1
c
5
5
0
5
5
d
5
5
5
0
5
e
5
5 5
5
0
The value 5 is chosen as
there are 5 nodes in graph
(recommended in
Botafogo et al. (1992).)
Diagram 4.14: Converted Distance Matrix (CDM)
The shortest path from node a to node b is one. When a node cannot directly reach
another node via a dedicated Hyperlink, the conversion constant (K) is used within the
distance matrix. For example in Figure 4.14, node b cannot directly reach node a so the
conversion constant is inserted in the distance matrix. However, by utilizing the Back facility
of a browser, the traversal from node b to node a requires only a single mouse click on the
back button. If the user’s point of entry to the Web site was the root node (Assumption 1), to
have arrived at node b the user must have traversed the Hyperlink a–b.
This traversal using the Back facility of the browser is a traversal of a single
Hyperlink, not as the Botafogo et al.’s conversion constant CDM suggests, 5 traversals.
This modification (CDMh) to Botafogo et al.’s Converted Distance Matrix design
removes the need for the infinity value or the conversion constant (Section 2.3.4.1) in the
CDM. The recognition of all links within a Web site as being bi-followable will change the
results of Botafogo et al.’s (1992) metrics when applied to strictly hierarchical Web sites.
Page 94
ANALYSIS OF DATA
__________________________________________________________________________________________
Following is an example of CDMh – Converted Distance Matrix for a Web site with
strict hierarchy, applied to the same small Web site.
Example using CDMh
a
b
d
c
e
a
b
c
d
e
a
0
1
1
2
2
b
1
0
2
1
1
c
1
2
0
3
3
d
2
1
3
0
2
e
2
1 3
2
0
Diagram 4.15: Strict Hierarchical Web Site Converted Distance Matrix (CDMh)
In diagram 4.15 the shortest path from node a to node b is one. When Back facility of
the browser is recognized and applied to the CDM all Hyperlinks present within a Web site
are treated as bi-followable Hyperlinks, therefore, only a single traversal is required to move
from node b to node a.
A further example of this is, the traversal from node e to node c. As stated this
modification (CDMh) assumes the user enters the Web site at the Homepage (root node), node
a in this case.
In this example, to reach node e the user traversed the path a – b, b – e. To traverse
the Web site from node e to node c requires the user to use the Back button twice (e – b, b –
a) and the traversal of a – c. This is a traversal of three Hyperlinks, as is reflected by the
CDM h.
Page 95
ANALYSIS OF DATA
__________________________________________________________________________________________
4.5.2 Application of Modification to the Experimental Red Web Site
The CDMh was applied to the experimental red Web site. The effect the modification
had on the results of Botafogo et al. (1992) metrics, when applied to the Web site is displayed
in Table 4.5. The complete CDM h and Botafogo et al.’s CDM using the conversion constant
are available in Appendix H. The blue Web site values are displayed as, the blue site contains
dedicated hyperlinks performing the job that the CDM h bi-followable links reflect, and
therefore a valid comparison can be made (blue Web site distance matrix available in
Appendix C).
Table 4.6:
Original and Modified red Web site metrics compared to blue Web site metrics
Red Web site
Blue Web site
Attribute
Botafogo et al.
CDM
CDMh
Botafogo et al.
CDM
Most central node:
Node 1
Node 1
Node 9
ROC value:
1,371.85
112.42
99.88
Next highest ROC
values:
Node 2 = 130.42
Node 2 = 112
Node 12 = 99.10
Node 3 = 130.42
Node 3 = 112
Node 14 = 98.31
Least central node:
Nodes 16-73
Nodes 40-44
Node 1
ROC value:
69.43
67.66
46.94
Cp value:
.0496
.9349
.9809
St value:
.01203
.00037
.02471
Number of Nodes:
73
73
73
Number of
Hyperlinks:
72
72
736
Cp (Compactness)
Page 96
St (Stratum)
ANALYSIS OF DATA
__________________________________________________________________________________________
Using the CDMh modification to the red Web site, the values of the ROC calculations
for the red and blue Web sites are much closer in absolute value. The results of the
calculations using the unmodified CDM have node 1 (red Web site) showing a value of
1,371.85 and node 9 (blue Web site) a value of 99.66. The calculations using the CDMh
modified values are 112.42 (red site) versus 99.66 (blue site). This provides a better
comparison of node centrality between the two sites.
With the modification to the CDM of the red Web site each of the 72 Hyperlinks on
the site is treated as bi-followable. Node 1 of the red Web site (Homepage), has the highest
ROC value (112.42) followed by nodes 2 and 3 (112) this reflects the fact that within this
structure there is only one hierarchy. This is further evidenced by the fact, that all the level 5
nodes (Appendix A for site map) have the low centrality values ranging from 67.66 (nodes
40-44) to 70.88 (nodes 23-32).
The Cp (Compactness) value for the red Web site using the unmodified CDM is
.0496. The Cp value for the blue Web site is .9809. These two values are at opposite ends of
the spectrum, as zero describes a totally disconnected structure and one describes a totally
connected structure (Botafogo et al., 1992).
With these values, it would be expected that
166
traversing the red site would be a much more
164
arduous and time-consuming undertaking
162
than traversing the blue site. The results of the
160
supposition. The mean time taken to complete
the task does show a difference, however, the
difference does not appear to be of the
magnitude reflected by the difference in
Task Time in seconds
158
experiment (Chapter 3) do not support this
compactness. Diagram 4.16 displays the mean
Task time per site.
156
154
152
150
148
Red site
Blue site
Diagram 4.16: Estimated Marginal
Means of Task Time per Site
Page 97
ANALYSIS OF DATA
__________________________________________________________________________________________
The calculated value for compactness of the red Web site using the CDMh
modification is .9349. This value reflects the fact that each node can easily reach another
node. The longest path from node to node within the red Web site traverses 8 Hyperlinks
(Appendix H for CDMh). The average* traversal path within the red Web site is across 5.69
Hyperlinks.
If the Botafogo et al.’s (1992) original distance matrix values are used (Appendix H
for CDM), the longest path in the red Web site would be a traversal of 73 Hyperlinks and the
average* path traversal 69.43 Hyperlinks. Obviously, this is not the case.
For comparison purposes the value for compactness of the blue Web site is .9809, this
value indicates that each node can reach any other node within the site easier than is the case
in the red Web site. The longest path from node to node in the blue Web site traverses 4
Hyperlinks (Appendix C for CDM). The average* traversal path within the blue Web site is
across 2.37 Hyperlinks.
The comparison between the two Web sites using the CDMh modification to the red
Web site clarifies and supports the result found in the Estimated Marginal Means of task time
on both sites (Diagram 4.16).
* Average is calculated using formula CD/(n2-n)
CD = Converted Distance
n = Number of nodes
The CDMh modified value for Stratum (St) on the red Web site (.0037) reflects that
there is little or no linear ordering in the site. In comparison however, the blue Web site
displays a higher order of magnitude result than the CDMh modified red Web site. This could
be the result of the linear ordering of the sequential structures contained in the level 5 nodes
of the blue Web site (see site map Appendix A).
The results of the application CDMh modification to the metrics developed by
Botafogo, Rivlin and Shneiderman (1992; 1994) would appear to provide a more accurate
description of the structural characteristics of the experimental red Web site.
Page 98
ANALYSIS OF DATA
__________________________________________________________________________________________
4.5.3 How the CDMh Affects the Methodology of this Thesis
In Chapter 3, it was stated the two Web sites were designed to be at the extremes of
Botafogo et al.’s, (1992) metrics. The metrics were applied to the two Web sites to
demonstrate the difference between the two Web sites. With the application of the CDMh to
the red Web site the value of all metrics pertaining to the red Web site changed.
Does this affect the methodology?
Whereas, the value of each of the metrics developed by Botafogo, Rivlin and
Shneiderman (1992; 1994) change for the red Web site, the structure they identify does not.
The application of the CDMh provides only for a better description of the characteristics of the
structure of a strictly hierarchical Web sites.
Even though the values of the metrics ROC and Cp, of the red Web site now show
much less difference in value to those of the blue Web site, the structures the experiment
described remain the same. The red site is a strict hierarchy and the blue site is a heavily
cross-linked (compromised), overlapping hierarchy using sequential (next) structures’ within
it. The experiment therefore, still accurate in such as it measures the difference of user
perceptions to these two different structures.
4.5.4 Multiple and Compromised Hierarchies and the CDMh
The CDMh modification is designed
Entry point
for use only on a strict hierarchy, where the
root node of the hierarchy is the only point of
b
entry to the site. The modification is
a
unsuitable to an overlapping hierarchical
structure (Section 2.3.3.1), as parts of the
c
d
structure would become unreachable. Diagram
4.17 demonstrates this; entry at node a allows
easy access to nodes c, d and e, however, node
b can not be accessed.
e
Diagram 4.17: Overlapping Hierarchy
Page 99
ANALYSIS OF DATA
__________________________________________________________________________________________
The same problem (unreachable nodes) can occur when entry to the Web site occurs
at any other location than the root node. This may occur if sub-nodes have been listed with
Web search engines or users of the Web site have added sub-nodes as a Favorite (Microsoft)
or Bookmark (Netscape).
The CDMh modification is also unsuitable for compromised (cross-linked) Web site
hierarchies (Section 2.3.3.1). Diagram 4.18 demonstrates the problem; the traversal from
node d to node a is dependent on the path the user took to reach node d. The user could have
taken the path a – c, c – d or the path a – b, b – c, c – d.
In either case, the return traversal to
Entry point
node a requires the same number of
traversals as the path to the current node.
a
The result is ambiguous (2 or 3 dependent
on users path), and therefore cannot be
b
c
used within the Converted Distance
Matrix as a static metric value.
d
Diagram4.18:
4.18:Compromised
CompromisedHierarchy
Diagram
4.5.5 Where to Use the CDMh Modification
The strict hierarchical modification to the converted distance matrix (CDMh) can
only be used on a Web site that meets the following criteria:
1.) Web site is a strict hierarchy.
2.) Web site has a single entry point, at the root node of the hierarchy.
The above criteria describes a considerable number of smaller Web sites, particularly
personal and small commercial Web sites, as the Homepage is the entry point to a strict
hierarchical site. In these cases the application of the CDMh modification to the Converted
Distance Matrix reflects a more appropriate measure of the structure, for comparison
purposes, when subjected to the metrics developed by Botafogo, Rivlin and Shneiderman
(1992; 1994).
Page 100
ANALYSIS OF DATA
__________________________________________________________________________________________
4.6 Examination of Lostness & Efficiency Indicators and the Web
As stated in section 3.6, Data Collected, it had been initially hoped that the user’s
traversal of the Web site could be tracked. This information would have allowed the metrics
developed by Smith (1996), described in section 2.3.5, to be applied to each subject of the
experiment. Due to the design of Internet Explorer 3.02 the exact traversals within a Web
site could not be accomplished within the scope of this thesis.
The intent of this section of the thesis is to show, using the information collected
from the server (Section 3.6), whether Smith’s (1996) indicators of “Lostness” and
“Efficiency” do in fact identify users who are lost. The details extracted from the server
provide the number of nodes visited by each user in both Web sites. A perfect search will be
conducted and the results compared with each experimental subject’s result. The following
of Smith’s (1996) indicators will be examined:
Lostness - Efficiency Indicator:
Number of visited nodes to complete required task (R) divided by number of
different nodes visited while searching (N).
Lostness - Efficiency = R/N
A value-approaching zero or greater
than one (task incomplete), indicate
the user is lost and inefficient.
Also, when considering a user’s efficiency, successful completion of the task is a
good indicator. A value is assigned for task accuracy (A) zero for incorrect answer, .5 for a
partially correct answer and one for a correct answer.
All 107 subjects who undertook the experiment provided the correct answers,
therefore, for all subjects A = 1.
Page 101
ANALYSIS OF DATA
__________________________________________________________________________________________
Before the Lostness - Efficiency indicators can be calculated, a perfect search must
be conducted to calculate the number of visited nodes to complete the required task. Below is
the result of a perfect search taken from the server’s “access_log” for both the red and blue
Web sites.
Table 4.7:
A perfect search – Results from Access_log
Machine ID
Date and Time
Node Visited
b204-17.scu.edu.au - - [10/Sep/1997:12:07:57 +1000] "GET /~wsmart10/red/marvel.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:07:59 +1000] "GET /~wsmart10/red/mmf.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:08:00 +1000] "GET /~wsmart10/red/mfem.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:08:01 +1000] "GET /~wsmart10/red/guardian.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:08:31 +1000] "GET /~wsmart10/red/dc.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:08:32 +1000] "GET /~wsmart10/red/dmf.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:08:34 +1000] "GET /~wsmart10/red/dmale.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:08:35 +1000] "GET /~wsmart10/red/cham.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:08:45 +1000] "GET /~wsmart10/blue/dc.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:08:47 +1000] "GET /~wsmart10/blue/dvmf.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:08:48 +1000] "GET /~wsmart10/blue/dvfem.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:08:50 +1000] "GET /~wsmart10/blue/clay.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:09:15 +1000] "GET /~wsmart10/blue/mvfem.htm
b204-17.scu.edu.au - - [10/Sep/1997:12:09:18 +1000] "GET /~wsmart10/blue/calypso.htm
Notes about perfect search:
•
Perfect search of red Web site requires 9 nodes be visited (includes index.html).
•
Total time to complete task in red Web site was 38 seconds (includes 25 seconds to
record answer).
•
Perfect search of blue Web site requires 7 nodes be visited (includes index.html).
•
Total time to complete task in blue Web site was 33 seconds (includes 25 seconds to
record answer).
Page 102
ANALYSIS OF DATA
__________________________________________________________________________________________
The indicators of Lostness and Efficiency use the same calculation. This calculation
is number of nodes required to be visited/number of nodes actually visited. The second
indicator of efficiency is completion of task; all 107 subjects completed the task in both Web
sites. The table below displays the 15 highest and 15 lowest results of the indicators of
Lostness and Efficiency (complete results available in Appendix I) for each Web site (red
and blue). Also shown is the preference (red or blue Web site) of subjects. Task time is
displayed for comparison to perfect search (38 sec. red Web site; 33 sec. blue Web site).
Table 4.8:
Excerpt - Table of Results Lostness/ Efficiency Indicator
Results ordered by visited nodes red site
Site
Preference
Blue
Red
Red
Red
Blue
Blue
Red
Blue
Red
Blue
Blue
Blue
Blue
Blue
Red
Visited
nodes red
site
9
10
10
10
10
10
10
10
11
12
12
12
12
13
13
Blue
Red
Red
Blue
Blue
Blue
Blue
Blue
Blue
Red
Blue
Blue
Blue
Red
Blue
18
18
19
19
20
20
20
20
20
20
21
21
21
26
27
Results ordered by visited nodes blue site
Lostness & Task time
Efficiency
red site
Indicator
1.00
0.90
0.90
0.90
0.90
0.90
0.90
0.90
0.82
0.75
0.75
0.75
0.75
0.69
0.69
40
59
89
244
104
100
70
63
81
168
181
72
121
235
74
0.50
0.50
0.47
0.47
0.45
0.45
0.45
0.45
0.45
0.45
0.43
0.43
0.43
0.35
0.33
199
136
146
203
252
226
285
512
339
196
189
393
163
270
322
Site
Visited
Preference nodes blue
site
Blue
8
Blue
9
Blue
9
Blue
9
Blue
9
Blue
10
Blue
10
Blue
10
Blue
10
Blue
10
Blue
10
Blue
10
Blue
10
Blue
10
Red
11
Red
Red
Red
Red
Red
Blue
Blue
Red
Red
Red
Red
Red
Red
Red
Red
Page 103
17
17
18
19
19
21
25
26
27
34
35
38
38
45
50
Lostness &
Efficiency Task time
Indicator
blue site
0.88
0.78
0.78
0.78
0.78
0.70
0.70
0.70
0.70
0.70
0.70
0.70
0.70
0.70
0.64
59
30
49
73
109
86
68
79
76
124
142
134
99
71
202
0.41
0.41
0.39
0.37
0.37
0.33
0.28
0.27
0.26
0.21
0.20
0.18
0.18
0.16
0.14
169
198
159
199
106
107
199
390
376
348
458
209
528
215
433
ANALYSIS OF DATA
__________________________________________________________________________________________
Table 4.8 demonstrates that these indicators in fact do, appear to measure lostness
and efficiency. This can clearly be seen in the 15 lowest lostness – efficiency indicator
results for each Web site.
Subjects who were most lost in the red Web site, as indicated by the low result on the
indicators of lostness and efficiency, prefer the blue Web site (11 of 15, or 73%).
Similarly subjects who were most lost in the blue Web site, as indicated by the low
result on the indicators of lostness and efficiency, prefer the red Web site (13 of 15, or 86%).
The group of 15 subjects reflected in the lowest Lostness and Efficiency indicators in
the blue Web site (Table 4.8) is interesting.
Out of the 107 subjects who undertook the experiment (Chapter 3), 33 preferred the
red Web site. The 15 subjects, who were most lost in the blue Web site, account for 39.4 %
of subjects in the experiment who preferred the red Web site. This clearly shows that these
subjects were uncomfortable / lost within the structure of the blue Web site.
As demonstrated, Smith’s (1996) indicator of lostness, when applied to the data
resulting from the experiment does reflect an accurate measure of a user’s lostness.
Since efficiency is dependent on lostness the first indicator of efficiency uses the
same formula as the indicator of lostness. The more lost a subject is the less efficient the
subject is. The average time of completion for the most efficient 15 subjects on both sites is
103.4 seconds. The average time of completion of task for the least efficient 15 subjects is
264.2 seconds. The above results also demonstrate that the efficiency indicator reflects an
accurate measure of the subject’s efficiency.
The second measure of efficiency is “Successful completion of task”. As every
subject completed the task set in both Web sites successfully, task accuracy (A) for all
subjects, in both Web sites = 1.
Page 104
ANALYSIS OF DATA
__________________________________________________________________________________________
4.7 Examination of Questions raised in Chapter 2
Several questions were raised in the course of the review of the relevant literature in
Chapter 2. Each of these questions will be addressed in this section of the thesis. The
conclusions reached for each question will be supported by the statistical analyses performed
in this chapter of the thesis or by the examination of the metrics discussed within this
chapter.
•
Do users utilize the features of modern browsers to aid navigation within a Web
site?
1. Back facility: As shown in section 4.3.4.5 (Red – Blue, Measures the use of Back Button
and History List), most users reported usage of this facility in both Web sites. It appears
that some subjects in this sample utilize this facility even when not required by the
structure of a Web site.
2. History facility: As shown in section 4.3.4.5 (as above), users appear not to avail
themselves of this facility frequently. This facility was used more often in the
experimental red Web site; however, the frequency of use for both Web sites was very
low.
3. Favorites facility: In section 4.2.1(Data Excluded) the results of the question dealing
with the utilization of the Favorites facility showed only 2.8 % of subjects (3 of the 107
subjects) in this sample used the facility. As a result the usage of the favorites facility
was dropped from the statistical analysis.
Some subjects in this sample appear to use the Back facility provided by modern
browsers to aid navigation within a Web site whether it is required or not (section 4.3.4.5).
The blue Web site was designed in such a fashion that users were not required to utilize this
facility. However, 31.8% of subjects reported high levels of usage of the Back facility
(frequency value >= 2.5 out of 5), with only 12.1% reporting no usage.
The History facility was infrequently used by the subjects in this sample, 40.1 % of
subjects reported they did not use this facility in the red Web site and 50.5 % of subjects
reported they did not use this facility in the blue Web site.
Page 105
ANALYSIS OF DATA
__________________________________________________________________________________________
The Favorite facility (Bookmark) was only used by 2.8 % of the subjects in this
sample. This would appear to suggest that users do not use this facility to navigate within a
Web site. The favorite facility would seem to be used more for navigation between Web
sites than within Web sites. This is supported by the following quote;
“Web users employ five general criteria to determine whether to bookmark a Web
page: (I) general usefulness, (ii) quality, (iii) personal interest, (iv) frequency of use, and (v)
potential future use.”
(Abrams, 1997)
Of the three users who used the favorite facility, two stated that they had
bookmarked the page for the next subject undertaking the experiment.
•
In the experiment detailed in Chapter 3, 69.2 % of subjects (74 of 107) preferred the
blue Web site. The blue Web site is a heavily cross-linked site using overlapping hierarchies
and sequential loops as part of its structure. The logistic regression analysis (Section 4.4)
identified 4 predictors of preference;
1. Lost; User Perception, Did you at any time feel lost navigating this site?
2. Comfort; User Perception, How comfortable were you with the structure of this site?
3. Task Time; Server Log, Time taken to complete the task.
4. Visited Nodes; Server Log, Number of Web pages (nodes) visited to complete task.
The model identified by the logistic regression is 88.8 % accurate, as it correctly
predicted 95 out of the 107 subjects’ preference, based on the model’s indicators.
Page 106
ANALYSIS OF DATA
__________________________________________________________________________________________
The doubly multivariate analysis of variance – repeated measures (Section 4.3)
identified several influences on the predictors of preference for this sample;
•
Lost;
1. Subjects with a low level of experience with both computers and the WWW were
more lost in both structures (Sections 4.3.5.1 and 4.3.5.3).
2. Women over 25 felt more lost in both structures (Section 4.3.5.4).
•
Comfort;
1. Women over 25 felt less comfortable with either structure, while men over 25 felt
more comfortable with both structures (Section 4.3.5.5).
•
Task Time;
1. Subjects with less experience with the WWW take longer to complete the task in a
strict hierarchy than a heavily cross-linked hierarchy (Section 4.3.4.1).
2. Subjects 25 and younger completed the task in the heavily cross-linked hierarchy
faster than their older counterparts, whereas, subjects over 25 completed the task
faster in the strict hierarchy than the younger group (Section 4.3.4.2).
3. Subjects with low levels of experience with computers take longer to complete the
task in both structures.
•
Visited Nodes;
1. Women over 25 visited more Web pages to complete the task in both structures,
whereas, men over 25 visited less Web pages in both structures to complete the task
(Section 4.3.5.6).
There is no doubt that these influences affected the outcome of some of the subject’s
preferences. However, 69.2 % to 30.8 % (74 to 33) of subjects preferred the heavily crosslinked structure.
Page 107
ANALYSIS OF DATA
__________________________________________________________________________________________
•
Which type of Web site -- strictly hierarchical or heavily crossed-linked -- increases
the user’s likelihood of experiencing the “Lost in Hyperspace” phenomenon?
The answer to this question appears to be both structures. The sample found some
subjects becoming lost and uncomfortable with the structure of both the Web sites. The
following diagrams demonstrate this;
5.0
Estimated Marginal Means of Comfort
Estimated Marginal Means of Lostness
5.0
4.0
3.0
2.0
1.0
preferred site
blue
0.0
red Web site
red
blue Web site
4.0
3.0
2.0
1.0
0.0
red Web site
preferred site
blue
red
blue Web site
Diagram 4.19: Estimated Marginal Means of Lost and Comfort
250
25
230
220
20
15
10
red Web site
preferred site
blue
red
Estimated Marginal Means of Task Time
Estimated Marginal Means of Visted Nodes
240
blue Web site
210
200
190
180
170
160
150
140
130
pref erred site
120
blue
110
100
red web site
red
blue Web site
Diagram 4.20: Estimated Marginal Means of Visited Nodes and Task Time
The diagrams above reflect several aspects of the sample. Overall subjects that
preferred the blue Web site were more lost, less comfortable, visited more nodes and took
longer to complete the task in the red Web site. The same pattern is reflected for those
subjects that preferred the red Web site. That is, subjects were more lost, less comfortable,
visited more nodes and took longer to complete the task in the blue Web site.
Page 108
ANALYSIS OF DATA
__________________________________________________________________________________________
Overall the strict hierarchy (red Web site) seems to cause a greater perception of
being lost in this sample, particularly when coupled with subjects inexperienced with
computers and the WWW (Sections 4.3.5.1, 4.3.5.3 and 4.3.5.4). However, those subjects
who preferred the red Web site reported the highest levels of feeling lost, visited more nodes
and took far longer to complete the task set in the blue Web site than the average for either
site.
Further evidence that both Web sites caused subjects to feel lost and uncomfortable
is demonstrated by the results of the lostness – efficiency indicators in section 4.6. The
indicators clearly showed that some subjects became lost and inefficient in both Web sites.
•
The metrics developed by Botafogo et al. (1992) for identifying structures in
HyperText were found to perform well (Sections 2.4.3 and 3.2.3) when applied to the
However, an anomaly appeared when the values of the metrics calculated for each
experimental Web site were compared to the results of the experiment (Section 4.5). The
metrics Relative Out Centrality and Compactness were specifically designed for
comparisons between Hypertext structures (Botafogo, Rivlin & Shneiderman, 1992; 1994).
When applied to Web sites accessed with modern browsers the comparisons appear
to be flawed. Specifically, the design of the distance matrix of a strict hierarchy seems to be
the cause of the anomaly as it ignores the browser’s navigational aids. A modification to the
distance matrix (CDMh) was detailed and tested in section 4.5.1 and 4.5.2. This modification
provides a more accurate measure of the structure of a strictly hierarchical Web site.
The metrics developed by Smith (1996) to measure lostness and efficiency in
HyperText could not be tested within the scope of this thesis, as the subjects’ paths through
the Web site could not be tracked. However, the indicators of lostness and efficiency could
be examined. These indicators were examined in section 4.6 and proved to be an accurate
indication of the subject’s lostness and efficiency within a Web site.
Page 109
ANALYSIS OF DATA
__________________________________________________________________________________________
4.8 Summary
This chapter subjected the data collected from the experiment outlined in Chapter 3
to two separate analyses. The first analysis performed (Section 4.3), a doubly multivariate
analysis of variance - repeated measures, identified influence effects on both experimental
sites.
The second analysis performed was a logistic regression (Section 4.4), this analysis
identified a model, using the measures lost, comfort, visited nodes and task time, that
predicts user preference of the experimental Web sites.
Section 4.5 of this chapter proposed a modification to the Converted Distance Matrix
of strictly hierarchical Web sites. The CDMh modification takes into account the Back
facility of modern browsers. This modification (CDMh) to the metrics developed by
Botafogo, Rivlin and Scheiderman (1992; 1994) provides a more accurate description of the
characteristics of a strictly hierarchical Web site.
Section 4.6 of this chapter tested the indicators of lostness and efficiency developed
by Smith (1996) and found them to be accurate when applied to Web sites.
Finally this chapter (Section 4.7) answered the questions raised in Chapter 2 of this
thesis.
The next chapter of this thesis will present the findings of this dissertation in relation
to the research question:
“What is the effect of Web site structure on the users’ perceptions of navigability?”
Page 110
SUMMARY AND CONCLUSIONS
__________________________________________________________________________________________
Chapter 5
5.1 Introduction
The overall Hypothesis of this dissertation is:
Ho:
Web site structure does not affect the user perception of navigability.
HA:
Web site structure does affect the user perception of navigability.
The experimental data collected from the experiment detailed in Chapter 3 and
analyzed in Chapter 4 does not support the null hypothesis. Therefore, the null hypothesis is
rejected and the alternative hypothesis is supported.
This chapter will present the findings of this dissertation in relation to the research
question:
“What is the effect of Web site structure on the users’ perceptions of navigability?”
In exploring the research question, four other related questions were raised;
1. Do users utilize the features of modern browsers to aid navigation within a Web site?
2. Which structure would users prefer -- heavily cross-linked or a strict hierarchy?
3. Which type of Web site -- strictly hierarchical or heavily crossed-linked -- increases the
Each of these questions will be briefly examined in section 5.3. The results of this
examination will be used in section 5.4 to provide conclusions to the overall research
question.
Page 111
__________________________________________________________________________________________
5.2 Summary of Methodology
Two Web sites labeled “red” and “blue” were created for comparison purposes
(detailed in Chapter 3). The content in both sites remained identical, with the exception of the
additional Hyperlinks in the blue Web site. The “red” Web site was a strict hierarchy, while
the “blue” Web site’s structure was a combination of compromised and overlapping
hierarchical structures (heavily cross-linked). The “blue” Web site also contained sequential
structures in the 5th level nodes. Appendix B provides a map for each Web site and a list of all
Hyperlinks for each node (page) of both sites.
Two tasks were set on each Web site for each of the 113 subjects who undertook this
experiment. These tasks required the subject to traverse the experimental Web sites for
information contained on a single page within the site. Each subject was required to answer a
questionnaire that collected details regarding;
•
General information (gender, age and experience with both computers and the WWW)
•
Perception information about each Web site’s structure (lost, comfort and preference)
•
Usage of facilities provided by modern browsers (Back button, History list and Favorites)
Additional data for each subject (time taken and pages visited) was taken from the
server log file. It had been initially hoped that the user’s traversal of the Web site could be
tracked. This information would have allowed the metrics developed by Smith (1996),
described in section 2.3.5, to be applied to each subject. However, to accomplish this the
browser itself would have to be modified (Section 3.6). Internet Explorer 3.02 caches each
Web page (HTML node) as it downloads. The next time a cached Web page is accessed,
whether via the back, history list or another Hyperlink, Internet Explorer accesses the client’s
cache, not the server (host). Using a modified browser that logged all URLs or sent a GET
command to the server with each access to a Web page could solve this. This modification
could not be accomplished within the scope of this thesis. The details that were extracted
from the server do, however, allow some of the indicators Smith (1996) developed for
lostness and efficiency to be calculated.
Six subjects were excluded from the sample (detailed in section 4.2) resulting in the
sample size being reduced to 107.
Page 112
__________________________________________________________________________________________
5.3 Summary of Findings
The objective of this research was to examine the effect of structure on users’
perceptions of navigability. To accomplish this, users compared two different structures
(detailed in previous section). The perceptions measured by the experiment were lostness,
comfort and preference of Web site. In the course of reviewing the current literature,
designing the Web sites and analyzing the data four additional questions were raised. These
questions and the conclusions reached for each follow;
1. Do users utilize the features of modern browsers to aid navigation within a Web
site?
Subjects in this sample appear to utilize the Back facility provided by modern
browsers to aid navigation within a Web site whether it is required or not The blue Web site
was designed in such a fashion that usage of the Back facility was not required, yet only
12.1 % of subjects reported no usage.. The History facility was infrequently used by the
subjects in this sample, 40.1 % of subjects reported they did not use this facility in the red
Web site and 50.5 % of subjects reported they did not use this facility in the blue Web site.
The Favorite facility (Bookmark) was only used by 2.8 % of the subjects in this sample, this
would appear to suggest that users do not use this facility to navigate within a Web site.
2. “Which structure would users prefer, heavily cross-linked or a strict hierarchy?
This question was answered by the preference of subjects within the sample. Possible
influences on this preference were investigated by examining the data set for influences using
a multivariate analysis (section 4.3), then determining the predictors of preference, section 4.4
(logistic regression analysis).
The results from the data set is that 69.2 % of subjects (74 of 107) in this sample
preferred the blue Web site. The blue Web site is a heavily cross-linked site using overlapping
hierarchies and sequential loops as part of its structure. The logistic regression analysis
(Section 4.4) identified 4 predictors of preference Lost, Comfort, Task Time and Visited
Nodes. The model identified by the logistic regression is 88.8 % accurate, as it correctly
predicted 95 out of the 107 subjects’ preference, based on the model’s indicators.
Page 113
__________________________________________________________________________________________
The doubly multivariate analysis of variance – repeated measures (Section 4.3)
identified several influences on the predictors of preference for the sample, these are listed
below;
•
Lost
Subjects with a low level of experience with both computers and the WWW were
more lost in both structures (Sections 4.3.5.1 and 4.3.5.3). Women over 25 felt more lost in
both structures (Section 4.3.5.4).
•
Comfort
Women over 25 felt less comfortable with either structure, while men over 25 felt
more comfortable with both structures (Section 4.3.5.5).
•
Task Time
Subjects with less experience with the WWW take more time to complete the task in a
strict hierarchy than a heavily cross-linked hierarchy (Section 4.3.4.1). Subjects 25 and
younger completed the task in the heavily cross-linked hierarchy faster than their older
counterparts, whereas, subjects over 25 completed the task faster in the strict hierarchy than
the younger group (Section 4.3.4.2). Subjects with low levels of experience with computers
take longer to complete the task in both structures.
•
Visited Nodes
Women over 25 visited more Web pages to complete the task in both structures,
whereas, men over 25 visited fewer Web pages in both structures to complete the task
(Section 4.3.5.6).
There is no doubt that these influences affected the outcome of some of the subjects’
preferences. However, 69.2 % to 30.8 % (74 of 107) of subjects preferred the heavily crosslinked structure (blue Web site).
Page 114
__________________________________________________________________________________________
3. Which type of Web site -- strictly hierarchical or heavily crossed-linked -- increases
the user’s likelihood of experiencing the “Lost in Hyperspace” phenomenon?
The answer to this question appears to be both. The sample found some subjects
becoming lost and uncomfortable with the structure of both the Web sites. Overall the strict
hierarchy seems to cause a greater perception of lost in this sample, particularly when coupled
with subjects inexperienced with computers and the WWW (Sections 4.3.5.1, 4.3.5.3 and
4.3.5.4). However, those subjects who preferred the red Web site, reported the highest levels
of the perception of being lost, visited more nodes and took far longer to complete the task set
in the blue Web site than the average for both sites.
Further evidence that both Web sites caused some subjects to feel lost and
uncomfortable is demonstrated by the results of the lostness – efficiency indicators in section
4.6. The indicators clearly showed that some subjects became lost and inefficient in both Web
sites.
The structures of both Web sites (red and blue) were subjected to the metrics
developed for HyperText by Botafogo, Rivlin and Shneiderman (1992; 1994) to identify the
hierarchical structures contained within each site (Section 3.2.3). The metrics clearly
identified the different hierarchical structures (Thuring et al., 1991; Woodhead, 1990; Section
2.3.3) within each Web site. However, when the values of the metrics for the red Web site
were compared with the results of the experiment an anomaly was discovered (Section 4.5).
Botafogo et al.’s (1992) metrics were developed before the introduction of modern
WWW browsers. As a result of this the metrics do not take into account the facilities offered
by modern browsers, particularly the Back facility.
Page 115
__________________________________________________________________________________________
The metrics Relative Out Centrality and Compactness were specifically designed
for comparisons between Hypertext structures (Botafogo, Rivlin & Shneiderman, 1992;
1994).
When applied to strictly hierarchical Web sites accessed with modern browsers the
comparisons appear to be flawed. Specifically, the design of the converted distance matrix of
a strict hierarchy seems to be the cause of the anomaly. A modification to the distance matrix
(CDMh) was detailed and calculated in section 4.5.1 and 4.5.2. This modification treats all
links within a strict hierarchy as bi-followable (two-way) links. This modification recognizes
the Back facility of modern browsers and provides for a more accurate description of the
characteristics of a strictly hierarchical Web site.
The metrics developed by Smith (1996) to measure lostness and efficiency in
HyperText could not be tested within the scope of this thesis, as the subject’s path through the
Web site could not be accurately tracked. However, Smith’s indicators of lostness and
efficiency were examined. These indicators were examined in section 4.6 and proved to be an
accurate indication of the subject’s lostness and efficiency within a Web site.
Page 116
__________________________________________________________________________________________
5.4 Conclusions About the Research Problem
In this study it was found that the structure of a Web site does affect the user
perception of navigability on the site. It was assumed (Section 1.6) that a user’s perception of
navigability is a function of perceptions lost, comfort and preference.
In section 4.4 a logistic regression was undertaken to identify what component(s) of
the data set predict user preference. The result of this analyses was lost, comfort, nodes
visited and task time were identified as the predictors of preference. Whereas, it can be argued
that there is interaction between these four predictors, the fact that the model could not
remove the perception based data, lost and comfort is a clear indication that user perception of
the structure is a factor in preference.
Users that experience a low level of lost, high level of comfort and prefer a particular
structure are considered to perceive that structure as more navigable. The inverse of this
condition, users who experience high level of lost, low level of comfort and do not prefer a
particular structure are considered to have experienced the “lost in hyperspace” phenomenon
within this structure.
Conclusions that can be drawn from this work:
•
The majority of users perceive a structure that allows multiple access (heavily crosslinked structure) to the content to be more navigable than a strict hierarchy.
•
Both types of Web sites tested -- strict hierarchy and heavily cross-linked structure -- can
cause a user to experience the “lost in hyperspace” phenomenon.
•
Inexperienced users of the WWW take longer to complete the task in a strict hierarchy
than a heavily cross-linked structure.
•
Users with low levels of experience with computers and the WWW feel more lost in both
structures tested.
•
Of the facilities offered by modern browsers, users most frequently use the Back facility
for navigation within a Web site. Users utilize this facility even if there are dedicated
Hyperlinks provided to perform this function.
•
The Favorites (Bookmarks) facility of modern browsers is not utilized by users as a
navigation aid within a Web site.
Page 117
__________________________________________________________________________________________
In exploring the structure of Web sites and HyperText, the metrics used to measure
these structures were brought into focus. The metrics developed by Botafogo, Rivlin and
Scheiderman (1992; 1994) were examined in detail. The evidence provided by the experiment
suggested an anomaly existed in the design of the Converted Distance Matrix (CDM) of a
strictly hierarchical Web site accessed by modern browsers. This anomaly is due to the fact
that the design of the CDM does not recognize the Back facility of the browser. A
modification (CDMh) which recognizes the Back facility by treating all Hyperlinks on a
strictly hierarchical Web site as bi-followable, was proposed to provide a more accurate
measure of the structure of a strictly hierarchical Web site. (Section 4.5).
5.5 Limitations
The experiment detailed in this thesis was structured to test the users’ perceptions
of navigability on two different Web site structures. No attempt has been made to use any of
the leading edge Web technologies, such as Java, Shockwave, Active X controls, database
connectivity or audio-visual media (Streaming video, real audio, etc), as the inclusion of any
of the previously mentioned technologies may alter the users’ perceptions of the Web site
structures.
Many types of Web site navigational aids are in use on the WWW (site maps,
navigation bars, etc.). However, the researcher’s intent was to test the users’ perceptions of
the structure of the Web site, rather than the various navigational aids available. Therefore, no
additional navigational aids were provided to the subjects.
The subjects were assumed to reflect the population of WWW users. The sample
contained a large number of 18 – 25 year old males (59 of the 107) and contained a low
number of female respondents (33 of the 107). Due to the small sample sizes in some of the
age/gender categories, the sample was transformed to have only two age categories 18-25 and
over 25.
Page 118
__________________________________________________________________________________________
The experiment was restricted to the campus of Southern Cross University as the
server and workstations, required to conduct the experiment were located on campus grounds.
The off campus delay in response time from the server would have distorted the measure of
time to complete the task for each Web site.
Smith’s (1996) metrics to determine the level of lostness in users could not be tested
due to the fact that each user’s exact traversal path could not be recorded. It would have been
desirable to test the effectiveness of Smith’s metrics.
Two types of structure were tested, a strict hierarchy and a heavily cross-linked
hierarchy (using compromised, overlapping and sequential structures). These structures were
chosen to be at the extremes of Botafogo et al.’s (1992) metrics. It is possible a structure
somewhere between the two would have been preferable to a larger majority of subjects.
5.6 Implications of Research for Web Site Design
When a Web site is being developed the design of the site should take into account the
users it is being designed for.
If the Web site has a specific target market the following should be considered:
•
Inexperienced users of both computers and the WWW felt more lost in both structures
tested -- strict hierarchy and heavily cross-linked hierarchy (Sections 4.3.5.1 and 4.3.5.3).
In the strict hierarchy, inexperienced WWW users took longer to complete the task set
(Section 4.3.4.1).
•
In both structures tested, women over 25 were found to feel more lost and uncomfortable,
and visited more pages within a site to complete the task set (Sections 4.3.5.4, 4.3.5.5 and
4.3.5.6).
The optimal type of Web site design for these groups of users would perhaps be a
carefully designed cross-linked structure, incorporating a sequential path through the site,
designed to assist users in exploration and comprehension of the site structure.
Page 119
__________________________________________________________________________________________
General Web site design considerations:
•
The majority of users perceive a structure, which allows multiple access to information to
be more navigable than a strict hierarchy. This was demonstrated by the fact that 69.2% of
subjects in the experiment preferred the heavily cross-linked structure.
•
In the experiment, users completed the task (located information) faster in the structure
that they preferred (Diagram 4.20, section 4.7). It must be noted that overall the lowest
mean time of task completion (on subject’s preferred site) was on the heavily cross-linked
(blue) Web site.
•
Of the facilities offered by modern browsers, the Back facility is most often used. In the
experiment, most users utilized this facility, even when structure of the site did not require
it. Also noteworthy, is the fact that all 107 subjects completed the task on the red Web
site. The design of the red Web site required a minimum of four traversals using the Back
button to complete the task.
The optimal design of a Web site should take into account these general design
considerations. Most users prefer a heavily cross-linked site, and yet have a complete
understanding of how to navigate a site with out the benefit
of the cross-linking. Heavily cross-linking a
site is time consuming for the developer, as
each link requires careful planning and testing.
A strict hierarchy is much simpler to design,
and as demonstrated users can navigate within
this type of site. Perhaps the optimal Web site
design is a combination of the three types of
hierarchical design; strict, compromised and
Compromised or Overlapping
Hierarchical Links
overlapping (Section 2.3.3.1). This would
balance the cross-linking (preferred by users)
Strictly Hierarchical Links
and the number of Hyperlinks the developer
would have to include.
Diagram 5.1 Combined Hierarchy
Page 120
__________________________________________________________________________________________
5.7 Further Research
There is much scope for further research on the subject matter covered within this
thesis. Such work should be undertaken on;
•
Additional subjects being added to the sample (provided in Appendix J). Effects were
found in the multivariate analysis (Sections 4.3.4.1, 4.3.4.2, 4.3.5.4, 4.3.5.5 and 4.3.5.6)
that dealt with age or gender by age. The five age categories in this sample were
transformed, due to small cell sizes (Section 4.2.2), into two age categories (18-25 & over
25). Additional subjects added to the sample would allow a more comprehensive
examination of these age-related effects.
•
Users’ perceptions to different levels of cross-linking within Web site structure. As
suggested in section 5.5 a structure between the extremes tested in this thesis may be
preferred by a larger majority of subjects.
•
Smith’s (1996) metrics should be tested experimentally for accuracy in reference to the
WWW. As stated in section 4.6 the indicators proposed by Smith accurately identify users
that are lost and inefficient. However, the other metrics proposed by Smith could not be
tested in the scope of this thesis.
•
Modifications to the metrics developed by Botafogo, Rivlin and Shneiderman (1992;
1994) are required, to recognize the facilities provided by modern WWW browsers. A
modification for use with strict hierarchies was suggested in section 4.5, further
modifications for other structures are required.
In an effort to assist further researchers, the data set collected from the experiment
detailed in Chapter 3 is included in Appendix J.
Page 121
__________________________________________________________________________________________
5.8 Conclusion
This thesis has demonstrated that the structure of a Web site does affect the user
perception of navigability.
Other interesting facts that were revealed in this thesis were;
Whereas, inexperienced WWW users took longer to complete the task in the red (strict
hierarchy) Web site than experienced users the blue Web site (heavily cross-linked structure)
demonstrated no significant difference in task time for either inexperienced or experienced
WWW users (Section 4.3.4.1).
Women over 25 felt less comfortable with either structure and visited more nodes
(pages) in both Web sites to complete the tasks while men over 25 felt more comfortable and
visited less pages with both structures to complete the task (Section 4.3.5.5).
The Back facility of a browser is used most frequently of all facilities of the browser
within a Web site, in fact, it appears to be utilized even when there is no requirement for this
facility to be used.
The History list facility is utilized infrequently by users for traversing within Web
sites. The Favorites (Bookmarks) facility is not utilized by users as a navigation aid within a
Web site.
Understanding a user’s perception of the structure of Web sites should allow Web site
developers to construct sites that are easier to understand and navigate. This would result in
an increase in speed of access to desired information on the site. The results of this thesis
show that a user who preferred a structure completed the task in that structure faster than in
the structure they did not prefer (Diagram 4.20, section 4.7). However, it must be noted that
overall the lowest mean time of task completion (on subject preferred site) was on the heavily
Page 122
BIBLIOGRAPHY
__________________________________________________________________________________________
BIBLIOGRAPHY
1. Abrams, D., 1997, “Human Factors of Personal Web Information Spaces”, Masters of
Science Thesis, Graduate Department of Computer Science, University of Toronto, URL
http://www.dgp.toronto.edu/~abrams/ revised 1997
2. Berghel, H., 1996, “The Client’s Side of the World-Wide Web”, Communications of the
ACM, USA, Vol. 39, No. 1, pp.30-40
3. Berners-Lee, T., 1992,93,94,95, “Style Guide for online hypertext”, URL
http://www.w3.org/Provider/Style/ revised 1997
4. Berners-Lee, T. & Calliau, R., 1990, “WorldWideWeb: Proposal for a HyperText
Project”, CERN, Geneva, Switzerland, 12th November 1990
5. Berners-Lee, T., Calliau, R., Luotonen, A., Nielsen, H.F., & Secret, A., 1994, “The World
Wide Web”, Communications of the ACM, USA, Vol 37, No. 8, pp. 76-82
6. Blythe, G., Montulli, L., Grobe, M. & Ware, S., 1996, “Lynx Users Guide Version 2.3”,
URL http://www.cc.ukans.edu/lynx_help/Lynx_users_guide.html revised 1996
7. Botafogo, R. A., Rivlin, E., & Shneiderman, B., 1992, “Structural Analysis of
Hypertexts: Identifying Hierarchies and Useful Metrics”, ACM Transactions on
Information Systems, USA, Vol. 10, No. 2, pp. 142-180
8. Bowman, C., Danzig, P., Mander, U., & Schwartz, F., 1994, “Scalable Internet Resource
Discovery: Research Problems and Approaches”, Communications of the ACM, August
1994, Vol.37, No.8, PP98-114
9. Brown, H., ed.,1991, “Hypermedia/Hypertext and Object-oriented Databases”, Chapman
& Hall, Great Britain
10. Bush, V., 1945, "As We May Think", The Atlantic Monthly, July 1945
11. Calliau, R., 1995, “Short History of HyperText”, URL
http://www.w3.org/pub/History.html, revised 1995
12. Calliau, R., 1995, “WWW Time Line”, URL
http://www.w3.org/pub/WWW/History.html, revised 3rd October 1995
13. Deemer, C., 1994, “What is HyperText?”, |T|E|L|E|C|I|N|E|, Summer 1995, Vol.3
14. Erickson, T., 1996, “The World-Wide Web as Social Hypertext”, Communications of the
ACM, Viewpoint, USA, Vol.39, No.1, pp. 15-17
15. Free, R. M., 1995, “MUD Frequently Asked Questions” URL
http://afarensis.graphcomp.com/mutt/mud.html revised 1995
Page 123
BIBLIOGRAPHY
__________________________________________________________________________________________
16. Gray, M., 1996, “Web Growth Summary”, URL
http://www.mit.edu/people/mkgray/net/web-growth-summary.html, revised 1996
17. GVU (Graphics Visualization & Usability Center), “GVU's 6th WWW User Survey”,
URL http://www.cc.gatech.edu/gvu/user_surveys/survey-10-1996/#highsum, GVU
Center, Georgia Institute of Technology, Atlanta, revised December 1996
18. Leiner, B.M, Cerf, V.G., Clark, D.D., Khan, R.E., Kleinrock, L., Lynch, D.C., Postel J.,
Roberts, L.G. & Wolff, S. 1997, “A Brief History of the Internet”, URL
http://www.isoc.org/internet-history.html , revised 20th February 1997
19. Lynch, P.L., & Horton, S.,1997, “Yale C/AIM Web Style Guide”, URL
http://info.med.yale.edu/caim/manual/contents.html revised 13th November 1997
20. Martin, J., 1990, “HYPERDOCUMENTS & How to Create Them”, Prentice Hall,
Englewood Cliffs, New Jersey, USA
21. NCSA (National Center for Supercomputing Applications), 1997, “NCSA Mosaic”, URL
http://www.ncsa.uiuc.edu/SDG/Software/Mosaic/, University of Illinois, revised January
1997
22. NCC (Netscape Communications Corporation), 1997, “Netscape Executive Team and
Mission”, URL http://home.netscape.com/comprod/exec_team.html revised April 1997
23. Nelson, T., 1992, “Literary Machines 93.1”, Mindful Press, Sausalito, California, USA
24. Norusis, M., 1997, “SPSS Professional Statistics 7.5”, SPSS Inc., SPSS Inc. USA
25. Proceedings 95, “AUUG’95 & Asia-Pacific World Wide Web’95 Conference &
Exhibition”, URL http://www.csu.edu.au/special/conference/apwww95, revised 1995
26. Phelps, K., 1994, “Xanadu: The Information Future”, URL
http://xanadu.com.au/xanadu/future.html, Xanadu Australia, revised 1994
27. Rada, R., 1991, “HYPERTEXT from TEXT to EXPERTEXT”, McGraw-Hill
International, Cambridge, United Kingdom
28. Ragett, D., 1997, “HTML 3.2 Reference Specification”, URL
http://www.w3.org/pub/www/TR/REC-html32.html revised 14th January 1997
29. Rivilin, E., Botafogo, R. & Shneiderman, B., 1994, “Navigating in Hyperspace:
Designing a Structure-Based Toolbox”, Communications of the ACM, USA, Vol. 37,
No. 2, pp. 87-96
30. Rose, H., 1996, “IRC Frequently Asked Questions”, URL
http://www.irchelp.org/irchelp/altircfaq.html, revised 13th December 1996
31. Shneiderman, B., 1992, “Designing the User Interface Strategies for Effective HumanComputer Interaction”, 2nd Edition, Addison-Wesley Publishing Co., USA
Page 124
BIBLIOGRAPHY
__________________________________________________________________________________________
32. Shneiderman, B. & Kearsley, G., 1989, “Hypertext Hands-On!: An Introduction to a
New Way of Organizing and Accessing Information”, Addison-Westley Publishing Co.
Inc. USA
33. Smith, P., 1996, “Towards a practical measure of hypertext usability”, Interacting with
Computers: the Interdisciplinary Journal of Human-Computer Interaction, British
Computer Society, Vol. 8, No. 4, December 1996
34. SPSS Inc., 1997, “SPSS Advanced Statistics 7.5”, SPSS Inc., USA
35. Sun Microsystems Inc, 1995, “Guide to Web Style” URL
http://www.sun.com/styleguide/tables/Welcome.html revised 2nd August 1996
36. Taylor, D., 1995, “Creating Cool Web Pages with HTML”, 2nd Edition, IDG Books
Worldwide Inc.
37. Thuring, M., Haake, J. M. & Hannemann, J., 1991 “What's Eliza Doing in the Chinese
room? Incoherent Hyperdocuments and How to Avoid Them”, Proceedings of Hypertext
'91, ACM Press, 1991.
38. WebTrends, 1997, “WebTrends Statistics: Web Browsers”, URL
http://www.webtrends.com/PRODUCTS/WEBTREND/REPORTS/industry/browser/feb9
7 /report.html, revised 8th July 1997
39. Woodhead, N., 1990, “Hypertext & Hypermedia Theory and Applications”, AddisonWesley Publishing Co., Great Britain
40. Zakon, R. (ed.), 1996, “Hobbes Internet Timeline”, URL
http://info.isoc.org/guest/zakon/Internet/History/HIT.html, revised 15th August 1996
Page 125
APPENDIX A
______________________________________________________________________________________
A Brief History of the Internet
“The Internet has revolutionized the computer and communications world like
nothing before. The invention of the telegraph, telephone, radio, and computer set the stage for
this unprecedented integration of capabilities. The Internet is at once a world-wide broadcasting
capability, a mechanism for information dissemination, and a medium for collaboration and
interaction between individuals and their computers without regard for geographic location.”
(Leiner et al. 1997)
The first recorded description of the social interactions through computer networking was
a series of memos about a “Galactic Network” by J.C.R. Licklider of MIT (Massachusetts
Institute of Technology) in August 1962. Licklider’s concept of a global interconnected network
though which everyone could access data and programs was very much like the Internet of today.
Leonard Kleinrock at MIT published the first paper on packet switching theory in July
1961. Kleinrock’s theory of computer communication using packets was a major step along the
path towards computer networking.
The National Physical Laboratory (NPL) in Middlesex, England set up the first test
packet-switching network in 1968. Shortly afterward, the Pentagon's Advanced Research Projects
Agency (ARPA) decided to fund a larger, more ambitious project in the USA.
In September 1969, the first node was installed in University of California, Los Angeles
(UCLA). By December that year, there were four nodes on the network, which was named
“ARPANET”, after its Pentagon sponsor (from ARPA and Network). The four nodes could
transfer data on dedicated high-speed transmission lines.
Page 126
APPENDIX A
______________________________________________________________________________________
In 1971 there were twenty-three hosts in ARPANET. Throughout the 70s, ARPA's
network continued to grow. ARPANET’s original standard for communication was known as
NCP (Network Control Protocol, NCP was superseded by the group of standards (protocols)
known as TCP/IP. TCP (Transmission Control Protocol) converts messages into streams of
packets at the source, then reassembles them into messages at the destination. IP (Internet
Protocol) handles the addressing, ensuring that packets are routed across multiple nodes and
networks.
In 1983, when TCP/IP superseded NCP the military segment of ARPA became MILNET.
However, TCP/IP still linked the networks. It was fairly easy to link other computers to the
growing network-of-networks. Since the TCP/IP software was public domain, and the basic
technology was decentralized by its very nature, it was difficult to stop people from joining the
network-of-networks. Actually, nobody wanted to stop them from joining this branching complex
of networks, which came to be known as the "Internet."
In 1986 the National Science Foundation (NSF) through its Office of Advanced Scientific
Computing, joined the evolving Internet. Using TCP/IP the new NSFNET (from NSF and
Network) set the pace for technical advancement, by linking newer, faster supercomputers,
through upgraded and expanded links. Other US government agencies joined the growing
Internet: NASA, the National Institute of Health and the Department of Energy.
“Such was the weight of the NSFNET program's ecumenism and funding ($200 million
from 1986 to 1995) - and the quality of the protocols themselves - that by
1990 when the ARPANET itself was finally decommissioned, TCP/IP had supplanted or
marginalized most other wide-area computer network protocols worldwide, and IP was well on
its way to becoming THE bearer service for the Global Information Infrastructure.” (Leiner et
al. 1997)
Page 127
APPENDIX A
______________________________________________________________________________________
From 1986 the Internet, with NSFNET as its backbone and TCP/IP as its protocol,
continued to grow and develop across the globe. The following table shows the connection of
countries to the NSFNET (Internet backbone).
Countries connecting to NSFNET (Internet - Backbone)
(Zakon 1996)
1988 Canada, Denmark, Finland, France, Iceland, Norway, Sweden.
1989 Australia, Germany, Israel, Italy, Japan, Mexico, Netherlands, New Zealand, Puerto Rico,
United Kingdom.
1990 Argentina, Austria, Belgium, Brazil, Chile, Greece, India, Ireland, South Korea, Spain,
Switzerland.
1991 Croatia, Czech Republic, Hong Kong, Hungary, Poland, Portugal, Singapore, South Africa,
Taiwan, Tunisia.
1992 Cameroon, Cyprus, Ecuador, Estonia, Kuwait, Latvia, Luxembourg, Malaysia, Slovakia,
Slovenia, Thailand, Venezuela.
1993 Bulgaria, Costa Rica, Egypt, Fiji, Ghana, Guam, Indonesia, Kazakhstan, Kenya,
Liechtenstein, Peru, Romania, Russian Federation, Turkey, Ukraine, Union of Arab
Emirates, Virgin Islands.
1994 Algeria, Armenia, Bermuda, Burkina Faso, China, Colombia, French Polynesia, Jamaica,
Lebanon, Lithuania, Macao, Morocco, New Caledonia, Nicaragua, Niger, Panama,
Philippines, Senegal, Sri Lanka, Swaziland, Uruguay, Uzbekistan
1995 NSFNET reverts back to a research network. Main US backbone traffic now routed
through interconnected network providers.
1996 Country domains registered: Qatar (QA), Vientiane (LA), Djibouti (DJ), Niger (NE),
Central African Republic (CF), Mauritania (MF), Oman (OM), Norfolk Island (NF),
Tuvalu (TV), French Polynesia (PF), Syria (SY), Aruba (AW), Cambodia (KH)
Page 128
APPENDIX A
______________________________________________________________________________________
The following table shows the growth of hosts (computers) on the Internet 1969 - July
1996. Network and Domain figures are only available from July 1989.
Internet growth 1969 - 07/1996 (Zakon 1996)
Date
Hosts
Date
Hosts
Networks
Domains
1969
4
07/89
130,000
650
3,900
04/71
23
10/89
159,000
837
N/A
06/74
62
10/90
313,000
2,063
9,300
03/77
111
01/91
376,000
2,338
N/A
08/81
213
07/91
535,000
3,086
16,000
05/82
235
10/91
617,000
3,556
18,000
08/83
562
01/92
727,000
4,526
N/A
10/84
1,024
04/92
890,000
5,291
20,000
10/85
1,961
07/92
992,000
6,569
16,300
02/86
2,308
10/92
1,136,000
7,505
18,100
11/86
5,089
01/93
1,313,000
8,258
21,000
12/87
28,174
04/93
1,486,000
9,722
22,000
07/88
33,000
07/93
1,776,000
13,767
26,000
10/88
56,000
10/93
2,056,000
16,533
28,000
01/89
80,000
01/94
2,217,000
20,539
30,000
07/94
3,212,000
25,210
46,000
10/94
3,864,000
37,022
56,000
01/95
4,852,000
39,410
71,000
07/95
6,642,000
61,538
120,000
01/96
9,472,000
93,671
240,000
134,365
488,000
07/96 12,881,000
Page 129
APPENDIX A
______________________________________________________________________________________
Uses of the Internet
As demonstrated the Internet has grown phenomenally during the last 30 years.
The services that the Internet has provided have matured over this time period. These services
include the following:
E-mail : (Electronic mail) In 1971, Ray Tomlinson invented an e-mail program to send
messages across a distributed network. In 1989 the first relays between a commercial electronic
mail carrier and the Internet occurred between MCI Mail through the Corporation for the
National Research Initiative (CNRI), and CompuServe through Ohio State University. Today,
using Multipurpose Internet Mail Extension (MIME) standard, users can send messages,
graphics, presentations, sounds and complete programs to most countries in the world. The
following map graphically demonstrates this.
Page 130
APPENDIX A
______________________________________________________________________________________
Telnet: Telnet Protocol Specification (Request For Comment (RFC) 854) is the virtual
terminal protocol of the TCP/IP protocol suite. Allows the connection of a remote user to the
host. The remote user is connected in such a manner that the host treats the remote user as a local
terminal. This was the original reason that ARPANET was created.
FTP: (File Transfer Protocol) As early as 1973, File Transfer specification (RFC 454)
was in use. The current FTP standard (RFC 959) has been in use since 1985. FTP allows the
transfer of both text and binary files from host to host across the Internet. Until Surpassed by the
World Wide Web, in 1995, FTP was the most popular application on the Internet. (Berghel 1996)
MUD’s & MOO’s: MUD is an acronym for either Multi-User Dungeon or Multi-User
Dimension. First MUD appeared in 1979 called MUD1, it was created by Richard Bartle and Roy
Trubshaw at the University of Essex. MUD’s are the first games ever to be played across the
Internet. MOO stands for MUD, Object-Oriented and are typically social text-based virtual
realities. Participants of a MUD or MOO use the Telnet virtual terminal protocol to access the
host.(Free, 1995)
USENET: (a.k.a. discussion groups, newsgroups) USENET was established in 1979
using UNIX-to-UNIX Copy Protocol. (UUCP) In 1986 Network News Transfer Protocol (NNTP)
was designed to enhance USENET news performance over TCP/IP. There are over 2,500
newsgroups available on Internet. Newsgroups are open discussion forums for a wide variety of
subjects. The user sends messages to the newsgroup host for posting to a newsgroup.
WAIS: (Wide Area Information Servers) WAIS was invented by Brewster Kahle and
released by Thinking Machines. WAIS databases are referred to as sources. The databases may
be organized in different ways, using various database systems, but the user is not required to
learn the query languages of the different databases. The WAIS client uses natural language
queries to find relevant documents and the result of a query is a set of documents, which contain
the words of the query.
Page 131
APPENDIX A
______________________________________________________________________________________
IRC : (Internet Relay Chat) IRC was originally written by Jarkko Oikarinen in 1988. IRC
is a multi-user chat system, where people convene on "channels" (a virtual place, usually with a
topic of conversation) to talk in groups, or privately. IRC has gained attention for its real time
communications capability as shown by the following quote.
“IRC gained international fame during the 1991 Persian Gulf War, where updates from
around the world came across the wire, and most IRC users who were online at the time gathered
on a single channel to hear these reports. IRC had similar uses during the coup against Boris
Yeltsin in September 1993, where IRC users from Moscow were giving live reports about the
unstable situation there.”
(Rose 1996)
Gopher: Released by Paul Lindner and Mark P. McCahill from the University of
Minnesota in 1991. Gopher gives access to FTP sites, normal documents, binary files, WAIS
(Wide Area Information Server) databases, Telnet sessions and is a way to give a simple,
consistent interface to a variety of services and information. In effect, it "goes for" things on the
Internet.
World Wide Web: (a.k.a. WWW, W3 and the Web)
“Without question the hottest part of cyberspace at this time is the World-Wide
Web. The Web is the Internet’s “killer protocol” mix: the client-server handshaking and
communications protocol, HyperText Transfer Protocol (HTTP), together with a protocol for
document formatting, HyperText Markup Language (HTML). The versatility of the resulting tools
have made the Web the focus of today’s cybernauts.”
(Berghel 1996 p.33)
The W3 is the most popular application in use on the Internet today. The growth of the
Internet has been by any measure astounding, the growth of the W3 has been extraordinary.
Page 132
APPENDIX B
Web Site Maps
______________________________________________________________________________________________________
Red Site - Strict Hierarchy
16
Level 1
Level 2
1
2
Level 3
4
17
18
19
20
21
22
Level 4
8
Level 5
23
24
25
26
27
28
29
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
54
55
56
57
58
59
60
30
31
52
53
32
9
5
10
11
3
6
12
Notes: 73 Nodes
72 Links
13
7
14
61
62
63
64
65
69
70
71
72
73
66
67
68
Hierarchical link
15
Page 133
APPENDIX B
Web Site Maps
______________________________________________________________________________________________________
Blue Site – Heavily Cross-linked Web Site
Level 1
1
Level 1, 2 and 3 cross connections
Level 2
2
Level 3
Level 4
4
8
9
5
10
11
Notes: All nodes are connected
to the level above and to
Root (Home) node.
3
6
12
7
Page 134
13
The nodes on levels 2, 3
and 4 are completely
cross connected. Level
4 cross connection not
shown.
14
Hierarchical link
Cross link
15
APPENDIX B
Web Site Maps
______________________________________________________________________________________________________
Blue Site – Heavily Cross-linked Web Site
Sample of level 4 and 5 cross connections
Level 4
4
Level 5
Hierarchical link
Cross link
Sequencing link
16
8
9
23
33
34
35
36
37
38
39
10
5
11
40
6
12
45
Notes:
13
All nodes are connected to the level
above and the Root (Home) node.
54
14
Level 5 nodes contain 11 links each.
Level 4 nodes contain 9 links each.
61
7
15
Total nodes 73
Total links 736
69
Page 135
APPENDIX B
Nodes and Hyperlinks of RED and BLUE Web sites (target nodes shown)_____________________________
Node
Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Title of
Node
COMIC SITE
Marvel Comics
DC Comics
Marvel Heroes and Heroines
Marvel Villains and Villainesses
DC Heroes and Heroines
DC Villains and Villainesses
Marvel Heroes
Marvel Heroines
Marvel Villains
Marvel Villainesses
DC Heroes
DC Heroines
DC Villains
DC Villainesses
Archangel
Beast
Spiderman
Captain America
Dare Devil
Shaman
Wolverine
Black Widow
She Hulk
Rogue
Filename
of Node
index.html
marvel.htm
dc.htm
mmf.htm
mvmf.htm
dmf.htm
dvmf.htm
mmale.htm
mfem.htm
mvmale.htm
mvfem.htm
dmale.htm
dfem.htm
dvmale.htm
dvfem.htm
archange.htm
beast.htm
spiderma.htm
captain.htm
daredevi.htm
shaman.htm
wolverin.htm
black.htm
shehulk.htm
rogue.htm
Hierarchical links
(RED and BLUE)
Nodes 2 and 3
Nodes 4 and 5
Nodes 6 and 7
Nodes 8 and 9
Nodes 10 and 11
Nodes 12 and 13
Nodes 14 and 15
Nodes 16 to 22
Nodes 23 to 32
Nodes 33 to 39
Nodes 40 to 44
Nodes 45 to 53
Nodes 54 to 60
Nodes 61 to 67
Nodes 68 to 72
None
None
None
None
None
None
None
None
None
None
Page 136
Cross links and
Sequential links (BLUE only)
None
Nodes 1 and 3
Nodes 1 and 2
Nodes 1, 2, 5, 6 and 7
Nodes 1, 2, 4, 6 and 7
Nodes 1, 3, 4, 5 and 7
Nodes 1, 3, 4, 5 and 6
Nodes 1, 4, 9, 10, 11, 12, 13, 14 and 15
Nodes 1, 4, 8, 10, 11, 12, 13, 14 and 15
Nodes 1, 4, 8, 9, 11, 12, 13, 14 and 15
Nodes 1, 4, 8, 9, 10, 12, 13, 14 and 15
Nodes 1, 4, 8, 9, 10, 11, 13, 14 and 15
Nodes 1, 4, 8, 9, 10, 11, 12, 14 and 15
Nodes 1, 4, 8, 9, 10, 11, 12, 13 and 15
Nodes 1, 4, 8, 9, 10, 11, 12, 13 and 14
Nodes 1, 8, 17, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 8, 18, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 8, 19, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 8, 20, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 8, 21, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 8, 22, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 8, 16, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 9, 24, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 9, 25, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 9, 26, 16, 23, 33, 40, 45, 54, 61 and 69
APPENDIX B
Node
Number
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
Title of
Node
Talisman
Guardian (target 1, red Web site)
Psylocke
Captain Marvel
Invisible Woman
Black Cat
Storm
Dr Doom
Dr Octopus
Electro
The Hulk
Sabertooth
Venom
Galactus
The Dream Queen
Enchantress
White Queen
Typhoid Mary
Calypso (target 1, blue Web site)
Flash
Batman
Superman
Robin
Chameleon (target 2, red Web site)
Green Lantern
Aquaman
Filename
of Node
talisman.htm
guardian.htm
psylocke.htm
captain1.htm
invisibl.htm
blackcat.htm
storm.htm
doom.htm
octopus.htm
electro.htm
hulk.htm
sabretoo.htm
venom.htm
galactus.htm
dream.htm
enchant.htm
white.htm
typhoid.htm
calypso.htm
flash.htm
batman.htm
superman.htm
robin.htm
cham.htm
greenla.htm
aqua.htm
Hierarchical links
(RED and BLUE)
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
Page 137
Cross links and
Nodes 1, 9, 27, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 9, 28, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 9, 29, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 9, 30, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 9, 31, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 9, 32, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 9, 23, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 10, 34, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 10, 35, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 10, 36, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 10, 37, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 10, 38, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 10, 39, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 10, 33, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 11, 41, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 11, 42, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 11, 43, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 11, 44, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 11, 40, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 12, 46, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 12, 47, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 12, 48, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 12, 49, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 12, 50, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 12, 51, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 12, 52, 16, 23, 33, 40, 45, 54, 61 and 69
APPENDIX B
Node
Number
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
Title of
Node
Superboy
Wildfire
Supergirl
Wonder Woman
Wonder Girl
Batgirl
Huntress
Umbra
Saturn Girl
Two Face
Joker
Luthor
Professor Zoom
Killer Croc
The Penguin
The Scarecrow
Savitor
Cheetah
Catwoman
Circe
Poison Ivy
Clayface (target 2, blue Web site)
Filename
of Node
superboy.htm
wildfire.htm
supgirl.htm
wonwom.htm
wongirl.htm
batgirl.htm
hunt.htm
umbra.htm
satgirl.htm
twoface.htm
joker.htm
luthor.htm
zoom.htm
croc.htm
penguin.htm
scare.htm
savitor.htm
cheetah.htm
cat.htmhtm
circe.htm
poison.htm
clay.htm
Hierarchical links
(RED and BLUE)
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
Page 138
Cross links and
Nodes 1, 12, 53, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 12, 45, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 13, 55, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 13, 56, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 13, 57, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 13, 58, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 13, 59, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 13, 60, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 13, 54, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 14, 62, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 14, 63, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 14, 64, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 14, 65, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 14, 66, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 14, 67, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 14, 68, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 14, 61, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 15, 70, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 15, 71, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 15, 72, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 15, 73, 16, 23, 33, 40, 45, 54, 61 and 69
Nodes 1, 15, 69, 16, 23, 33, 40, 45, 54, 61 and 69
APPENDIX C
Red Site Distance Matrix
NODE
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
Contrastatus
1
2
3
4
5
6
7
8
9
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
0
1
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
1
1
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
1
2
1
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
3
2
1
73
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
3
2
73
1
73
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
3
2
73
1
73
73
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
3
3
2
73
1
73
73
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
6
3
2
73
1
73
73
73
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
6
____________________________________________________________________________________________________
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73
3
3
3
3
3
3
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
2
2 73 73 73 73
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73
2
2
2
2 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
73 73 73 73 73 73
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
2
2
2
2
2
2
2
2
2
2
2
2 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
2
2
2
2
2
2
2
2
2
2
2
2
2
73 73 73 73 73 73
1
1
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
1
1
1 73 73 73 73 73
73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0
6
6
6
6
6
6 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10
Converted Distance = 364912
Page 139
COD
266
2798
2798
3905
4260
3976
4189
4752
4536
4752
4896
4608
4752
4680
4896
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
364912
ROC
Status
Prestige
1371.85
266
266
130.42
97
96
130.42
97
96
93.45
36
33
85.66
26
23
91.78
34
31
87.11
28
25
76.79
7
1
80.45
10
4
76.79
7
1
74.53
5
-1
79.19
9
3
76.79
7
4
77.97
8
5
74.53
5
-1
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
1170
Absolute Prestige = 1170
APPENDIX C
Blue Site Distance Matrix
NODE
1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
Contrastatus
____________________________________________________________________________________________________
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73
0
1
1
2
2
2
2
3
3
3
3
3
3
3
3
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
1
0
1
1
1
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
1
1
0
2
2
1
1
3
3
3
3
2
2
2
2
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
1
1
2
0
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
1
1
2
1
0
1
1
2
2
1
1
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
1
2
1
1
1
0
1
2
2
2
2
1
1
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
1
2
1
1
1
1
0
2
2
2
2
2
2
1
1
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
2
2
1
2
2
1
2
2
2
0
1
1
1
1
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
2
2
1
2
2
2
1
0
1
1
1
1
1
1
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
2
2
2
1
2
2
1
1
0
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
2
2
2
1
2
2
1
1
1
0
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
2
2
2
2
1
2
1
1
1
1
0
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
2
2
2
2
1
2
1
1
1
1
1
0
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
1
2
2
2
2
2
1
1
1
1
1
1
1
0
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
2
2
2
2
2
1
2
2
2
2
2
1
1
1
1
1
1
1
1
0
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
2
2
2
3
3
3
1
2
2
2
2
2
2
2
0
1
2
2
2
2
2
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
1
2
2
2
2
2
2
2
1
0
1
2
2
2
2
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
1
2
2
2
2
2
2
2
1
2
0
1
2
2
2
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
1
2
2
2
2
2
2
2
1
2
2
0
1
2
2
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
1
2
2
2
2
2
2
2
1
2
2
2
0
1
2
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
1
2
2
2
2
2
2
2
1
2
2
2
2
0
1
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
1
2
2
2
2
2
2
2
1
2
2
2
2
2
0
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
2
1
2
2
2
2
2
2
1
2
3
3
3
3
3
0
1
2
2
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
2
1
2
2
2
2
2
2
1
2
3
3
3
3
3
1
0
1
2
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
2
1
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
0
1
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
2
1
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
2
0
1
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
2
1
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
2
2
0
1
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
2
1
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
2
2
2
0
1
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
2
1
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
2
2
2
2
0
1
2
2
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
2
1
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
2
2
2
2
2
0
1
2
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
2
1
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
2
2
2
2
2
2
0
1
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
2
3
3
3
2
1
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
2
2
2
2
2
2
2
0
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
1
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
0
1
2
2
2
2
2
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
1
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
0
1
2
2
2
2
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
1
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
0
1
2
2
2
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
1
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
2
0
1
2
2
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
1
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
2
2
0
1
2
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
1
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
2
2
2
0
1
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
1
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
2
2
2
2
0
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
2
1
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
0
1
2
2
2
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
2
1
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
0
1
2
2
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
2
1
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
0
1
2
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
2
1
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
2
0
1
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
2
3
3
2
2
2
1
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
2
2
0
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
1
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
0
1
2
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
1
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
0
1
2
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
1
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
0
1
2
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
1
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
2
0
1
2
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
1
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
2
2
0
1
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
1
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
2
2
2
0
1
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
1
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
2
2
2
2
0
1
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
1
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
2
2
2
2
2
0
1
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
1
2
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
2
2
2
2
2
2
0
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
2
1
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
0
1
2
2
2
2
2
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
2
1
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
0
1
2
2
2
2
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
2
1
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
0
1
2
2
2
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
2
1
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
2
0
1
2
2
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
2
1
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
2
2
0
1
2
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
2
1
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
2
2
2
0
1
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
2
3
2
2
2
2
2
1
2
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
2
2
2
2
0
1
2
3
3
3
3
3
3
1
2
3
3
3
1
2
2
3
3
3
2
2
2
2
2
2
2
1
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
0
1
2
2
2
2
2
2
1
2
3
3
3
1
2
2
3
3
3
2
2
2
2
2
2
2
1
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
0
1
2
2
2
2
2
1
2
3
3
3
1
2
2
3
3
3
2
2
2
2
2
2
2
1
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
0
1
2
2
2
2
1
2
3
3
3
1
2
2
3
3
3
2
2
2
2
2
2
2
1
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
2
0
1
2
2
2
1
2
3
3
3
1
2
2
3
3
3
2
2
2
2
2
2
2
1
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
2
2
0
1
2
2
1
2
3
3
3
1
2
2
3
3
3
2
2
2
2
2
2
2
1
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
2
2
2
0
1
2
1
2
3
3
3
1
2
2
3
3
3
2
2
2
2
2
2
2
1
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
2
2
2
2
0
1
1
2
3
3
3
1
2
2
3
3
3
2
2
2
2
2
2
2
1
2
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
2
2
2
2
2
0
1
2
3
3
3
1
2
2
3
3
3
2
2
2
2
2
2
2
2
1
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
0
1
2
2
2
1
2
2
3
3
3
2
2
2
2
2
2
2
2
1
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
0
1
2
2
1
2
2
3
3
3
2
2
2
2
2
2
2
2
1
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
0
1
2
1
2
2
3
3
3
2
2
2
2
2
2
2
2
1
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
2
0
1
1
2
2
3
3
3
2
2
2
2
2
2
2
2
1
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
1
2
3
3
3
3
3
3
3
1
2
3
3
3
3
3
1
2
3
3
3
3
3
3
1
2
2
2
0
72 140 140 179 184 180 183 131 128 131 133 129 131 130 133 94 150 201 201 201 201 201 94 150 198 198 198 198 198 198 198 198 94 150 201 201 201 201 201 94 150 203 203 203 94 150 199 199 199 199 199 199 199 94 150 201 201 201 201 201 94 150 200 200 200 200 200 200 94 150 203 203 203
Page 140
COD
266
231
231
178
183
179
182
128
125
128
130
126
128
127
130
174
173
173
173
173
173
174
171
170
170
170
170
170
170
170
170
171
174
173
173
173
173
173
174
176
175
175
175
176
172
171
171
171
171
171
171
171
172
174
173
173
173
173
173
174
173
172
172
172
172
172
172
173
176
175
175
175
176
12486
ROC
Status
Prestige
46.94
266
194
54.05
231
91
54.05
231
91
70.15
178
-1
68.23
183
-1
69.75
179
-1
68.60
182
-1
97.55
128
-3
99.89
125
-3
97.55
128
-3
96.05
130
-3
99.10
126
-3
97.55
128
-3
98.31
127
-3
96.05
130
-3
71.76
174
80
72.17
173
23
72.17
173
-28
72.17
173
-28
72.17
173
-28
72.17
173
-28
71.76
174
-27
73.02
171
77
73.45
170
30
73.45
170
-28
73.45
170
-28
73.45
170
-28
73.45
170
-28
73.45
170
-28
73.45
170
-28
73.45
170
-28
73.02
171
-27
71.76
174
80
72.17
173
23
72.17
173
-28
72.17
173
-28
72.17
173
-28
72.17
173
-28
71.76
174
-27
70.94
176
82
71.35
175
25
71.35
175
-28
71.35
175
-28
70.94
176
-27
72.59
172
78
73.02
171
21
73.02
171
-28
73.02
171
-28
73.02
171
-28
73.02
171
-28
73.02
171
-28
73.02
171
-28
72.59
172
-27
71.76
174
80
72.17
173
23
72.17
173
-28
72.17
173
-28
72.17
173
-28
72.17
173
-28
71.76
174
-27
72.17
173
79
72.59
172
22
72.59
172
-28
72.59
172
-28
72.59
172
-28
72.59
172
-28
72.59
172
-28
72.17
173
-27
70.94
177
83
71.35
175
25
71.35
175
-28
71.35
175
-28
70.94
176
-27
2403
APPENDIX D
__________________________________________________________________________________________
Red Web Site Sample Pages
Level 1 (Homepage)
Sample from level 2
Note: Appendix B contains Web site maps
Page 141
APPENDIX D
__________________________________________________________________________________________
Sample from level 3
Sample from level 4
Page 142
APPENDIX D
__________________________________________________________________________________________
Level 5 (target page)
Page 143
APPENDIX D
__________________________________________________________________________________________
Blue Web Site Sample Pages
Level 1 (Homepage)
Sample from level 2
Page 144
APPENDIX D
__________________________________________________________________________________________
Sample from level 3
Sample from level 4
Page 145
APPENDIX D
__________________________________________________________________________________________
Page 146
APPENDIX E
______________________________________________________________________________________
SOUTHERN CROSS UNIVERSITY
Effects of Web site Structure on user’s perception of navigability.
You are invited to participate in a study of Web site navigation.
Any information that is obtained in connection with this study and that can be identified with
you will remain confidential and will be disclosed only with your permission.
If you decide to participate, you are free to withdraw your consent and to discontinue
participation at any time without prejudice.
If you have any questions, I expect you to ask me. If you have any additional questions later,
Bill Smart (phone: 83 2762) will be happy to answer them.
In this study I hope to learn details of user’s perception of navigation in different Web site
structures. If you decide to participate, you will be asked to fill out a questionnaire and
perform a task, that will require you to navigate two Web sites. This will take approximately
20 - 25 minutes. Questionnaires will be collected upon completion of the experiment. Data
pertaining to the navigational steps you have taken in both Web sites will be logged.
To undertake this experiment you must be familiar with navigating the WWW.
Internet Explorer 3.02 is the browser that will be used for navigating the two Web sites.
If you are unfamiliar with the use of the BACK, HISTORY, FAVOURITES or the VISITED
LINK facilities in Internet Explorer. Please let the researcher know.
Thank you, for your time.
W.J. Smart
BAppSci (Computing)
Machine ID
Page 147
APPENDIX E
______________________________________________________________________________________
Questionnaire General Information
1. Are you ? Male
2. What is your age ?
18 - 25
Female
Please tick one box.
26 - 32
33 - 40
41 -47
Over 47
3. How experienced are you with the WWW ? Put a mark on the line.
Inexperienced
Very experienced
4. How experienced are you with computers ? Put a mark on the line.
Inexperienced
Very experienced
5. How many Marvel / DC Comic Book characters are you familiar with ?
Please tick one box.
1-3
4-6
7-9
10-15
15-20
20-25
25-30
30-40
> 40
Page 148
APPENDIX E
______________________________________________________________________________________
Questionnaire Site Red
Instructions
Please open the URL http://spike.scu.edu.au/~wsmart10/red
Task
Search the Web site to find the answer to the following questions:
1. What is the secret identity of Guardian ?
_________________________________
2. What is the secret identity of Chameleon ? _________________________________
After you have completed the task. Please answer the following questions:
1. Did you at any time feel lost navigating this site? Put a mark on the line.
Not at all
Frequently
2. How often did you use Internet Explorers BACK facility? Put a mark on the line.
Not at all
Frequently
3. How often did you use Internet Explorers HISTORY facility? Put a mark on the line
Not at all
Frequently
4. How comfortable were you with the structure of this site ? Put a mark on the line.
Not at all
Very
Page 149
APPENDIX E
______________________________________________________________________________________
Questionnaire Site Blue
Instructions
Please open the URL http://spike.scu.edu.au/~wsmart10/blue
Task
Search the Web site to find the answer to the following questions:
1. What is the secret identity of Clayface? ____________________________________
2. What is the secret identity of Calypso? ____________________________________
After you have completed the task. Please answer the following questions:
1. Did you at any time feel lost navigating this site? Put a mark on the line.
Not at all
Frequently
2. How often did you use Internet Explorers BACK facility? Put a mark on the line.
Not at all
Frequently
3. How often did you use Internet Explorers HISTORY facility? Put a mark on the line
Not at all
Frequently
4. How comfortable were you with the structure of this site? Put a mark on the line.
Not at all
Very
Page 150
APPENDIX E
______________________________________________________________________________________
Questionnaire Conclusion
Please tick the appropriate box.
1. Did you at any stage use Internet Explorers Favorites facility?
Yes
No
2. Overall which Web site structure did you prefer?
Red
Blue
Thank you for your time and cooperation
W. J. Smart
Page 151
APPENDIX F
________________________________________________________________________________________________
Sample Data From access_log
Machine ID
Date and Time
Nodes Visited
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
-
-
[24/Jul/1997:15:59:44
[24/Jul/1997:15:59:48
[24/Jul/1997:15:59:55
[24/Jul/1997:16:00:03
[24/Jul/1997:16:00:12
[24/Jul/1997:16:00:36
[24/Jul/1997:16:00:39
[24/Jul/1997:16:00:42
[24/Jul/1997:16:00:45
[24/Jul/1997:16:01:06
[24/Jul/1997:16:01:07
[24/Jul/1997:16:01:11
[24/Jul/1997:16:01:16
[24/Jul/1997:16:01:22
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
/~wsmart10/blue/dc.htm HTTP/1.0" 200 1165
/~wsmart10/blue/dvmf.htm HTTP/1.0" 200 1566
/~wsmart10/blue/dvmale.htm HTTP/1.0" 200 1970
/~wsmart10/blue/dvfem.htm HTTP/1.0" 200 1746
/~wsmart10/blue/clay.htm HTTP/1.0" 200 1757
/~wsmart10/blue/cheetah.htm HTTP/1.0" 200 1797
/~wsmart10/blue/cat.htm HTTP/1.0" 200 1750
/~wsmart10/blue/circe.htm HTTP/1.0" 200 1744
/~wsmart10/blue/poison.htm HTTP/1.0" 200 1764
/~wsmart10/blue/marvel.htm HTTP/1.0" 200 1162
/~wsmart10/blue/mmf.htm HTTP/1.0" 200 1583
/~wsmart10/blue/mvmf.htm HTTP/1.0" 200 1588
/~wsmart10/blue/mvfem.htm HTTP/1.0" 200 1778
/~wsmart10/blue/calypso.htm HTTP/1.0" 200 1674
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
203.2.34.220
-
-
[24/Jul/1997:16:07:26
[24/Jul/1997:16:07:27
[24/Jul/1997:16:07:29
[24/Jul/1997:16:07:30
[24/Jul/1997:16:07:37
[24/Jul/1997:16:07:59
[24/Jul/1997:16:08:10
[24/Jul/1997:16:08:13
[24/Jul/1997:16:08:24
[24/Jul/1997:16:09:00
[24/Jul/1997:16:09:02
[24/Jul/1997:16:09:04
[24/Jul/1997:16:09:16
[24/Jul/1997:16:09:25
[24/Jul/1997:16:09:27
[24/Jul/1997:16:09:28
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
+1000]
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
"GET
/~wsmart10/red/marvel.htm HTTP/1.0" 200 981
/~wsmart10/red/mmf.htm HTTP/1.0" 200 999
/~wsmart10/red/mfem.htm HTTP/1.0" 200 1294
/~wsmart10/red/guardian.htm HTTP/1.0" 200 975 **
/~wsmart10/red/mmale.htm HTTP/1.0" 200 1242
/~wsmart10/red/shehulk.htm HTTP/1.0" 200 810
/~wsmart10/red/mvmf.htm HTTP/1.0" 200 1020
/~wsmart10/red/mvfem.htm HTTP/1.0" 200 1000
/~wsmart10/red/mvmale.htm HTTP/1.0" 200 1183
/~wsmart10/red/dc.htm HTTP/1.0" 200 976
/~wsmart10/red/dvmf.htm HTTP/1.0" 200 1002
/~wsmart10/red/dvmale.htm HTTP/1.0" 200 1236
/~wsmart10/red/dvfem.htm HTTP/1.0" 200 968
/~wsmart10/red/dmf.htm HTTP/1.0" 200 1044
/~wsmart10/red/dmale.htm HTTP/1.0" 200 1267
/~wsmart10/red/cham.htm HTTP/1.0" 200 962
**
* Blue target nodes
** Red target nodes
Information gleaned from sample data:
•
Blue site 15 nodes visited (one added for index.html not reflected on server log). Time taken to
complete blue task is 98 seconds.
•
Red site 17 nodes visited (one added for index.html not reflected on server log). Time taken to
complete red task is 122 seconds.
Page 152
*
*
APPENDIX G
______________________________________________________________________________________
Univariate Tests of Between Subjects Effects
Source
Measure
Intercept
Exp. With Comp.
Exp. With WWW
Gender
Age
Order
Gender * Age
Gender * Order
Age * Order
Error
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num nodes visit
Task time
Transformed Variable: Average
Page 153
df
F
Significance
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
98
98
98
98
98
98
241.772
672.639
23.909
1272.884
1574.804
713.538
5.644
0.008
0.129
0.362
0.000
15.937
9.836
0.058
1.526
0.924
0.019
1.085
0.160
0.052
0.987
1.604
0.009
0.232
0.002
0.204
0.099
7.833
0.762
0.682
5.439
0.963
0.060
5.827
0.690
0.039
5.471
1.408
0.257
7.183
4.300
0.796
0.042
0.420
0.637
3.601
0.114
0.036
0.176
0.003
0.215
3.316
0.772
1.554
0.000
0.000
0.000
0.000
0.000
0.000
0.019
0.927
0.720
0.549
0.983
0.000
0.002
0.810
0.220
0.339
0.892
0.300
0.690
0.820
0.323
0.208
0.926
0.631
0.963
0.653
0.753
0.006
0.385
0.411
0.022
0.329
0.806
0.018
0.408
0.844
0.021
0.238
0.613
0.009
0.041
0.374
0.838
0.518
0.427
0.061
0.736
0.851
0.676
0.957
0.644
0.072
0.382
0.216
APPENDIX G
______________________________________________________________________________________
Univariate Tests of Within-Subjects Contrasts
Source
RB (red-blue)
RB * Exp. With Comp.
RB * Exp. With WWW.
RB * Gender
RB * Age
RB * Order
RB * Gender * Age
RB * Gender * Order
RB * Age * Order
Error (RB)
Measure
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Lost rating
Back Button
History List
Comfort rating
Num. nodes visit
Task time
Page 154
df
F
Significance
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
98
98
98
98
98
98
0.703
159.72
6.371
0.423
0.194
4.666
0.278
4.596
0.447
0.206
0.365
2.348
0.025
2.228
0.817
0.234
0.271
5.477
0.807
0.554
1.994
1.113
0.067
0.000
1.710
0.220
0.016
1.629
0.009
7.403
0.004
0.234
1.400
0.816
2.273
22.920
0.012
0.369
0.562
0.026
1.601
7.330
0.002
2.243
0.119
0.456
0.765
0.691
0.455
0.235
0.427
0.531
0.076
0.006
0.404
0.000
0.013
0.517
0.661
0.033
0.599
0.035
0.505
0.651
0.547
0.129
0.876
0.139
0.368
0.630
0.604
0.021
0.371
0.459
0.161
0.294
0.797
0.984
0.194
0.640
0.900
0.205
0.925
0.008
0.948
0.630
0.240
0.369
0.135
0.000
0.914
0.545
0.455
0.872
0.209
0.008
0.964
0.137
0.731
0.501
0.384
0.408
0.502
0.629
0.515
0.468
0.783
0.938
APPENDIX H
Red Site Distance Matrix
NODE
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
Contrastatus
1
2
3
4
5
6
7
8
9
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
0
1
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
1
1
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
1
2
1
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
3
2
1
73
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
3
2
73
1
73
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
3
2
73
1
73
73
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
3
3
2
73
1
73
73
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
6
3
2
73
1
73
73
73
73
0
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
73
6
____________________________________________________________________________________________________
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73
3
3
3
3
3
3
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
2
2 73 73 73 73
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73
2
2
2
2 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
73 73 73 73 73 73
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
2
2
2
2
2
2
2
2
2
2
2
2 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
2
2
2
2
2
2
2
2
2
2
2
2
2
73 73 73 73 73 73
1
1
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
1
1 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
1
1
1 73 73 73 73 73
73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
1
1
1
1
1
73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0 73
73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
0
6
6
6
6
6
6 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10
Page 155
COD
266
2798
2798
3905
4260
3976
4189
4752
4536
4752
4896
4608
4752
4680
4896
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
5256
364912
ROC
Status
Prestige
1371.85
266
266
130.42
97
96
130.42
97
96
93.45
36
33
85.66
26
23
91.78
34
31
87.11
28
25
76.79
7
1
80.45
10
4
76.79
7
1
74.53
5
-1
79.19
9
3
76.79
7
4
77.97
8
5
74.53
5
-1
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
69.43
0
-10
1170
APPENDIX H
Red Site Distance Matrix Using CDM h Modification
NODE
1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
Contrastatus
____________________________________________________________________________________________________
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73
0
1
1
2
2
2
2
3
3
3
3
3
3
3
3
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
1
0
2
1
1
3
3
2
2
2
2
4
4
4
4
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
1
2
0
3
3
1
1
4
4
4
4
2
2
2
2
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
2
1
3
0
2
4
4
1
1
3
3
5
5
5
5
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
4
4
4
4
4
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
1
3
2
0
4
4
3
3
1
1
5
5
5
5
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
2
2
2
2
2
2
2
2
2
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
3
1
4
4
0
2
5
5
5
5
1
1
2
2
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
4
4
4
4
4
4
4
4
4
4
4
4
4
2
3
1
4
4
2
0
5
5
5
5
3
3
1
1
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
2
2
2
2
2
2
2
2
2
2
2
2
2
3
2
4
1
3
5
5
0
2
4
4
6
6
6
6
1
1
1
1
1
1
1
3
3
3
3
3
3
3
3
3
3
5
5
5
5
5
5
5
5
5
5
5
5
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
3
2
4
1
3
5
5
2
0
4
4
6
6
6
6
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
5
5
5
5
5
5
5
5
5
5
5
5
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
3
2
4
3
1
5
5
4
4
0
2
6
6
6
6
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
1
1
1
1
1
1
1
3
3
3
3
3
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
3
2
4
3
1
5
5
4
4
2
0
6
6
6
6
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
2
2
2
2
2
2
2
1
1
1
1
1
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
3
4
2
5
5
1
3
6
6
6
6
0
2
4
4
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
1
1
1
1
1
1
1
1
1
3
3
3
3
3
3
3
5
5
5
5
5
5
5
5
5
5
5
5
5
3
4
2
5
5
1
3
6
6
6
6
2
0
4
4
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
3
3
3
3
3
3
3
3
3
1
1
1
1
1
1
1
5
5
5
5
5
5
5
5
5
5
5
5
5
3
4
2
5
5
3
1
6
6
6
6
4
4
0
2
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
1
1
1
1
1
1
1
1
3
3
3
3
3
3
4
2
5
5
3
1
6
6
6
6
4
4
2
0
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
3
3
3
3
3
3
3
3
1
1
1
1
1
4
3
5
2
4
6
6
1
3
5
5
7
7
7
7
0
2
2
2
2
2
2
4
4
4
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
1
3
5
5
7
7
7
7
2
0
2
2
2
2
2
4
4
4
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
1
3
5
5
7
7
7
7
2
2
0
2
2
2
2
4
4
4
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
1
3
5
5
7
7
7
7
2
2
2
0
2
2
2
4
4
4
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
1
3
5
5
7
7
7
7
2
2
2
2
0
2
2
4
4
4
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
1
3
5
5
7
7
7
7
2
2
2
2
2
0
2
4
4
4
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
1
3
5
5
7
7
7
7
2
2
2
2
2
2
0
4
4
4
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
3
1
5
5
7
7
7
7
4
4
4
4
4
4
4
0
2
2
2
2
2
2
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
3
1
5
5
7
7
7
7
4
4
4
4
4
4
4
2
0
2
2
2
2
2
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
3
1
5
5
7
7
7
7
4
4
4
4
4
4
4
2
2
0
2
2
2
2
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
3
1
5
5
7
7
7
7
4
4
4
4
4
4
4
2
2
2
0
2
2
2
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
3
1
5
5
7
7
7
7
4
4
4
4
4
4
4
2
2
2
2
0
2
2
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
3
1
5
5
7
7
7
7
4
4
4
4
4
4
4
2
2
2
2
2
0
2
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
3
1
5
5
7
7
7
7
4
4
4
4
4
4
4
2
2
2
2
2
2
0
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
3
1
5
5
7
7
7
7
4
4
4
4
4
4
4
2
2
2
2
2
2
2
0
2
2
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
3
1
5
5
7
7
7
7
4
4
4
4
4
4
4
2
2
2
2
2
2
2
2
0
2
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
2
4
6
6
3
1
5
5
7
7
7
7
4
4
4
4
4
4
4
2
2
2
2
2
2
2
2
2
0
6
6
6
6
6
6
6
6
6
6
6
6
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
1
3
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
0
2
2
2
2
2
2
4
4
4
4
4
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
1
3
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
0
2
2
2
2
2
4
4
4
4
4
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
1
3
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
0
2
2
2
2
4
4
4
4
4
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
1
3
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
2
0
2
2
2
4
4
4
4
4
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
1
3
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
2
2
0
2
2
4
4
4
4
4
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
1
3
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
2
2
2
0
2
4
4
4
4
4
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
1
3
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
2
2
2
2
0
4
4
4
4
4
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
3
1
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
0
2
2
2
2
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
3
1
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
2
0
2
2
2
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
3
1
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
2
2
0
2
2
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
3
1
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
2
2
2
0
2
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
3
5
4
2
6
6
5
5
3
1
7
7
7
7
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
2
2
2
2
0
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
5
3
6
6
2
4
7
7
7
7
1
3
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
0
2
2
2
2
2
2
2
2
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
1
3
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
2
0
2
2
2
2
2
2
2
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
1
3
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
2
2
0
2
2
2
2
2
2
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
1
3
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
2
2
2
0
2
2
2
2
2
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
1
3
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
2
2
2
2
0
2
2
2
2
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
1
3
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
2
2
2
2
2
0
2
2
2
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
1
3
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
2
2
2
2
2
2
0
2
2
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
1
3
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
2
2
2
2
2
2
2
0
2
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
1
3
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
2
2
2
2
2
2
2
2
0
4
4
4
4
4
4
4
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
3
1
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
4
4
4
4
4
4
4
4
0
2
2
2
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
3
1
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
4
4
4
4
4
4
4
4
2
0
2
2
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
3
1
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
4
4
4
4
4
4
4
4
2
2
0
2
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
3
1
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
4
4
4
4
4
4
4
4
2
2
2
0
2
2
2
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
3
1
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
4
4
4
4
4
4
4
4
2
2
2
2
0
2
2
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
2
4
7
7
7
7
3
1
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
4
4
4
4
4
4
4
4
2
2
2
2
2
0
2
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
4
2
7
7
7
7
3
1
5
5
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
4
4
4
4
4
4
4
4
4
2
2
2
2
2
2
0
6
6
6
6
6
6
6
6
6
6
6
6
6
4
5
3
6
6
4
2
7
7
7
7
5
5
1
3
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
0
2
2
2
2
2
2
2
4
4
4
4
4
4
5
3
6
6
4
2
7
7
7
7
5
5
1
3
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
0
2
2
2
2
2
2
4
4
4
4
4
4
5
3
6
6
4
2
7
7
7
7
5
5
1
3
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
0
2
2
2
2
2
4
4
4
4
4
4
5
3
6
6
4
2
7
7
7
7
5
5
1
3
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
2
0
2
2
2
2
4
4
4
4
4
4
5
3
6
6
4
2
7
7
7
7
5
5
1
3
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
2
2
0
2
2
2
4
4
4
4
4
4
5
3
6
6
4
2
7
7
7
7
5
5
1
3
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
2
2
2
0
2
2
4
4
4
4
4
4
5
3
6
6
4
2
7
7
7
7
5
5
1
3
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
2
2
2
2
0
2
4
4
4
4
4
4
5
3
6
6
4
2
7
7
7
7
5
5
1
3
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
2
2
2
2
2
2
2
0
4
4
4
4
4
4
5
3
6
6
4
2
7
7
7
7
5
5
3
1
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
4
0
2
2
2
2
4
5
3
6
6
4
2
7
7
7
7
5
5
3
1
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
4
2
0
2
2
2
4
5
3
6
6
4
2
7
7
7
7
5
5
3
1
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
4
2
2
0
2
2
4
5
3
6
6
4
2
7
7
7
7
5
5
3
1
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
4
2
2
2
0
2
4
5
3
6
6
4
2
7
7
7
7
5
5
3
1
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
4
2
2
2
2
0
266 267 267 300 310 304 306 357 351 367 371 355 359 362 368 427 427 427 427 427 427 427 422 422 422 422 422 422 422 422 422 422 437 437 437 437 437 437 437 442 442 442 442 442 426 426 426 426 426 426 426 426 426 430 430 430 430 430 430 430 434 434 434 434 434 434 434 434 440 440 440 440 440
Page 156
COD
266
267
267
300
310
300
308
357
344
367
364
355
359
363
369
428
428
428
428
428
428
428
422
422
422
422
422
422
422
422
422
422
438
438
438
438
438
438
438
442
442
442
442
442
426
426
426
426
426
426
426
426
426
430
430
430
430
430
430
430
434
434
434
434
434
434
434
434
440
440
440
440
440
29904
ROC
Status
Prestige
112.42
266
0
112.00
267
0
112.00
267
0
99.68
300
0
96.46
310
0
99.68
300
-4
97.09
308
2
83.76
357
0
86.93
344
-7
81.48
367
0
82.15
364
-7
84.24
355
0
83.30
359
0
82.38
363
1
81.04
369
1
69.87
428
1
69.87
428
1
69.87
428
1
69.87
428
1
69.87
428
1
69.87
428
1
69.87
428
1
70.86
422
0
70.86
422
0
70.86
422
0
70.86
422
0
70.86
422
0
70.86
422
0
70.86
422
0
70.86
422
0
70.86
422
0
70.86
422
0
68.27
438
1
68.27
438
1
68.27
438
1
68.27
438
1
68.27
438
1
68.27
438
1
68.27
438
1
67.66
442
0
67.66
442
0
67.66
442
0
67.66
442
0
67.66
442
0
70.20
426
0
70.20
426
0
70.20
426
0
70.20
426
0
70.20
426
0
70.20
426
0
70.20
426
0
70.20
426
0
70.20
426
0
69.54
430
0
69.54
430
0
69.54
430
0
69.54
430
0
69.54
430
0
69.54
430
0
69.54
430
0
68.90
434
0
68.90
434
0
68.90
434
0
68.90
434
0
68.90
434
0
68.90
434
0
68.90
434
0
68.90
434
0
67.96
440
0
67.96
440
0
67.96
440
0
67.96
440
0
67.96
440
0
36
APPENDIX I
__________________________________________________________________________________________
Table of Results Lostness/ Efficiency Indicator
All 107 results ordered by visited nodes red site
site
Site
Preference
Blue
Red
Red
Red
Blue
Blue
Red
Blue
Red
Blue
Blue
Blue
Blue
Blue
Red
Blue
Red
Red
Blue
Blue
Blue
Red
Red
Red
Red
Red
Blue
Blue
Red
Red
Red
Red
Red
Blue
Blue
Blue
Red
Blue
Blue
Blue
Blue
Red
Red
Red
Blue
Blue
Blue
Blue
Blue
Blue
Blue
Red
Blue
All 107 results ordered by visited nodes blue
Lostness &
Site
Visited
Preference nodes blue Efficiency
Indicator
site
Blue
8
0.88
Blue
9
0.78
Blue
9
0.78
Blue
9
0.78
Blue
9
0.78
Blue
10
0.70
Blue
10
0.70
Blue
10
0.70
Blue
10
0.70
Blue
10
0.70
Blue
10
0.70
Blue
10
0.70
Blue
10
0.70
Blue
10
0.70
Red
11
0.64
Blue
11
0.64
Blue
11
0.64
Blue
11
0.64
Blue
11
0.64
Blue
11
0.64
Red
11
0.64
Blue
11
0.64
Blue
11
0.64
Blue
12
0.58
Blue
12
0.58
Blue
12
0.58
Blue
12
0.58
Blue
12
0.58
Red
12
0.58
Red
12
0.58
Blue
12
0.58
Blue
12
0.58
Blue
12
0.58
Red
12
0.58
Blue
12
0.58
Blue
13
0.54
Blue
13
0.54
Blue
13
0.54
Blue
13
0.54
Blue
13
0.54
Blue
13
0.54
Blue
13
0.54
Blue
13
0.54
Blue
13
0.54
Red
13
0.54
Blue
13
0.54
Red
13
0.54
Blue
13
0.54
Blue
13
0.54
Blue
13
0.54
Red
13
0.54
Blue
13
0.54
Blue
13
0.54
Visited
red site
nodes red Efficiency
Indicator
site
9
1.00
40
10
0.90
59
10
0.90
89
10
0.90
244
10
0.90
104
10
0.90
100
10
0.90
70
10
0.90
63
11
0.82
81
12
0.75
168
12
0.75
181
12
0.75
72
12
0.75
121
13
0.69
235
13
0.69
74
13
0.69
139
13
0.69
200
13
0.69
89
13
0.69
424
13
0.69
181
13
0.69
117
13
0.69
99
13
0.69
97
13
0.69
80
13
0.69
69
13
0.69
112
13
0.69
75
13
0.69
217
13
0.69
84
13
0.69
152
13
0.69
145
13
0.69
123
13
0.69
158
14
0.64
191
14
0.64
86
14
0.64
129
14
0.64
76
14
0.64
72
14
0.64
65
14
0.64
222
14
0.64
95
14
0.64
175
14
0.64
158
14
0.64
72
14
0.64
114
15
0.60
274
15
0.60
274
15
0.60
118
15
0.60
173
16
0.56
106
16
0.56
106
16
0.56
147
16
0.56
116
Page 157
Task time
blue site
59
30
49
73
109
86
68
79
76
124
142
134
99
71
202
130
148
194
72
49
198
88
51
144
153
83
151
65
205
127
79
140
97
164
90
108
108
153
152
152
93
100
94
91
261
92
105
122
127
195
151
120
94
APPENDIX I
__________________________________________________________________________________________
All 107 results ordered by visited nodes red site
site
Site
Preference
Blue
Blue
Blue
Blue
Blue
Blue
Blue
Blue
Blue
Red
Blue
Blue
Blue
Blue
Red
Red
Blue
Blue
Red
Blue
Blue
Blue
Blue
Blue
Blue
Blue
Blue
Blue
Blue
Blue
Blue
Red
Blue
Blue
Blue
Red
Blue
Blue
Blue
Blue
Red
Red
Blue
Blue
Blue
Blue
Blue
Blue
Red
Blue
Blue
Blue
Red
Blue
Visited
nodes red Efficiency
red site
site
Indicator
16
0.56
175
16
0.56
147
16
0.56
140
16
0.56
135
16
0.56
117
16
0.56
176
16
0.56
257
16
0.56
507
16
0.56
384
16
0.56
183
16
0.56
102
17
0.53
104
17
0.53
126
17
0.53
281
17
0.53
188
17
0.53
168
17
0.53
142
17
0.53
129
17
0.53
248
17
0.53
176
17
0.53
58
17
0.53
143
17
0.53
76
17
0.53
191
17
0.53
221
17
0.53
232
17
0.53
146
17
0.53
145
17
0.53
126
17
0.53
122
17
0.53
169
17
0.53
202
18
0.50
171
18
0.50
199
18
0.50
92
18
0.50
169
18
0.50
359
18
0.50
217
18
0.50
281
18
0.50
199
18
0.50
136
19
0.47
146
19
0.47
203
20
0.45
252
20
0.45
226
20
0.45
285
20
0.45
512
20
0.45
339
20
0.45
196
21
0.43
189
21
0.43
393
21
0.43
163
26
0.35
270
27
0.33
322
All 107 results ordered by visited nodes blue
Site
Visited
Lostness &
Preference nodes blue Efficiency
site
Indicator
Blue
13
0.54
Blue
13
0.54
Blue
14
0.50
Blue
14
0.50
Red
14
0.50
Red
14
0.50
Red
14
0.50
Red
14
0.50
Red
14
0.50
Blue
14
0.50
Blue
14
0.50
Red
14
0.50
Red
14
0.50
Blue
15
0.47
Blue
15
0.47
Blue
15
0.47
Blue
15
0.47
Blue
15
0.47
Blue
15
0.47
Red
15
0.47
Red
15
0.47
Blue
15
0.47
Blue
15
0.47
Blue
15
0.47
Blue
15
0.47
Blue
15
0.47
Blue
16
0.44
Blue
16
0.44
Red
16
0.44
Blue
16
0.44
Blue
16
0.44
Blue
16
0.44
Blue
16
0.44
Blue
16
0.44
Blue
16
0.44
Blue
16
0.44
Red
16
0.44
Red
17
0.41
Blue
17
0.41
Red
17
0.41
Red
17
0.41
Red
18
0.39
Red
19
0.37
Red
19
0.37
Blue
21
0.33
Blue
25
0.28
Red
26
0.27
Red
27
0.26
Red
34
0.21
Red
35
0.20
Red
38
0.18
Red
38
0.18
Red
45
0.16
Red
50
0.14
Page 158
Task time
blue site
141
110
144
156
137
166
93
100
118
146
69
126
72
123
91
65
90
105
72
122
152
139
98
117
173
82
195
229
132
106
135
190
89
245
141
343
287
196
176
169
198
159
199
106
107
199
390
376
348
458
209
528
215
433
APPENDIX J
Experimental Data Set
General Information
Gender Age Exp with Exp with Prefer
WWW
Computer
0
1
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
1
1
1
1
0
1
1
1
1
1
1
2
1
3
1
1
1
1
4
1
3
4
2
1
1
4
1
1
1
4
1
1
1
2
1
5
2
1
3.73
2.16
2.93
2.57
2.9
3.63
4.47
3.5
3.37
0.63
5
3.07
2.27
0.73
5
2.57
0.6
3.7
3.73
3.66
1.8
3.16
4.76
4.6
4.96
4.96
0.53
1.36
2.4
Note: Legend on last page
4.5
1.96
2.8
2.5
3.3
3.87
4.73
3.57
4.3
2.4
5
3.77
3.73
3
5
2.63
2.33
4.2
4.2
3.66
3.2
3.4
4.76
4
4.96
4.1
3.56
2.96
3.06
________________________________________________________________________
Red Web site – Strict Hierarchy
Blue Web site – Heavily Cross-linked
Back History Comfort Nodes Task
Back History Comfort Nodes Task Lost
Red or Lost
visited time
visited time rating facility facility rating
rating facility facility rating
Blue
usage usage
usage usage
First
0
1
0.93
4.23
1.23
1.37
17 104
0.87
3.87
0.06
3.93
13 108
0
0
0.07
2.03
0
4.33
13 235
0.06
0.1
0
5
13 108
0
0
0.83
5
0.13
0.67
27 322
0.6
0
0
4.8
13 153
0
1
0
4
0.13
3.1
21 189
0
0
0
5
14 144
0
0
0.23
4.87
0.03
4.83
14 191
0.07
0.17
0.03
4.93
14 156
0
0
1.63
3.43
0
3.3
15 274
0.2
0.07
0
3.53
13 152
0
0
0.53
4.63
0
4
14
86
1.2
4.86
0
4.43
15 123
0
0
1.37
3.23
3.47
2.7
14 129
0.73
0.87
0.43
3.17
12 144
0
1
2.17
1.37
0.1
1.6
20 252
0.03
0.4
0
1.47
12 153
0
1
3.37
4.3
0
1.27
20 226
3.03
1.4
0
1.3
16 195
1
0
0.73
0.63
0.63
0.63
10
59
0
0
1.03
5
14 137
1
1
0.03
3.6
0.7
3.97
13
74
0.73
2.83
0
2.1
14 166
0
1
1.27
2.83
0.03
4.43
15 274
0.43
0.23
0.03
4.9
13 152
0
1
0.23
3.57
0.03
3.93
21 393
0.53
1.7
0.23
3.86
13
93
0
1
1.5
5
0
3.67
17 126
2.43
5
0
2.5
15
91
0
0
1.2
4
0
2.3
17 281
0.5
0.7
0
4.27
13 100
0
0
2.03
3.8
0
4.33
12 168
0.93
0.5
0
4
12
83
1
1
1.77
3.73
1.57
3.77
17 188
4.23
4.5
0.17
0.63
19 199
1
0
0.86
3.53
0.06
3.43
26 270
2.7
1.33
0.16
3.23
45 215
1
0
1.33
3
0.1
4
17 168
1.06
0.23
0.23
1.33
38 209
0
1
1.3
2.76
0.1
3.9
17 142
3.33
4.2
0.03
4
16 229
0
1
3.46
4.86
0.3
3.86
18 171
2.46
4.96
0.08
2.56
25 199
0
0
0
4.1
0
4.46
13 139
0
0
0
5
13
94
1
0
0
4.8
0
5
19 146
2.6
4.9
0
2.8
14
93
0
0
0.13
3.36
4
3.8
17 129
0.56
1.53
0.8
3.76
13
91
0
1
0.1
0
0
5
12 181
0.36
0.86
0
5
12 151
1
0
1.96
4.26
0
2.4
17 248
0.16
0.26
0.06
0.7
17 196
0
0
3.3
4.4
0.03
3.46
18 199
3.7
0.15
0.06
2.58
12
65
0
1
1.93
3.53
0
3.63
17 176
2.13
1.53
0
3.3
17 176
Page 159
APPENDIX J
General Information
WWW
Computer
0
0
0
0
0
0
0
0
0
0
0
1
1
0
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
2
3
2
1
3
4
1
1
1
2
2
2
2
2
1
3
1
1
3
1
4
5
1
1
3
1
1
1
1
3.4
3.16
1.76
3.73
4.9
1.96
1.56
3.23
3.8
3.03
2.36
1.4
0.7
2.26
1.26
3.9
4.33
2.16
0.08
3.4
0.8
4
5
4.26
0.86
2.6
4
3.66
4.5
3.33
3.63
2.8
3.96
4.9
2.4
2.46
5
3.83
3.03
3.13
2.5
2.56
4.1
1.8
4.76
3.1
2.16
2.23
4.03
1.96
3.93
5
4.41
1.43
3.6
4.3
4.33
3.93
________________________________________________________________________
Red or Lost
visited time
Blue
usage usage
usage usage
First
1
0
0.73
0.66
0.5
3.63
10
89
3.86
0.7
0
4.13
11 202
1
0
0.06
3.66
0
4.76
13 200
0.53
3.46
0.83
3.8
16 132
1
1
4.9
3.16
0.03
4.4
13
89
3.16
0.36
0
4.16
13 261
1
1
1.96
4.1
2.83
1.73
10 244
2
0.33
0
2.63
12 205
0
0
0.13
2.56
0.06
4.93
16 106
0.06
0.63
0.1
4.93
15
65
0
1
0.66
4.06
1.06
4.03
17
58
0.21
1.16
0.46
3.86
9
30
0
1
2.93
5
0
2.43
16 106
2.6
0
0
2.53
16 106
0
0
3.41
5
0.96
1.63
13 424
0
0.33
0
5
10
86
0
1
3.36
4.96
0
1.03
13 181
1.06
0.43
0
0.8
13
92
1
1
0.6
4.46
0.3
3.8
14
76
2.9
0.73
0
3.83
13 105
1
1
0.06
4.8
0.06
4.13
16 147
2.8
1.66
0
2.4
27 376
0
0
2.93
4.8
0
2.03
13 117
3.2
1.23
0
3.86
16 135
1
1
0.1
4.73
0.03
4
11
81
4.53
0.1
0.03
0.46
12 127
0
0
0.66
4.1
0.05
4.73
10 104
3.43
0.36
0.05
4.76
10
68
0
1
2.7
5
0
0
14
72
0
0
0
5
8
59
0
1
2.6
4.8
0
1.4
18
92
0.23
0
0
4.33
9
49
0
1
2.23
4.93
0.06
1.33
17 143
0.73
4.96
0.06
3.33
13 122
0
0
0.03
2.6
0.03
5
16 116
0.03
0.93
0.03
5
10
79
0
1
4.93
4.98
0.1
1.56
16 175
4.93
0.05
0
5
13 127
0
1
1.93
4.03
0.03
2.43
17
76
2.5
2.5
0.03
2.26
10
76
0
0
0
3.1
0
4.5
10 100
0
0.06
0
4.6
11 130
0
1
0.26
3.66
0.1
3.13
17 191
0.6
3.1
0.1
3.83
13 195
0
0
0
2.66
0
2.7
16 147
0.13
1.26
0
4.06
9
73
0
0
0.2
4.96
0
1.3
16 140
0.3
2.46
0
3.41
11 148
0
0
3.86
4.83
0.06
2.5
17 221
2
0.8
0.03
3.8
10 124
1
0
0.03
5
1.3
3.66
13
99
1.86
0
0
3.13
19 106
0
1
0.13
4.03
0
1.4
14
65
1.36
1.3
0
2.13
15
90
0
0
2.13
4.53
0.1
1.66
14 222
1.43
0.56
0.13
3.06
16 190
0
1
2
2.6
0
1.08
16 135
0.63
0
0
1.16
11 194
Page 160
APPENDIX J
General Information
WWW
Computer
0
0
0
0
0
0
0
0
0
1
0
0
0
0
1
1
1
1
1
0
1
1
1
1
1
1
1
0
0
1
1
1
1
4
1
1
1
1
1
1
1
1
1
1
4
2
1
3
1
1
3
1
1
1
3
4
1
2
2.4
3.73
4.06
3.56
2.43
3.8
1.7
3.86
4.43
2.43
3.03
1.66
2.73
2.5
4.86
2.56
0.3
0.2
2.9
3.86
5
1.53
1.36
0.06
0.06
1.73
2.3
2.8
2.3
3.56
4.9
4.23
3.63
2.43
4.1
2.76
3.76
4.83
3.33
3.1
1.7
2.5
2.53
4.83
2.5
0.6
1.16
4.1
3.23
3.53
5
1.4
0.96
0.08
2.53
4.8
2.96
2.3
________________________________________________________________________
Red or Lost
visited time
Blue
usage usage
usage usage
First
1
0
0.16
4.2
0
3.76
13
97
1.1
0.16
0.03
1.3
14 100
1
1
0.6
4.2
0
2.96
13
80
3.1
0
0
1.43
14 118
1
1
0.2
4.73
0.23
4.46
13
69
0.76
0.5
0.33
2.3
17 169
1
1
0.8
3.9
0
4.9
13 112
0.8
4.36
0
3.1
17 198
1
1
0.36
4.5
0.1
4.13
10
70
3.1
0.13
0.1
2.56
13 151
0
0
0.03
1.8
0
4.93
16 117
0.33
0.26
0.03
4.53
13 120
0
0
0.33
3.13
0
3.6
17 232
0.3
0.5
0
4.63
13
94
0
0
0.47
4.9
0.13
0.16
21 163
4.96
4.93
0.1
0.2
15 105
0
0
1.03
2.86
0.2
3.13
13
75
0.76
2.83
0.23
2.63
15
72
0
0
2.03
3.26
0.3
2.9
14
95
1.5
2.06
2.53
2.9
16
89
0
1
1.83
4.2
4.26
4.1
17 146
0.26
3.9
0.16
4.2
13 141
0
0
1.33
4.66
0.53
3.4
13 217
3.5
4.66
0.76
4.26
11
72
1
1
1.6
4.7
0
3.53
13
84
2.63
3
0
2.93
15 122
0
1
2.36
4.16
0.5
0
16 176
0
0.76
0
3.83
13 110
0
1
0
0.25
0
3.1
9
40
0
0
0
4.83
11
49
1
1
4.73
4.36
0
4.8
14 175
3.23
1.4
0
4.46
38 528
1
1
2.53
4.76
0
4.46
13 152
4.73
2.53
0.03
4.1
35 458
0
0
0.3
4.8
0
3.73
20 285
0.3
1
0
4.8
12
79
0
1
0.8
3.6
0.33
4.9
15 118
0.08
4.96
0.05
3.6
9 109
1
1
0.1
4.5
0.03
4.66
13 145
0.9
0.2
0
1.9
15 152
0
0
2.43
5
0
2.73
12
72
0
0
0
5
10 142
0
1
3.16
4.2
0.1
1.86
16 257
2.4
3.7
0.1
3.7
15 139
0
0
1.03
3.2
2.3
3.03
20 512
2.86
3.96
1.36
4.5
16 245
0
0
1.5
4.07
0
2.53
16 507
1.33
0
0
3.56
12 140
0
0
3.9
4.5
0.1
1.66
16 384
0.53
1.43
0.06
2.06
12
97
1
0
1.56
4.6
0.26
2.6
13 123
3.33
1.06
0.23
1.16
12 164
1
0
3.73
4.16
0.1
3.93
14 158
3.46
4.5
0
3.76
18 159
1
1
0.97
4.6
0.2
3.83
18 169
4.43
0.07
0.1
0.4
26 390
0
0
2.37
4.9
0.07
4.95
18 359
1.2
2.87
0.07
1.23
14 146
Page 161
APPENDIX J
General Information
WWW
Computer
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
1
1
4
1
1
1
1
1
1
1
1
1
1
1
1
2
1
2
1
2.97
2.53
2.76
2.8
3.76
3.5
4.3
3.53
3.5
3.43
1.26
1.8
0
4.73
3.3
4.57
3.3
5
3.57
3.2
2.93
3.36
2.66
2.93
4.5
4.9
4.26
3.53
5
3.6
1.33
1.77
1.63
4.8
3.33
4.63
3.13
5
4.83
3.3
________________________________________________________________________
Red or Lost
visited time
Blue
usage usage
usage usage
First
0
0
1.47
4.37
0.1
3.17
15 173
1.33
0.03
0.03
4.37
16 141
0
0
1.76
1.27
0.26
0.97
20 339
3.4
0.2
0.2
2.46
12
90
1
1
3.03
2.53
0.63
1.4
16 183
3.3
2.03
0.47
1.4
11 198
0
0
1.37
4.43
0.2
2.2
17 145
1.7
4.1
1
1.63
10 134
1
1
0
4.95
0
4.3
14
72
4.83
0.26
0
0.33
50 433
0
1
2.06
4.47
0.03
0.13
17 126
2.23
0.3
0.03
2.5
11
88
0
0
3.3
3.73
0.67
2.33
19 203
1.3
3.23
0.33
4.23
14
69
0
1
2.3
3.8
0.1
2.17
17 122
0.46
3.9
0.03
3.2
15
98
0
0
1.03
3.23
0
3.63
17 169
0.77
1.87
0
3.23
15 117
1
0
0.6
4.1
0
4.2
20 196
2.5
0.1
0
2.73
14 126
0
1
1.7
2
2.7
2.6
12 121
1.46
2.5
1.83
2.46
16 343
0
0
1.16
4.97
0
3.73
18 217
1.47
0.43
0.03
4.07
15 173
0
1
3.6
4.4
0
3.13
18 281
2.07
3.7
0
2.6
10
99
1
0
3.4
3.97
0.23
3.1
17 202
4.17
4.33
0.13
2.43
16 287
0
0
0.13
4.5
0.07
4.77
18 199
0.03
0.7
0.05
4.87
11
51
0
0
3.23
3.43
0
2.4
10
63
0.97
0
0
3
21 107
0
0
0.5
2.06
0.2
3.06
14 114
0.46
2.93
0.3
3.7
10
71
0
0
0
3.3
0
5
16 102
0
2.73
0
5
15
82
1
0
0.87
4.77
0
4.3
18 136
2.67
4.1
0
1.13
14
72
1
1
4.77
4.8
0.1
2.87
13 158
3
0.1
0.06
2.37
34 348
Legend
Gender 0 = male; 1 = female
Age 1 = 18-25; 2 = 26-32
3 = 33-40; 4 = 41-47
5 = Over 47
Prefer 0 = Blue Web site
1 = Red Web site
Page 162
Red or Blue First 0 = Red Web site first
1 = Blue Web site first

the entire thesis - Southern Cross Business School

Transcription

Similar documents

The Continuous World of Dungeon Siege

Rotterdam pilot location

distributed detectection of alicious nodes in wirless sensor networks

DVRT-Link - MicroStrain

Basic network measures and visualization

(Scale Out NAS System)

Jimcor Agencies - Montvale, NJ - I