Online Visual Landscape Assessment Using Internet Survey

Online Visual Landscape Assessment Using Internet Survey

Online Visual Landscape Assessment Using Internet Survey
Techniques
Michael ROTH
1
Introduction
The conservation, preservation and development of the variety, particularity and beauty of
nature and landscapes are an important historical root of nature conservation and landscape
planning (cf. GASSNER, 1989; FISCHER-HÜFTLE, 1997; WÖBSE, 2002, p. 167). To achieve
these aims, the scenic beauty and visual quality of landscapes have to be evaluated. For
visual landscape assessment, landscape planners can use either non-empirical “expert
methods” or empirical methods (surveys). Both types of methods have serious
disadvantages: The so called “expert methods” normally represent the assessment of one
person, which can lead to arbitrary results depending on the scheme chosen from the huge
variety of possible scenic quality estimation methods (GRUEHN 2001). The empirical
methods for gathering data on visual landscape quality cause an immense practical effort,
resulting in either small samples or high costs.
The situation mentioned (non-validated expert methods on the one hand and highly
laborious and expensive empirical methods on the other) has led to the insufficient
consideration of visual aspects in environmental planning (HEHL-LANGE & LANGE, 1992;
KÖHLER & PREISS 2000; BREUER 2001; NOHL 2001; JESSEL & FISCHER-HÜFTLE 2003).
Since the mid-nineties of the last century, “we are in the midst of an Internet revolution in
experimental research” (REIPS, 2002). In landscape evaluation, especially visual quality
assessment, the aforementioned revolution hasn’t taken place yet. Landscape planning is
just about to discover the vast potentials of digital media as means of public participation in
the planning process (cf. von HAAREN et al., 2002). So it is about time to expand the
knowledge on the use of up-to-date Internet survey techniques in empirical visual
landscape assessment, which was the main goal of the study presented in this paper.
2
Method
To (a) test the objectivity, reliability and validity of visual landscape assessment gained
through Internet surveys and (b) find out whether there are (demographic, methodological
or technical) factors affecting the rating of the participants, two main steps had to be taken:
First, suitable criteria had to be found to judge the validity of the Internet survey and the
data acquired by it. The results of a recent on-site survey and photograph-based survey
(GRUEHN et al., 2003, unpublished) were taken for this purpose, because the validity of in-
2
Michael Roth
field and photograph-based scenic quality evaluations is acknowledged by experts and
demonstrated by scientists (cf. HUNZIKER & KIENAST, 1999).
Second, a web questionnaire was developed incorporating up-to-date standards for
Internet-based experimenting (e.g. REIPS, 2002; ADM, 2000, 2001a and 2001b) into
landscape perception research. Figure 1 shows an example of the online questionnaire.
Fig. 1:
Screenshot of the online questionnaire.
Several open source/freeware software systems were used for the creation of the online
survey: Hyper-Text Markup Language (HTML) web pages, small JavaScript programs
(used for client-side response time measurement, user input check-up and hardware
Online Visual Landscape Assessment Using Internet Survey Techniques
3
identification) and programs written in Practical Export and Reporting Language (PERL)
running on the Apache web server to generate dynamic web-pages (online questionnaire)
and to save the transferred data to a MySQL database running on another server. The
technical realization is illustrated in figure 2.
Fig. 2:
System architecture of the online questionnaire.
As no commercial software was used, no licence fees had to be paid. No plug-ins or special
software was needed on the participants’ (clients’) computer. Several other means were
taken to ensure the acceptance of the online survey and to increase the number of
participants by reducing the amount of dropout (for the first three techniques cf.
REIPS, 2002):
• high-hurdle technique (e.g. collection of personal data at the beginning, decrease of the
amount of text page per page),
• warm-up technique (practice of rating before the real experiment start),
• one-item-one-screen design (each rating on a separate web page),
• incentive (raffle of two gift certificates for books, 10 € each),
• response time measurement.
Michael Roth
4
The obligation to maintain secrecy was ensured by the possibility to leave all demographic
data fields blank and the separate storage of demographic data and evaluation results. For
identification only the e-mail address, and for localisation of the participant only the postal
code or the country, was used.
To investigate potential methodical biases resulting from different means of input, every
second person was presented a text field to directly enter their scoring instead of the
graphical rating scale with numeric marks shown in figure 1.
The participants had to evaluate up to 17 digital landscape photographs regarding
25 criteria that were all taken from either the German Federal Nature Conservation Act or
various scenic beauty evaluation methods used in landscape planning. The experiment was
online from November 4, 2003 for two months. Respondents were acquired through
advertisements on several web sites, in online forums and e-mail newsletters. A
documentation version of the web questionnaire can be visited at
http://www.tu-berlin.de/~landschaftsbild/umfrage_01/doku.html in the Internet.
3
Results
3.1 Participants
774 people visited the online survey’s web site. 558 of them viewed further pages of the
online questionnaire. 424 started with the evaluation of the landscape photos and 321
completed the evaluation of at least one photo regarding the 25 criteria presented. Overall,
1477 complete evaluations of photos were collected.
The age of the participants ranged from 11 to 78 years, the average age was nearly
30 years. (Coincidentally) 50 % of the participants were male; the other 50 % were female.
Most of the participants came from Germany (337), Austria (29), Switzerland (9), but even
though the questionnaire was only presented in German language some participants came
form Italy (2), the USA (1), Finland (1), Japan (1) and Turkey (1).
The participants differ from the general public regarding their average age (just under
30 years), their above-average school graduation and above-average professional
graduation. Only about one quarter of the participants deal with landscape assessment
occupationally.
3.2 Dropout analysis
Dropout analysis is important to judge the generalizability of the online survey’s results. As
the participants could stop the experiment whenever they wanted, and demographic and
technical data was collected right at the start and each rating and response times were saved
separately (and immediately after the rating) to the database, dropout could be analysed
regarding the dependence on all these factors.
Online Visual Landscape Assessment Using Internet Survey Techniques
5
None of the technical factors (screen resolution, colour depth, operating system, browser)
of methodological factors (type of rating input) was found to have a significant influence
on dropout (n = 558, α = 0,05).
From the demographical factors, only age, nationality and the role of nature and
environment for the participant’s life were found to have significant (p ≤ 0,05), but very
small (0,008 ≤ ε² ≤ 0,04), influences on dropout. Using COHEN’S (1988) thresholds for
effect sizes as cited in BORTZ (1999, p. 137), these effects have only low practical
relevance.
3.3 Response time analysis
The filling in of the web questionnaire for the assessment of 2 or 3 different landscape
photographs took about 10 to 15 minutes. The rating for one criterion took between 4 and
6 seconds (medians of response times) except when a new photograph was presented (the
median of response time then was 11 seconds). Several factors have a strongly significant
(p ≤ 0,001) influence on response times:
• Participants using a text field for data input needed 15 seconds less for the assessment
of one photograph (25 criteria).
• The average time for the assessment of one photograph decreases with the course of the
survey from about 3 minutes (first photograph) to about 2 minutes (17th photograph).
• Women need about 25 seconds less for the complete rating of one photograph.
• Users of bigger screens (higher screen resolution) and newer browsers need less time
for the evaluation of the landscape photographs.
3.4 Objectivity of the online survey
Regarding the objectivity of online visual landscape assessment, we have to distinguish
between the view of the individual and the view of a group or population. PLACHTER et al.
(2002, p. 364) define methods of landscape assessment as objective when they produce
results that are independent of the evaluator. On an individual level, preferences, previous
knowledge, experiences, expectations etc. play an important role for scenic quality
estimation. On a group level, visual landscape assessment is nearly independent of the
factors used for group formation, as the results of the online survey show: sex, nationality,
request for information on the results of the study, participation in the raffle, school
graduation, professional occupation with visual landscape assessment, screen resolution,
colour depth, operating system, browser, type of input instrument, sequence of
photographs, response time and web site linking/referring to the internet survey1 have
either insignificant (p > 0,05) or irrelevant (ε² < 0,04) effects on the evaluation results. The
independence of the evaluation results of the type of input instrument (graphical rating
scale, text field) corresponds with the results of HUNZIKER & KIENAST (1999) who found
out that landscape preference values are independent of the evaluation method (pair
comparison/direct scoring).
1 The referring web site can be used in the so-called “multiple site entry technique” (REIPS, 2002) to
calculate an estimate of biasing potential through self-selection of the participants.
6
Michael Roth
With the other factors (age, geographical origin within Germany, importance of nature and
environment for the personal life, frequency of outdoor trips) the statement mentioned
above (no significant or no relevant effect) is true for 21 to 24 of the investigated
25 criteria. The strongest significant influence explains about 6,5 % of the variance of one
criterion (ε² = 0,065). The effect of the geographical origin within Germany hasn’t been
considered problematic, because the sample of respondents is representatively distributed
throughout the 16 German federal states (Kendalls τb = 0,669, p < 0,005).
3.5 Reliability of the online survey
The reliability of the online survey was tested with 15 participants who filled in the
questionnaire for a second time after a certain span of time (few days up to nearly two
months). These people produced 84 duplicate photograph evaluations (25 criteria each).
Reliability was computed both on an individual and on a group level. Spearman’s rho (rank
correlation) and Pearson’s r (absolute correlation) were used as reliability measures and
produced similar results. Figure 3 shows the reliability values for each criterion on
individual and group level. For all criteria except “confused” and “typical”, satisfying
results for reliability have been achieved (p < 0,01).
Fig. 3:
Results of the reliability test (rank correlation for each criterion on first and
second assessment). This figure also shows the 25 criteria used for visual
landscape assessment.
Online Visual Landscape Assessment Using Internet Survey Techniques
7
3.6 Validity of the online survey
The validity of the online survey and its results was tested by comparing (correlating) the
results of the online study with the ones of the on-site survey and the colour print-based
survey. Again, Spearman’s rho and Pearson’s r were used and produced very similar
results. Validity (Pearson’s r > 0,4, according to the thresholds given by BORTZ & DÖRING
(2003, p. 201)) was achieved for 14 of the 25 criteria, high validity (Pearson’s r > 0,6) for
11 amongst those (cf. table 1). Through cluster analysis, these criteria could be grouped
into “experience dimensions” (cf. Bauer et al., 1979) according to the categories mentioned
in the German Federal Nature Conservation Act in connection with scenic qualities. Those
are: variety (“diversified”, “variform”) and beauty (“aesthetic”, “beautiful”, “total aesthetic
value”). The potential criteria to record the peculiarity of landscape (“typical”,
“characteristic”, “peculiar”) have not proved to be valid or must be doubted because of the
ambiguous meaning (positive and negative) of the criteria “peculiar(ity)” in German and
English language and the lack of information needed to assess the characteristic
(geographical and historical background knowledge on the site presented on the digital
photograph). The dimension of visual naturalness, often used in scenic quality estimation
methods instead of beauty (e.g. ADAM et al., 1986), could be validly recorded through the
use of the criteria “natural” and “original”.
Tab. 1:
Results of the validity test showing the correlation of group means for each
photo/landscape (*** p < 0,001, ** p < 0,01, * p < 0,05)
criterion
correlation (Pearson’s r)
online survey – on-site survey
online survey – photo-based survey
diversified
0,793***
0,888***
aesthetic
0,601*
0,681**
snug
0,723**
0,691**
beautiful
0,665**
0,755***
generic
0,499*
0,566*
graceful
0,625**
0,670**
variform
0,797***
0,882***
peculiar
0,593*
0,558*
wild
0,644**
0,720**
depressing
0,838***
0,877***
natural
0,683**
0,695**
monotonous
0,696**
0,767***
original
0,502*
0,652**
total aesthetic value
0,617**
0,707**
Michael Roth
8
4
Conclusions and Outlook
The Internet survey has proved to be a cost-efficient, objective (on group level), reliable
and valid instrument to gather data on landscape perception and visual landscape
assessment. Results of such a survey have a high generalizability because of the
independence of most demographic, methodological and technical factors and the ease of
drawing a representative sample regarding the geographic distribution of participants. For
those reasons, online scenic quality evaluation can be used to acquire broadly empirically
based assessments of scenic quality needed for planning purposes and environmental
impact assessment. This democratic legitimation (and validation) of visual landscape
assessment can help to give visual aspects a higher weight in environmental planning.
Landscape assessment experts are not made dispensable by the method presented, because
it needs to be professionally prepared, conducted and interpreted. The evaluation of the
peculiarity of nature and landscapes can – according to the results of this study – not be
assessed by the general public through Internet surveys, because a certain level of expert
and background knowledge is needed for this purpose.
Furthermore, online visual quality estimation delivers data useable for fundamental
research as well as the development, improvement and validation of non-empirical scenic
beauty estimation methods, especially GIS-based procedures (e.g. Bishop & Hulse, 1994;
Hunziker & Kienast, 1999; Augenstein, 2002; Roth, 2002).
Up-to-date standards and methods of Internet-based experimenting were applied. The
research taken enhances past landscape preference studies using the Internet (e.g.
Bishop, 1997; Wherrett, 1999), because not only one aspect (colour difference or overall
landscape preference), but 25 criteria representing different experience and perception
dimensions were investigated and the results of the online survey were validated by using
the ones of a recent on-site study.
Further research should be done to examine the effect of other landscape representations in
the Internet (e.g. 360° panoramic images, video, visual landscape simulations, 3D models
or interactive presentations) and cross-cultural comparisons of online visual landscape
assessment.
5
References
Adam, K., Nohl, W. & Valentin, W. (1986): Bewertungsgrundlagen für
Kompensationsmaßnahmen bei Eingriffen in die Landschaft. Düsseldorf.
ADM – Arbeitskreis Deutscher Mark- und Sozialforschungsinstitute e.V. (Ed.) (2000):
Richtlinie für Online-Befragungen. Frankfurt am Main.
ADM – Arbeitskreis Deutscher Mark- und Sozialforschungsinstitute e.V. (Ed.) (2001a):
Standards zur Qualitätssicherung für Online-Befragungen. Frankfurt am Main.
ADM – Arbeitskreis Deutscher Mark- und Sozialforschungsinstitute e.V. (Ed.) (2001b):
Checklist for Clients Commissioning Online Surveys. Frankfurt am Main.
Online Visual Landscape Assessment Using Internet Survey Techniques
9
Augenstein, I. (2002): Zur Berücksichtigung des landschaftsästhetischen Potentials in der
planerischen Umweltvorsorge: Entwicklung eines GIS-gestützten Verfahrens am
Beispiel des Regierungsbezirkes Dessau. Doctoral thesis, Rostock University, Faculty
of Agricultural and Environmental Sciences.
Bauer, F., Franke, J. & Gätschenberger, K. (1979): Flurbereinigung und
Erholungslandschaft. Empirische Studie zur Wirkung der Flurbereinigung auf den
Erholungswert einer Landschaft. Bundesminister für Ernährung, Landwirtschaft und
Forsten (Ed.): Schriftenreihe B: Flurbereinigung, Heft 68. Landwirtschaftsverlag,
Münster-Hiltrup.
Bishop, I.D. (1997): Testing perceived landscape colour difference using the Internet.
Landscape and Urban Planning 37 (3-4): 187-196.
Bishop, I.D. & Hulse, D.W. (1994): Prediction of scenic beauty using mapped data and
geographic information systems. Landscape and Urban Planning 30 (1-2): 59-70.
Bortz, J. (1999): Statistik für Sozialwissenschaftler. 5th edition. Springer, Berlin.
Bortz, J. & Döring, N. (2003): Forschungsmethoden und Evaluation für Human- und
Sozialwissenschaftler. 3rd edition. Springer, Berlin.
Breuer, W. (2001): Ausgleichs- und Ersatzmaßnahmen für Beeinträchtigungen des
Landschaftsbildes. Vorschläge für Maßnahmen bei Errichtung von Windkraftanlagen.
Naturschutz und Landschaftsplanung 33 (8): 237-245.
Cohen, J. (1988): Statistical Power Analysis for the Behavioral Sciences. 2nd edition.
Erlbaum, Hillsdale, NY.
Fischer-Hüftle, P. (1997): Vielfalt, Eigenart und Schönheit der Landschaft aus der Sicht
eines Juristen. Natur und Landschaft 72 (5): 239-244.
Gassner, E. (1989): Zum Recht des Landschaftsbildes. Natur und Recht 11 (2): 61-66.
Gruehn, D. (2001): Einschätzung der Bedeutung von Landschaftselementen für das
Landschaftserleben. In: Paar, P. & Stachow, U. (Eds.): Visuelle Ressourcen Übersehene ästhetische Komponenten in der Landschaftsforschung und –entwicklung.
ZALF-Bericht 44. Zentrum für Agrarlandschafts- und Landnutzungsforschung e.V.,
Müncheberg.
Gruehn, D., Roth, M. & Kenneweg, H. (2003, unpublished): F&E-Studie "Entwicklung
eines Ansatzes zur Einschätzung der Bedeutung von Landschaftselementen für das
Landschaftserleben als Grundlage für die Beurteilung des Landschaftsbildes".
Abschlussbericht i.A. des Sächsischen Landesamtes für Umwelt und Geologie.
TU Berlin.
Hehl-Lange, S. & Lange, E. (1992): The insufficient consideration of visual aspects in
environmental planning. Association of European Schools of Planning, VI. AESOPCongress, Stockholm.
Hunziker, M. & Kienast, F. (1999): Potential impacts of changing agricultural activities on
scenic beauty - a prototypical technique for automated rapid assessment. Landscape
Ecology 14 (2): 161-176.
Jessel, B. & Fischer-Hüftle, P. (2003): Bewältigung von Eingriffen durch
Verkehrsvorhaben in das Landschaftsbild. Naturschutz und Landschaftsplanung 35
(12): 373-383.
Köhler, B. & Preiss, A. (2000): Erfassung und Bewertung des Landschaftsbildes.
Grundlagen und Methoden zur Bearbeitung des Schutzguts "Vielfalt, Eigenart und
Schönheit von Natur und Landschaft" in der Planung. Informationsdienst Naturschutz
Niedersachsen 20 (1): 1-60.
10
Michael Roth
Nohl, W. (2001): Landschaftsplanung. Ästhetische und rekreative Aspekte. Konzepte,
Begründungen und Verfahrensweisen auf der Ebene des Landschaftsplans. Berlin,
Patzer-Verlag.
Plachter, H., Bernotat, D., Müssner, R. & Riecken, U. (2002): Entwicklung und Festlegung
von Methodenstandards im Naturschutz. Ergebnisse einer Pilotstudie. Bundesamt für
Naturschutz (Ed.): Schriftenreihe für Landschaftspflege und Naturschutz 70.
Landwirtschaftsverlag, Münster-Hiltrup.
Reips, U.-D. (2002): Standards for Internet-Based Experimenting. Experimental
Psychology 49 (4): 243-256.
Roth, M. (2002): Möglichkeiten des Einsatzes geographischer Informationssysteme zur
Analyse, Bewertung und Darstellung des Landschaftsbildes. Natur und Landschaft 77
(4): 154-160.
Von Haaren, Chr., Knickrehm, B., Kunze, K. & Redslob, M. (2002).: Interaktiver
Landschaftsplan. Verbesserungsmöglichkeiten für die Akzeptanz und Umsetzung von
Landschaftsplänen.
Bundesamt
für
Naturschutz
(Ed.):
Angewandte
Landschaftsökologie 43. Landwirtschaftsverlag, Münster-Hiltrup.
Wherrett, J.R. (1999): Issues in using the Internet as a medium for landscape preference
research. Landscape and Urban Planning 45 (4): 209-217.
Wöbse, H.H. (2002): Landschaftsästhetik. Ulmer Verlag, Stuttgart.