Masterthesis The impact of prior attitudes and prior knowledge on

Transcription

Masterthesis
The impact of prior attitudes and prior knowledge on the problem formulation stage
of information problem solving.
Gerdo J. Velthorst
Februari 2015
Master Onderwijswetenschappen
Open Universiteit Nederland
Begeleiders: Prof. dr. F.L.J.M. Brand-Gruwel and J. Frerejean, PhD
Examinator: Prof. dr. H.P.A. Boshuizen
Studentnummer: 850659995
THE IMPACT OF PRIOR ATTITUDES AND PRIOR KNOWLEDGE ON THE PROBLEM FORMULATION STAGE OF INFORMATION PROBLEM SOLVING
2
Table of contents
Abstract
3
3.
4
4.
5.
6.
Introduction
3.1
Information problem solving
5
3.2
Prior attitudes and prior knowledge
6
3.3
Research questions and hypotheses
10
Method
11
4.1
Participants
11
4.2
Materials
11
4.3
Measures
12
4.4
Procedure
16
4.5
Data analysis
17
Results
18
5.1
Consistency of bias in information seeking with prior attitudes
18
5.2
The relationship between consistency and attitude strength
21
Discussion
28
6.1
Conclusions
28
6.2
Limitations and future research
29
6.3
Implications
30
References
32
3
Abstract
Information problem solving (IPS) with the purpose of learning is common practice in higher
education. Most of the information problems students engage in can be characterized as ill-defined.
The first and pivotal step in solving these types of information problems constitutes the formulation of
a problem statement that reflects an understanding of the essential characteristics of the information
problem. Descriptive models of the IPS-process emphasize the importance of this first stage. Research
has shown that individuals differ in solving information problems. Limited prior knowledge impedes a
thorough IPS-process, and IPS is prone to bias due to prior attitudes. However, most studies do not
take problem statement formulation into account and primarily study bias in the selection of
information with artificial search engine result pages. The aim of this study was to determine the
impact of prior knowledge and prior attitudes on the first stage of IPS, problem formulation,
considering a wider range of information seeking activities.
Seventy freshmen pre-service teachers were given an ill-defined information problem regarding the
controversial issue of climate change. Students were asked to make preparations for writing an
informative article. After a short period of online information searching to explore the topic, students
formulated an essential question, provisional answer and selected four useful sources. The result on
these tasks was considered to be a problem statement. A logging tool recorded information searching
unobtrusively and students were asked to take notes. Ten days before, students were questioned about
their attitudes towards the topic at hand. Analysis of the information seeking activities in the first stage
of IPS showed that participants demonstrate bias in information seeking, consistent with prior
attitudes. Most consistency was found in the keywords students used to search for information and to a
lesser extent in the follow-up activities: the essential questions, provisional answers and selection of
information. Attitude strength explained the relationship between prior attitudes and bias in
information seeking only in the keywords and central questions. Participants with stronger attitudes
demonstrated less consistency and participants with weaker attitudes demonstrated more consistency
of prior attitudes with bias in information seeking. These results show that prior attitudes and the
strength of these attitudes affect the first stage of information problem solving. When using ill-defined
information problems in higher education, educators should be aware that students are not unbiased
when they engage in these problems. As a result of prior attitudes, intended learning outcomes may be
out of reach right from the start. Regulatory activities that address prior attitudes have to be part of
information problem solving.
Keywords: higher education, information problem solving, problem formulation, prior attitudes, prior
knowledge, attitude strength.
4
3. Introduction
In most contemporary educational programs in higher education, students work on assignments that at
some point involve working on information problems. The availability of the Internet in education has
considerably changed the process of information seeking. Nowadays students’ information seeking
activities are almost always mediated by the World Wide Web. The purpose of information seeking in
higher education is not just to retrieve information or to find the right answer, but to engage in
information seeking for learning. The information seeking behaviour aimed for in these tasks exceeds
fact-finding and requires investigative activities in a more exploratory manner (Marchionini, 2006).
Students are expected to adopt an information seeking approach in which they analyse and scrutinize
information to deepen their understanding of a particular topic (Bowler, 2010; Limberg, 1999). In
practice however, students appear to gather facts from the start, rather than use information to organise
ideas and reflect upon them (Kulthau, 2004; Todd, 2006).
Learning from information problems is not only pertinent at the end when students have all
the information they need, but also during the information seeking process, particularly in the initial
stages of the information seeking process as described by Kuhlthau (2004). In these stages, students
first explore their information problem. Often, they experience uncertainty because they operate on the
limits of their understanding and the boundaries of the topic are yet to be discovered (Anderson,
2006). While exploring, students’ thoughts develop, resulting in a clear, concise and well-structured
information problem statement, which is the starting point for further collection and use of
information. At this point, as a result of learning, students’ problem statements should reflect a deeper
understanding of the topic and the task at hand (Attfield, Blandford, & Dowel, 2003). The extent to
which problem statements are well-structured and concise determine the success and efficiency of
forthcoming information seeking activities (Kulthau, 2004; Marchionini & White, 2007). A significant
part of working on information problems should therefore be devoted to thinking and learning in the
initial stages (Todd, 2003). Learners do not always perceive the importance of activities that lead to a
clear problem statement and often expect to be able to express focus right from the start (Isbell &
Kamerlocher, 1998).
Most process models of information seeking acknowledge the importance and complexities of
the initial stages of information seeking (Brand-Gruwel, Wopereis, & Walraven, 2009; Kuhlthau,
2004; Marchionini, 1997; Sutcliffe & Ennis, 1998). Research efforts have led to increasing refinement
of these models by clarifying cognitive attributes that account for individual differences in information
seeking. It is clear that prior knowledge and prior attitudes influence information seeking (Hart et al.,
2009); information seeking is prone to ignorance and bias. Most research on prior knowledge and
attitudes, though, is primarily devoted to the post-focus stage of information seeking. It stands to
5
reason, however, that bias in information seeking originates earlier, in the early pre-focus stages of
information seeking where students engage in exploratory activities and construct their problem
statement. The purpose of this study is to determine the impact that prior attitudes toward the topic of
an information problem and prior topic knowledge have on this initial stage of information seeking.
The results may be useful for educators responsible for tutoring students working on informationbased tasks or writing task descriptions for information based assignments.
3.1 Information problem solving
Information seeking behaviour is concerned with the methods people employ to discover and gain
access to information in order to satisfy a goal (Marchionini, 1997; Spink & Cole, 2004; Wilson,
1999; Wilson, 2000). Information seeking is common in professional practice, and often embedded in
a wider set of actions or a larger process of task completion (Saracevic et al., 1988; Spink & Cole,
2006; Vakkari, 1997). In accordance with the seminal work by Dervin and Nilan (1986), the
information seeker can be seen as an active agent in finding and using information. Moreover,
individuals vary considerably in the information seeking behaviour they demonstrate (Ingwersen &
Järvelin, 2005). Information seeking is a complex process encompassing a wide variety of information
seeking activities and corresponding behaviours. This becomes clear when information seeking is
perceived as problem solving.
Information problem solving can be seen as a particular instance of problem solving, dealing
with information problems. Information problems are problems or problem situations that can be
resolved by gathering and using information (Brand-Gruwel, Wopereis, & Vermetten, 2005; Case,
2012). There are several problem-solving methods tailored for solving information problems, for
example the Information Search Process model (Kuhlthau, 2004), the Big6 model (Eisenberg, 2008)
and IPS-I model (Brand-Gruwel, Wopereis, & Walraven, 2009). Although these models identify
different stages, this does not imply that information problem solving is a linear process (Foster, 2006;
Foster & Urquhart, 2012). The different stages and activities are highly interrelated, the outcomes of
each activity depend on the outcomes of adjacent activities, and each activity requires an intelligent
coordination of cognitive skills (Brand-Gruwel, Wopereis, & Vermetten, 2005; Hill, 1999; Moore,
1995; Wilson, 1999).
Most information problems students work on in higher education can be characterised as illdefined, because tasks are open-ended, task descriptions are poorly specified or afford multiple
interpretations, and when used for learning, students start as novices with regard to the topic at hand.
Therefore, these information problems cannot be solved in an obvious and straightforward manner
(Anderson, 1993; Chi & Glaser, 1985; Jonassen, 2000; Mayer & Wittrock, 2006; Shuell, 1990; Simon,
1973; Simon, 1978). Problem definition is a prerequisite first step for solving these ill-defined
6
information problems (Byström & Järvelin, 1995; Vakkari, 1999). That is, students first need to
identify, understand and represent the essential characteristics of the information problem, thoughtful
structure the problem elements, and plan the problem solving process (Chi & Glaser, 1985;
Marchionini, 1997; Ormerod, 2005; Wopereis, Brand-Gruwel, & Vermetten, 2008).
Constructing a problem definition of information problems is done by performing several
tasks (Brand-Gruwel, Wopereis, & Walraven, 2009). Performing these tasks results in a
comprehensive, inclusive and concise description of the information problem, with the use of relevant
and differentiated concepts in an integrated fashion (Vakkari, 2000). A first task, after reading the task
description, is to express the information need in essential and subsidiary questions (Brand-Gruwel,
Wopereis, & Vermetten, 2005; Taylor, 1962). These questions set the first boundaries of the problem
space and reflect prior subject knowledge. A second task is to construct an expectation of what the
answer will, by and large, look like (Marchionini, 1997). A third task is to derive the information
requirements: ‘What information do I need?’ These requirements relate to the type and amount of
information needed and the relevance criteria to judge the information found. A problem definition
resulting from these tasks provides a clear sense of direction that guides further information seeking
activities (Kuhlthau, 2004).
3.2 Prior attitudes and prior knowledge
There is little doubt about the importance of the initial stage of information problem solving. But the
extent of our understanding of the cognitive processes involved in the first stage of information
problem solving is limited. It is important to note that each stage in information problem solving
yields its own information seeking behaviour. Vakkari and Hakala (2000), for example, found that in
the pre-focus stage students, writing their master’s thesis, reject specific document types like literature
reviews and conference papers because these types of information contain very specific information.
Instead, students look for general background information, for example in encyclopaedias, which give
a helicopter view of the topic. In the following stages of information problem solving, when a high
degree of focus is attained, students look for more faceted background information, information that is
situated in subtopics of the general topic (Vakkari, 2000). A specific focus on the different stages in
information problem solving is therefore warranted.
Prior attitudes and information seeking. People bring their own views with them when solving an
information problem. When a search strategy is adopted, these views are visible in the keywords
people use, the particular links that are followed in a search engine result page, and even in the reasons
why people search for information. For example, White (2013) found that when people are questioned
in retrospect about their beliefs before and after searching for the answer to a self-generated yes-no
7
question, the majority of the people have the goal of confirming their prior-held belief, finding yes for
an answer, and reaffirming their initial answer when engaging in multiple results. After searching,
beliefs appear to be even stronger. In spite of the amount and the diversity of online information,
information seeking behaviour is not open-minded and free from bias.
Research on decision making shows that decision makers who engage in information seeking
have tendencies to look up information consistent with or in support of their preliminary choice of
decision and give more positive evaluations to confirmatory information (Fisher & Greitemeyer,
2010). This phenomenon is known as confirmatory information seeking and is most pronounced in
situations where decision makers sequentially seek pieces of information (Jonas, Schulz-Hardt, Frey,
& Thelen, 2001). In these situations, decision makers remind themselves repeatedly of their
preliminary choice in every information seeking event. Consequently, decision makers suffer from
selective exposure to information consistent with their preliminary choice.
Generally speaking, people tend to be biased in favour of their own arguments and judge
confirming views as more persuasive and convincing. This not only holds true for simple fact-finding
tasks or preliminary decisions, but also for prior attitudes regarding the topic at hand. Attitudes
concerning particular objects or thoughts consist of evaluations with a cognitive component and an
affective component (Bohner & Dickel, 2011; Krosnick, Judd, & Wittenbrink, 2005). The cognitive
component is based on the ideas, beliefs and perceptions about an object. The affective component is
based on the preference and feelings toward an object. Attitudes may differ in the degree to which the
cognitive or the affective component is dominant (Azjen, 2001; Crano & Prislin, 2006). Therefore,
aside from attitude extremity – the degree to which an idea or belief deviates from the mid-point of a
continuum like favour-oppose – attitudes can also be described in terms of attitude strength (see for
example Brannon, Tagler, & Eagly, 2007).
Most research on attitudinal preferences in information seeking concentrates on one particular
information seeking activity, the selection of information. Hart et al. (2009) confirm in their metaanalysis that people, across studies, demonstrate a moderate confirmation bias in information
selection; people prefer attitude-consistent information. This is explained mostly by defence motives,
which is the desire to defend one’s prior attitude. Confirmation bias is larger when people have high
commitment and attach personal relevance to their attitudes. Confirmation bias is smaller when people
have high confidence in their attitudes, because in that case people belief in their capacity to
effectively deal with inconsistent information.
In a study regarding political issues like gun ownership, Knobloch-Westerwick and Meng
(2009) show that students, as expected, are more likely to choose information consistent with their
attitudes and spent more time with consistent information. With decreasing certainty students choose
inconsistent information more often and spend less time reading consistent information. Their
8
explanation is that people with uncertain attitudes expect to gain more certainty in their point of view
when looking into inconsistent information, as briefly as they do. Brannon, Tagler, and Eagly (2007)
also demonstrate that psychology students’ strength of prior attitude, a composite measure that
includes importance and certainty, predicts the preference for attitudinally consistent information.
When students were asked to choose sources from a list of titles, similar to search engine result pages,
increasing strength of prior attitudes leads to increasing preference for attitude consistent information.
Findings concerning the impact of strength-related characteristics of prior attitudes on
information seeking are not conclusive and depend on which combinations of strength dimensions are
used and how relationships between these characteristics are conceptualized. For example, Sawicki et
al. (2011) found that only when undergraduate students are not familiar with the information
presented, decreasing levels of attitude confidence - when students are uncertain about their attitudes
as the authors express it - lead to increasing bias aligned with prior attitudes. The explanation is that
unfamiliar and consistent information helps to ascertain views founded on weak attitudes and therefore
might be processed more actively. This is in line with Hart et al. (2009). For students that report
familiarity with the presented information, it is increasing levels of attitude confidence that lead to
bias toward consistency. The same relationship was found by Knobloch-Westerwick and Meng (2009)
and Brannon et al. (2007).
Specifically with regard to science topics, Jang (2013) found that adults more often select
news articles referring to attitude-inconsistent information and spend more time reading this
information. This phenomenon was explained by schema theory, because novel or schemainconsistent information is more salient to people. Also, the preference for challenging information is
most visible when news articles discuss risks and threats. Thus, under some conditions, prior attitudes
can even enforce a disconfirmation bias, where people avoid certain types of information. Further
analysis shows that certainty of attitudes, which is explained as perceived science knowledge,
increases the preference for attitude-consistent information.
Cognitive bias is not only visible when people select information, but also in students’ essay
writing when information is processed and used (Edwards & Smith, 1996; Kobayashi, 2010). If
students disagree with one of two opposing views, arguments against the opposing view in texts are
engaged in and used in essay texts. Students seem to give more attention and scrutinize arguments
incompatible with their attitudes whereas they uncritically accept arguments compatible with their
attitudes. Students seem to put extensive cognitive effort in defending their position by disconfirming
or negating incompatible arguments when they are asked to take a stance in their writings.
Van Strien, Brand-Gruwel, and Boshuizen (2014) found that secondary school students take
views consistent with prior attitudes in their essays. Students were given time to read multiple texts
with conflicting views about violent videogames, and afterward students were given an open-ended
9
essay writing task. Students with neutral prior attitudes were compared to students with positive
attitudes toward the issue. They found that students with neutral prior attitudes were more likely to
reflect a balanced view in their writings and summarize information at hand. Students with positive
prior attitudes were considerably more likely to take a positive view consistent with their attitudes and
to base their writings on personally added information.
Prior knowledge and information seeking. Aside from differences in prior attitudes, students’ prior
knowledge and understanding also varies. Some students may have initial ideas about the topic under
investigation and can detail rich concept maps of their prior knowledge with a high number of
concepts and relationships, whereas others may not (Chung & Neuman, 2007). Prior knowledge can
be defined as the whole of declarative (‘knowing that’) and procedural knowledge (‘knowing how’),
structured in schemata, which a person has available when starting a task. Prior knowledge can be
described in quantitative and qualitative qualities: completeness, amount, accuracy (or
misconceptions), accessibility, availability, and structure (Dochy, Segers, & Buehl, 1999). Information
seeking is complicated when students are domain novices with limited prior knowledge, which is often
the case in education. Specifying the information problem, or even constructing queries for that
matter, in domain specific keywords that are not known and describing what information is needed are
difficult tasks (Belkin, 2000).
Willoughby et al. (2009) compared the quality of essays written by students with high and low
prior domain knowledge that were evenly distributed to a thirty minute search and no-search
condition, prior to writing. Students with high prior knowledge that were allowed to search the
Internet outperformed students with high prior knowledge who were not allowed to search the
Internet. The scores of the latter group were comparable to those of students with low prior
knowledge, with or without internet exposure. It appears, then, that the value of information searching
for learning is only visible for students with high prior knowledge.
High levels of prior knowledge, next to system expertise, differentiate successful from less
successful searchers, because topic knowledge contributes to the relevance and precision of using
keywords and keyword combinations, finding the information needed on a webpage and the
evaluation of information (Belkin, 2000; Makinster, Beghetto, & Plucker, 2002). Shiri and Revie
(2003) found that people with a moderate or high topic familiarity are slightly (but not significant)
more involved in the conceptual analysis of terms or documents, in the information found. They also
use system features of the search environment more often. When the information seeker is familiar
with the subject, he extracts more (additional) terms from the information that is found. These
additional terms subsequently lead to highly useful information sources when used in search engines,
because these terms reflect the words and language used in those documents to be found (Pennanen &
10
Vakkari, 2003). Broader search terms are dropped – i.e. the information seeker becomes more
selective – and an increasingly larger and specific vocabulary is used. This facilitates the use of more
specific search terms and greater query specificity.
Regarding prior knowledge, a review study of Chen and Macredie (2010) shows that domain
novices differ from experts in their capacity to process online information. Novices benefit from
hierarchical text structure, hierarchical maps and conceptual overviews to browse through hypertexts,
because they lack a conceptual schema to map onto the information. Also, domain novices use fewer
(meta)cognitive strategies and spend less time with these strategies while searching. With the
exception of fact-finding tasks, novices need more time and actions to find the information and deal
with information problems differently.
Low levels of prior topic knowledge evoke data-driven strategies where information seeking is
broad and superficial (Land & Greene, 2000). At the very beginning of discovering topic ideas datadriven exploration is fruitful, but strategies have to become goal-directed where the information
problem is focused and information is consciously judged in light of project goals, hypotheses and
information needs. When data-driven strategies consolidate, topic understanding remains fragmented
and most of the learning time is spend on undirected searches for information.
The knowledge base of prior attitudes. Prior knowledge and attitudes are also related, because
attitudes are embedded in knowledge structures. When these structures are extensive and wellorganized, they can foster strong attitudes toward an attitude object. Wood, Rhodes, and Biek (2014)
reason that in case of high prior knowledge, the strength of affect moderates preference for attitude
consistent information. When affect is strong, positive or negative, information processing is selective
and more attention is given to attitudinal consistent information. When affect is moderate or low, an
elaborated knowledge base leads to open-mindedness and enhanced capacity to process new
information. Knowledgeable people, they found, are more capable of recalling counter-attitudinal
information and evaluating information critically. Low prior knowledge induces closed-mindedness,
because new information cannot be processed adequately. In combination with moderate or low affect,
information is more likely to be processed in accordance to prior attitudes because the effort that is
invested in information processing is likely to be minimal.
3.3 Research questions and hypotheses
In summary, it is evident that prior attitudes and prior knowledge affect information problem solving.
As suggested by Smith Fabrigar, and Norris (2008), processing information consistent with prior
attitudes toward the topic of an information problem might even be a common strategy when
information is abundantly available and people have to be selective. This is the case when people can
11
use the Internet for solving information problems. There appears to be a paucity of research of the
impact of prior attitudes on particular stages of the information problem solving process. We argue
that from the start of information problem solving, bias could exert its influence. The present study
tries to fill this gap and addresses the question to what extent problem formulation, and the
corresponding information seeking activities, is influenced by prior attitudes and whether attitude
strength can explain differences between participants in the degree of consistency of bias in
information seeking with prior attitudes.
First, it was expected that participants demonstrate bias in all the information seeking
activities regarding problem formulation, consistent with their prior attitudes. Second, we
hypothesized that consistency of bias in information seeking with prior attitudes is higher in the first
information seeking activity (searching for information), compared to consistency in the remaining
information seeking activities (the selection and use of the information found). Third, it was expected
that attitude strength accounted for differences in consistency scores between participants. We
assumed that, in line with Sawicki et al. (2011), participants were relatively unfamiliar with the
information that would be found. Therefore, when controlling for prior knowledge, participants with
weaker prior attitudes were expected to demonstrate bias that was more consistent with prior attitudes
in all information seeking activities, and participants with stronger prior attitudes were expected to
demonstrate bias less consistent with their prior attitudes.
4. Method
4.1 Participants
All participants were first-year pre-service teachers of one Dutch teacher training institute. After four
years of higher education, qualified teachers teach all subjects to children between four and twelve
years of age in primary education. A total of 70 participants took part in this study (23 men, 47
women; Mage = 18.99 years, SD = 1.53). Because this study was embedded in a mandatory course, all
first-year students participated. All students were Dutch speaking and started with their second
semester of the first year.
4.2 Materials
Search task. Participants were given an ill-defined information problem. All participants received the
following task description: ‘You were asked by the editors of National Geographic magazine to write
a two-page article about the causes of climate change. You are completely free to elaborate on your
own ideas. Because you do not have time to speak with experts, you will have to find your information
on the Internet. You have decided to take fifteen minutes to explore this topic on the Internet. That
12
will help you to come up with a focused question that you find interesting for the article and that you
are going to answer later on with the information you can find on the Internet. After exploring the
topic, you will first share your ideas with the editors.’
Task environment. All participants worked individually on a work station: a desk with a personal
computer with the operating system Windows 7 and a 19” monitor. The screen resolution was set on
1280 x 1024 pixels per inch. The Internet browser Firefox was used, version 24.0.
4.3 Measures
Prior attitudes. Prior attitudes were measured with one paper-and-pencil-questionnaire. The
questionnaire served two goals: participant selection and determination of prior attitudes.
Participant selection. Only those participants who actually acknowledged the issue of climate
change were included in this study, because participants’ stance in the controversy regarding the
causes of climate change (natural vs. anthropological causes) was used to investigate bias. Seven items
questioned participants’ beliefs about the existence of climate change, for example: 'I don’t believe the
earth is warming up’. Participants rated their agreement with these items on a 7-point Likert scale
ranging from completely disagree to completely agree. Total scale reliability was high but should be
interpreted with caution, because of high inter-item correlation of three item pairs (α = .80, r ranging
from .55 to .62). The scores on the seven items were averaged to produce a composite measure.
Participants with mean scores equal or above scale mid-point, were included.
Prior attitudes. Two self-report scales assessed attitude position and attitude strength.
First, attitude position was measured with four items regarding two related controversies. Two items
questioned participants about their position in the controversy about causes of climate change. The
first item emphasised anthropological causes and the other item natural causes: ‘Humans are
responsible for the earth warming up’ and ‘Climate change is a natural phenomenon’. The other two
items questioned participants’ beliefs about the scientific evidence regarding climate change: “There is
enough scientific evidence to warrant the claim that man is responsible for climate change” and
“There are only few scientists who truly believe that our climate is changing due to anthropological
causes”. Principal component analysis confirmed that the items measured two distinct controversies,
explaining 68% of the variance. Next, for each item pair a proposition score was calculated. By
subtracting participants’ position with regard to the natural stance from the position toward the
anthropological stance, an attitude position score toward the proposition ‘climate change is caused by
man’ was calculated. The attitude position score, ranging from -6 to 6, was rescaled to 0 to 1, with
stepwise increments of 1/12. On this recoded scale a 0 reflected complete disagreement, 0.5 reflected a
neutral attitude position, and 1 complete agreement with the proposition.
13
Second, attitude strength was measured with a 6-items 7-point Likert scale. The dimensions
that were taken into account are importance, certainty, interest and ego-involvement. The items were
derived from previous research by Brannon, Tagler, and Eagly (2007) and Pomerantz et al. (1995).
Importance is operationalized with the statement ‘Climate change is an important issue for me’.
Certainty is operationalized with the statements ‘My opinions of climate change are not likely to
change’ and ‘I am sure that my opinions about climate change are right. Interest in relevant
information is operationalized with the statement ‘I am interested in information about the issue of
climate change’ and Ego-involvement is measured with two statements: ‘My opinions of climate
change are representative of my values’ and ‘My opinions of climate change are central to my selfimage’ (Pomerantz et al., 1995). The items of the prior attitude strength measure were subjected to
principal components analysis, which confirmed the presence of a one-factor-structure, explaining
56% of variance and with factor loadings ranging from .52 to .82. Scale reliability was high (α = .84),
but should be interpreted with caution, because of high inter-item correlation of four item pairs (r
ranging from .53 to .67).
Bias measures information seeking activities. The participants undertook four information seeking
activities. After information searching, participants were asked to use the information found to
formulate a central question for their article, a provisional answer and participants selected four useful
online sources. Each activity was assessed for the presence of bias, and therefore four bias measures
were established.
Bias in information searching was evaluated by looking at the queries within a search session.
Each query was judged as neutral or biased toward natural or anthropological causes. Only queries
with the keywords ‘human’ and ‘natural’, corresponding synonyms or direct searches into specific
anthropological or natural causes were judged as biased. For example, the queries ‘causes climate
change humans’ and ‘fossil fuels climate change’ were judged as queries biased in favour of
anthropological causes. Queries like ‘causes climate change’ and ‘enhanced greenhouse effect’ were
judged as neutral. Next, time spent within each query was taken into account. This was done by the
following formula: 0.5 + 0.5 x (ptime anthro - ptime natural), where ptime is the proportion of time spent
within a biased query relative to time spent within all queries. This resulted in a continuous scale, with
bias scores ranging from 0 to 1. For example, a score of 0.5 reflected a neutral session. In this case, the
participant only spent time within neutral queries or an equal amount of time was spent within
anthropological and natural biased queries.
Bias in participants’ questions was evaluated by assigning one of three ratings to each
question. A rating of 0.5 was assigned to questions that reflected a neutral position, 0 to biased
questions toward natural causes and 1 to a biased question toward anthropological causes. Questions
14
could not be scored if they only reflected consequences of climate change, for example ‘What can we
do to reduce the effects of climate change’. In addition, all questions were divided under two
independent subject matter specialists. A short scoring guideline with three examples were given. The
subject matter specialists were also asked to mark the questions they could not score. Agreement,
including agreement on questions that could not be scored, was substantial, κ = .611 (95% CI, .437 to
.785), p < .000. If inter-rater agreement was only computed based on questions that could be scored,
then 47 cases were included and κ increased to .720 (95% CI, .489 to .951), p < .000).
Bias in participants’ answers was evaluated by assigning one of three ratings to each answer.
A rating of 0.5 was assigned to answers that reflected a neutral position, 0 to biased answers toward
natural causes and 1 to a biased answer toward anthropological causes. In addition, all answers were
divided under two independent subject matter specialists. A short scoring guideline with three
examples were given. There was substantial agreement between the two judgments, κ = .683 (95% CI,
.489 to .877), p < .000.
Bias in participants’ selection of online sources was evaluated by rating the content of listed
webpages. Each webpage was examined by intuitive content analysis and a rating of 0.5 was assigned
to webpages that reflected a neutral position, 0 to biased pages toward natural causes and 1 to biased
webpages toward anthropological causes. Websites that predominantly engaged in anthropological or
natural causes were judged as biased. Bias meant that a website was selective to one viewpoint with
regard to the causes of climate change. A website was judged neutral when no outspoken viewpoint
was presented or both viewpoints were equally presented. Next, time spent on each page was taken
into account. This was done by the following formula: 0.5 + 0.5 x (ptime anthro - ptime natural), where
ptime is the proportion of time spent on a webpage relative to time spent within all selected pages. This
resulted in a continuous scale, with bias scores ranging from 0 to 1. For example, a score of 0.5
reflected a neutral selection. In this case, the participant only spent time on neutral webpages or an
equal amount of time was spent within anthropological and natural biased webpages.
Consistency with attitude position. Finally, four consistency scores were calculated by relating
participants’ (prior) attitude position to the bias found in each of the information seeking activities.
Because both attitude position and all four bias measures were projected on a 0-1 scale, consistency
was calculated by the following formula: 1 - | attitude position – bias in information seeking activity |.
The consistency scores reflected the degree of consistency of attitude position with bias found in the
information seeking activities, where 0 reflected no consistency and 1 reflected maximum consistency.
Consistency scores were interpreted within these two extremities. Consistency of prior
attitudes with bias in information seeking was viewed as less evident in the low range from 0 to 0.50
and more evident in the high range from 0.50 to 1. Instead of pursuing scale analysis, the theoretical
15
mean was chosen to establish these ranges because attitude position was measured with neutral items,
an odd-length Likert-scale was used, and bias measures were established by either calculation or
judgements corroborated by a subject matter specialist.
Prior knowledge. Different methods exist for assessing prior knowledge. Association tests and free
recall tests are two of these methods. Although association and free recall tests are less objective in
comparison with multiple choice test and results are known to confound with verbal abilities, they are
widely used because these tests afford in-depth investigation of prior knowledge (Dochy, Segers, &
Buehl, 1999). A combination of these methods is found in mapping techniques like mind mapping
(Eppler, 2006; Hay, Kinchin, & Lygo-Baker, 2008). Mapping techniques can be used to assess the
extent and, more importantly, the structure of declarative knowledge (Shavelson, Ruiz-Primo, &
Wiley, 2005).
In this study, fully student-generated mind maps were used. A mind map has a ‘central image’
with main themes that radiate as branches from the central image. Mind maps are more associative and
less structured than concept maps. Mind maps focus more on free concept and relationship exploration
without a superimposed structure (Davies, 2010). Less directed mapping techniques are shown to elicit
more higher-level thinking like content-relevant explanations of concepts and their relationships
(Ruiz-Primo et al., 2001b; Shavelson, Ruiz-Primo, & Wiley, 2005).
Prior to judging the mind maps, an expert mind map was construed and validated by a subject
matter teacher. The rating of mind maps was initially based on different characteristics: the number
concepts, the position of concepts in the hierarchy and the relationships between concepts (see also
D’Antoni, Zipp, & Olson, 2009; Evrekli, Inel, & Balim, 2010). The quality of participants’ mind
maps, however, did not afford a fine-grained scoring protocol. Also, participants were not asked to
explain their mind maps and they used different central keywords (‘climate change’ and ‘causes of
climate change’). A modest scoring protocol was therefore used. Only keywords or short phrases were
allowed as nodes in the mind maps. With exception of the central keyword, the number of nodes in
each mindmap was counted. This total score was corrected by subtracting nodes that could not be
related to the expert mind map. Nodes related to consequences of climate change were marked as an
example, and possible misconceptions were marked, but neither was subtracted from the corrected
total score.
A random sample of 36 mind maps was scored by a second judge. A two-way mixed, absolute
agreement, intra-class correlation coefficient was calculated to assess inter-rater agreement.
Agreement proved to be good (single measures ICC = .882, 95% CI, .782 to .938, p < .000). Second,
an indication of the accuracy of the prior knowledge assessment was established by comparing the
results with the results of two National entry-level exams freshmen pre-service teachers had to take.
16
These tests measure subject matter knowledge on geography and science (biology and physics) and
were considered to approximate prior knowledge on a more general level. A weak correlation was
found for the science test, r(57) = .34, p < .01. A marginal correlation was found between corrected
prior knowledge scores and the geography test, r(57) = .19, p = .146.
4.4 Procedure
This cross-sectional one shot study followed two stages: the assessment of prior attitudes and a
classroom session in which students worked on an information problem.
Assessment of prior attitudes. To avoid priming effects, ten days before the information seeking
session the 17-item questionnaire was administered on paper, which inquired about participants’
attitudes towards the issue of climate change and the causes of climate change. The questionnaire was
handed out together with an exam students had to take. At the end of the exam, each student filled out
the questionnaire. The questionnaire started with a short introduction on paper and after the
questionnaire was filled out, the questionnaire was handed in and students left the classroom.
Classroom session. All participants took part in a forty minute classroom session. These sessions
were scheduled in four groups of approximately 20 students. After a short word of welcome, each
student was seated behind a computer an asked to work individually. The classroom session was
divided in four parts, and the instruction for every part was displayed on the students’ computer
screens. Participants were prompted to start and finish each of the four parts simultaneously to ensure
equal time on task.
For the first part, participants were given a task description: a short story of 120 words that
invited them to prepare a two-page article for a magazine about the causes of climate change.
Participants read the task description on their screens. After two minutes, when every participant had
finished, they were asked to proceed with the next part.
Second, participants were instructed to draw a mind map using as much of their prior
knowledge about the topic climate change as possible. This task was introduced as a sensible first step.
Participants were told to place ‘climate change’ in the centre of their mind map. The mind map was
constructed in the digital environment Mindmaps, an open source and web-based mind mapping
application. All students had practiced working with this application once in preparation of this
session but did not receive training in constructing mind maps. After five minutes, students were asked
to end their mind mapping activities and save their mind map.
Third, every participant was given fifteen minutes for exploratory activities with use of the
Internet. Participants had been pre-alerted to the last part in which they would be asked to write down
17
an essential question for the article that also seemed worthwhile investigating further. Participants
could take notes. They were instructed to use Firefox as their Internet browser and were told that their
browser use was recorded. The information seeking process was recorded with recording software, a
Firefox plug-in. This plug-in recorded their browser use, use of search engines, search terms, and
browsed webpages. Hyperlinks that referred to useful information resources could be copied and
pasted in an empty, digital worksheet.
For the fourth and last part, participants were given stepwise instruction for three short
assignments. The first assignment was to write down the essential question. “Write down the main
question about the causes of climate change that you are going to answer in your paper with a search
for information on the Internet”. This question reflects the information problem statement after the
exploratory activities. The second assignment was to write down a possible answer in two or three
sentences to that question. “Write down, in three sentences, a possible answer to that question that you
will probably arrive at.” This reflects the student’s hypothesis. The third assignment was to select four
sources from the previous encountered information sources that seem worthwhile for further
investigation in light of the question posed. “Select four sources that you encountered and you
probably will use to answer the question of your paper.”
4.5 Data analysis
The first hypothesis, that participants demonstrate bias consistent with their prior attitudes in all four
information-activities, will be tested with one sample t-tests, testing the highest and lowest consistency
scores to the defined lower and upper range of consistency scores.
The second hypothesis, that consistency of prior attitudes with bias in information seeking is
most evident in information searching, in comparison to the remaining information seeking activities,
will be tested using a one way multivariate analysis of variance. The multivariate model consists of
one independent variable with four levels, representing the information seeking activities, and the
consistency scores regarding proposition 1 and 2 as dependent variables. This hypothesis will be tested
with Helmert-contrasts.
Prior to analysis, the assumptions were checked. Two univariate outliers (z = -3.72 and z = 3.51; | z | > 3.29) and one multivariate outlier (Mahalanobis D = 21.02; D > 10.828, χ2 with df = 1, p <
.001) were removed. Only for query formulation (queries), the assumption of multivariate normality
was not met (W = 0.913, p < .001), which can reduce power. However, the sample sizes in each cell
were approximately equal and further examination of the variance-covariance matrices showed that
the assumption of homogeneity of variance was met (ratiomin:max variance 1:2.4 and covariance 1:3.9).
The third hypothesis, that variance in consistency scores could be explained, when controlling
for prior knowledge, by a linear relationship between consistency and attitude strength, will be tested
18
using multiple regression. In this model prior knowledge and attitude strength are entered as predictor
variables and, repeated for all four information seeking activities, consistency as criterion variable.
With 60 respondents and two predictor variables, the number of cases was below minimal requirement
of 66 (50 + 8 times 2) for testing multiple correlations with anticipated medium-size relationships (ƒ2
=.15) and substantially below the required 106 (104 + 2) for testing the individual predictors
(Tabachnick & Fidell, 2013). The confidence level was set to 95%. Statistical power to detect
medium-size relationship was of concern. A priori power calculation with G*Power revealed
statistical power of .747 (<.80) and therefore results were considered to be a tentative exploration of
relationships.
Prior to analysis, the assumptions for regression analysis were checked. For each combination
of dependent and independent variables scatterplots were produced that indicated linear relationships.
A restrictive decision rule was used for deletion of univariate outliers, biased toward inclusion of data
(| z | > 3.29). Scatterplots and Mahalanobis D values were used to identify outliers in the regression
model and to assess discrepancy. The influence of the outliers was assessed by Cook’s D. Only high
discrepant outliers (Mahalanobis D values greater than 9.21 (2 distribution with df = 2, p < .01)) that
were influential (Cook’s D values greater than 1) were deleted. Identified outliers were also inspected
for their impact on each of the predictors and high influential cases on predictors were deleted
(standardized DFBeta’s > 1). Only one case was deleted in the regression analysis of the information
seeking activities question, because it was influential on the predictor attitude strength. Both predictors
were uncorrelated, r(58) = .029 and VIF values were acceptable. Multicollinearity was not an issue.
Independence of residuals was checked with the Durbin-Watson statistic and for each regression
model found to be around 2.
5. Results
A total of 70 students participated in this study. Six participants were excluded, because they did not
complete two or more tasks. Also, items of the attitude questionnaire were examined prior to analysis
for accuracy of data entry and missing values. Minimum and maximum values were checked and only
one missing value was found. Second, only participants who acknowledged the issue of climate
change were included, leaving 60 participants for further analysis.
5.1 Consistency of bias in information seeking with prior attitudes
First, it was expected that participants demonstrated bias in all four information-seeking activities,
consistent with their prior attitudes. Participants’ prior attitudes were questioned by means of two
propositions. For each proposition and information seeking activity, table 1 displays the mean
19
consistency of prior attitudes with bias demonstrated in information seeking. Consistency was
expressed on a scale from 0 to 1.
A paired-sample t-test was conducted to compare consistency scores on proposition 1 with
proposition 2. There was not a significant difference between the scores on proposition 1 (M = .75 SD
= .19) and proposition 2 (M = .77 SD = .19); t(223) = -1.41, p = .161. These results suggest that
consistency of prior attitudes with bias in information seeking, that has been made visible by means of
two propositions, appeared equal. Therefore, the highest and lowest cell-mean of the four informationseeking activities in proposition 1 was used to classify consistency scores in the upper or lower range
(figure 1). Most consistency was visible in participants’ queries (M = .84), though significantly below
scale maximum of 1 (t(59) = -9.34, p < .000). Least consistency was found in the provisional answers
(M = .66), which was significantly above the lower bound of the upper range of 0.5 (t(54) = 5.71, p <
.000). Consistency ranged in the upper range of the consistency scale. As expected, participants
demonstrate bias in information seeking activities, consistent with their prior attitudes (hypothesis 1).
Table 1.
Descriptive Statistics Consistency of Bias in Information Seeking with Prior Attitudes
Proposition 1
Proposition 2
N
M
SD
N
M
SD
Information searching
Queries
60
.84
.13
59
.83
.15
Using the information
Questions
52
.77
.21
52
.79
.20
Answers
55
.66
.21
54
.72
.22
Sources
60
.72
.20
59
.73
.17
.75
.19
.77
.19
Mean
Note. Consistency scores ranged from no consistency to maximum consistency (0-1)
20
Figure 1.
Distribution of Consistency Scores for Proposition 1 and 2
Second, consistency of bias in information seeking with prior attitudes was expected to be higher in
the first information seeking activity, when participants search for information, compared to
consistency in the remaining information seeking activities that entail the selection and use of the
information found. In order to examine this difference, a multivariate analysis of variance was
conducted. The analysis showed that the degree of consistency differed with respect to the four
information seeking tasks (Pillai’s T = .139, F(6,424) = 5.269, p < .001). Approximately 6.9% of the
variance in the degree of consistency is associated with the information seeking activities. In a followup analysis, the results of the between-subject test indicated that the consistency scores differed within
each proposition (F(3, 212), p < 0.001, η2 = .117 and F(3, 212), p <0.01, η2 = .078).
A planned comparison with Helmert contrasts was used to assess the difference between the
consistency scores of the first information seeking activity, query formulation, with the other
information seeking activities. The results showed, for both propositions, that bias in information
seeking was more consistent with prior attitudes in query formulation, the first information seeking
activity, compared to consistency in the remaining information seeking activities (95% CI, .057 to
.163, p < 0.01 and .037 to .142, p < 0.01). Generally speaking, bias in information seeking is most
consistent with prior attitudes when searching for information and to a lesser extent in the information
seeking activities after information searching (hypothesis 2).
21
5.2 The relationship between consistency and attitude strength
We assumed that the degree of consistency of bias in information seeking with prior attitudes was
influenced by attitude strength; a linear relationship was expected. When controlling for prior
knowledge, participants with weaker prior attitudes were expected to demonstrate bias that was more
consistent with prior attitudes and participants with stronger prior attitudes were expected to
demonstrate bias less consistent with prior attitudes.
Queries. Participants were given the task to engage in exploratory information searching activities.
Before analysis, an auxiliary description of the search sessions was made. Session duration was fifteen
minutes and each session was recorded in a search log. A search log can be described through session,
query, and term level analysis (Jansen, 2006; Jansen, 2009).
On average, recorded session duration was 14 minutes and 26 seconds. A session started on
first browser interaction and stopped when participants were instructed to stop. A total of 396 queries
were run with exclusion of moving back and forth from the search engine result page and session
length, the number of queries per searcher that was build, was 5,58 (SD = 3.58). Participants were
instructed to use Google as their search engine, but in 9.60% of the queries other search engines like
Google Scholar were used.
Query analysis showed that participants build 145 unique queries. High usage queries were
klimaatverandering (climate change; 58 times), oorzaken klimaatverandering (causes climate change;
58 times), broeikaseffect (greenhouse effect; 15 times), gevolgen klimaatverandering (consequences
climate change; 14 times), and natuurlijke oorzaken klimaatverandering (natural causes climate
change; 14 times). Participants started with 17 unique initial queries. The most occurring initial
queries were oorzaken klimaatverandering (causes climate change; 22 times), klimaatverandering
(climate change; 19 times), de oorzaken van klimaatverandering (the causes of climate change; 6
times), oorzaken van klimaatverandering (causes of climate change; 4 times), and
klimaatveranderingen (climate changes; 4 times). Term analysis showed that participants together
used 861 terms in their queries, from which 134 unique terms. High usage terms were
klimaatverandering (climate change; 250 times), oorzaken (causes; 125 times), broeikaseffect
(greenhouse effect; 30 times), gevolgen (consequences; 23 times) and opwarming (warming; 19
times).
A standard multiple linear regression was performed between consistency as the dependent
variable and prior knowledge and attitude strength as independent variables (Table 2). For proposition
1, the results of the regression indicated the two predictors explained 8.6% of the variance (R2adj =
.054, F(2,57) = 2.69, p = .077). It was found that attitude strength accounted for a significant
proportion of the variance in consistency, after controlling for the effect of prior knowledge, R2 change
22
= .084, F(1, 57) = 5.256, p < .05. Attitude strength was a significant predictor of consistency ( = 2.90, t(57) = -2.29, p < .05). For proposition 2 the results of the regression indicated the two predictors
explained 14,7% of the variance (R2adj = .116, F(2, 56) = 4.816, p = .012). It was found that attitude
strength accounted for a significant proportion of the variance in consistency, after controlling for the
effect of prior knowledge, R2 change = .146, F(1, 56) = 9.571, p < .01. Attitude strength was a
significant predictor of consistency ( = -.382, t(56) = -3.094, p < .01).
Table 2.
Results Regression Analysis – Queries
Proposition 1 (N = 60)
Zero-Order r
PKN
ATS
Attitude strength (ATS)
Prior knowledge (PKN)
.029
Z-O r
CON

sr2
b
CON

sr2
b
-.289*
-.290*
.084
-.024
-.382**
-.382**
.146
-.049
.045
.053
.003
.001
-.031
-.030
.000
-.001
Intercept =
Mean
9.40
3.78
.873
SD
4.00
1.13
.092
.950
Intercept =
1.029
.834
R2 = .086
.146
R2 = .147*
Note * p < .05 and ** p < .01; Consistency scores (CON) based on statement set 1 after transformation
As expected, these results suggested that attitude strength was associated with consistency. Bias in
information searching was consistent with prior attitudes, but consistency was higher for participants
that hold weaker attitudes and lower for participants that hold stronger attitudes. In proposition 2, an
even more pronounced influence of attitude strength was visible (hypothesis 3). Figure 2 displays
consistency scores used in the regression analysis for both propositions in relation to attitude strength.
Due to ceiling effects, results of the regression analysis should be interpreted with caution.
23
Figure 2.
Plot of the Relationship between Consistency and Attitude Strength for IS-activity Queries
Questions. A standard multiple linear regression was performed between bias consistency as the
dependent variable and prior knowledge and attitude strength as independent variables (Table 3).
For proposition 1 the results of the regression indicated the two predictors explained 3.1% of the
variance (R2adj = -.009, F(2,48) = .776, p = .466). It was found that attitude strength accounted for a
significant proportion of the variance in consistency, after controlling for the effect of prior
knowledge, R2 change = .028, F(1, 48) = 1.402, p < .05. Attitude strength was only a marginal
predictor of consistency ( = -.018, t(48) = -1.18, p = .242). For proposition 2 the results of the
regression indicated the two predictors explained 11.4% of the variance (R2adj = .076, F(2,47) = 3.021,
p = .058). Results should be interpreted with caution because of mild heteroskedasticity (bow-tie
shaped; variance of residuals decreases as ŷ increases). It was found that attitude strength accounted
for a significant proportion of the variance in consistency, after controlling for the effect of prior
knowledge, R2 change = .097, F(1, 47) = 5.072, p < .05. Attitude strength was a significant predictor
of consistency ( = -.310, t(47) = -2.25, p < .05).
24
Table 3.
Results Regression Analysis – Questions
Zero-Order r
PKN
ATS
.076
Z-O r
CON

sr2
SD
9.47
3.89
3.67
1.04
CON

sr2
b
-.310*
.095
-.035
.113
.012
.003
-.167
-.018
.028
-.168
-.312*
-.055
-.002
.003
-.059
.135
Intercept =
Mean
b
.841
.114
.925
Intercept =
.851
R2
= .031
.118
R2 = .114
Note * p < .05 and ** p < .01; Consistency scores (CON) based on statement set 1 and 2 after transformation;
In consistency scores, derived from proposition 1, a marginal trend was found for attitude strength,
with a slope similar to the regression line in the first information seeking activity. However, in
proposition 2 the influence of attitude strength was pronounced. Bias in essential questions was
consistent with prior attitudes, and attitude strength partially explained differences in participants’
consistency scores in line with the expectation (hypothesis 3). Figure 3 displays consistency scores
used in the regression analysis for both propositions in relation to attitude strength. Again, due to
ceiling effects, results of the regression analysis should be interpreted with caution.
Figure 3.
Plot of the Relationship between Consistency and Attitude Strength for IS-activity Questions
.948
25
Provisional answers. A standard multiple linear regression was performed between consistency as the
dependent variable and prior knowledge and attitude strength as independent variables (Table 4).
For proposition 1, the results of the regression indicated the two predictors explained 3.4% of the
variance (R2adj = -.003, F(2,52) = .921, p = .404). It was found that attitude strength did not explain
the variance in consistency, after controlling for the effect of prior knowledge, R2 change = .001, F(1,
52) = .046, p = .832. Attitude strength did not influence consistency scores ( = -.029, t(52) = -.214, p
= .832). For proposition 2, linearity of the relationship between consistency scores and attitude
strength is doubtful (Figure 4). The regression indicated the two predictors explained 0.9% of the
variance (R2adj = -.029, F(2,51) = .236, p = .791). It was found that attitude strength did not explain the
variance in consistency, after controlling for the effect of prior knowledge, R2 = .000, F(1, 51) = .000,
p = .998. Attitude strength did not influence consistency scores ( = .000, t(51) = -.003, p = .998).
Results should be interpreted with caution, because common transformations did not lead to a normal
distribution of residuals.
Table 4.
Results Regression Analysis – Answers
Zero-Order r
PKN
ATS
.001
Z-O r
CON

sr2
b
CON

sr2
b
-.029
-.029
.000
-.005
.002
.000
.000
.000
-.183
-.183
-.033
-.010
-.096
-.096
-.096
-.005
Intercept =
Mean
9.22
3.78
.662
SD
3.90
1.16
.211
.773
Intercept =
.772
.722
R2 = .034
.216
R2 = .009
Bias in provisional answers was consistent with prior attitudes. In contrast with queries and questions,
attitude strength did not explain differences in consistency scores (hypothesis 3). Prior knowledge has
a notable but marginal influence. Participants with relative higher prior knowledge appeared to be
inclined to formulate provisional answers that are less consistent with their prior attitudes. Figure 4
displays consistency scores based on both statement sets in relation to attitude strength.
26
Figure 4.
Plot of the Relationship between Consistency and Attitude Strength for IS-activity Answers
Source selection. Participants were given the task to select four sources. Consistency of bias in
sources with prior attitudes was calculated. A standard multiple linear regression was performed
between consistency as the dependent variable and prior knowledge and attitude strength as
independent variables (Table 5). For proposition 1, the results of the regression indicated the two
predictors explained 3.3% of the variance (R2adj = -.002, F(2, 56) = .951, p = .392). It was found that
attitude strength did not explain the variance in consistency, after controlling for the effect of prior
knowledge, R2 change = .025, F(1, 56) = 1.465, p = .231. Attitude strength did not influence
consistency scores ( = .160, t(56) = 1.21 p = .231. Results should be interpreted with caution,
because common transformations did not lead to a normal distribution of residuals. For proposition 2,
the results of the regression indicated the two predictors explained 0.9% of the variance (R2adj = -.026,
F(2, 56) = .252, p = .778). It was found that attitude strength did not explain the variance in
consistency, after controlling for the effect of prior knowledge, R2 change = .007, F(1, 56) = 3.85, p =
.537. Attitude strength did not influence consistency scores ( = .013, t(56) = .621 p = .537) .
27
Table 5.
Results Regression Analysis – Source selection
Zero-Order r
PKN
ATS
.076
Z-O r
CON

sr2
b
CON

sr2
b
.152
.160
.159
.029
.083
.083
.083
.013
-.087
-.099
-.099
-.005
.046
.046
.046
.002
Intercept =
Mean
SD
9.46
4.01
3.73
1.06
.724
.193
.661
Intercept =
.659
.725
R2
= -.002
.173
R2 = -.026
Bias in source selection was consistent with prior attitudes. In contrast with queries and questions, but
in line with answers, attitude strength did not explain differences in consistency scores (hypothesis 3).
Figure 5 displays consistency scores based for both propositions in relation to attitude strength.
Figure 5.
Plot of the Relationship between Consistency and Attitude Strength for IS-activity Sources
28
6. Discussion
6.1 Conclusions
The main purpose of this study was to provide insight into the impact of prior attitudes towards and
prior knowledge of the controversial topic of an information problem on the information seeking
activities performed by students in higher education when defining an information problem. Students
use keywords to search, find and explore online information. They then take notes, formulate central
questions and develop general ideas of the answer. These activities, which are often short of duration
in educational practice, result in a concise description of the information problem that guides the
subsequent steps of information problem solving (Brand-Gruwel, Wopereis, & Walraven, 2009). This
first stage of information problem solving was studied using an ill-defined information problem, and
students were given considerable freedom in task performance.
The most important finding is that students demonstrate bias right from the start of the process
of information problem solving. Students act upon their prior attitudes, even when the assignment is to
make preparations for an informative journal article about a science topic like climate change. This
verifies and extends the findings of Hart et al. (2009). The influence of prior attitudes is not limited to
students’ selection of information; these attitudes in fact surface in all information seeking activities.
When students favour a particular position, information seeking behaviour is in alignment with that
particular position.
Most consistency of prior attitudes with bias in information seeking is found in the first
activity, when students search for information. In the follow-up activities, when students use the
information found, consistency is also visible, albeit to a lesser extent. It is possible that congruency
between adjacent information seeking activities gains importance over prior attitudes when progress is
made in the information seeking activities. Additional analysis shows that consistency scores decrease,
but correlations between consistency scores of adjacent information seeking activities increase. These
correlations exceed those between non-neighbouring information seeking activities. This finding
emphasises the importance of the (cognitive) activities students engage in, bridging information
searching and the follow-up activities in problem formulation.
Furthermore, the degree of consistency partially depends on the strength of prior attitudes, at
least in the first two information seeking activities (queries and questions). Mediated by search
engines, participants have vast amounts of information at their disposal. Those who are presumed to
be unfamiliar with all the online information demonstrate information seeking behaviour increasingly
consistent with their own prior attitudes, when strength of prior attitudes decreases. This finding is in
resonance with Sawicki et al. (2011). In this particular constellation of low familiarity and weak prior
attitudes, high consistency might be explained by students’ heightened motivations to validate their
29
weak attitudes. Hart et al. (2009) state in their meta-analysis that low confidence in attitudes is related
to heightened defence motivations, resulting in stronger bias. On the other hand, low consistency
might be explained by dissonance theory (see for example Smith et al, 2008). Students with strong
attitudes might experience less dissonance when engaging in information reflecting other views and
may therefore be less motivated to avoid or more motivated to open up counter-attitudinal
information.
Finally, prior knowledge does not appear to have influence on the consistency between prior
attitudes and bias in any of the information seeking activities. This was unexpected in two ways. First,
given the argument that strong attitudes can be the result of high prior knowledge (Wood, Rhodes, &
Biek, 2014), at least a moderate correlation between the two predictors was expected. This was not the
case. Second, because prior knowledge impacts information seeking and processing, it was
hypothesized that the moderating impact should be visible in the full range of activities related to
problem formulation. Again, this was not the case.
It is doubtful that topic knowledge is of less importance in the first stage of information
problem solving. The most obvious explanation comes from methodological difficulties of using mind
maps to assess prior knowledge, aside from concerns over statistical power. The mind map task did
not successfully elicit issue relevant prior knowledge. Most participants used ‘climate change’ as
central image instead of ‘causes of climate change’, which entails that the measurement lacks
precision. Irrelevant variation might be introduced because of differences in verbal fluency, which is
known to confound with the assessment of prior knowledge via mind maps. Finally, the quality of
most mind maps was poor. It can be argued that most participants, even participants with relatively
high prior knowledge scores, have to be classified as novices with regard to the chosen topic.
Considering all participants as novices, combined with the exploratory nature and short duration of the
information-searching event, could explain the absence of the influence of prior knowledge.
6.2 Limitations and future research
When studying bias in information seeking while giving participants freedom in searching for
information, considerable thought has to go to choosing an issue or attitude object. The degree of bias
in the field of expertise, the availability of information regarding different views, and the
representation of information in search engine result pages (for example, sponsored links), were not
taken into account. It is very reasonable to assume that the rank position of online sources on the
search engine result pages introduced bias in all four collections of consistency scores (White, 2013).
In this study, students visited webpages which were predominantly presented on the first search engine
result page. A strong negative skewed frequency distribution was visible when investigating rank
30
position of the 794 visited websites (Mdn = 4) and 85.6% of the hyperlinks that were followed by
students was presented in the top ten positions of the search engine result page.
The second concern regards the attitude strength construct. Because the primary goal was to
take the full range of information activities into account, a general measure of attitude strength was
constructed. Krosnick et al. (1993) advise caution in using composite scales to measure attitude
strength due to a lack of evidence for one uniform factor structure. Also, conflicting findings regarding
strength related attributes of attitudes warrants a more fine-grained approach. However, scale
reliability was high, and both certainty and confidence were part of the strength construct in this study.
Third, external validity is limited. This study was done in one particular teacher education
college in the Netherlands. In addition to the specific population, first-year pre-service teachers, the
context in which this study took place might induce a particular kind of information seeking
behaviour. A global goal for the information seeking activities was given, and students had
experienced in their first semester that particular types of information (and maybe even particular
types of views) are valued. In addition to cognitive characteristics, these contextual variables may also
explain different findings.
Finally, the results suggest a more complex relationship between attitude position, strength
and knowledge. Although attitude strength and prior knowledge were not correlated, an inspection of
only the 32 participants with pro-anthropological attitudes shows a positive relationship between
attitude strength and prior knowledge, which is absent in both the neutral and contra-anthropological
group. This is probably due to differences in accessibility of pro-anthropological information in
comparison to contra-anthropological information pur sang. Sample size unfortunately did not afford
further analysis taking specific positions into account, but participants or particular groups of
participants may differ in the extent to which their attitudes are grounded in relevant topic knowledge.
Future research could disclose the impact prior knowledge, differentiating between attitude positions
and stages of information problem solving.
6.3 Implications
When lecturers in higher education use information problems as their pedagogical strategy, they
should realise that students differ in how they engage in these problems. More specifically, students
differ in their attitudes regarding the topic under investigation, and this affects information problem
solving. Ample time should therefore be devoted to problem formulation, and lecturers should make
their students aware of different views that might exist with regard to the topic under investigation.
Fortunately, the first stage of problem formulation is suitable to be subject of classroom collaboration
and discussion. When working in groups on the same (ill-defined) information problem, problem
formulations can be exchanged, and as differences and cognitive bias surface they can become subject
of further inquiry. Providing insight into the pitfalls, like cognitive bias, may also render more openmindedness. When engaging in the first stage of information problem solving, it is worthwhile to
evaluate the process and results: to what extent did attitudes or particular views affect the activities
that result in problem formulation? This activity can be added as a specific regulatory activity in the
IPS-I model of Brand-Gruwel, Wopereis, and Walraven (2009). From practitioners’ perspective, an
interesting question arises: which pedagogical interventions can effectively reduce bias?
31
32
References
Ajzen, I. (2001). Nature and operation of attitudes. Annual Review of Psychology, 52(1), 27-58. doi:
0066-4308/01/0201-0027
Anderson, J. R. (1993). Problem solving and learning. American Psychologist, 48(1), 35–44.
doi:10.1037/0003-066x.48.1.35
Anderson, T. D. (2006). Uncertainty in action: Observing information seeking within the creative
processes of scholarly research. Information Research, 12(1), paper 283. Retrieved from
http://informationr.net/ir/12-1/paper283.html
Attfield, S., Blandford, A., & Dowell, J. (2003). Information seeking in the context of writing: A
design psychology interpretation of the “problematic situation. Journal of Documentation, 59(4),
430–453. doi:10.1108/00220410310485712
Belkin, N. J. (2000). Helping people find what they don’t know. Communications of the ACM, 43(8),
58–61. doi:0002-0782/00/0800
Bohner, G., & Dickel, N. (2011). Attitudes and attitude change. Annual Review of Psychology, 62,
391–417. doi:10.1146/annurev.psych.121208.131609
Bowler, L. (2010). A taxonomy of adolescent metacognitive knowledge during the information search
process. Library & Information Science Research, 32(1), 27–42. doi:10.1016/j.lisr.2009.09.005
Brand-Gruwel, S., Wopereis, I., & Vermetten, Y. (2005). Information problem solving by experts and
novices: analysis of a complex cognitive skill. Computers in Human Behavior, 21(3), 487–508.
doi:10.1016/j.chb.2004.10.005
Brand-Gruwel, S., Wopereis, I., & Walraven, A. (2009). A descriptive model of information problem
solving while using internet. Computers & Education, 53(4), 1207–1217.
doi:10.1016/j.compedu.2009.06.004
Brannon, L. A., Tagler, M. J., & Eagly, A. H. (2007). The moderating role of attitude strength in
selective exposure to information. Journal of Experimental Social Psychology, 43(4), 611–617.
doi:10.1016/j.jesp.2006.05.001
33
Byström, K., & Järvelin, K. (1995). Task complexity affects information seeking and use. Information
Processing & Management, 31(2), 191-213. doi:10.1016/0306-4573(95)80035-r
Case, D. O. (2012). Looking for information: A survey of research on information seeking, needs and
behavior. Bingley, UK: Emerald Group Publishing.
Chen, S. Y., & Macredie, R. (2010). Web-based interaction: A review of three important human
factors. International Journal of Information Management, 30(5), 379–387.
doi:10.1016/j.ijinfomgt.2010.02.009
Chi, M. T. H., & Glaser, R. (1985). Problem-solving ability. In R. Sternberg (Ed.), Human abilities:
An information processing approach (pp. 227–250). San-Francisco, CA: Freeman.
Chung, J. S., & Neuman, D. (2007). High school students’ information seeking and use for class
projects. Journal of the American Society for Information Science and Technology, 58(2001),
1503–1517. doi:10.1002/asi
Crano, W. D., & Prislin, R. (2006). Attitudes and persuasion. Annual Review of Psychology, 57, 345–
74. doi:10.1146/annurev.psych.57.102904.190034
D’Antoni, A. V, Zipp, G. P., & Olson, V. G. (2009). Interrater reliability of the mind map assessment
rubric in a cohort of medical students. BMC Medical Education, 9, 19. doi:10.1186/1472-6920-919
Dervin, B., & Nilan, M. (1986). Information needs and uses. In M. Williams (Ed.), Annual Review of
Information Science and Technology (pp. 3–33). White plains, NY, USA: Knowledge Industry
Publications, Inc.
Dochy, F., Segers, M., & Buehl, M. M. (1999). The relation between assessment practices and
outcomes of studies: The case of research on prior knowledge. Review of Educational Research,
69(2), 145–186. doi:10.3102/00346543069002145
Edwards, K., & Smith, E. E. (1996). A disconfirmation bias in the evaluation of arguments. Journal of
Personality and Social Psychology, 71(1), 5–24. doi:10.1037/0022-3514.71.1.5
Eisenberg, M. B. (2008). Information literacy: Essential skills for the information age. Journal of
Library & Information Technology, 28(2), 39–47. doi:10.14429/djlit.28.2.166
34
Eppler, M. J. (2006). A comparison between concept maps, mind maps, conceptual diagrams, and
visual metaphors as complementary tools for knowledge construction and sharing. Information
Visualization, 5(3), 202–210. doi:10.1057/palgrave.ivs.9500131
Evrekli, E., İnel, D., & Balim, A. G. (2010). Development of a scoring system to assess mind maps.
Procedia - Social and Behavioral Sciences, 2(2), 2330–2334. doi:10.1016/j.sbspro.2010.03.331
Fischer, P., & Greitemeyer, T. (2010). A New Look at Selective-Exposure Effects: An Integrative
Model. Current Directions in Psychological Science, 19(6), 384–389.
doi:10.1177/0963721410391246
Foster, A. (2006). A non-linear perspective on information seeking. In A. Spink, & C. Cole (Eds.),
New Directions in Human Information Behavior (pp. 155-170). Dordrecht, Netherlands:
Springer.
Foster, A., & Urquhart, C. (2012). Modelling nonlinear information behaviour: Transferability and
progression. Journal of Documentation, 68(6), 784–805. doi:10.1108/00220411211277046
Hart, W., Albarracín, D., Eagly, A. H., Brechan, I., Lindberg, M. J., & Merrill, L. (2009). Feeling
validated versus being correct: A meta-analysis of selective exposure to information.
Psychological Bulletin, 135(4), 555–88. doi:10.1037/a0015701
Hay, D., Kinchin, I., & Lygo-Baker, S. (2008). Making learning visible: The role of concept mapping
in higher education. Studies in Higher Education, 33(3), 295–311.
doi:10.1080/03075070802049251
Hill, J. R. (1999). A conceptual framework for understanding information seeking in open-ended
information systems. Educational Technology Research and Development, 47(1), 5-27.
doi:10.1007/bf02299474
Ingwersen, P., & Järvelin, K. (2005). The turn: Integration of information seeking and retrieval in
context. Dordrecht, Netherlands: Springer.
Isbell, D., & Kammerlocher, L. (1998). Implementing Kuhltau: A new model for library and reference
instruction. Reference Service Review, 26(3/4), 33–44. doi:10.1108/00907329810307722
35
Jang, S. M. (2013). Seeking congruency or incongruency online? Examining selective exposure to
four controversial science issues. Science Communication, 36(2), 143-167.
doi:10.1177/1075547013502733
Jansen, B. J. (2006). Search log analysis: What it is, what’s been done, how to do it. Library &
Information Science Research, 28(3), 407–432. doi:10.1016/j.lisr.2006.06.005
Jansen, B. J. (2009). The methodology of search log analysis. In B.J. Jansen, A. Spink, & I. Taksa
(Eds.), Handbook of research on Web log analysis (pp. 100-123). Hershey, PA: Information
Science Reference.
Jonas, E., Schulz-Hardt, S., Frey, D., & Thelen, N. (2001). Confirmation bias in sequential
information search after preliminary decisions: An expansion of dissonance theoretical research
on selective exposure to information. Journal of Personality and Social Psychology, 80(4), 557–
571. doi:10.1037/0022-3514.80.4.557
Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology
Research and Development, 48(4), 63–85. doi:10.1007/BF02300500
Knobloch-Westerwick, S. (2009). Looking the other way: Selective exposure to attitude-consistent
and counterattitudinal political information. Communication Research, 36(3), 426–448.
doi:10.1177/0093650209333030
Kobayashi, K. (2010). Strategic use of multiple texts for the evaluation of arguments. Reading
Psychology, 31(2), 121–149. doi:10.1080/02702710902754192
Krosnick, J. A., Boninger, D. S., Chuang, Y. C., Berent, M. K., & Carnot, C. G. (1993). Attitude
strength: One construct or many related constructs? Journal of Personality and Social
Psychology, 65(6), 1132–1151. doi:10.1037/0022-3514.65.6.1132
Krosnick, Jon, A., Judd, C. M., & Wittenbrink, B. (2005). The measurement of attitudes. In D.
Albarracin, B. T. Johnson, & M. P. Zanna (Eds.), Handbook of attitudes and attitude change (pp.
21–76). Mahwah, NJ: Erlbaum.
Kuhlthau, C.C. (2004). Seeking meaning. A process approach to library and information services.
Westport: Libraries Unlimited, Inc.
36
Land, S.M., & Greene, B.A. (2000). Project-based learning with the world wide web: A qualitative
study of resource integration. Educational Technology Research and Development, 48(1), 45–66.
doi:10.1007/BF02313485
Limberg, L. (1999). Three conceptions of information seeking and use. In Proceedings of the Second
international conference on research in information needs, seeking and use in different contexts.
(pp. 116–135).
Makinster, J. G., Beghetto, R. A., & Plucker, J. A. (2002). Why can’t I find Newton's third law? Case
studies of students' use of the web as a science resource. Journal of Science Education and
Technology, 11(2), 155–172. doi:10.1023/A:1014617530297
Marchionini, G. (1997). Information seeking in electronic environments (No. 9). Cambridge
University Press.
Marchionini, G. (2006). From finding to understanding. Communications of the ACM, 49(4), 41–46.
doi:0001-0782/06/0400
Marchionini, G., & White, R. (2007). Find what you need, understand what you find. International
Journal of Human-Computer Interaction, 23(3), 205–237. doi:10.1080/10447310701702352
Mayer, R. E., & Wittrock, M. C. (2006). Problem solving. In P. A. Alexander & P. H. Winne (Eds.),
Handbook of educational psychology. (Second edition, pp. 287–304). New York: Lawrence
Erlbaum Associates.
Moore, P. (1995). Information problem solving: A wider view of library skills. Contemporary
Educational Psychology, 20, 1–31. doi:10.1006/ceps.1995.1001
Novick, L. R., & Bassok, M. (2005). Problem solving. In K. J. Holyoak & R. G. Morrison (Eds.), The
Cambridge Handbook of Thinking and Reasoning (pp. 321–350). New York, NY, USA:
Cambridge University Press.
Ormerod, T.C. (2005). Planning and ill-defined problems. In R. Morris, & G. Ward (Eds.), The
cognitive psychology of planning (pp. 53-70). New York: Psychology press.
Pennanen, M., & Vakkari, P. (2003). Students’ conceptual structure, search process, and outcome
while preparing a research proposal: A longitudinal case study. Journal of the American Society
for Information Science and Technology, 54(1993), 759–770. doi: 10.1002/asi.10273
37
Pomerantz, E. M., Chaiken, S., & Tordesillas, R. S. (1995). Attitude strength and resistance processes.
Journal of Personality and Social Psychology, 69(3), 408–419. doi:10.1037/0022-3514.69.3.408
Pretz, J.E., Naples, A.J., & Sternberg, R.J. (2003). Recognizing, defining, and representing problems.
In J.E. Davidson & R.J. Sternberg (Eds.), The psychology of problem solving (pp. 3-30). New
York: Cambridge University Press.
Ruiz-Primo, M. A., Schultz, S. E., Li, M., & Shavelson, R. J. (2001a). Comparison of the reliability
and validity of scores from two concept-mapping techniques. Journal of Research in Science
Teaching, 38(2), 260–278. doi: 10.1002/1098-2736(200102)38:2<260::AIDTEA1005>3.0.CO;2-F
Ruiz-Primo, M. A., Shavelson, R. J., Li, M., & Schultz, S. E. (2001b). On the validity of cognitive
interpretations of scores from alternative concept-mapping techniques. Educational Assessment,
7(2), 99–141. doi:10.1037/e650052011-001
Saracevic, T., Kantor, P., Chamis, A. Y., & Trivison, D. (1988). A study of information seeking and
retrieving. I. Background and methodology. Journal of the American Society for Information
Science, 39(3), 161–176.
Sawicki, V., Wegener, D. T., Clark, J. K., Fabrigar, L. R., Smith, S. M., & Bengal, S. T. (2011).
Seeking confirmation in times of doubt: Selective exposure and the motivational strength of
weak attitudes. Social Psychological and Personality Science, 2(5), 540–546.
doi:10.1177/1948550611400212
Shavelson, R. J., Ruiz-Primo, M. A., & Wiley, E. W. (2005). Windows into the mind. Higher
Education, 49(4), 413–430. doi:10.1007/s10734-004-9448-9
Shiri, A. A., & Revie, C. (2003). The effects of topic complexity and familiarity on cognitive and
physical moves in a thesaurus-enhanced search environment. Journal of Information Science,
29(6), 517–526. doi:10.1177/0165551503296008
Simon, H. A. (1973). The structure of ill structured problems. Artificial Intelligence, 4, 181–201. doi:
doi:10.1016/0004-3702(73)90011-8
Simon, H. A. (1978). Information-processing theory of human problem solving. In W.K. Estes (Ed.),
Handbook of Learning and Cognitive Processess, Volume 5 (pp, 271–295). Hilssdale, NJ:
Erlbaum.
38
Shuell, T. J. (1990). Teaching and learning as problem solving. Theory into Practice, 29(2), 102–108.
doi:10.1080/00405849009543439
Smith, S. M., Fabrigar, L. R., & Norris, M. E. (2008). Reflecting on six decades of selective exposure
research: progress, challenges, and opportunities. Social and Personality Psychology Compass,
2(1), 464–493. doi:10.1111/j.1751-9004.2007.00060.x
Spink, A., & Cole, C. (2006). Human information behavior: Integrating diverse approaches and
information use. Journal of the American Society for Information Science and Technology, 57(1),
25–35. doi:10.1002/asi
Sutcliffe, A., & Ennis, M. (1998). Towards a cognitive theory of information retrieval. Interacting
with Computers, 10(3), 321–351. doi:10.1016/s0953-5438(98)00013-7
Tabachnick, B.G., & Fidell, L.S. (2013). Using multivariate statistics. Upper Saddle River (NJ):
Pearson.
Taylor, R. S. (1962). The process of asking questions. American Documentation, 13(4), 391–396.
doi:10.1002/asi.5090130405
Todd, R. J. (2003). Adolescents of the information age: Patterns of information seeking and use.
School Libraries Worldwide, 9(2), 27–46.
Todd, R. J. (2006). From information to knowledge: Charting and measuring changes in students’
knowledge of a curriculum topic. Information Research, 11(4), paper 264. Retrieved from
http://informationr.net/ir/11-4/paper264.html
Van Strien, J. L., Brand-Gruwel, S., & Boshuizen, H. (2014). Dealing with conflicting information
from multiple nonlinear texts: Effects of prior attitudes. Computers in Human Behavior, 32, 101111. doi: 10.1016/j.chb.2013.11.021
Vakkari, P. (1997). Information seeking in context: A changing and challenging metatheory. In P.
Vakkari, R. Savolainen, & B. Dervin (Eds.), Information seeking in context (pp. 451-464).
London: Graham Taylor.
Vakkari, P. (1999). Task complexity, problem structure and information actions: Integrating studies on
information seeking and retrieval. Information Processing & Management, 35(6), 819–837.
doi:10.1016/s0306-4573(99)00028-x
39
Vakkari, P. (2000, July). Relevance and contributing information types of searched documents in task
performance. In Proceedings of the 23rd annual international ACM SIGIR conference on
Research and development in information retrieval (pp. 2-9). ACM.
Vakkari, P., & Hakala, N. (2000). Changes in relevance criteria and problem stages in task
performance. Journal of Documentation, 56(5), 540–562. doi:10.1108/eum0000000007127
White, R. (2013, July). Beliefs and biases in web search. In Proceedings of the 36th international
ACM SIGIR conference on Research and development in information retrieval (pp. 3-12). ACM.
Willoughby, T., Anderson, S. A., Wood, E., Mueller, J., & Ross, C. (2009). Fast searching for
information on the Internet to use in a learning context: The impact of domain knowledge.
Computers & Education, 52(3), 640–648. doi:10.1016/j.compedu.2008.11.009
Wilson, T. D. (1999). Models in information behaviour research. The Journal of Documentation,
55(3), 249–270. doi:10.1108/eum0000000007145
Wilson, T. D. (2000). Human information behaviour. Informing Science, 3(2), 49–56.
Wood, W., Rhodes, N., & Biek, M. (2014). Working knowledge and attitude strength: An
information-processing analysis. In R.E. Petty, & J.A. Krosnick (Eds.), Attitude strength:
antecedents and consequences (pp. 283-314). East Sussex: Psychology Press.
Wopereis, I., Brand-Gruwel, S., & Vermetten, Y. (2008). The effect of embedded instruction on
solving information problems. Computers in Human Behavior, 24(3), 738–752.
doi:10.1016/j.chb.2007.01.024

Masterthesis The impact of prior attitudes and prior knowledge on

Transcription

Similar documents

Dr. Stefan Tinuta

How to become an Effective Teacher? S.Fariha Hasnain

Chapter 4 Social and Cultural Environments

POSITIVE ATTITUDE BUILDING T.M.JAYASEKERA B Sc. Eng.,C Eng., MBA,MICE,MSLIM,MSLITD,FIE,FIM,FCIWEM

an information sheet on the school

Attitude Formation and Change

Chapter 6 The importance of market research © Hudson & Hudson.

What’s an Attitude? How do I get one?

Olashore International School

Attitudes Chapter 8 Pages 265-293