chyung

APPLYING THE TONGRUENCE"PRINCIPLE
OE BLOOM'S TAXONOMY TO DESIGNING
ONLINE INSTRUCTION
Seung-Youn (Yonnie) Chyung and Donald Stepich
Boise Stale Universiiv
Bloom and his colleagues developed the laxonomy of educaliona! objcclivcs in 1936, and 45 years later a
group of psychologists and educatoi^s irvised the one-dimensional taxonomy lo a two-dimensional Cixonomy,
The developers offhi; laxonomy lhcori/.i:d Ihal ihc taxonomy ofcducalional objectives could be u.'^ed wilh any
suhjcci matter and for any levels of learners. However, lillie has been written aboiii how ihe laxonomy can be
lilfeeiivcly used in designing asynchronously-delivcred online inslmetion. We have found Ihat Ihe laxonomy
is an efleclive guideline for designing graduale-lcvel online inslmclion, because il helped us maintain ihe congruence among inslmclional components. In this article, we present a case study that explains how the use ol
the taxonomy of educational objectives was instrumental in the developinenl of graduate-level online insiniction.
INTRODUCTION
Instructional design (ID) is a prescriptive science in that its fundamental purpose is to identily instructional methods that are likely to
lead to desired learning goals in a given situation (Reigeluth. 1983). One principle that
helps instructional designers accomplish this
purpose is the principle of congruence, which
says that, in any situation, learning goals,
instructional strategies, and assessment tnethods should be carefully matched, or congruent
(Dick, Carey, & Carey, 2()OI; Smith & Ragan,
1999). To achieve ihis congruence, many
instructional design models suggest identifying the intended learning outcomes (objectives) and determining the types of learning
represented by those objectives (Dick. Carey,
& Carey, 2(K)I; Morrison, Ross. & Kemp.
2001; Smith & Ragan. 1999). In this way,
objectives serve as a cornerstone for the
instructional design process, helping to determine the instructional strategies and assessinent methods that will be used which, in turn,
helps to ensure the congruenee of the instruction.
• Stuns-Youii (Yoniiiel Chyung, Department of Instruclional & Performance Technology, College ol" Engineering. Boise
State Universily. 1910 University t)r.. Boise. ID 83725-2070. H-mail: [email protected]
TheQuaiierly Review of Dislance Hducation. Volume 4(.l). 2(KI\ pp. 317-130
ISSN 1,528-3518
Copyright © 2003 Inlormalion Age Publishing, Inc.
All rights of reproduction in any form resei-ved.
The Quuiicrly Rcviuw iil' Disiaiicc hducaiian
Various classification schemes have been
developed lo describe dirfeieni types ol Icurninj; outcomes (Gagnc. 1985; Morrison. Ross.
& Kemp, 2(K)I). One of the most well known
is Bloom's Taxonomy of Educational Objoctivcs (Bloom. Engclhart. Furst. Hill. & Krathwohl. 1956: Anderson & Krathwohl. 2(K)I).
When it was deveUtpcd. Blooin and his colleagues iheori/ed that the taxonomy ct>uld be
used wilh any learners and in any content area.
Because of its broad appllcabilily, the taxonomy has been widely used as a guide lor
designing inslruclion in traditional face-toface learning environments (e.g.. Ainsworth,
1994: Calhoun. Davidson. Sinioris. Vincent, &
Gritlith. 2(K)2; Chcethan & Chivers. 2(K)I:
Niehoff & Whitney-Bammerlin. 1995).
However, ihe growth and development of
the World Wide Web as an instructional delivery system has brought a new dimension lo the
design of instruction. In contrast to traditional
face-io-facc inslruelion. asynchront)us online
learning environments: provide a "reduced
cues environment" {Haythornthwaitc. Ka/incr.
Robins. & .Shoemaker, 2000) in which facial
expressit)n, vocal inllection. and olher nonverbal cues are absent: involve mostly asyncbronous communication among participants,
which means that participants do not interact
in real time: and ask students to dcmonstrale
their understanding of course content mostly
via writing. These characteristics have a number of benefils. For example, students can participate at a time and place that is convenient
for them; there is redueed time pressure and
students can think about others" messages, as
well as their own. before participating: and it Is
more dilTieult for one person to dominate the
interaction. At the same time, asynchronous
online learning presents the instructional
designer with a number of ehallengcs. One of
these challenges is developing instructional
activities that will engage and inlorm students,
who have different knowledge and experience
levels, at a distance. While Bloom's taxonomy
has been used as a guide in the design of
facc-tolace instruclion. little has been written
abt)ut its use in the design o\ asynchronous
Vol. 4. No. .1, 2(K)3
online inslruction. This article describes one
possible approach. I'he article contains two
main sections: a brief overview of BlcHim's
taxonomy, and a description of how we have
used Bloom's taxonomy in the design of the
instructional materials for a graduate-level
online course.
BLOOM'S TAXONOMY
The original purpose for developing a taxonomy of edueational objectives was to "provide
for classification ol the goals of our educational systotn" {Blootn. et al., 1956, p. 1). This
grew out of a need, expressed by a group of
eollcge examiners, fur a iheoretical framework
that would allow educators to eommunicatc
and share testing-relaled information with itne
;mother. The resulting discussions, held
between 1949 and 1953. lead to the conclusion
that evaluation criteria and testing materials
should be based on well defined educational
objectives. Bloom and his colleagues deliberately cbose the term "taxonomy" for their classification system to emphasize the hierarchical
nature oi learning. From their research, they
concluded that, "there is an unmistakable trend
pointing toward a hierarchy of classes of
behavitsr which is in aecordancc with our
present tentative classifieation of Ihese behaviors" (Bloom, ct al.. 1956. p. 19). As a result,
they idcntilled six levels of cognitive skill,
from siinple to complex, and argued that
instructional objoetives should be ordered to
tbilow that hierarchical sequence (see Table
1). Their hypothesis was that learning complex
cognitive skills, such as the ability to synthesize interrelated intormution, would be based
on learning simpler cognitive skills, such as
understanding concepts and principles, wbich
would be built based on learning even much
simpler cognitive skills, such as remembering
specific facts.
Ultimately, three taxonomies were developed: a cognitive taxonomy deseribing intclleclual abilities (Bloom, et al.. 1956), an
affective taxonomy describing values and atti-
Applying ihe 'Congruence"Prim iple of Bloom' s Taxonomy lo
Online Instruction
319
TABLF. I
The Original Taxonomy of Ehe Cognitive Domain (Bloom, cl al.. 1956)
Uvel
Evaluation
Synthesis
Analysis
Applieation
Comprehension
Knowledge
Description
Makingjinlgmciits aboui ihe value of ideas, works, solutions, rnclhods, materials, etc.
Putting logoiher of elements and piuts so as to form a whole.
Breakdown of the material into its eonstitiicnt p;«1sand deteetion of the relationships of the pads
and of the way they are organized.
The use of abstraciions in particular and concrete silualioiis.
An understanding of (he literal message contained in a eommunication. Three lypcs of
comprehension behavior are translation, inlerprelalion, and extrapolalion.
The recall ot specifie.^ and universal^, the recall of methmls and processes, or the reeall of a
pattern, structure, or setting,
tudes (Kralhwohl, Bloom, & Masia. 1964);
and a psychomotor taxonomy describing muscular movements and the manipulation of
objects (Simpson, 1972; Harrow. 1972). The
development of three taxonomies was ba.sed
on the premise that there are three qualitalively
different types, or "domains." of learning. In
this view, learning a cognitive skill (for example, reading music) is nol the same as learning
a physical .skill (for example, plucking the
strings of a guitar) or learning an afiective skill
(for example, appreciating music played on a
guitar). Each domain includes a different hierarchy of component skills and requires different instruetionai strategies.
While three taxonomies were developed,
the cognitive taxonomy has received the most
attention, because it is often seen as the major
focus of secondary and post-secondary education. In a recent revision (Anderson & Krathwohl, 2001), the levels have been reordered
slightly. More importantly, a second, knowledge dimension has been added to provide a
more comprehensive system for classifying
edueationat objectives. Definitions for the
knowledge and cognitive process dimensions
are provided in Table 2.
While the taxonomy has been widely used,
it has also been criticized (Bereiter & Seardamalia. 1998; Moore, 1982; Sugrue. 2002), At
the heart of the criticistns is the idea that ihe
taxonomy is analytic, meaning that it attempts
to hreak learning down into component parts
and identify prerequisite relationships among
those components. Critics argue that learning is
synthetic or holistie and thatcognitive skills are
dynamically interdependent rather than hierarchically related. For example, understanding a
concept or principle (the understand level) may
occur in conjunction with learning how to use it
(the apply level) or determining its value in a
given situation (the evaluation level).
These criticisms reflect changes in the way
learning is eonceptualized (Cunningham,
1992: Jonassen. 199!: Jonassen, Peck. & Wilson. 1999) and instructional designers have
begun to incorporate these changes into the
instructional design process (Tennyson. 1995;
Wilson. 1997: Young, 1993), However, we
believe that Bloom's taxonomy still has merit
as a guide for instructional planning for two
specific reasons. First, it reminds us that the
key 10 effeetive instruction is the congruence
or '"degree of correspondence among the
objectives, instruction, and assessment"
(Anderson & Krathwohl, 2001. p. 10), Second,
because it is analytical, it helps remind us that
learning is made up of a complex array of cognitive skills. At the same time, it doesn't prevent us from designing instruction in a more
dynamic way, in which a low-level cognitive
skill can be learned in conjunction witb a
higher-level cognitive skill.
In the next sections, we describe our efforts
to use Bloom's taxonomy as a guide for
designing and developing graduate-level
online instruction and how the taxonomy
helped us maintain the congruence atnong the
components of online instruction.
The Quarterly Review oi" Disiajiec Hducacion
-^
c
I - ^ 15
y
U
0-
c
c
•c
o
CL
«< 'P
,r
S
O
-j
H
3
=
-3
=
< a: C 5 a
<u
P c
? >
a. '5b
-5 E
T3 S
C —
•3 '.J
?
o
/? "71 .tS
::> u
>i
B
o
o
X
rt
E•a
2 I
<:2. 3. °
H
'-
Vol. 4. No .^, 2003
Applying the 'Congruence"Primiple of Bloom' v Tctximoniy to
BACKGROUND OF
THE COURSE
Onlim- Insininiou
(Gagne. 1985). collapsed into ihi: loMowing
five major sections:
The Instruciional & Performance Technology
(IPT) Depanmenl at Boise State Universiiy
offers both online and on-campus courses
leading lo a master's degree. AM online
courses are delivered asynchronousiy via the
Internet using Lotus Notes software. During
ihc fall 2()02 semester, both authors taught a
course titled. "Introduction to IPT." The goal
of the course was to help new students acL|uire
information ahout the historical and theoretical
Ibundations of the IPT field. Enrollment was
13 students in each class.
Within Lotus Notes, the database holding
all of the course documents is referred to as a
"course room."' Unlike other Weh-based
course management systems. Lotus Notes does
not structure a course according lo a pre-dctermined set of placeholders. Instead, instructors
can construct their course rooms in whatever
way they choose. We used a calendar approach
to construct our course rporn. With this
approach, the semester was divided into modules with each module designed to last 7 days,
from Sunday to Saturday. Each mcxJule was
based on Gagne's "nine events of instruction"
A "Read Me First" section. This section
was used to post announcements and to
present the instructional objectives for the
module, as a way of telling students what
they were expected to accomplish during
the week. In addition, students were given
an advanced organizer designed to provide them with a clear direction for the
week and help them construct relationships between information from previous
weeks and the current week's topics.
A section for new content. This section
was used to post multimedia instructional
materials, designed to serve as "lecture
notes." along with multimedia demonstration liles and links to additional learning
resources on the Web.
One or more sections for planned learning
activities. These sections were used to
present the learning activities for the
module. Learning activities included a
variety of think-out-loud exercises, ongoing threaded discussions, small group
projects, or individual assignments. For
example, students might be asked to use
assigned readings to post answers to spe-
«rPTsnfc 4K-'-lse ' AllDoEvmenti
'tf»
»
;oi
, '•.: IPT'>;;f,4'i:i.irii -AM Dru.nnBnr
"
11*1536 •»W/4M
_JlmiiuOa'iV«
_J My FavwM!
t _J BSU Catipus
Main TspK nei(>a»e - flsipanig
t Priwaie Discussion Area tut Gmii
Kennedy
WfDk I Read Me First
WeRk I Tectinnlnqy-HiHd 1 rinll
• 5. Private discussion area
• 1. Wockiy 'Road Me Fir^f area
•2. Weekly mstruciionai material nioa
* I 'i Systemic d Synlemntif Apf)lif:oti(in^»i^3, I/Vcckty learning activity aroa
rntry Kniiwledigu AssessmHMl
Wook t. Wiap Up
4 * * r •*• *'^*^^'' " " P - " P afoa
FtCURK 1
The Bask Weekly Structure of ihe hitrtxluctory IPT Course Room.
The Quarterly Review of Disiance Hducaiioii
322
Vol. 4. No. 3. 2<K)3
TABLE 3
A Summarv of the Inslrudional Modules
Module sections
Irislnitiioiial liveiils
I. Wcokly "Read Me First" area
1. Gain attention
2. Intorni !e;irners olohjeetive
3. Stimulate recall of prior knowledge
2.Weeklv instiiiciional material area
4,Present new content
3.Weekly learning aetivity area
5.Provide learning guidance
6.Blicil perlbrmanee
4.Wcckfy wrap-up area
9.Hnhance retention and transfer
.Private discussion area for each sliident7.Provide feedback
4.
5.
cific questions. Or they might be asked to
commenl on a short posted sccnurio atid
then comment on the ideas others post as
the discussion progresses.
A wrap-up section. This section was used
to present a sutnmary of [he key points
from the tiiodulc.
A private discussion section for each student, aceessihie only to that student and
the instructor. These private areas were
used each week to present students with
individual performance feedback. This
feedback was designed to help students
tiionitor their t)wn learning progress and
to eoaeh them toward higher-level, more
critical thinking about the content of the
course.
Table }< summarizes these five major sections and the events o\ instruction that are represented within each section. While Gagne's
"'nine events ol" instruction" and Bloom's taxonomy were combined to guide the design of
the mcxluies. our foeus in this article is on
Bloom's taxonomy. Readers who would like
more infoniialion about the "nine events of
itistruction" are referred to Gagnc (1985) and
Gagne and Medsker (1996).
By combining this calendar approach with
Gagne's nine events of instruction, new students received a consistent instructional struc-
Types of InsimvlUmal Aiih'iiies
Weekly annimncemcms
Inslriiciional topics and objeciives
Ferlbrmanee criteria
Advanced orgutii/ers
Lecture noles
Multimedia denionstratiotis
Links to additional leaining resources on
the Web
Think-oui-loud aetivities
Threaded, collaboraiive discussions
Individual assignments
A sumimu^y of weekly aetivities
PosI organizers
Weekly pcrfonnanee evaluations
ture each week, which helped them develop
sell-regulative online behaviors. See Figure 1.
USING BLOOM'S TAXONOMY TO
DEVELOP CONGRUENT
INSTRUCTION
Onee we had established the overall structure
of the online environment and identified the
weekly modules, our next steps were to
develop objectives, learning activities and
materials, and evaluation criteria for each
module. This is the point at which the c(ingruence principle becomes particularly important.
Maintaining the congruence among the objectives, learning activities, and evaluation criteria is critical to the effectiveness of the
instruction. Congruent instruction means that
learning activities are designed to support Ihc
objectives and that the evaluation methods are
designed to assess important learning outcomes represented by the objectives. Bloom's
taxt)nomy of educational objectives was
instrumental in making sure there was congruence among the components of each module.
In this case study, the Week 4 tnodule will be
used to illustrate how we designed our online
instruction to maintain the congruence among
instructional eomponents (see Figure 2).
Applying the Vongrufiur'Trinciple
_ J ByAulho.
Ji B^ Dale
_ ] Iniliucloi'sVien
I Mv Favomes
of Bloom' s Taxonomy to Desif-iiinf- Online hisinuiiim
Pnvote Discussion Area lor George
Kennedy
» _ l BSU CawKis
Week 4 Read Me First
' Weekly ovaluatien •
^
Week 1: Behavior Engineering Model
' 4-1 Worthy Pcrtormancc - Tho
rLConomir:Vniue
' 4-? BEM as u Diaqnustic Tinil
323
• Woohiy instructional
objectives
> - Congment
yWeekly learning
activities
• A~3 The DiMusion ol Effect
FrGURE 2
Making the Congruence Among Instructional Components.
Developing Objectives
We used Bioom's original taxotiomy lo
determine the levels of the objeetives for each
module and to design learning aetivities
through which students would accomplish
those ohjectives. [Note: The introductory [PT
course was originally developed by one of the
authors of this article in 1997 using Bloom's
original taxonomy, and has been revised every
semester. After the revised Bloom's taxonomy
was published in 2001. the authors of this article adopted some of the new concepts of the
revised taxonomy in their course design; however, they continued to use Bloom's original
taxonomy as the framework of their course
design because they did not fmd compelling
reasons to modify their course using the
two-dimensional taxonomy.] Depending on
the weekly instruetional topic, the levels of the
objectives ranged from remembering facts and
understanding new concepts to synthesizing
and evaluating information. For example, the
topic of the Week 4 module was Thomas Gilbert's Behavior Engineering Model and we
identified four instructional objectives for that
module. Prior to developing learning aetivities.
we determined the levels in the taxonomy for
each ohjective. Because we acknowledge that
learning sequence and processes are interdependent, we listed the highest level from the
taxonomy, in conjunction with lower, supporting levels. The objectives and levels for this
module are summari/.ed in Table 4.
After we developed the objectives and categorized them using Bloom's taxonomy, we
compared those objeetives against the entry
knowledge levels of our students. At the begin-
TABLK 4
Week 4 Objectives and Levels
Objectives
1. To explain "worthy pcriormance" ac-corcJing lo Gilbcn's
Behavior Engineering Model
2. To iilili/,c (he liehavior engineering model as a diagnostic
loo! 10 analyze ihe sources tor improving hum;ui
eompeienee
3. To apply the coneepl of "ihe greatest leverage" lo behavior
engineering silualions
4. To predici "JilTusion of etfeet" in various behavior
engineering situations;
Levels in Bloom's li
Knowledge and Comprehension
Application and Analysis
Appiiealion and Analysis
Analysis, Synthesis, and t^valuation
The Quarterly Review of Dislanw: hducaiion
334
ning ol" the scmosler, we adminislcrcd an
en try-knowledge assessment to the students in
order to assess their existing knowledge on
specific instructional topics. This was done
with a seit-assessment questionnaire, made up
ol' 40 items (hat asked students to rale how
familiar they were with information that would
be covered in the course. A 3-point scale was
used:
1.
2.
3.
1 have never heard of it.
I have heard of it, hut I can't explain to
other people exactly what it is.
I'm Janiiliai" with it and I can explain to
other people exactly what it is.
Scores were summarized hy the topic. This
information about students* existing knowledge helped us jusiify the levels of the
pre-determined weekly instructional objectives or modify them if necessai^. For example, data obtained from the entry knowledge
assessment showed that students' entry knowledge levels on the three items related to the
Week 4 module (#5 about Thomas Gilhen. #6
ahout the Behavior Engineering Model, and #7
about the concept of worthy performance)
were extremely low (see Figure 3). Since most
of the students had not even heard of the information (81%, 657c. 88% respectively), we
concluded that the prescribed objectives for
the Week 4 module were appropriate for the
targeted learners and decided to use the objectives without modification.
Developing Learning Activities
We posted instructional materials for each
module on Friday and students participated in
the learning activities anytime during the following week. The learning activity areas for
each module provided various activities that
were designed to facilitate learning and help
students achieve the instructional objectives
for (hat module. For example, we included
three learning activities for (he Week 4 module
on Gilbert's Behavior Engineering Model:
activity 4-1 for objective #1, activity 4-2 for
objectives #2 and #3, and activity 4-3 for
objectives #3 and #4 (see Figure 4 ).
A Learning Activity for the Knowledge
and Comprehension Levels
The first learning activity during week 4
(4-1: Worthy Performance—The Economic
Value) was designed to help students understand Gilbert's use of the term, "worthy performance." Students were asked. "What does
Gilbert mean by producing 'worthy performance'?" The following messages illustrate
t%of sludents) 1tH>%
Q5: Thomas Gilbert
06:BEM
U7: Worthy Perf,
b # 1 Never heard of il
CJ #2 Heard ot il
Vol. 4. No, 3. 2(X)3
35%
» #3 C a i exi^an i(
FlGtJRK 3
Results olEhc Enlry Knowledge Assessment on Questions 5, 6. and 7.
Applying the 'Conf-nwiite "Principle of Bloom' ,v Ttixonumy to
Online Insirinlinn
©
IT
(5)N».H«.Iwie (Jl^j
^ By Author
-IBytl-.
Main I npiE - naupDnia - Ftiitpatiis
Kennedy
V _ 1 BSU C
J
Week 4
learning
acttvrtios
Week. 4 Rund Me Firsl
Week 1 Ilithnviiir Enginnnnnc) Mndel
» ^ - l " Worthy''"fl^^'^Q'icH Iho
^r
KrsinomirVnIuH
> 4-2 BEM n i n OioqnDslii: Tiiiil - 4 - '
WeekiWrnp up
knowledgo & comprehension
iipplicatior) & analysis
llor tho obicctivos 12 and >3|
-—b. analysis, syitlhcsts & evaluation
d, (lorlhc otJicclivo M and Mf
FiGUKh 4
Learning Acliviiies Used in ihe Wcuk 4 Module.
how studenLs iJemonsiratcd their knowledge
and comprchensitin. (Please niHe that the
names of the students have been changed in all
of the messages that follow. In addition, the
messages arc written as they were posted hy
the students, with errors in spelling and syntax
left intact.)
Lisa: Worthy performance is the cost effective function of accompMshnicni over behavior, where behavior is the combination
of environment and a person's behavior
reperlory. This is based on :i management
or business concept and stated as his 3rd
Leisurely Theorem or Management.
Translation: Worthy perfitrmance is the
most efficient use of the individual interacting with their environment and its resources lo achieve a desired yoai. Gilbert
extends this to say that wt)rthy perftirmance is also the ultimate responsibility
of management systems, and the absence
of worthy performance and accomplishment is a failure of the management system.
Lisa's message can be divided into two sections, corresponding to the knowledge and
cotiiprehension levels of Bloom's taxonomy.
Lisa first recalls what "worthy performance" is.
using terms that arc taken largely from Gilbert
(the knowledge level). She then elaborates on
that dctlnition and translates it into terms thai
are more her own {the comprehension level).
Eugene: Any accomplishment is the result of
some effort expended by an individual or
team who have some level of competence
in the sequence of events either from prior
experience, training, or even leamini!
while completing the task. Additionally,
the activities leading up to the accomplishment has to take plaee in some locale, presumable well-suited lor the
endeavor. These two things combined
yield the final product. Gilbert recognizes
this fact and makes them the basis of
"wiirthy performance" . . . . For example,
trying to have ."i year olds packing Hour
into bags outside during a high wind is
clearly not a worthy performance. Providing a stable interior environment for a
well-trained, mature staff to pack those
bags using automated equipment is far
more worthy. It is this disparity that Gilbert is describing and providing insights
on improving.
Eugene's message also shows comprehension, but in a slightly different way. Like Lisa.
Eugene begins with an explanation of what
worthy performance is. pointing out that two
things "yield the final product." He then adds a
simple example that highlights important
aspect.s of the concept. For boih students, ihc
The Quarterly Kevicw of Oisiancc Hducalion
learning activity cliciled the knowledge and
comprehension thai was the inlent of the
objective.
Although this first learning activity was
designed to facilitate students" knowledge and
comprehension, we have observed examples
of "slep-up learning" in ihe students'
responses, in which students demonstrated the
intended level ol' learning (for example, comprehension) and then went beyond that to demonstrate an unanticipated higher level of
learning {for example, application, analysis,
synthesis, or evaluation). The following messages illustrate this step-up learning.
Ken: 1 have held training sessions forihemio
go, but thi;y don't attend. Thai is one of
the other reasons I gel Trustraled. So I
guess I need lo use Gilbert's BEM and
find (he root cause of their absence at my
training .sessions. Maybe these people
who don't attend need some son of incentive to attend - like a stipend or some type
of "freebie". I have to look into this a little
deeper.
Sandra: I think ihe BEM model would be a
useful iool in (hat one can visualize and
weigh what will happen if you change one
or more ol the parameters.
Both students go beyond eomprebension. In
bis message. Ken is beginning lo apply Gilbert's "Behavioral Engineering Model"
(BEM) lo a particular situation. Sandra's message redeets her evaluation of the usefulness
olGilherl's model. This phenomenon reflccls
ihe kind of "sell-directed learning" that is
facihlatcd by ibe self-directed, relleclive
nature of ihc asynchronous online learning
cnvironnicnl. Because there are fewer lime
constraints than in (raditional. faec-to-face
instruction, students can think about the messages thai have previously been posted, along
with whal they wanl to add. before posting
their message. This encourages students lo
reflect on ihcir growing knowledge as a part of
the discussion.
Voi, 4, No. 3. 2(H)3
A Learning Activity for the Application
and Analysis Levels
The seeond learning aelivity during Week 4
{4-2: BEM as a Diagnostic Tool) asked students to:
Look around al your workplaec and
ohsent:' a typical pcrlormer and an exemplary performer in ierms of iiccomplishing
aspecitle Usk. Howdifftircnt ;irc ihey.' If i(
is desirable, inierview wiih your typieal and
exemplary performers lo see their own percepiions aboui ihe 6 issues. Report wha(
yoL Ibund and brainsiorm/discuss wha(
needs lo be done in order lo help ihe typieal
performer reduee his/her PIP (Polenlial for
Improving Perlornuuice).
To formulate a response, students would have
to apply Gilbert's mode! to a particular situation and analyze the data they obtained from
their observations and interviews. The following messages illustrate this application and
analysis ot Gilbert's model.
(lary: The situation I have observed includes
two individuals in a sales/account management setting
I was unable to interview the Exemplary Performer. bu( have
interacted enough wiih ihe person to understand their curren( performance. My
discussions were primarily with the Typical Performer.... In a brief synopsis, it
appears that the Typical Performer is not
performing in an exemplary manner because he does not I'eel thai a support and
feedback system is in place. Greater iacilitalion by management of a communication and feedback system may help lo
remedy this situation.
,|uliet:... I think the lack of incentive shown
for both performers by ihe diagnostic table is lypieal '.'>\' military medicine. This
has been discussed frequently among.st
staff. There is no reward for performing
adequately let alone well. The only incentive seems to be internal professional satislaetion . . . . The knowledge difference
between performers points to training as a
solution. A training program thai provided instruction and opportunities to siari
Applying ihe 'Congruence "Principle of Bloom' s Tiixonomy lo Oesi_ii>ii>ii; Oniine Insiniciion
IV's on the very young and old would
likely elose this perfortTiance gap. However, my impression is that if the gap were
calculated, it would not be worth the development costs of the training program.
Gary and Juliet have eonsidered different
situations. But they have followed a similar
process of applying Gilbert's model and separating the results into components of Ihe
model. In Gary's situation, the major problem
appears lo be the amount of feedback ihc
employees reeeive. Juliet describes incentives
and knowledge as the primary problem areas
within her situation.
When we used learning activities designed
to help students aequire analysis skills, we
often asked them to provide feedback to one
another. This method helped students consider
different views of the same situation. The following example of a threaded discussion
shows how this kind of inter-student feedback
promoted collaborative analysis.
Susan: Here is an example from my office . .
. The exemplary performer finds contact
information on the web and contacts the
submitter for additional information. Exemplary performers use feedback to enhance their own performance. The typical
performer places missing information orders at ihe back of the pile and only deals
with them when they absolutely have to
do so. Exemplary performers receive pay
increases to reward them for their work.
Exemplary performers seek ways to speed
up their own processing and prevent incomplete orders from even reaching them.
Typical performers simply process orders. Exemplary performers process orders faster ihan the expected rate. Typieal
perfonners process orders at or below Ihe
expected rate.
Dan: vSusan. Did you do interviews? \i seems
that motivaiion may be an issue here. You
didn't indicate if incentives were available. The task seems simple enough and
assuming they have the tools, motivation
and incentives could be the problem.
What do you think?"
327
Kristi: I agree Dan that incentive and motive
could be the problem. Also, the data factor indicates that a typical performer puts
incomplete orders in the back ot the pile.
This behavior must be eliminated and replaced with a more aggressive approach.
Perhaps an order rejection process that result.s in properly submitted orders. This
wotild force ihe number of iniprojierly
submitted orders to decrease.
Dan: Kristi. an intervention is needed to stop
the workers from putting incomplete orders in the back of the pile. Your process
of "order rejection" would work and the
system (instrument) could force that to
happen. Just don't accept it unlit it's right.
The incentive for having the most complete orders monthly sounds good. But
could other factors cause the same person
to always win this contest? Such as typing
speed. You would need to evaluate Ihe
outcome.
Susan: Kristi, this is a great idea. As Dan has
stated, some intervention is required. I
like the incentive idea and agree Ihat we
would need to evaluate who wins each
month so we can try lo learn from iheir
motivation and their process and implement everywhere. Remember, we want
worthy performance!"
In this excerpl. the students are discussing
how to interpret ihe results Susan obtained in
her example. In her initial analysis, she compares different behaviors exhibited by the
exemplary performer and the typieal performer. Dan and Kristi analyze the situation
and propose a possible cause of the different
behaviors, suggesting ihat incentives may be
an important issue. Susan agrees and begins lo
consider ihe practical implications of ihis recommendation.
A Learning Activity for the Analysis,
Synthesis, and Evaluation Level
The third learning activity during Week 4
(4-3: The Diffusion of Effect) was designed to
help students synthesize information about
The Quarterly Review oI'Dislanee Hducation Vol. 4. No. 3. 2003
Gilbert's Behavior Engineering Model and
assess the value of Gilbert's model. Students
were asked to respond to the fallowing question: "What does Gilbert mean by "the dilTusion of effect'?' Is it important lor pt:rlbrniancc
technologists to be aware of the diffusion of
effect? If so, why? If not. why not? How do
performance technologists apply ihis principle
to their practice?" The following messages
illustrate student synthesis and evaluation.
Colleen: It's very imporlanl for HP Technologists to he aware ol" Ihc diffusion of effect. Throughout the article, Gilbert
explains that the behavior engineering
model doesn't apply scientific methods,
bui engineering methods, which means
every HP Technologist w'\\\ have a different style when using Ihe model. Gilbert
later states, "the behavior engineering
model cannot tell whether or not a motivational effect is present, but lead him to
ask certain qiie.stion,s." These questions
are important in getting to the true performance opportunity and also detennining
the most effective and co.st efficient solution.
David: As far as PTs applying this principle
to their practice. I think it is important to
be aware t)f the phenomena. However, 1
question its usefulness, particularly if the
diffusion results in positive change in the
PIP. It seems to me that Gilbert feci.s that
as long as he is getting results, he doesn't
really eare how it is happening. Any
thoughts?
The evaluation level involves making Judgments about the value or u.scfulncss of ideas,
solutions, inctbods. etc. The centerpiece of
both of these messages is an evaluation of Gilbert's model, though the students come Lo different conclusions about the model. Colleen
notes that ibc questions tbc model asks are critical to a "true" understanding of a particular
situation. In contrast, David questions the
value of tbc model because it doesn't, in his
view, adequately address issues of process.
Evaluating Weekly Online Learning
Outcomes Based on Bloom's Taxonomy
Bloom's taxonomy also guided us in setting
evaluation criteria that would be congruent
with the obicctives for each module (see Table
5).
At the end of each weekly module, we evaluated individual students' pcrlbrtnancc and
provided feedback to tbcm using their private
discussion area. Two types of feedback were
provided to tbc learners: the number of points
thai they earned and qualitative, constructive
feedback about their pcribnnancc during the
week. Below are examples of this qualitative
feedback that we provided to students:
Instructor: Your BEM example |Note: a
document link to the student's original
message is inserted in the feedback statement so that the student will review her
own message and reflect on her performance as comparing it to the instructor's
feedback.] about increasing referrals
shows that you cleariy undersiand the difference between behavior and accomplishment. You did a nice job identifying
several solutions and indicating which
one would have the greatest impact, at the
lowest cost.
In this message, tbc instructor provides confirmation to tbc student thai she has acquired
the basic knowledge and comprehension levels
of the objective. In addition, the instructor
compliments the student for the analytic
thought she shows in her response. This serves
to reinforce the student for her analysis and
increase the confidence she feels in her ability
to master the objectives.
Instructor: Dan, you've demonstrated in
your messages that you have a goixJ understanding aboul Gilbert's ideas [a link to
his document is inserted here]. You also
did a good job analyzing your work situation using the HEM [a link to his document is insertetl)—the daia you collected
is prelly interesting, ll's a great start in
your needs assessment and PI processes. I
Applying ihe 'Omgruence "Principle of Bloom' s Tcvconomy lo Deslf;ttii\i' Online Instrm-tian
329
TABLH 5
Week 4 Ohjeetives, Learning Activity, and Evaluiilion Criteria
Instriiclional Ohjertives
1. To explain "worthy performance"
according lo Gilbert's Hchuvior
Hngineering Model
2. To utilize ihe behavior engineering
mode! as a diagnostic tool to analy/c
Ihc sources tor improving human
compeicnce
3. To apply the concepi of "the greatest
leverage" lo behavior engineering
situations
4. Ti) predict "diffusion of effect" in
various behavior engineering
situations
Learning Aciivity & the Inieiided
Level ol
Learning aciivity # 4-1
Comprehension
Leiiming activity #4-2
Analysis
Learning activity #4-3
B valuation
can see that a successful intervenlion on
this stop will have a domino effect {tipple
effect) on other end results since ihe work
How is such that it has lo be done correclly from the beginning to the end.
In the above message, the instructor points
out to the student that he has demonsiratcd
basic knowledge and analytic skills about the
topic, [n addition, the latter part of the tnessage
is intended to reinforce the student for his synthesis oithe data and to help hitn predict possible effects of his suggested intervention.
With weekly feedback from the instructor.
students were able lo self-monitor their learning progress and build close rapport with the
instructor.
CONCLUSION
Designing instruction can be a challenge to
instructional designers for a number of reasons. The recent proliferation of the Wtsrld
Wide Web as an instructional delivery inediutTi
has brought new challenges to the task of
designing instruction. However, in the design
ol Internet-based online instruction, thectueial
Evitluation iij Studenls' Performance on
Acliieviiif; ihe Hiiihes! Level ofLeuming
in the Learning Aciivity
Accurate description of the meaning of
"wonhy performance" in the context
of performance improvement
Concrtite and coherent application of the
six elements in the behavior
engineering model lo analyze real
performance impmvement situations
Logieat analysis of selecting cosieffeetive solutions lo the performance
problems
Analysis and synthesis of various factors
thai contribute lo systemic effects of a
solution
Jusiifieation of their selection as the
cost-effective solution
Judgment on the usefulness of
behavioral engineering model
instructional design principle still applies;
instructional components such as instructional
objectives, instructional activities, and assessment methods should be carefully matched to
help students achieve the intended learning
outeomes. We have found that Bloom's taxonomy can be a helpful guide in achieving this
congruenee in online instruetion.
REFERENCES
Ainsworlh. P. (1994). Restructuring the introductory aceounling courses: The Kansas Slate University experience, .foumul of Aceounling
Education. /2(4). 305-32.1
Anderson. L.. & Krathwohl. D. (Eds.). (2(K)I). A
laxonoiny for learning, teaching, and as.'ifs.sing:
A revision of liloom 's taxonomy of cdttcalional
ohjvciives. New York: Longman.
Berciter. C . & Scardamalia. M. (1998). Beyond
Bloom's taxonomy: Rethinking knowledge for
the knowledge age. In A. Hargreaves. A, Lieberman. M. Fullcn, & D. Hopkins (Eds.). Iniernolioiml hamilwok of educational change. Boston:
Kluwer Acadetiiie.
Bloom. B.. Engelhart. M.. Furst, E., Hill. W . &
Kralhwohl. D. (19,56). Taxonomy of editcaiional
objeelives: The classification of educational
TheQuancrly Kcvicw ofDislancc Education
goals tHandbook I: Cognitive domain). New
York; McKay.
Calhoun. J.. Davidson. P., .Sinioris. M.. Vincenl. E..
& GrilTiih, J. (2002). Toward an understanding
of competency identificalion and assessment in
hcallh care managemenl. Qualiry Mamigemeni
in Heallh Can: //(D. 14-38.
Cheeihan. G.. & Chivcrs. G, (2001), How protessionals learn: An invesiigaiion ofinformal learning amongst people working in professions.
Journal of European Indcislrial Training. 25{5).
248-292.
Cunningham, D. J, (1992). Beyond educational psychology: Steps toward an educational semiotic.
Educalional Psychology Review. 4{1). [65-194.
Dick. W., Carey. L.. & Carey. J. (). (2001). The syslemaiic design of inslniclion (5th ed.). New
York: Longman.
Gagne. R. (1985). The conditions of learning (4th
Ed.). New York: Holt. Rinehart and Winston.
Gagne- R.. & Medsker. K. (1996). The conditions of
(earning: Training applications. Forl Worth.
TX: Harcourt Brace College Publishers.
Harrow. A. (1972). A taxonomy of the psxchomoior
domain: A guide for developing behavioral
objectives. New York: McKay.
Haythomthwaile, C. Kazmer. M. M.. Robins, J.. &
Shoemaker, S. {2000). Community development
among distance learners: Temp(ual and technological dimensions. Journal of Computer-Mediated Communication. 6( I). Retrieved October 2,
2(X)0, from the World Wide Web: http://
www,ascusc,org/icmc/vol6/issuel/haythomthwaiie.html
Jonassen. D. H. (1991). Objectivism versus constructivism: Do we need a new philosophical
paradigm? Educational Technology Research
and Development. 39{}). 5-14.
Jonassen. D. H.. Peck. K. L.. & Wilson. B. G.
(1999). Learning with teclmologv: A constructivist perspective. Upper Saddle River. NJ: Merrill.
Vol. 4, N»i, 3. 2003
Krathwohl. D.. Bloom, B.. & Masia. B. (1964). Taxonomy of educational objectives: The cla.ssificalion of educational goals (Handbook If:
Ajjective domain). New York: David McKay.
Moore. D. (1982). Reconsidering Bloom's taxonomy of educational objectives, cognitive
domain. Educational Theory, 32( 1). 29-34.
Morrison. G., Ross. S.. & Kemp. J. (2001). Designing effective instruction (3rd Ed.). New York:
Wiley.
Nichoff. B.. & Whitney-Bammerlin. D. (1995).
Don't let your training process derail your journey to total qualiiy management. S.A.M
Advanced Management Journal. 6()( I). 39-45.
Reigeluth. C. M. (1983). Instructional design: What
is it and why is it? In C. M. Reigeluth (Ed.).
Instructional-design theories and models: An
overview of their current status. Hiilsdale. NJ:
ErlbauniSimpson. E. (1972). The classification of educational objectives in the psychomotor domain:
The psychomotor domain (Vol. 3). Washington,
DC: Gryphon House.
Smith. P. L.. & Ragan. T. J. (1999), Instructional
design (2nd cd.). Upper Saddle River, NJ: Merrill.
Sugrue, B. (2002). Problems with Bloom's taxonomy. Retrieved December 14. 2002. from the
World Wide Web: http://www,perfonTiancexpress.org/()212/
Tennyson. R. D, (1995). The isnpact of the cognitive
science movement on instmctional design fundamentals. In B. B, Seels (Ed.I. Instructional
design jundametitals: A reconsideration. Englewood Cliffs. NJ: Educational Technology Publicalit)ns.
Wilson, B. G. (1997), Rejlections on constructivism
and instructional design. Retrieved March 4,
2003, from the World Wide Web: http://carbon.cudenver.edu/~bwiison/construct.html
Young, M. (1993), Instructional design for situated
learning. Educational TechnoU>gy Research and
Development. 410). 43-58,