The Way Entrepreneurs Learn: Migration of a 30

The Way Entrepreneurs Learn: Migration of a 30-Second Elevator Pitch Exercise from
Instructor-Led Delivery to Blended Learning Technology
Dan Suhr, (inXsol, LLC), Sharon Ballard (EnableVentures, Inc.), Ricardo Gomez (NCIIA),
Henry Ryng (inXsol, LLC), Angela Shartrand (NCIIA)
Abstract
inXsol was awarded an NSF SBIR Grant (0945987)1 to research migration of established
entrepreneur education (EE) tools, Supercoach® Entrepreneurial Training2 (SET), from an
instructor-led to a blended learning environment. One part of the project was converting a 30Second Elevator Pitch exercise to an online module that familiarized students with sample
pitches, clarified the key components of an elevator pitch and allowed the students to video and
share their own pitch through the module. In addition, an innovative assessment protocol was
designed so that the student 30-Second Elevator Pitches were electronically reviewed by a panel
of industry and EE experts using a rubric specifically designed for this project. This paper will
identifies why this exercise was selected, the key learning objects, user experience for the
module, the assessment strategies embraced and the commercialization lessons learned.
Introduction
The objective for the National Science Foundation (NSF) Small Business Innovation Research
(SBIR) Phase I entrepreneur education (EE) project was to develop scaleable, compelling,
interdisciplinary curricula and delivery methods that provide broader access to curriculum and
reduce the entrepreneur’s time from "innovative idea to market implementation." To achieve
those goals inXsol, with Henry Ryng serving at the project’s Principle Investigator, proposed to
determine the efficacy of web-based simulations and exercises with the Supercoach®
Entrepreneurial Training (SET) curriculum that has historically been delivered in an instructorled format. The proposed research addressed the question:
Can migrating a portion of the SET curriculum from instructor-led delivery to web-based
delivery shorten the duration of courses and workshops and lead to significantly
improved EE throughput while retaining (or increasing) the participant’s performance
levels?
Body
The development of the 30-Second Elevator Pitch was one component of the SBIR project. The
product resulting from the Phase I effort was a web-based treatment of two SET exercises (30Second Elevator Pitch and the One-Page Strategic Business Plan) traditionally presented by a
qualified instructor in classes and workshops. We called the resulting product SET Academy
1
NSF GRANT SUPPORT AND DISCLAIMER -­‐ The project described was supported by Grant Number 0945987 from the National Science Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. 2
Supercoach® Entrepreneurial Training was developed by Sharon Ballard of EnableVentures, Inc. and licensed to inXsol, LLC for NSF SBIR Grant (0945987) development. (www.setacademy.com). The 30-Second Elevator Pitch exercise was developed with Level 2
interaction characteristics while the One-Page Strategic Business Plan EE exercise was
developed with Level 1 interaction characteristics. Levels of Interactivity are general classes of
richness, sophistication and realism of interactivity the participant experiences in eLearning and
simulations. Level 1 has very limited interactions while Level 4 would be a free play simulation
where the participant is not limited by predetermined paths and there are high levels of
interaction. After the modules were developed the assessment protocol was designed compare a
control cohort (workshop 1) with a test cohort (workshop 2). The test cohort was asked to
complete the two web-based exercises before attending a SET workshop. The control cohort was
not exposed to these web-based exercises before attending the SET workshop. Rather, the
control cohort was given the same pre-workshop materials traditionally provided. The two 4-day
workshops were conducted about a month apart. Both workshops used the same instructor,
facilities, guest speakers and “dream panel” of live reviewers.
The educational model depicted below is a progression structure where students are exposed to
the EE material and exercises prior to the class or workshop, proceed to live instructor-led
workshops and classes, move on to a series of one-on-one coaching sessions and then are finally
in their own. For this project we envisioned EE exercises and simulations leading the student
through the process with a collaborative thread and cumulative exercises that would support the
student through the progression. inXsol’s experience is that this supported structure helps
normalize student knowledge and allows for the instructor to address the most complex
information instead of using valuable time to review foundational information.
Figure 1 – Proposed Structure
inXsol’s experience supports this approach. During the last 15 years inXsol has developed
simulations with Level 1 to 4 interactions mimicking the performance and operation of aviation
and industrial machinery. Simulations are useful when training on “live” equipment is too
dangerous, expensive or scarce to use for operator training. These simulations were structured to
embrace experiential learning concepts advanced by Kolb and Fry (1975). The experiential
learning model they described included four elements: concrete experience, observation and
reflection, the formation of abstract concepts and testing in new situations (Kolb and Fry 1975).
This model, although somewhat simplistic, describes the rationale behind inXsol’s hard skill
simulators developed for our customers. inXsol’s experience with complex simulations is easily
transferrable to the experiential learning required for EE.
inXsol uses scaffolding strategies (Bransford et al. 2000) to support a student while learning
separate components of new materials in our simulations and exercises. The characteristics of
scaffolding that advanced this project included:
•
•
•
Model progression. Learning environments using model progression are a major design
framework for both classroom and eLearning applications. A learning objective is broken
into chunks and presented with more and more content or complexity included as student
performance and mastery progresses.
Use of domain specific content. Access to domain-specific information is provided
through questions and answers, reference materials, web-links, and inspirational
entrepreneurial stories. Adult professionals gather information on their own, and the
mastery of the literature of a profession is a key skill for long-term success.
Engaging assignments. The learning environment provided engaging assignments using
authentic scenarios that provided the situations, questions and exercise situations used to
motivate the user in a variety of situations. These assignments are embedded in the
learning environments with model progression as noted above.
Finally, inXsol’s experience shows that significant efficiencies can be gained for both the
facilitator and the student by moving proven instructor-led SET workshop to an interactive online infrastructure. However, it is not just inXsol that have found these efficiencies in eLearning
and simulations. Setaro (2001) summarized the studies that examined the training compression
experienced by companies that adopted web-based delivery of training.
Fletcher (1990) after carefully reviewing over forty independent studies found that
Technology Based Training (TBT) yielded time savings of 35-45% over traditional
classroom instruction while obtaining equivalent or better gains in learning retention
and transfer. Adams (1992), Cantwell (1993) Bradley (1994) and Hofstetter (1994) later
confirmed these results across several years and hundreds of studies. Likewise Hall’s
(1997) in-depth review of over 130 case studies found that computer-based training
required significantly less time than instructor-led training. The amount of reduction in
time ranged from 20-80% with 40-60% being the most common range for timesaving.
None of the studies and meta-studies reported a decrease in training effectiveness and
most reported a substantial increases in training effectiveness (both learning retention
and transfer). More recently, Hemphill (1997) found that while CBT saves time, it does
not negatively impact effectiveness of learning. (Setaro 2001)
These characteristics and inXsol’s experience in developing training solutions, coupled with the
SET experience with entrepreneurial training and development, provided compelling reasons for
the NSF to award the proposal.
The purpose of our research was to answer the following sets of research questions that
supported our primary question. First, we posed questions in context of the test cohort as
compared to the control cohort. Those questions included:
• How can web-based exercises/simulations be incorporated in the experiential learning of
SET?
• Was a participant’s performance in the workshop enhanced by the web-based exercises
completed prior to their participating in the instructor-led course or workshop?
• Did the individuals in the test cohort begin the workshop with consistent understanding of
the SET process and expectations of the workshop as measured through comparison of the
workshop pretests? The theory was that the test cohorts’ workshop pretest scores would be
clustered with less variance in performance than the control cohorts.
• Would a test cohort perform measurably better than the control cohort?
• What were the differences in course evaluations supplied by both the test cohort and
control cohort?
Our second set of research questions addressed the test cohort’s experience with the Level 1
exercise interaction characteristics compared to the exercise developed to Level 2 interaction
standards:
• How much time did each test subject spend completing or interacting with each exercise?
• What was the pre-test and post-test performance delta for the differing levels of
interaction?
• Did the interactivity level of the web-based exercises result in different observable inworkshop performance?
• Using test cohort survey results, were participants more likely to complete a web-based
exercise if there is Level 2 interactions present compared to Level 1 interactions?
In summary, we expected to determine how web-based exercises and simulations could enhance
the learning process in entrepreneurs. We also established how these tools and technologies
could supplement the experiential learning environment in traditional SET training. Our focus
was on developing qualitative metrics for demonstrating the efficacy of our tools.
Module Development
Two exercises were selected to convert to web-based experiential learning modules: 30-Second
Elevator Pitch and the One-Page Strategic Business Plan. The criteria used to select these
exercises out of dozens of potential exercises included: sequence in the workshop (early
exercises in the workshop would better isolate the impact of the online modules), interactivity
(interactive exercises provide an opportunity to better demonstrate the effectiveness of the
experiential simulation approach), and “hooks” (exercises that had components that would be
used in later workshop exercises would provide a foundation for additional future exercises and
also allow for the participant’s constituency to be evaluated). Rubrics were designed to evaluate
each exercise. While is it important to describe the criteria used to select the modules for the
NSF SBIR project, the remainder of the paper will focus on one exercise – the 30-Second
Elevator Pitch.
30-Second Elevator Pitch Module. This exercise was chosen because it is the first step in the
Supercoach® process as illustrated below and compared to the standard business planning
process. The Supercoach® process relies on oral communication skills and builds towards writing
a business plan when one is needed for financing or strategic alliances; whereas, the traditional
process the written business plan is the first task undertaken - arguably the most difficult task of
the process. The Supercoach® process allows an idea to be focused through short exercises.
When a student completed the 30-Second Elevator Pitch module, they know the components of
an elevator pitch, will have evaluated at least nine pitches, and, finally, recorded their own pitch.
The 30-Second Elevator Pitch was divided into three rounds of interactions. The first round the
student watches nine videos of actual pitches of startup companies. From those nine pitches they
would select five to promote to the next round where they will better understand evaluation
criteria. Below are screen captures from round one and round two.
Figure 2 – 30-Second Elevator Pitch Example of Module
Round two includes more detailed evaluation of the five pitches that were promoted using the
following criteria.
Criteria
Market Opportunity? Was the value proposition of their opportunity clear? What is the problem
they are solving? Was the size of their opportunity clear and quantified? Was the target customer
of their opportunity identified?
Why you? Was their competitive advantage clear? Did they express why their management team
is qualified for their opportunity?
Impact? Did they explicitly request some form of support (time, expertise or money)? Were they
memorable? Could you repeat the main ideas of their venture? Was it 30-seconds or less?
Good Story? Did their pitch arouse your attention and hold it? Would you share their story with
others?
Compelling? Was their argument persuasive and forceful? Did you learn something?
After the student scores each video, the coach’s perspective becomes available. That perspective
is an evaluation of the video using the coaching philosophy of asking questions and not lecturing
or advising. This process help the student understand and evaluate the criteria.
Finally, it is the student’s turn to create and practice their pitch in the third round of the module.
Five different scenarios were created to put the student in a situation where they may be called
upon to deliver their elevator pitch. The system check the user’s computer for a microphone and
web camera and then provides a way for the student to record their pitch and play the pitch back
at a later point. It was planned that in subsequent versions of the SET Academy the user will be
able to allow their team members, coaches or instructors to view and provide feedback on the
recording. The following screenshot illustrates the interface for round three of the 30-Second
Elevator Pitch Module. The coach’s perspective in this round provides a recap of the evaluation
criteria for a 30-Secnd Elevator Pitch. It was anticipated that this module would take 30-45
minutes for a student to complete.
Figure 3 – 30-Second Elevator Pitch – “It’s Your Turn”
Evaluation Design for the 30-SecondElevator Pitch Module
The evaluation utilized a non-equivalent post-test design (quasi-experimental). Recruitment and
assignment of workshop attendees was managed by inXsol with the help of Arizona State
University.
Workshop Participants
Attendees were randomly selected to participate in the workshops through recruitment efforts of
Arizona State University (ASU) SkySong. ASU advertised the workshop opportunity on their
website and through their network of advisors that serve the greater Phoenix entrepreneurial
community. The only limitation inXsol placed on recruiting was that no more than half of the
workshop participants would be student teams. Attendees were assigned on a rolling basis to
workshop 1 (comparison group), unless that did not work with the attendees’ schedule, in which
case they were assigned to workshop 2 (treatment group). The recruitment for workshop 2 was
undertaken to reflect the demographic mix attained in the first workshop in terms of student vs.
community participants.
The demographics of the two cohorts were similar except for two characteristics: education and
work experience. In the case of education, in workshop 1 (comparison) 56 percent of the group
had completed some graduate work or had an advanced degree. For workshop 2, 83 percent of
the participants had undertaken graduate work or completed an advanced degree. The other
characteristic that stood out was Entrepreneurship Experience. Forty-four percent of workshop 1
participants reported holding leadership roles in a start-up, while workshop 2 only reported 33
percent of the participants holding a leadership role in a start-up. Finally, 31 percent of the
workshop 1 participants reported exiting a start-up enterprise through sale or merger compared to
17 percent for workshop 2 participants. Generally speaking, it appears workshop 1 had more
start-up experience and workshop 2 reported greater education levels.
Scoring of Student Samples: Online and Live Judging
Two simple instruments were used to measure the quality of work samples. One five-item
checklist was created to score the 30-Second Elevator Pitch. Twenty-seven judges were recruited
to score the quality of work samples via a web-based tool. Three of the judges scored both types
of work samples (elevator pitch and five-minute presentation), and the other 24 were randomly
assigned to score either the pitch or presentation exercise. Each judge was given a random
selection of work samples from each exercise that they were asked to score using the above
checklist or rubric as appropriate. Two types of work samples were judged: a) a video of 30Second Elevator Pitch and b) a five-minute video of a final presentation). To minimize
procedural confounds, work samples were created and presented in an identical manner (e.g.,
video clips used the same backdrop, audio, and video settings). Judges logged into the website
and were given instructions to score the work samples. Judges were instructed to view the
elevator pitch or final presentation video, and then were presented with a scoring sheet to enter
their scores for each. When one work sample was scored, the judge had the option to continue
scoring another work sample, or to quit the session. The number of work samples judges chose to
score varied. Judges scored between three and 24 of the 30-Second Elevator Pitches) and
between three and 19 of the five-minute presentations. [Below are screen captures of the online
evaluation tool created for this NSF Phase I project.]
Figure 4 – 30-Second Elevator Pitch – Evaluator Interface
Major Findings Resulting from these Activities
The panel of 15 judges independently assessed the 30-second pitches, which were delivered by
24 participants. In total, judges submitted 358 scores (224 for the comparison and 134 for the
treatment) using the five-item checklist (see Figure 2). Each item had three categories of
performance “Yes”(11-5), “No” (0-5), and “Needs Work” (6-10).
Individual scores were calculated for each of the items, and the final score was made up of the
scores in the five items. The final scores are the basic unit of analysis (n=358). The result of the
independent t-test shows no significant statistical difference between the treatment (M=10.54,
SD=2.53) and comparison group (M=10.16, SD=2.37); t(356)=---­‐1.44, p=0.214 (95%CI ---­‐0.90
to 0.14).
Summary of Research
In summary, no differences were found between judges’ ratings on the elevator pitches delivered
by workshop participants in each group. The treatment participants performed as well as the
comparison group verifying the research hypotheses. From a practical standpoint that means that
over 2 hours of the instructor-led course was replaced by 30-45 minute eLearning module that
the students completed before the workshop began. This freed the instructor to expose the
students to more complex subjects or for more one-on-one instruction. As a result, students that
took the online modules prior to attending the workshop exhibited higher performance in the
subsequent exercises including the 5-Minute final presentation.
There may be several explanations for this finding. In the elevator pitch, judges may focus on
public speaking abilities and the quality of oral communication skills. These skills may not be
readily improved in a short time period, especially when there is a 30 second time limit. Also, we
did not account for differences in participants’ backgrounds that may have influenced their oral
communications skills prior to the workshops. For example, we did not ask participants about
how much prior experience they had with public speaking, nor did we ask about their English
language abilities or experience developing elevator pitches in the past. These prior experiences
are likely to be a strong influence on performance in the exercise and should be controlled for in
future studies.
Other suggestions for future work included ensuring that participants would be provided with the
rubric and checklist beforehand so they can prepare their presentations based on the criteria upon
which they are going to be evaluated later on. It would be worthwhile also to ensure that when
online participants do the online exercises, they have the opportunity to apply the same rubric to
the presentations or pitches they are rating.
Conclusion
Entrepreneurial Education has been identified as a key building block in the entrepreneurial
process for scientists and engineers. inXsol has shown that technology can be used to develop
scaleable, compelling, interdisciplinary curricula and delivery methods providing broader access
to the target audience and compression of the entrepreneurial process duration from "innovative
idea to market implementation." By making the process more efficient, the community gains
through the jobs created by these new institutions. If the best ideas can be advanced more rapidly
through the process, then job creation and economic growth will accelerate. Better access to
innovative training will allow this to happen.
There are three components of the EE process that can be enhanced providing better outcomes
for the country’s entrepreneurs: curriculum, delivery, and collaboration. In the past the
Supercoach® curriculum and its innovative approach to support the scientist and engineer
entrepreneur has been proven a more effective training method than traditional approaches.
inXsol’s Phase I research demonstrated that engaging, interactive web-based modules can
compress the training time or delivery for the student and increase instructors’ capacity. inXsol
believes that future research will show that the efficiency and quality can be extracted from the
collaboration process – medium-term support for the students or venture teams through ongoing
interaction with coaches. Unfortunately, NSF did not fund the collaboration approach outlined
and submitted by inXsol for the Phase II proposal.
The most important conclusion for the Phase I research is the interest the marketplace has
exhibited for a technology-assisted approach to entrepreneurial education – especially more
highly interactive approaches. One example of this is the enthusiasm in which many people were
enthusiastically participating in this project – the two workshops (18 participants), live
evaluators (4), electronic evaluators (27) and a four-hour focus group with the project board of
advisors. However, the interest never converted into commercial subscriptions to the modules.
Of the four commercial enterprises that expressed interest in subscribing to these prototype
modules during the Phase I project only one organization actually paid the modest fee to expose
their students to the modules.
How could a product with so much promise fail in the EE marketplace? There seemed to be three
primary reasons the EE organizations hesitated subscribing to the modules. First, there was an
element of the “not invented here” mentality. Potential customers liked what they saw but
wanted to tweak the modules one way or another without even a modest budget for
redevelopment. Speaking of money, the institutions most interested in the modules were hesitant
to pay a subscription even for the modest pricing of the modules. Those institutions did not want
to pay, during a transition period, for the traditional paper-based SET curriculum and additional
electronic modules to support the traditional SET curriculum. Which ties into the final barrier to
adoption – availability of all the modules. In all instances, the institutions wanted the total
curriculum available online as a condition of subscription purchases. Therefore, once the NSF
Phase II funding was not available to complete the curriculum, the institutions were unwilling to
fund the remaining development.
Highly interactive eLearning modules and simulations are the way of the future. The industry is
moving that direction but there is still the necessity to demonstrate an ROI for the customer and
also overcome the threat EE instructors feel about a portion of their instruction replaced by
eLearning modules. The 30-Second Elevator Pitch module was innovative and effective when it
was created and the modules are still recognized for innovation and instructional design
creativity.
References
Adams, G. L. 1992. "Why interactive?" Multimedia & Videodisc Monitor. (March).
Bradley Associates. 1994. Multimedia made easy: Guide to developing interactive multimedia
for training. Palo Alto, CA: Bradley Associates.
Bransford, J. D., Brown, A. L., & Cocking, R. R. 2000. How People Learn: Brain, Mind,
Experience, and School. Expanded Edition.
Cantwell, S. 1993. "Multimedia transforms Union Pacific's training strategy." Tech Trends.
(November/December).
Fletcher, J.D. 1990. Effectiveness and cost of interactive videodisc instruction in defense training
and education. Washington DC: Institute for Defense Analyses. (July).
Hall, B. 1995. "Multimedia training's return on investment." Workforce Training News.
(July/August).
Hemphill, H. H. 1997. The impact of training on job performance. Scottsdale, AZ: Thompson
NETg.
Hofstetter, F. T. 1994. "Is multimedia the next literacy?" Educator's Tech Exchange. Inside
Technology Training. Louisville, KY: Ziff-Davis. (Winter).
Kolb. D. A. and Fry, R. 1975. 'Toward an applied theory of experiential learning”, in C. Cooper
(ed.) Theories of Group Process, London: John Wiley.
Kuratko, Donald F. 2005. “The Emergence of Entrepreneurship Education: Development,
Trends, and Challenges” Entrepreneurship Theory & Practice, Vol 29 #5: 577-598.
Katz, J.A. 2003. “The Chronology and Intellectual Trajectory of American Entrepreneurship
Education.” Journal of Business Venturing, 18, 283-300
Kuratko, Donald. 2003. “Entrepreneurship Education: Emerging Trends and Challenges for the
21st Century.” Coleman Foundation White Paper Series for the U.S. Association of Small
Business & Entrepreneurship in cooperation with The Entrepreneurship Program College
of Business Ball State University.
Mache, D. and D. Markley. 2004. “A County Guide: Energizing an Entrepreneurial Economy,”
Lincoln, NE, Center for Rural Entrepreneurship.
Raffo, C., A. Lovatt, et al. 2000. "Teaching and learning entrepreneurship for micro and small
businesses in the cultural industries sector," Education and Training (6): 356 – 365]
Setaro, John. 2001. Many Happy Returns: Calculating E-Learning ROI. [online]:
http://www.learningcircuits.org/2001/jun2001/Elearn.htm [referred: 27.09.2005]
Stacy, W., Walwanis Jacobs, M., Colanna-Romano, J. 2007. “Using pedagogical information to
provide more effective scenarios.” Proceedings of the Interservice/Industry Training,
Simulation & education Conference (I/ITSEC). Orlando, FL.
Yaw, D. C. (2005). An evaluation of e-learning in industry at Level Three based upon the
Kirkpatrick model. (Doctoral dissertation, Indiana State University, 2005). Dissertation
Abstract International, A66/12, 136.