Is Rosela Stone the future of language learning?

 Is Rose%a Stone the future of language learning? Gillian Lord University of Florida glord@ufl.edu A(n important) side note… •  The study reported here was conducted with Rose3a Stone’s knowledge and technical support. •  They were not involved in the design, data collec<on or analysis. •  The Rose3a Stone licenses were purchased at regular price. Why this study? •  Omnipresent and powerful marke9ng* makes Rose<a Stone … •  En7cing to the average language learner •  Temp7ng to administrators •  Appealing to educators? •  The best known language program on the market *“RoseBa Stone … spent $98.5 million on adver7sing in 2011, up from $70.5 million in 2010, according to Kantar Media” www.ny7mes.com/2012/06/20/business/media/roseBa-­‐stone-­‐ads-­‐emphasize-­‐fun-­‐not-­‐efficiency.html of rReviews osetta Sotone What ddo o raeviews cademic f say? Stone say? Rosetta •  Lafford, Lafford & Sykes (2010) •  Evaluate if programs provide the tools necessary for effec7ve language learning, based on features that research has shown to be important (interac7on, relevant contextualiza7on of language, etc.) •  “… these products do not incorporate a number of the [necessary] research-­‐based insights (e.g., the need for culturally authen7c, task-­‐
based ac7vi7es) that informed SLA scholars might have given them.” •  Santos (2011) •  Lack of context •  General inability to respond to spontaneous student speech •  What Rose3a Stone calls interac7on is “a rather poor and limited version of what one would encounter in a real-­‐life conversa7on” •  DeWaard (2013) •  “Not a viable replacement of current instruc7on at the postsecondary level” •  Based on personal experience, professional reac7ons What do empirical studies show? •  Vesselinov (2009) – commissioned by RoseBa Stone; RS beginning users demonstrated increased knowledge of the language afer 55 hours of use •  Nielson (2011) – self-­‐study programs in workplace; some success but remarkable aBri7on; lack of community (e.g., Rovai, 2002) •  Stevenson & Liu (2010) – lack of ability to engage learners in true interac7on; users do not take advantage of Web 2.0 tools to network •  This study – Phase 1 of analysis indicated that first-­‐
semester gains in some areas are comparable between RS users and a classroom control. Participants •  Par7cipants were University of Florida students enrolled in Beginning Spanish 1 (avg. age = 20) •  L1 English •  No other L2 proficiency (beyond h.s. requirement) •  Course designed for those with NO prior Spanish instruc<on •  Par7cipants belonged to one of 3 environments: Original popula7on had 20-­‐25 par7cipants in •  Classroom (C): N=4 each of the three groups. •  Rose3a Stone (RS): N=4 •  Classroom+Rose3a Stone (RS+C): N=4 Participants Control (Classroom) group (C) •  In-­‐tact sec7on of Beginning Spanish •  Followed regular syllabus with standard materials •  Carried out standard classroom assessment materials •  Met with researcher 3x during semester Participants RoseBa Stone group (RS) •  Self-­‐selected (required by IRB) •  Not required to aBend any regular class •  Used Rose3a Stone package (“Conversa7onal Spanish”): •  16-­‐week course designed to cover material comparable to a face-­‐to-­‐face beginning class •  6 units of Rose3a Stone® Version 4 TOTALe® Spanish, each has 4 lessons [Level 1, half of Level 2] •  Minimum of 6 RoseBa StudioTM sessions •  Minimum of 8 hours in RoseBa WorldTM •  Monitoring of program access and 7me on task •  Followed predetermined deadlines in progressing through the material •  Met with researcher 3x during semester Participants Classroom + RoseBa Stone group (RS+C) •  In-­‐tact sec7on of Beginning Spanish class •  Same instructor as control group •  Used Rose3a Stone materials as their textbook (including all features described for RS group) •  Met with researcher 3x during semester Data collected •  General oral and wri<en proficiency and skills •  January, March, May •  Par9al CLEP test (30 items) •  May •  Versant Automated proficiency test •  May •  Assessment of aYtudes •  January, May •  Discussion of experiences •  January, March, May Phase 1 Results: CLEP test Average scores (converted to %) 100.00 90.00 p = 0.165 80.00 70.00 60.00 50.00 40.00 38.65 39.17 Classroom RoseBa Stone 47.50 30.00 20.00 10.00 0.00 RS+class Phase 1 Results: Versant test Average scores (converted to %) 100.00 90.00 p = 0.615 80.00 70.00 60.00 50.00 40.00 30.00 27.08 26.25 Classroom RoseBa Stone 20.00 20.00 10.00 0.00 RS + Class PHASE 2 ANALYSIS Final interview -­‐ Classroom INTERVIEWER: Cuéntame, ¿qué te gusta hacer en tu 7empo libre, o los fines de semana? SL: Repitas, please. INTERVIEWER: ¿Qué te gusta hacer? SL: Qué te gusta hacer… INTERVIEWER: ¿Te gusta ir a películas? ¿Te gusta escuchar música? SL: Uh, ¿fin de semana? INTERVIEWER: Sí. SL: Uh, sí. En fin de semana, yo… yo estudio, uh, mucho. INTERVIEWER: ¿Sí? SL: Uh, para mis exámenes. Sí. Yo tengo muchos examines en química orgánica, biología, y laboratorio. Uh, sí. Mucho, uh… no, muy ocupado. So, no películas, no, uh, deportes. INTERVIEWER: ¿Cuál fue la úl7ma película que viste? SL: Cuál te… INTERVIEWER: La úl7ma vez, the last 7me, que viste una película. SL: Phew… Hmm. Let’s see… dos menses. INTERVIEWER: Meses, mhm. SL: Meses. Ago. ¿Cómo se dice “ago”? INTERVIEWER: Hace. Hace dos meses. SL: Hace, sí. INTERVIEWER: Wow. SL: Yo no… yo no veo muchas películas en Gainesville. INTERVIEWER: ¿Qué película fue esa, hace dos meses? ¿Cómo se llamaba? SL: Uh, el pelí—la película… ¿cómo se dice “was”? INTERVIEWER: Era, o fue. SL: Era. La película era… INTERVIEWER: ¿No te acuerdas? SL: Yo no… sí. Final interview – Rosetta Stone INTERVIEWER: Mhm, ¿y qué haces en Gainesville? SH: Um… you’re going to have to forgive me, my mind’s like blown… Um, yo estoy estudiar. INTERVIEWER: ¿Tú estudias? ¿Y qué más? SH: Yo trabajo en un restaurante de Dragonfly. INTERVIEWER: Y, ¿con mucha frecuencia, vas de compras? SH: Yo no entendí, repe7rlo, por favor. INTERVIEWER: ¿Con mucha frecuencia, vas de compras? “Ir de compras” significa go shopping. SH: All right, say that one more 7me, please. INTERVIEWER: ¿Con mucha frecuencia, vas de compras? SH: Uh, no, uh, no voy a… what did you, how did you say “to go shopping”? INTERVIEWER: Ir de compras. SH: No voy de compras. INTERVIEWER: Y, ¿qué vas a hacer este verano? SH: Este verano, yo voy a visitar Brazil. INTERVIEWER: Vas a visitar Brazil, y ¿vas a estudiar en Brazil? SH: No, um, yo voy a trabajar en Brazil. INTERVIEWER: Y, em, ¿qué más vas a hacer en Brazil? ¿Vas a leer, vas a jugar deportes? SH: What am I going to do in Brazil? I thought I just answered that. INTERVIEWER: ¿Solo trabajar? SH: I don’t know, I’m going on a missions trip, I don’t know how to express that in Spanish, but… INTERVIEWER: Pues, buena suerte, muchas gracias. Fluency analysis •  All interviews •  3 groups x 4 par7cipants = 12 par7cipants x 3 interviews = 36 •  Transcribed and analyzed for fluency measures •  “Fluency” • 
• 
• 
• 
• 
Total number of words spoken Number of Spanish words; Number of English words Number of dysfluencies Lexical density (number of unique Spanish words) Number of fillers/non-­‐lexical items General observations GROUP Classroom Average RS+C Average RS Average Total # words # Spanish words # Fillers Repe99ons/ #Unique false starts words 147.78 109.97 16.14 2.81 50.22 90.61 47.53 8.33 2.14 27.22 131.21 88.53 8.72 5.54 45.32 Ratio of L1/L2 words Ra9o of English-­‐to-­‐Spanish words used, by group 1.00 0.83 0.90 0.80 0.68 0.70 0.60 Classroom
Average Control Average 0.50 RS + class Average 0.40 RoseBa Stone Average 0.30 0.26 0.20 0.10 0.00 0 = no English words produced 1 = 1 English word produced for every Spanish word Assistance requests Average # of clarifica9on / assistance requests by group 3.24 3.50 3.00 2.50 2.06 1.75 2.00 1.50 1.03 1.00 0.50 0.61 0.11 0.00 # Clarifica7on requests in Spanish # Clarifica7on requests in English Control Average RS + class Average RoseBa Stone Average Assistance requests Average # of clarifica9on / assistance requests by group 3.24 3.50 3.00 2.50 2.06 1.75 2.00 1.50 1.03 1.00 0.50 0.61 0.11 0.00 # Clarifica7on requests in Spanish # Clarifica7on requests in English Control Average RS + class Average RoseBa Stone Average Other considerations (1) Language learning motivation •  All learners indicated that their primary mo7va7on was to fulfill their language requirement •  One third (4/12) reported that they chose Spanish, specifically, as a language that would be useful in their par7cular careers •  RS volunteers par7cipated out of curiosity or convenience, but shared same degree of language learning mo7va7on. Motivation – Classroom Group Group Classroom Classroom High school language background No Spanish French 3 No Spanish La7n 3 Classroom No Spanish Classroom No Spanish La7n AP Why Spanish? I am required to take a language for my major. Language requirement and for myself since I feel Spanish is a good asset as a physician. Fulfill [college requirement]. I am going to Panama on a service trip. I believe formal classes would help me gain a beBer grasp of the language than picking it up on my own. Motivation – RS+C Group Group Rose<a Stone + Class High school language background No Spanish Rose<a Stone + Class No Spanish Rose<a Stone + Class No Spanish French 2 Rose<a Stone + Class No Spanish French 4 Why Spanish? I am taking Spanish because I feel like it will be beneficial later on in life. As a requirement and to benefit my future jobwise. I need two semesters of a foreign language to graduate. College requirement. Motivation – RS Group Group Rose<a Stone High school language background No Spanish Rose<a Stone No Spanish La7n 3 Rose<a Stone No Spanish French 2 No Spanish ASL 3 Rose<a Stone Why volunteer for RS? Why Spanish? Foreign language Heard a lot of requirement. good things about RoseBa Stone so decided to try it. CLAS requirement Can beBer manage my 7me and schedule and move more at my own pace without dealing with class. Required for Sounded major. beneficial. Spanish is useful in I was going to use my state/needed my own to FL requirement. supplement educa7on anyway. Other considerations (2) Time on task GROUP Comple9on Rate Classroom 96.99% Average Score Total Course Total Class Usage (hours) Time (hours) 90.77% 70.00 39.00 RS+C 93.67% 98.63% 32.81 37.25 RS 97.67% 95.88% 30.69 NA Other considerations (3) Attitude survey •  Few changes (pre-­‐post) in any group •  Significant changes (RS+C, C) on item #3: •  “I am enjoying my Spanish-­‐learning experience this semester.” •  Significant changes (RS) on item #11: •  “Interac<ng via chat or telephone is comparable to interac<ng face-­‐to-­‐face.” •  Changes (RS, RS+C) on item #19: •  “I would prefer to learn a language on my own <me and at my own pace than in a group or classroom seHng. “ Conclusions •  Across environments… •  Comparable outcomes on some measures •  Different development of conversa7onal skills, discourse strategies •  Skep7cism is jus7fied •  More research (always!) needed •  Larger, more varied sample size •  Different proficiency levels •  Broader student group •  More sensi7ve/appropriate tes7ng measures •  Assess cultural awareness and competence Thank you. glord@u[l.edu •  Special thanks to: •  UF College of Liberal Arts and Sciences •  UF Humani7es Scholarship Enhancement fund •  Carlos Enrique Ibarra (sta7s7cs) •  Caroline Reist, Keegan Storrs, Diana Wade (RA) •  Laura Bradley (RoseBa Stone) QR code here Works Cited Bley-­‐Vroman, R. (1988). “The fundamental character of foreign language learning.” In W. Rutherford & M. Sharwood Smith (Eds.), Grammar and second language teaching (pp. 19-­‐30). Rowley, MA: Newbury House. Bley-­‐Vroman, R. (2009). “The evolving context of the Fundamental Difference Hypothesis.” Studies in Second Language Acquisi<on 31(2), 175-­‐198. DeWaard, L. (2013). “Is RoseBa Stone a viable op7on for L2 learning?” Forthcoming in ADFL Bulle<n. Godwin-­‐Jones, R. (2007). “Emerging technologies; Tools and trends in self-­‐paced language instruc7on. Language Learning and Technology,” 11(2), 10-­‐17. Retrieved 26 September 2012 from hBp://llt.msu.edu/vol11num2/emerging/ Godwin-­‐Jones, R. (2009). “Emerging technologies: Speech tools and technologies. Language Learning and Technology,” 13(3), 4-­‐11. Retrieved 26 September 2012 from hBp://llt.msu.edu/vol13num3/emerging.pdf Krashen, S. D. & Terrell, T. D. (1983). The Natural Approach: Language acquisi<on in the classroom. Hayward, CA: Alemany Press. Lafford, B., Lafford, P. & Sykes, J. (2007). “Entre dicho y hecho …: An assessment of the applica7on of research from second language acquisi7on and related fields to the crea7on of Spanish CALL materials for lexical acquisi7on.” CALICO Journal, 24(3), 427-­‐529. Nielson, K. B. (2011). “Self-­‐study with language learning sofware in the workplace.” Language Learning and Technology, 15(3), 110-­‐129. Retrieved 26 September 2012 from hBp://llt.msu.edu/issues/october2011/nielson.pdf Rovai, A. P. (2002). “Development of an instrument to measure classroom community.” The Internet and Higher Educa<on, 5, 197-­‐211. Santos, V. (2011). “Review of Rose3a Stone Portuguese (Brazil) levels 1, 2, & 3.”CALICO Journal, 29(1), 177-­‐194. Stevenson, M. P. & Liu, M. (2010). “Learning a language with web 2.0: Exploring the use of social networking features of foreign language learning websites.” CALICO Journal, 27(2), 233-­‐259 Vesselinov, Roumen. Measuring the Effec<veness of RoseBa Stone. hBp://resources.roseBastone.com/CDN/us/pdfs/Measuring_the_Effec7veness_RS-­‐5.pdf. Rosetta Stone interface Rosetta Stone interface (vocabulary) Rosetta Stone interface (grammar) Rosetta Stone interface (pronunciation) Rosetta Stone interface (World – “play”) Rosetta Stone interface (World – “talk”) Rosetta Stone interface (World – “explore”) Rosetta Stone interface (Studio)