Theory – practice gap in CAQDAS users` experience in Poland
Transcription
Theory – practice gap in CAQDAS users` experience in Poland
Theory – practice gap in CAQDAS users’ experience in Poland KAROL HARATYK, UNIVERSITY OF WARSAW [email protected] ANNA KORDASIEWICZ, UNIVERSITY OF WARSAW, GOLDSMITHS COLLEGE (VISITING SCHOLAR) [email protected] CAQDAS 2014 CONFERENCE: PAST, PRESENT & FUTURE: 25 YEARS OF CAQDAS: THE ROLE OF METHODS TEACHING IN LEARNING TO USE QUALITATIVE SOFTWARE HORSLEY, 1-3 MAY 2014 The CAQDA vicious triangle in Poland… 2 CAQDA's image in the literature users opinions on CAQDA CAQDA practice THEORY – PRACTICE GAP Karol Haratyk , Anna Kordasiewicz Presentation plan 3 Description of a research project on qualitative analysis in Poland 2. The CAQDA viscious triangle: 1. 1. 2. 3. CAQDAS’ image – state-of-the-art CAQDAS – users’ opinions CAQDAS – users’ practice 3. Theory-practice gap 1. Grounded theory applied? – An illustration 4. Conclusions Karol Haratyk , Anna Kordasiewicz 1. Description of the research idea and project 4 a. IDEA OF THE PROJECT b. DESCRIPTION OF THE RESEARCH PROJECT Karol Haratyk , Anna Kordasiewicz Point of departure/some assumptions 5 The critics of the qualitative research Lack of clarity of qualitative analysis procedures (i.a. Silverman 2004, 2007, Lofland et al. 2010). Explicit accounts of analytical proceeding is a rarity „anegdotalism” (Silverman 2004) or qualitative data serving only illustrative purposes. Few systematic methodological studies on the practice and theoretical background of qualitative analysis Karol Haratyk , Anna Kordasiewicz The theory – technique connection 6 Some but still few systematic methodological studies on the practice and theoretical background of qualitative analysis Examples pertain mostly to the computer assisted analysis, like Franzosi et al. (2012), MacMillan (2005), Koenig (no data), Saillard (2011), Schönfelder (2011), Bielinski, Iwińska, Kordasiewicz (2007), KWALON experiment (FQS 2011), Jones, Diment (2010), Theoretical background of analysis outside the context of CAQDA has been presented in Frost et al. (2010), Stubbe et al. (2003). BUT: „Technique is theory” - Mauthner&Doucet (2003), Ruppert et al. (2013) Karol Haratyk , Anna Kordasiewicz Stubbe et al. 2003 Frost et al. 2010 7 Lonkila 1995, MacMillan 2005 Bringer et al. 2006 Jones & Diment 2010 Some papers based on KWALON experiment, FQS 2011 Franzosi et al. 2012 Niedbalski&Slezak 2012 Theory Technique Bielinski et al. 2007 Results KWALON experiment, FQS 2011 Karol Haratyk , Anna Kordasiewicz The idea of the metamethodological project 8 The comprehension of the process of qualitative analysis is vital for the enhancement of the quality of qualitative research We may find inspiration in the field of quantitative methodological studies, e.g. experiments on the question phrasing in survey research It is necessary to investigate how we conduct our analysis and what its connection with the theoretical background is We focus on computer assisted QDA but we include and compare paper-based with the CAQDA Karol Haratyk , Anna Kordasiewicz Research project consisted of: 9 CAQDA's image in the literature users’ opinions on CAQDA CAQDA practice Literature review (1) CAQDA’s image in the literature Online survey on qualitative research CAQDA users’ (2) opinions and (3) practice Open-ended and closed-ended questions 201 academic and non-academic Polish researchers Conducted between 1.10.2012-31.12.2012. Karol Haratyk , Anna Kordasiewicz Online survey on CAQDA users’ opinions and practice 10 Young and academic researchers tied to major academic centres in Poland and women were overrepresented CAQDAS users - 66 people declared use of CAQDAS (35% of the sample, overrepresentation) CAQDAS used were mostly paid software: Atlas.ti used by 55% of CAQDAS users MaxQDA 42% Nvivo 27% QDAminer 22% And some freeware: Weft QDA 9% Open code 6% Karol Haratyk , Anna Kordasiewicz Who are the CAQDAS users? 11 43/65 aged 25-34 CAQDASUs younger then total 48/65 Ph.D. students (25) and Ph.D. (23) bigger share of Ph.D. among CAQDASUs 55/65 connected to academia, but 28/65 combine academic and extra-academic jobs bigger share than in total 38/65 CAQDASUs declared that research activities were most important in their professional life in last 3 years CAQDASUs more focused on research than total 38 women, 26 men/65 women slightly underepresented among CAQDASUs Karol Haratyk , Anna Kordasiewicz Research project - continued 12 Qualitative research in progress Practice: own analytical projects, software workshops, teaching and training discourse analysis, autoethnography, IDI and observation of analytical practice Karol Haratyk , Anna Kordasiewicz 2. The CAQDA viscious traingle 13 CAQDA's image in the literature users opinions on CAQDA CAQDA practice THEORY – PRACTICE GAP Karol Haratyk , Anna Kordasiewicz The CAQDA's image in the literature 14 users opinions on CAQDA CAQDA's image in the literature CAQDA practice THEORY – PRACTICE GAP Karol Haratyk , Anna Kordasiewicz 2. CAQDA’s image in the literature 15 a. FOCUS ON LEGITIMIZING CAQDA NOT ON CRITICAL ASSESMENT b. LEGITIMIZNG CAQDA THROUGH: a. b. c. Manual-like approach Focus on CAQDA advantages Relatively superficial references to grounded theory strategy Karol Haratyk , Anna Kordasiewicz Manual-like approach 16 Karol Haratyk , Anna Kordasiewicz Uniformely positive CAQDA image 17 Karol Haratyk , Anna Kordasiewicz Superficial references to grounded theory strategy 18 Papers on CAQDA: Tie CAQDA to the grounded theory approach irrespective of the actual software potential Use references to grounded theory more as a general qualitative analysis legitimization strategy (conf. Kelle 1997) than to systematically confront GT procedures with software functionalities Karol Haratyk , Anna Kordasiewicz CAQDA’s image in the literature - summary 19 Papers on CAQDA in Poland focus on Introducting CAQDA Legitimizing CAQDA And so they have an introductory character which impedes critical methodological studies of CAQDA Result: lack of critical assessment of CAQDA Karol Haratyk , Anna Kordasiewicz The users opinions on CAQDA 20 CAQDA's image in the literature users opinions on CAQDA CAQDA practice THEORY – PRACTICE GAP Karol Haratyk , Anna Kordasiewicz 3. CAQDAS – users’ opinions 21 CAQDAS ARE O.K., BUT… a. ADVANTAGES OF THE CAQDA b. DISADVANTAGES OF THE CAQDA c. SUMMARY Karol Haratyk , Anna Kordasiewicz CAQDAS – users’ opinions 22 Users (participants of the web survey) seem to share part of the literature’s enthusiasm in that they are able to list a number of advantages of CAQDA BUT Users also acknowledge many disadvantages of the CAQDA not discussed in the literature Karol Haratyk , Anna Kordasiewicz Among the advantages of the CAQDAS… 23 advantages Speeding up of analysis (25) Working with big stock of data and efficient data archiving (19) Ease of information retrieval (6) Systematicity (7) Team work (4) Visualisations (3) Karol Haratyk , Anna Kordasiewicz Among the disadvantages of the CAQDAS… 24 disadvantages Alienation from research material Adjustment of data choice to software capabilities Prices (Atlas.ti): 505 euro Basic Ph.D. salary at the University of educational Warsaw is 1725 euro 560 euro / commercial month Karol Haratyk , Anna Kordasiewicz Lack of holistic approach due to focus on coding Lack of coding standardized procedures Difficulties in mastering the software Costs Screen dimensions Users’ opinions – summary (1) 25 Among the good sides of CAQDA, technical issues (39 users) prevailed over methodological impact (15 users) Among bad sides two main groups of arguments received equal attention: negative methodological effects (19 users), extramethodological constraints of the CAQDA (20 users) Karol Haratyk , Anna Kordasiewicz Users’ opinions – summary 26 Slightly more users listed advantages (43 users) than disadvantages (36), BUT Few listed only good (10 users) or only bad (3 users) sides of the CAQDAS So the opinion of users is much more ambivalent than the image produced in the literature According to our users, the aim of qualitative analysis is to deepen understanding but apparently CAQDAS are not the tool helpful in achieving this aim… Is it a result of users’ more critical approach or of imperfect practice? Karol Haratyk , Anna Kordasiewicz The CAQDA practice 27 CAQDA's image in the literature users opinions on CAQDA CAQDA practice THEORY – PRACTICE GAP Karol Haratyk , Anna Kordasiewicz Practice 28 N=201; n=61 Coding of texts Coding of images Coding of recordings Automatic coding Retrieval of fragments coded with one code Retrieval of fragments coded with combination of two or more codes 97% 28% 23% 51% 69% 58% Looking at the number of uses of a code Looking at the size of fragments coded with a code Visualizations Reliability tests 51% 23% 33% 23% Karol Haratyk , Anna Kordasiewicz CAQDAS learning process 29 Participants learned to use the software: On their own (34/59) On an academic course (11/59) On a commercial training (8/59) From a friend (6/59) Karol Haratyk , Anna Kordasiewicz Impact of the learning mode on analytical practice 30 Those who learned on their own used more advanced options (e.g. retrieval of a combination of codes, visualisation – valid also for the academic courses) Those who learned by themselves and on an academic course analysed non-textual data more often. Those who learned on commercial training relied more heavily on automated or quantitative options, like „automatic coding”, analysis of number of uses and the scope of codes. “…teaching the use of the programs to novice researchers has to be embedded in a pedagogy which has a sense of the exemplars of qualitative analysis, rather then as skills and techniques to be mechanically aplied”, Lee&Fielding 1991 Karol Haratyk , Anna Kordasiewicz 3. The theory – practice gap 31 CAQDA's image in the literature users opinions on CAQDA CAQDA practice THEORY – PRACTICE GAP Karol Haratyk , Anna Kordasiewicz Methodological and theoretical pluralism 32 In our survey domination but not monopoly of Grounded theory (45/201 mentioned spontanously, 123/201 confirmed when prompted) • • • • • • • Interactionism (13), Anthropology (12), Discourse analysis (11) and critical discourse analysis (6) interpretative sociology (11), phenomenology (11), triangulation (11), visual sociology (10) We will focus on GT as most widely represented Karol Haratyk , Anna Kordasiewicz CAQDAS and grounded theory approach 33 CAQDAS and grounded theory approach - good old marriage or dangerous liaisons? Not obvious nor necessary a connection. GT commonly associated with CAQDA (Lonkila (1995), Niedbalski, Slezak (2012), Bringer (2006) The very design and „language” of the GT present in CAQDAS Procedures commonly analysed as corresponding: Coding procedure Memoing Linking Kołtun (2011) – reception of the grounded theory in Poland – also through CAQDA university courses Karol Haratyk , Anna Kordasiewicz CAQDAS and grounded theory approach 34 GT commonly associated with CAQDA BUT: CAQDAS’ use does not have to follow GT strategy „purely rhetorical” references to GT (Lonkila 1995, Kelle 1997) „There is a danger that the choice of techniques available (computer programs like ATLAS/ti or NUD.IST may also suggest the choice of method (grounded theory)” (Lonkila 1995). Karol Haratyk , Anna Kordasiewicz Grounded theory and CAQDAS use 35 N=201; n=181 In the last three years the strategy of grounded theory... ...has not constituted for me Count important reference point in qualitative research ...has constituted for me one Count of the reference points in qualitative research ...has constituted for me the Count most important reference point in qualitative research Total Count Karol Haratyk , Anna Kordasiewicz Do you use specialized software for qualitative analysis at the stage of the analysis of the textual data? Yes No 14 45 Total 59 22% 38% 33% 38 55 93 60% 47% 51% 11 18 18% 15% 63 100% 118 100% 29 29,0 16% 181 100% Coding as a key GT practice 36 GT is a rich and heterogeneous tradition, although commonly coding is considered as one of the key elements. It is understood not as a sheer technical operation of a label atribution, but as analysis itself (Lonkila 1995, Richrds & Richards 1991), through: Open, axial and selective coding connected with: Categories and their properties and dimensions identification Which is achieved through a process of iterative analysis and confrontation (comparison) of the data and generating the analytical categories from data (Strauss 1987, Charmaz 2009) Karol Haratyk , Anna Kordasiewicz What is the GT analytical style/coding practice? 37 Questions on practice testing GT or non-GT analytical style: Coding with a ready-made list or a code list elaborated alongside analysis? Coding data fragments once or more? Indexing or theoretical character of codes? Analysis after coding phase or simultaneosly? Karol Haratyk , Anna Kordasiewicz Usually in the last three years (please, indicate which sentence describe better your practice in the scale 1 to 4) Does the strategy of grounded theory constituted a reference po int in qualitative research in the last three years? No X 1 – firstly I have created codes and next I have coded data rather without changing the code list or changing it slightly 38 4 – code list I have created only during the process of coding data [31%] Yes N X Total N X N 2,21 14 2,88 49 2,73 63 1 – I have coded data only once (rather I have not returned to fragments coded 2,00 14 2,84 49 2,65 previously in order to code them again) 4 – I have coded data for many times (I have returned to fragments coded previously in order to code them again) [29%] 63 1 – mainly I have used codes in order to find fragments interesting for me 1,93 14 2,63 49 2,48 4 – codes have constituted for me mainly a tool to generate generalizations: after coding data I have created subsequent, mo re general codes which I have used to code again data or fragments o f data [22%] 63 1 – I have analyzed data already during the process of coding: I have browsed data, I 2,50 14 2,51 49 2,51 have made crosstabs or I have analyzed co-occurrence of codes etc. (not including reliability tests) [14%] 4 – I have analyzed data only after coding them: : I have browsed data, I have made crosstabs or I have analyzed co-occurrence of cod es etc. (not including reliability tests) 63 Karol Haratyk , Anna Kordasiewicz Grounded theory applied? 39 There are no big differences in coding styles between users who refer to GT and those who do not Generally, the results indicate a slight propensity for GT coding style, so maybe it is a sign of GT as a paradigm for CAQDA BUT Not even GT proponents get close to the „ideal type” of GT coding style, so the paradigm is weak and applied rather loosely. This results have to be confronted with an analistparticipant perspective in a qualitative study. Karol Haratyk , Anna Kordasiewicz Inspiration from other studies versus GT approach in the analytical pracitce 40 Table. Inspiration from other studies versus GT approach in the analytical pracitce COMPARISON WITH OTHER THOERIES IN THE COURSE OF ANALYSIS In the course of the last three years… …I aimed to compare my results with the state-of-the-art in the course of the analysis ... I aimed not to comp my results with the state-of-the-art in the course of the analysis Total Karol Haratyk , Anna Kordasiewicz Has the grounded theory approach constituted a reference point in your research for last three years? NO YES 39% 73% 62% 27% 13 100% 48 100% Total 40 66% 21 34% 61 100% 4. Conclusions 41 1. Limited use of possibilities offered by CAQDAS. 2. Users’ ambivalent attitude towards CAQDAS. 3. Methodological texts: legitimization of CAQDA. 4. Domination of grounded theory in methodological texts as well as users declarations. 5. Weak application of grounded theory as far as coding style is concerned – theory – practice gap. Karol Haratyk , Anna Kordasiewicz strategy of legitimization of CAQDA in texts -> insufficient support users’ ambivalent attitude limited use of the possibilities (Grounded) theory – practice gap Karol Haratyk , Anna Kordasiewicz 42 43 Karol Haratyk, University of Warsaw [email protected] Anna Kordasiewicz, University of Warsaw [email protected] Authors 44 Karol Haratyk, Ph.D. student at the Institute of Sociology, Univeristy of Warsaw, uses CAQDAS (MAXqda, ATLAS.ti) in his research projects, conducts academic courses on qualitative data analysis; Anna Kordasiewicz, assistant professor at the Centre of Migration Research, University of Warsaw, uses CAQDAS (mainly ATLAS.ti) in her research projects, carries out academic cources and commercial trainings in qualitative data analysis (also using freeware, opencode); We initiated an informal network of CAQDAS users in Poland, that involves analysts from the University of Łódź, Jagiellonian University in Cracow and University of Gdańsk; Inspiration to take up the research subject was our analytical and training experience, as well as our interest in the theoretical background of qualitative analysis Karol Haratyk , Anna Kordasiewicz
Similar documents
Art, Science Technology of QDA
To develop theories based on accepted standards of good scientific method so as to understand social phenomena.
More information