Recenzija naučnog rada
Transcription
Recenzija naučnog rada
Napisati, objaviti, prezentovati i vrednovati naučno delo Recenzija naučnog rada Jelena Marinković Institut za medicinsku statistiku i informatiku novembar, 2011. godina Recenzija u cilju publikovanja naučnog rada – sadržaj predavanja 1. 2. 3. 4. 5. Kako autor vidi svoj rad? Kakvi su trendovi? Kakva su pravila / vodiči? Recenzija i recenzenti A kakve su činjenice – nekoliko primera? Autor(i): Od ideje ... preko diskusije, ... Merenja, prikupljanja podataka, i njihove obrade (eksploracije i analize), do pisanja rada ... Koraci u pisanju rada 1. 2. 3. 4. 5. 6. Napravi plan samog rada. Skiciraj rad. Uključi “podatke”. Napiši prvu verziju. Obogati prvu verziju. Revidiraj! Da li je dovoljno? Revizija Revizija Okamov brijač u pisanju Princip naučne štedljivosti - važan princip u logičnom i naučnom razmišljanju. Broj aksioma ili pretpostavki neophodnih da se ustanovi istina ili objasni neki fenomen treba da je što manji (Vilijam od Okama). l Dobar pasus nema suvišnih rečenica. l Dobra rečenica nema suvišnih reči. l Dobri radovi zahtevaju više tabela i ilustracija a manje reči. l Svaka referenca treba da ima razumnu svrhu. Da li je dovoljno? WC Roberts Am J Cardiol, 1986 Kako autor, urednik i recenzent čitaju rad? A E R E R E E E A Od ideje do ... ? Prihvatanja Dragi ... sa zadovoljstvom Vas obaveštavamo da prihvatamo Vaš rad za objavljivanje u... Odbacivanja Dragi ... sa žaljenjem Vas obaveštavamo da ne prihvatamo Vaš rad za objavljivanje u... Recenzija u cilju publikovanja naučnog rada – očekuje se: l Da je pripremljen u skladu sa uputstvom za pripremu rukopisa za odgovarajući časopis! l Da je zanimljiv i adekvatno metodološki urađen i napisan za dati časopis! Najopštije odluka o prihvatanju rada za publikovanje zavisi od: 1. Da li su autori izabrali važan problem? 2. Da li rad pomera granice našeg znanja o predmetu? 3. Da li je rad metodološki udobro urađen? 4. Da li će ovo istraživanje verovatno potaknuti dodatna istraživanja? 5. Da li će istraživanje pomoći proces donošenja odluka? 6. Sve u svemu, da li je rad vredan publikovanja? Odluka recenzenta l Rad se ne prihvata za publikovanje l Rad se mora korigovati po zahtevima recenzenta ako je to po mišljenju autora moguće i opravdano i ponovo poslati recezentu na razmatranje (manja i veća revizija) l Rad se prihvata bez izmena za publikovanje Korekcije na zahtev recenzenta ¡ Moraju biti prezentovane u samom radu i ¡ Moraju detaljno biti objašnjene u pratećem pismu Recenzija u cilju publikovanja naučnog rada 2. Kakvi su trendovi? 3. Kakva su pravila / vodiči? 4. Kako čitati i recenzirati naučni i stručni članak? 2. Kakvi su trendovi? l Pre skoro 15 godina kada je Frank Davidoff, FACP, postao glavni i odgovorni urednik Annals of Internal Medicine, ¡ medicina zasnovana na dokazima bio je potpuno nov koncept za mnoge lekare, ¡ Internet publikovanje futuristički scenario a ¡ pitanje opstanka opštih medicinskih časopisa visokog kvaliteta upitno zbog visoko specijalizovane podele rada u okviru sistema zdravstvene zaštite. l Pitanje: Kako se promenio sadržaj Anala od 1995. godine? l Odgovor FD: ¡ Medicina zasnovana na dokazima bila je tada tek u povoju. Danas je standardni dokaz — dakle ono što se smatra adekvatnim dokazom — mnogo rigoroznije. ¡ Za uređivanje časopisa ovo znači da su se naši standardi o tome šta smatramo interesantnim i prihvatljivim promenilo. A ova činjenica je sa svoje strane uvećala važnost statističke recenzije. Našu reputaciju smo realizovali zato što naši statističari — tri glavna i sedam pomoćnih — pažljivo pregleda svaki rad jednom kada smatramo da je problem zanimljiv. Autori tih radova moraju uzeti u obzir sve njihove komentare. Vodiči dobre statističke prakse l Naučni metod = Statistički metod statističke metode) l Statistika kao naučni jezik ¡ semantika: gde (na kojim mestima u radu), šta, zašto, značenje i interpretacija ¡ sintaksa: u kom obliku (≠ 3. Kakva su pravila / vodiči ? l Uniform Requirements for Manuscripts Submitted to Biomedical Journals (International Committee of Medical Journal Editors) l CONSORT (Consolidated Standards of Reporting Trials) l QUOROM (Quality of Reporting of Meta-analyses) l MOOSE (Meta-Analysis Of Observational Studies in Epidemiology) l STROBE (The Strengthening the Reporting of Observational Studies in Epidemiology) l STARD (Standards for Reporting of Diagnostic Accuracy) l TREND (Transparent Reporting of Evaluations with Nonrandomized Designs) … UNIFORM REQUIREMENTS FOR MANUSCRIPTS SUBMITTED TO BIOMEDICAL JOURNALS (International Committee of Medical Journal Editors). Dostupno na http://www.icmje.org/ CONSORT (Consolidated Standards for Reporting Trials): a 22-item checklist for reporting simple two group, parallel, randomized controlled trials (Moher, Schulz, & Altman, 2001). Dostupno na MS Word format ] http://www.consortstatement.org/Downloads/Checklist.doc ili [ PDF format ] http://www.consort-statement.org /Downloads/checklist.pdf. QUOROM (Quality of Reporting of Meta-Analyses): a 17-item checklist for reporting systematic reviews (Moher et al., 1999). Dostupno na http://www.consort-statement.org/QUOROM.pdf. QUOROM je dostupan samo u .pdf formatu. MOOSE (Meta-Analysis Of Observational Studies in Epidemiology): a 35-item checklist for reporting observational studies (Stroup et al., 2000). Dostupno na http://www.greenjournal.org/misc/moose.pdf ili http://jama.ama-assn.org/cgi/content/full/283/15/2008. STROBE (The Strengthening the Reporting of Observational Studies in Epidemiology). Dostupno na http://www.strobe-statement.org/ TREND (Transparent Reporting of Evaluations with Nonrandomized Designs): a 22-item checklist for nonrandomized designs (2004). Dostupno na http://www.trend-statement.org/asp/documents/statements/AJPH_ Mar2004_Trendstatement.pdf. TREND je dostupan samo u .pdf formatu. STARD (Standards for Reporting of Diagnostic Accuracy): a 25-item checklist for diagnostic test accuracy (STARD, 2001). Dostupno na http://www.clinchem.org/cgi/reprint/49/1/7.pdf ili http://www.clinchem.org/cgi/content/full/49/1/7/. 4. Recenzija i recenzenti l Recenzija (peer review) je proces u kome se naučni rad, projekat, ideja pregledaju i procenjuju od strane stručnjaka u istoj oblasti, bez materijalne naknade. l Procena se po pravilu odnosi na: značaj, originalnost, prezentaciju, valjanost, etičke aspekte i druge karakteristike studije. l Fundamentalni korak u biomedicinskom publikovanju. l Sistem recenzija institucionalizovan je u potpunosti od 40-tih godina prošlog veka. Recenzija – malo istorije l Praksa slična procesu recenzije - Ethics of the Physician autora Ishaq bin Ali al-Rahwi (854–931) iz Al-Rahe, Sirija. l Prvi zabeleženi proces recenzije – Kraljevsko društvo 1665 od strane urednika Philosophical Transactions of the Royal Society Henrija Oldenburga. l U skladu sa prihvaćenom definicijom - Medical Essays and Observations koje je 1731.g. štampalo Edinburško Kraljevsko društvo. Recenzija – zašto recenzenti? l “Četiri oka više vide nego dva”. l Odluka (da li publikovati ili ne, šta menjati) je uvek na glavnom uredniku / uređivačkom odboru časopisa. l Urednici se oslanjaju na recenzente zbog: ¡ Količine posla. ¡ Različitosti mišljenja. ¡ Ograničene ekspertize. Recenzija – ključna pitanja o recenzentima l Koliko recenzenata? l Ko su recenzenti: “Anonimni” i nezavisni l Otvorena ili zatvorena? (i autori i recenzenti) l Vreme potrebno za recenziju (online) Recenzija – procedura l Urednik šalje kopije dostavljenog rada recenzentima (2-3) epoštom ili preko web-zasnovanog sistema za procesovanje rukopisa (web-based manuscript processing system). l Recenzenti vraćaju evaluirani naučni rad uredniku naglašavajući slabosti, probleme i sugestije za unapređenje rada. Skoro uvek uključuje i eksplicitne preporuke: bezuslovno prihvatanje, uslovno prihvatanje ako ga autori rada unaprede na određeni način, odbacivanje ali sa ohrabrivanjem revizije rada i ponovnog slanja i bezuslovno odbacivanje. l Urednik evaluira komentare recenzenata, uključuje sopstveno mišljenje o radu, kontekst časopisa, i šalje odluku autorima po pravilu sa komentarima recenzenata. Recenzija – procedura (2) l Da li je urednik obavezan da prihvati mišljenje recenzenata? l Nesaglasnost recenzenata? ¡ treći recenzent ¡ odgovor autora na negativnu recenziju uredniku ¡ odgovor autora na negativnu recenziju recenzentu ali bez uključivanja drugog recenzenta Recenzija – procedura (3) l “Open access model” – na Internetu je čitava istorija naučnog rada. l Tradicionalno recenzenti su anonoimni. l Promena ovog standarda – opcija recenzentima da potpišu svoju recenziju (Zahvalnost anonimnim ili imenovanim recenzentima na kraju rada). Recenzenti l Kako se postaje recenzent? l Sukob interesa. l Autori predlažu recenzente. l Autori saopštavaju ko recenzent ne može da bude. l Anonimni ili ne? l Demografske karakteristike recenzenata. l “Veliki” i “mali” časopisi. Recenzenti (2) l Evaluacije recenzenata – Journal Clubs i pisma uredništvu. l Da li je recenzija odlika valjanosti ili samo prihvatljivosti rada? l A autori, njihove kritike rada recenzenata? ¡ Vreme, vreme, ... (od nekoliko nedelja do nekoliko godina) ¡ Šta je prihvatljivo u oblasti a šta ne? Recenzenti (3) l Kako se menja praksa? ¡ dobri časopisi traže originalne podatke ¡ dvostepeni proces recenzije ¡ open peer review (od 90-tih, Nature 2006) ¡ ekstenzija – posle publikovanja koncept Open Peer Commentary 5. A kakve su činjenice – nekoliko primera? SVE ZAVISI OD ČASOPISA U KOJI SE ŠALJE RAD Agenda jednog recenzenta u 2010.g.: Journal l JACC l AJC l AJC l AJC l Circulation l Cardiologia l GIC l Circulation l Cardiologia l JACC l AHJ l AJC l AJC l JACC l AJC l ASE l Pediatric Res Month Feb Feb March March March Apr Apr Apr Apr June July Sept Oct Nov Dec Dec Dec Suggestion REJECT Major Revision ACCEPT REJECT REJECT REJECT ACCEPT REJECT REJECT REJECT Major Revision ACCEPT ACCEPT REJECT ACCEPT ACCEPT ACCEPT Editorial Decision REJECT REJECT ACCEPT REJECT REJECT REJECT ACCEPT REJECT ACCEPT REJECT REJECT ACCEPT REJECT REJECT ACCEPT ACCEPT ACCEPT Peer review time: how late is late in a small medical journal? Archives of Medical Research, 2003, Volume 34, Issue 5, Pages 439-443 Methods We analyzed 1,346 editorial requests for manuscript peer review sent from the Croatian Medical Journal to 690 reviewers from February 1998 to December 2001. Results Peer reviewer response rate was 78.6% (1,057 of 1,346 requests for review) and median review time was 29 days (95% confidence interval [95% CI]=28–31 days). Of returned reviews, 554 (52.4%) arrived on time (before the deadline established by the editor) and the others arrived late. Median delay was 12 days (95% CI=10–15 days). There was no difference in delay for Croatian- and non-Croatian-authored manuscripts, but more reviews of articles on clinical medicine were delayed than those on public health. More reviews from non-Croatian or male reviewers were never returned to the editors. For reviews that arrived late, those from male reviewers were more delayed than those from female reviewers (median, 13 days, 95% CI=11–15 days vs. median, 8 days, 95% CI=5– 12 days, respectively). Logistic regression analysis revealed that odds for reviewer's positive response to an editorial request for manuscript review were greater for female Croatian reviewers, who received up to three review requests. Conclusions Peer review time in a small general medical journal may be comparable to review times in larger and more prestigious journals. Choice of reviewers may improve the expediency of peer review: editors from small journals may profit from building and educating local peer reviewers, bearing in mind that female reviewers may provide more expedient reviews. An Evaluation of Time to Publication for Randomized Trials Submitted to the BMJ Sara Schroter, Douglas G. Altman, and John P. A. Ioannidis Objective To evaluate the publication rate of randomized controlled trials (RCTs) submitted to the BMJ and to understand how many and which trials remain unpublished a long time after submission. Design We evaluated all 660 reports of RCTs submitted to the BMJ between January 1, 1998, and December 31, 2001. We identified the published articles or, if we could not locate any published reports, we contacted the authors. Results We found that 602 of 660 RCTs (91%) were published: 150 (23%) in the BMJ and 452 (68%) in other journals. All except 6 of the trials published elsewhere were in journals with lower impact factor than BMJ (median, 2.13; interquartile range [IQR], 1.53-3.03). The median time from submission to publication was 1.36 years (IQR, 0.89-2.15; and 1.30, IQR, 0.87-1.91, excluding the 40 unpublished trials). About 25% of the RCTs submitted to the BMJ remained unpublished for 2 or more years after submission. Excluding the BMJ-published papers, a higher impact factor was associated with more rapid publication (HR, 1.08 per point, P = .002). Unpublished trials were significantly less likely to go to external review, have external reviews returned, and reach the BMJ editorial "hanging" committee than published trials but were not significantly different in their country of origin. Conclusions Results of all RCTs should be publicly available since they are level I evidence. The large majority of the trial reports submitted to the BMJ get published in time, but about 25% remained unpublished after 2 years. Among those remaining unpublished, negative results are not perceived as a prime reason for failure to publish. Editors “play dice” with the paper? In a sample of 100 papers sent out for review each one to 2 reviewers, the same evaluation was given only in 27 cases, and in 8 cases the assessment was totally split: reject outright for 1 reviewer and accepted outright for the other. Robert and Suzanne Fletcher Editors Ann Int Med, 1993 Copyright ili pravo na kopiranje? …The present meeting stems from the same philosophy: a more sophisticated.. …This supplements, stems from the same philosophy: a more efficacious… There was no ambition of completeness, exhaustivity, or even balance in choosing topics to discuss. Rather, special attention was devoted to the most elusive aspects … and the most innovative diagnostic approaches There was no ambition of completeness exhaustivity, or even balance in choosing topics to discuss. Rather, special attention was devoted to the most elusive or controversial issues … and the most innovative therapeutic approaches …The contributions of the participants, both old and new friends who promptly joined in this fourth venture... …The efforts of participants, both old and new friends who promptly joined in this venture... Circulation 1991; 83(5):III-I Am Heart J 1999; 138:S47 What Makes a Good Reviewer and a Good Review for a General Medical Journal? (JAMA. 1998;280:231-233) Nick Black, MD; Susan van Rooyen, BSc; Fiona Godlee, MRCP; Richard Smith, FRCP; Stephen Evans, MSc Context.—Selecting peer reviewers who will provide high-quality reviews is a central task of editors of biomedical journals. Objectives.—To determine the characteristics of reviewers for a general medical journal who produce high-quality reviews and to describe the characteristics of a good review, particularly in terms of the time spent reviewing and turnaround time. Design, Setting, and Participants.—Surveys of reviewers of the 420 manuscripts submitted to BMJ between January and June 1997. Main Outcome Measures.—Review quality was assessed independently by 2 editors and by the corresponding author using a newly developed 7-item review quality instrument. Results.—Of the 420 manuscripts, 345 (82%) had 2 reviews completed, for a total of 690 reviews. Authors' assessments of review quality were available for 507 reviews. The characteristics of reviewers had little association with the quality of the reviews they produced (explaining only 8% of the variation), regardless of whether editors or authors defined the quality of the review. In a logistic regression analysis, the only significant factor associated with higher-quality ratings by both editors and authors was reviewers trained in epidemiology or statistics. Younger age also was an independent predictor for editors' quality assessments, while reviews performed by reviewers who were members of an editorial board were rated of poorer quality by authors. Review quality increased with time spent on a review, up to 3 hours but not beyond. Conclusions.—The characteristics of reviewers we studied did not identify those who performed high-quality reviews. Reviewers might be advised that spending longer than 3 hours on a review on average did not appear to increase review quality as rated by editors and authors. US and Non-US Submissions An Analysis of Reviewer Bias (JAMA. 1998;280:246-247) Ann M. Link, MA Context.—Reviewers increasingly are asked to review manuscripts from outside their own country, but whether they are more likely to recommend acceptance of such manuscripts is not known. Objective.—To assess whether US reviewers or non-US reviewers evaluate manuscripts differently, depending on whether the manuscripts are submitted from outside the United States or from the United States. Design and Setting.—A retrospective analysis of all original submissions received by Gastroenterology in 1995 and 1996. Reviewers ranked manuscripts in 4 decision categories: accept, provisionally accept, reject with resubmission, or reject. Main Outcome Measure.—Ranking of papers based on nationality of authors and reviewers. Results.—The percentage of non-US manuscripts placed in each decision category by US (n=2355) and nonUS reviewers (n=1297) was nearly identical (P=.31). However, US reviewers recommended acceptance of papers submitted by US authors more often than did non-US reviewers (P=.001). Non-US reviewers ranked US papers slightly more favorably than non-US papers (P=.09), while US reviewers ranked US papers much more favorably (P=.001). Conclusions.—Reviewers from the United States and outside the United States evaluate non-US papers similarly and evaluate papers submitted by US authors more favorably, with US reviewers having a significant preference for US papers. Changes to Manuscripts During the Editorial Process - Characterizing the Evolution of a Clinical Paper (JAMA. 1998;280:227-228) Gretchen P. Purcell, MD, PhD; Shannon L. Donovan, MA; Frank Davidoff, MD Context.—Biomedical manuscripts undergo substantive change as a result of the peer review and editorial revision processes. Objective.—To characterize quantitatively problems in manuscripts identified during peer review and changes made to address these problems. Design and Setting.—Descriptive analysis of manuscripts submitted to and articles published by the Annals of Internal Medicine. A taxonomy of problems that occur in reporting clinical research was developed from analysis of changes made to 7 manuscripts between submission and publication (published October 15, 1996, and November 1, 1996). The taxonomy was used to characterize changes to 12 additional manuscripts (published January 15, 1997, to April 1, 1997). Main Outcome Measure.—Types of problems necessitating changes to manuscripts during peer review and revision. Results.—Changes occurred because of 5 types of problems: too much information, too little information, inaccurate information, misplaced information, and structural problems. Changes most often occurred because information was missing or extraneous. The distribution of changes seemed to be influenced by the type of information involved (such as background or conclusions). Conclusion.—The proposed framework may be useful for characterizing quantitatively the effects of peer review and for comparing those effects across editors, journals, and specialties. Effect of Blinding and Unmasking on the Quality of Peer Review A Randomized Trial (JAMA. 1998;280:234-237) Susan van Rooyen, BSc; Fiona Godlee, MRCP; Stephen Evans, MSc; Richard Smith, FRCP; Nick Black, MD Context.—Little research has been conducted into the quality of peer review and, in particular, the effects of blinding peer reviewers to authors' identities or masking peer reviewers' identities. Objective.—To determine whether concealing authors' identities from reviewers (blinding) and/or revealing the reviewer's identity to a coreviewer (unmasking) affects the quality of reviews, the time taken to carry out reviews, and the recommendation regarding publication. Design and Setting.—Randomized trial of 527 consecutive manuscripts submitted to BMJ, which were randomized and each sent to 2 peer reviewers. Interventions.—Manuscripts were randomized as to whether the reviewers were unmasked, masked, or uninformed that a study was taking place. Two reviewers for each manuscript were randomized to receive either a blinded or an unblinded version. Main Outcome Measures.—Mean total quality score, time taken to carry out the review, and recommendation regarding publication. Results.—Of the 527 manuscripts entered into the study, 467 (89%) were successfully randomized and followed up. The mean total quality score was 2.87. There was little or no difference in review quality between the masked and unmasked groups (scores of 2.82 and 2.96, respectively) and between the blinded and unblinded groups (scores of 2.87 and 2.90, respectively). There was no apparent Hawthorne effect. There was also no significant difference between groups in the recommendations regarding publication or time taken to review. Conclusions.—Blinding and unmasking made no editorially significant difference to review quality, reviewers' recommendations, or time taken to review. Other considerations should guide decisions as to the form of peer review adopted by a journal, and improvements in the quality of peer review should be sought via other means. Predictors of Time to Publication of Manuscripts Rejected by Major Biomedical Journals Kirby Lee,1 Nicholas Lehman,2 I'Alla Brodie,3 and Lisa Bero1 Objective To evaluate publication rates and predictors of time to publication for manuscripts rejected by major biomedical journals. Design Cohort study of 1008 manuscripts reporting original research submitted for publication and subsequently rejected at 3 major biomedical journals (BMJ, Lancet, and Annals of Internal Medicine) during 2003-2004. Our main predictor of publication was statistical significance of results (P < .05) reported for the primary outcome. We also abstracted manuscript characteristics including study design, sample size, funding source, and whether the manuscript was outright rejected vs rejected after external peer review. The primary outcome of our study was subsequent publication of the rejected manuscripts. We determined publication status and time from rejection to publication in the medical literature by searching PubMed, Cochrane Library, and the Cumulative Index for Nursing and Allied Health Literature through June 30, 2008 (minimum follow-up time of 4.3 years). Predictors of time to publication were analyzed using multivariable Cox proportional hazards regressions. All analyses were planned a priori. Results Seventy-six percent (767) of manuscripts were published (median, 1.25 years, range, 0.01-5.32 years). The majority of manuscripts were published in specialty journals (85%, 654/767) with lower impact factors than the original rejecting journal. Manuscripts were more likely to be published if they had larger sample sizes and less likely if they did not disclose a funding source or did not report results using statistical tests for comparisons, particularly after 1.25 years from initial rejection. Conclusions A quarter of manuscripts initially rejected by major biomedical journals remained unpublished. Although some methodological characteristics and disclosing the funding source were associated with publication, articles with statistically significant results were not more likely to be published. Publication of Research Reports After Rejection by the New England Journal of Medicine in 2 Time Periods Michael Bretthauer, Pam Miller, Edward W. Campion, and Jeff Drazen Objective In the past 2 decades, the number of peer-reviewed medical journals that publish original research reports has increased. We compared the subsequent publication of original research manuscripts after rejection at a major general medical journal, the New England Journal of Medicine, with an acceptance rate of less than 10%. Data were obtained for both 1995 and 2003. Design All original research manuscripts rejected after external peer review at the index journal during the calendar years 1995 and 2003 were identified from our electronic databases. All manuscripts were tracked for subsequent publication in a medical or scientific journal by searching for similar titles and authors in PubMed. Searches were performed between November 25, 2008, and January 30, 2009, for the 1995 manuscripts. For the 2003 manuscripts, searches were completed in the first week of February 2009. Descriptive statistics were computed by the use of SPSS 15.0. Results Of 1423 manuscripts rejected in 1995, we identified 1273 that were subsequently published (89.5%), as compared to 1040 of 1205 (86.3%) for those in 2003. The manuscripts were published in 384 different journals in 1995, as compared to 319 in 2003. The median time from rejection by the index journal to eventual publication in another journal was 437 days (interquartile range [IQR]: 313-652 days) in 1995 vs 398 days (IQR: 286-564 days) in 2003. Table 7 shows the 5 types of journals that published the papers after rejection at the index journal for the 2 time periods. Conclusions The vast majority of research manuscripts that are rejected by a general medical journal after peer review are published elsewhere. However, there is substantial delay in this process. The time from rejection at the index journal to publication elsewhere declined only slightly from 1995 to 2003. Odluka recenzenta l Rad se ne prihvata za publikovanje l Rad se mora korigovati po zahtevima recenzenta ako je to po mišljenju autora moguće i opravdano i ponovo poslati recezentu na razmatranje (manja i veća revizija) l Rad se prihvata bez izmena za publikovanje Korekcije na zahtev recenzenta ¡ Moraju biti prezentovane u samom radu ¡ Moraju detaljno biti objašnjene u pratećem pismu Literatura l 1. Različita uputstva, vodiči l 2. Evaluacija medicinske literature U: Marinković J, Dotlić R, Janošević S i sar.: Statistika za istraživače u oblasti medicinskih nauka, Medicinski fakultet, Beograd, 2001. l 3. Pyrczak F. Evaluating Research in Academics Journals. A Practical Guide to Realistic Evaluation. Second Edition. Pyrczak Publishing, Los Angeles, 2003. pp 141-14