How to Write the Results and Discussion Sections

Transcription

How to Write the Results and Discussion Sections
E NGLISH 1102/66

P ROF . H AGIN

K ENNESAW S TATE U NIVERSITY

S PRING 2009
This document will help you learn how to properly aggregate your raw data into
tables that address your scientific questions and hypotheses. Once your field
research is completely collected, you should start creating data tables to begin the
process of interpreting your findings. Once that task is complete, you may write your
Results and Discussion sections of the paper. The bulleted items below in green are
verbatim passages from the APA Style Manual, 5th edition.
What Is the Results Section?
The Results Section reveals your research findings, displayed as both tabulated
numbers and a written narrative that walks the reader through the data.
(1.10) “The Results section summarizes the data collected and the statistical or data analytic
treatment used. Report the data in sufficient detail to justify the results. Mention all
relevant results, including those that run counter to the hypothesis.”
(1.10) “Discussing the implications of the results is not appropriate here.”
Step 1: Create Your Tables
Tables are created in a scientific paper to organize your results and provide a
mechanism for interpreting your data. Scientific papers require tabulated data,
which will be placed in the Results section. You have NO obligation to create any
graphs or charts. Data presented in this paper should be numerical.
(1.10) “To report the data, choose the medium that presents them more clearly and economically.
Tables commonly provide exact values and, if well prepared, can present complex
data and analyses in a format that is familiar to the reader […].”
(1.10) “Figures of professional quality attract the reader’s eye, provide a quick visual impression,
and best illustrate complex relationships and general comparisons but are not
intended to be as precise as tables.”
Tables must be created in a standard format, including a title and a numeric identity.
(3.66) “Give every table a brief but clear and explanatory title.”
(3.62) “Tables that communicate quantitative data are effective only when the data are arranged so
that their meaning is obvious at a glance […]. A table should be organized so that
entries that are to be compared are next to one another.”
(3.76) “The standards for good figures are simplicity. A good figure
• augments rather than duplicates the text;
• conveys only essential facts;
• omits visually distracting detail;
• is easy to read […];
• is easy to understand […];
• is consistent with and is prepared in the same style as similar figures in the same
article […]; and
• is carefully planned and prepared.”
(3.80) “Note. For legibility, a sans serif typeface […] is recommended.”
Step 2: Interpret Your Data
Your data will answer the research questions and test the hypotheses. Your first task,
after creating all of your tables, is to understand what the data is telling you. The
most important issue is to determine if your hypotheses have been confirmed or
denied by the research. Don’t worry if all of your hypotheses were disproved – you
still have plenty to writer about (and in some cases, you’ll have more). You gain
nothing by being “right” about your prediction, so let the data tell their story.
Step 3: Write the Text for the Results Section
The Results section addresses WHAT you found, NOT what it means or why you
found it (save that for the Discussion section). If you have created your tables first,
then you will know what to WRITE about in the Results section, which then outlines
what you should develop in the Discussion. Here are some aspects of the content of
this section:
(1.10) “Summarizing the results and the analysis in tables or figures instead of text may be
helpful; for example, a table may enhance the readability of complex sets of analysis of
variance results. Avoid repeating the data in several places and using tables for data that
can be easily presented in a few sentences in the text.”
(1.10) Tables and figures supplement the text; they cannot do the entire job of communication.
Always tell the reader what to look for in tables and figures, and provide sufficient
explanation to make then readily intelligible […].”
A good rule of thumb is to treat the text of the Results section as the guidebook to
understanding the tables. For each table that you create, devote a paragraph that
highlights the valuable data, preferably by pointing out numbers that confirm or
reject your hypothesis. In later tables, you will wish to point out and discuss the data
that helps you to determine whether demographic variables affected your results.
Walk the reader through the table and point out relationships, trends, and anomalies
in your data data.
DO NOT write something about every number in every table (unless your tables are
very small). Not every data figure is useful, so focus on the big items first, and only
point resort to the nit-picky things if you happen to run out of things to say.
(1.10) “When you use tables or figures, be certain to mention all of them in the text. Refer to all
tables as tables and to all graphs, pictures, or drawings as figures.
(3.63) “In the text, refer to tables by their numbers:”
as shown in Table 8, the responses were ….
children with pretraining (see Table 5) ….
(3.62) “For several reasons, it is worthwhile to be selective in choosing how many tables to include
in your paper.”
Step 4: Write the Discussion Text
The Discussion section discusses WHY your results occurred in the ways that they
did. This is the section that will reveal your conclusions and the validity of your
hypotheses. In this section, you should interpret your data by establishing causes
and effects that tell us the story of human behavior. Remember, no interpretations
should be placed in the Results section — save that for the Discussion. Facts and
opinions are kept in separate sections in a scientific paper.
(1.11) “After presenting the results, you are in a position to evaluate and interpret their
implications, especially with respect to your original hypothesis. You are free to
examine, interpret, and qualify the results, as well as to draw inferences from them.
Emphasize any theoretical consequences of the results and validity of your
conclusions.”
Using the same sequence that you used for the Results Section, begin your
Discussion with the same approach. Go through each table, one by one, and explain
the meaning behind the data points that you chose to highlight in the Results section.
(1.11) “Open the Discussion section with a clear statement of the support or nonsupport for
your original hypothesis. Similarities and differences between your results and the
work of others should clarify and confirm your conclusions. Do not, however, simply
reformulate and repeat points already made; each new statement should contribute to
your position and to the reader’s understanding of the problem. Acknowledge
limitations, and address alternative explanations of results.”
Discuss the most important issues first, then the secondary ones, and then any other
points of interest, including the inclusion of any anecdotal observations from the
field that did not appear in data form. Start your discussion with confirmation or
rejection of each hypothesis, starting with the main hypothesis (the one that best
reflects your independent and dependent variables). Refer back to Table 1 directly
and repeat the same data that you incorporated into your written Results narrative.
This time, however, take the initiative to discuss what the data is telling you.
After assessing your hypotheses (which may take more than one paragraph), you
should then explore the other research questions and demographic impacts that will
be represented in your tables. Results that express absolute clarity about an issue are
easy to analyze, but values that fall closer together need further analysis. Explore
multiple causes if your results deviate from the ways that you had predicted.
Take time to explain your rationales, even if they are mere guesses. When you are
guessing, please be clear to the reader by using words such as “suggest,” “possible,”
“likely,” “may be caused by,” or other language choices that leave room for
alternative influences. Use as many paragraphs as you need to carefully explore your
findings. Navigate the reader through the sea of numbers. Since the Discussion is
the appropriate location for your interpretations, opinions, and conclusions, it should
be the longest section of the paper, perhaps about three full pages (but potentially
many more if you are inspired).
End the Discussion section with a brief CRITIQUE of your project. Comment on
what worked well and what could have been done differently. Your assessment of
your project’s scope and validity can assist future scientists to learn from your
mistakes and to create a better research design. Remember, academics work together,
so you are not competing with other researchers, nor are you going to provide the
definitive answer to anything. Use facts, be fair, and utilize good judgment.
By the way, writing the Discussion section may force you to reconsider your table
sequence. If you realize that you prefer to discuss your tables in a different order,
then I recommend switching your tables in the Results section to better conform to
the sequence of your discussion. This is common, so be prepared to reassess a logical
order for your discussion.
(1.11) “You are encouraged, when appropriate and justified, to end the Discussion section with
commentary on the importance of your findings. This concluding section may be
brief or extensive, provided that it is tightly reasoned and self-contained. In this section
you might address the following sorts of issues:
• Problem choice: Why is this problem important? What larger issues […] hinge on
the findings? What propositions are confirmed or disconfirmed by the
extrapolation of these findings to such overarching issues?
CONTINUED Æ
• Levels of analysis: How can the findings be linked to phenomena at more complex
and less complex levels of analysis? What needs to be known for such links to
be forged?
• Application and synthesis: If the findings are valid and replicable, what real-life
psychological phenomena might be explained or modeled by the results?”
How to Create Tables
E NGLISH 1102/66

P ROF . H AGIN

K ENNESAW S TATE U NIVERSITY

S PRING 2009
To illustrate the creation of your tables, let’s look at a sample experimental design. This
design is original and does not reflect anyone’s design in class.
To create your first table, recall your research questions and hypotheses, the
independent and dependent variables, and the manner in which you structured your
groups. Your table should arrange your raw data in ways that answers your research
questions and test your hypotheses.
SAMPLE EXPERIMENTAL DESIGN
Research question: “Will increasing text size enhance a subject’s recall?”
Literature Review: Research says yes.
Hypothesis #1: Increased text size will positively affect a subject’s recall.
Method: Generate a paragraph of text with accompanying questions to answer in a
controlled environment. Subjects will be asked 4 comprehension questions about the
content of the narrative and their recall frequency assessed.
Three Subject Groups::
Control:
Experiment 1:
Experiment 2:
plain text (Times New Roman, 12 point font)
Demographics Collected:
Age:
Gender:
Race:
Vision:
(18-25, 26-40, 41+)
(male/female)
(Caucasian, African, Near Eastern, Asian, Hispanic)
(20/20, glasses, contacts, LASIK)
Begin creating your tables by locating the data that can answer your main hypothesis.
Arrange this data in rows and columns. You may use the x and y axis any way that best
fits your paper, but try to arrange all of your tables in the same way. I’ll show you two
different ways to arrange this data in a moment.
Before I show an example, remember that the APA Style Manual dictates certain rules
about creating tables:
•
•
•
•
•
Tables must be numbered, starting with 1
Tables must be titled
The tables should be formatted into a SANS SERIF font, such as Arial
Both raw values and percentages (or averages) should be displayed
Percentages MUST be expressed with two digits past the decimal (e.g., 13.00%)
So, let’s make a table. Remember, this data is entirely made up and not based on
anything. Do not refer to any of these fictitious numbers in your papers.
Start with Table 1 that will answer the main hypothesis: did the independent variable
have the impact on the dependent variable that you anticipated? Since our main
hypothesis suggests that text size will affect recall, let’s contrast the data for the three
groups (a control group and two experimental). Here are two ways of arranging this
information into tables. The first places the groups on the vertical axis and the data
categories on the horizontal:
Table 1. The Effects of Text Size on Recall
Correct Answers Number Possible
CONT (12 pt)
143
208
EXP 1 (24 pt)
142
196
EXP 2 (6 pt)
103
180
% Recall
68.75%
74.45%
57.22%
Also, you may create the table with the groups across the top and the data categories on
the left. This will allow the percentages to appear at the bottom of the table rather than
to the right. Neither has an advantage, but choose the option that works best for you:
# Correct
# Possible
% Recall
CONTROL
143
208
68.75%
EXP 1
142
196
74.45%
EXP 2
103
180
57.22%
Once you settle on an arrangement, you should ask yourself how your chosen
arrangement will work across the other tables? Try to plan your tables out before
committing to them. Foresight and careful thinking can save you a lot of time having to
reformat your tables. Because of my planning ahead, I will opt for my original
configuration, but notice in the example for Table 2 below that I will also need to utilize
a version of the alternative format as well.
Your second table should contain information that is secondary to the main hypothesis,
but not as trivial as race or gender demographics. If you asked additional research
questions or generated additional hypotheses, then create data tables that help you
explore your claims. For my sample table 2 below, I decided to set the recall data into
four categories for the individual questions that the subjects answered about the text.
The three sub-tables that follow provide deeper investigation into the percentages.
Table 2. The Effects of Text Size on Recall Per Question: Total % Recall
CONTROL
EXP 1
EXP 2
Q1
87.04%
86.54%
66.67%
Q2
69.23%
78.00%
54.35%
Q3
44.00%
37.50%
30.95%
Q4
73.08%
86.96%
75.00%
Table 2a. The Effects of Text Size on Recall Per Question: Control Group
# Correct
# Possible
% Recall
Q1
47
54
87.04%
Q2
36
52
69.23%
Q3
22
50
44.00%
Q4
38
52
73.08%
Table 2b. The Effects of Text Size on Recall Per Question: EXP 1. Group
# Correct
# Possible
% Recall
Q1
45
52
86.54%
Q2
39
50
78.00%
Q3
18
48
37.50%
Q4
40
46
86.96%
Table 2c. The Effects of Text Size on Recall Per Question: EXP 2. Group
# Correct
# Possible
% Recall
Q1
32
48
66.67%
Q2
25
46
54.35%
Q3
13
42
30.95%
Q4
33
44
75.00%
You are not required to create sub-tables for every table, but I wanted to show the raw
data and I lacked the space to do this. The smaller versions of Table 2 are lettered a, b,
and c, so please refer to tables clearly, especially if you have more than one version.
Now we can see the sample size and realize that the totals for each question differ. This
means that some subjects did not attempt certain questions. This may need to be
explained in your Discussion section if it causes a problem, such as a deviant or very
small sample size.
As you decide how many tables to create, try to designate one table for each
demographic item. In this example, I would be wise to make a table for age, race,
gender, and vision quality. I collected this data, so I might as well see what I found.
Therefore, my Results section may contain 5 or more tables, and you may create even
more by combining and extracting specific figures from various tables for comparisons.
Any method of articulating your tabulated results in a comprehendible way will clarify
your purpose when writing the narrative. If your tables are clear, the paper becomes
very easy to write. This is why I was so demanding about your research design.
You may create your tables in Microsoft Word by simply tabbing and spacing your
numbers as you need. Don’t overlook some tools in Word that can make your tables
appear more professional. You may boldface your data, but avoid italics, etc.
The ruler at the top has many features that allow you to set additional customized tabs.
In the left corner of the screen you will see a box with an “L”-shaped corner icon at a
right angle. You can click this to set your customized tab to be flush left, centered at the
tab, or flushed right. Click your mouse on the icon, then at the location on the ruler that
you wish to apply the customized tab. Play around with it for a while.
Another method of creating the table is by inserting one from the upper menu options.
Find the “Table” menu (it is a drop-down on Word 2003). Once you click that feature,
you can set the exact number of rows and columns that you desire. The table can then
be placed in the open space by your cursor and can be formatted easily by highlighting
the squares. By visiting the “Format: Borders and Shading” option that allows you to
colorize the border, you can click “None” to designate no border lines, thus creating a
professional looking table without the black guidelines.
Remember that you need to add an additional row and column for the margin headings
that allow the reader to know what each axis represents. So, in my sample Table 1, I
needed to create a 4x4 table, even though I only had 3 rows and columns of data.
You may also insert a table that you create in Excel. If you know how to use Excel, I
would highly suggest doing so. I can show you easy ways of calculating your data that
will save you time and ensure greater accuracy.
Analyzing Your Data
Use the tabulated data to confirm or reject your hypothesis. If you based your
hypothesis on a real research conclusion, then your experiment should be predisposed
to confirming your hypothesis. However, because your subject pool is limited and
unique, you may find that other realities exist. Comparing your research findings to
those of the research community is one way to analyze your data outside the numbers
that you collected.
But the data must tell the main story, not anyone’s opinions or biases. If your
experimental design were well conceived and executed, then you can generally trust the
validity of your results, assuming that your subject pool is healthy. Therefore, the data
must lead the way.
Recall Table 1 below that shows the total results for the subjects’ overall performance on
the recall task (all four questions combined together).
CONT (12 pt)
143
208
EXP 1 (24 pt)
142
196
EXP 2 (6 pt)
103
180
% Recall
68.75%
74.45%
57.22%
The first question ought to be this: is my hypothesis confirmed or denied? In order to
answer this, we need to recall the hypothesis:
Hypothesis #1: Increased text size will positively affect a subject’s recall.
Does the data from Table 1 support or reject the hypothesis? The answer is YES. In
building your Table 1, the researcher chose to reveal the three main statistics from the
results: the overall percentage of recall across three different text size groups. The
rightmost column clearly juxtaposes the three important data values, and this allows us
to draw a clear conclusion about the influence of font size and recall ability.
Please show your raw data to reveal your work, but use the percentages for
comparative purposes. Notice that Experimental Group 1 displayed the highest recall
(likely due to the increased font size) at 74.45% compared with the Control Group at
68.75%. However, had you used the raw numbers instead of the percentages, you
might have erroneously claimed that the Control performed better with 143 correct
answers as opposed to the 142 by Experimental Group 1. Per every 100, EXP 1 wins.
Sample Results and Discussion Paragraphs
Since you will be writing about Table 1 in the Results section first, remember to limit
your narrative ONLY to the facts and the revelation of various percentages and numeric
data. Here is a sample paragraph that this researcher might have generated from the
analysis of Table 1:
RESULTS SECTION:
Table 1 shows the total results of the subjects’ recall rate (all four comprehension
questions combined). Experimental Group 1 encountered the largest font size (24
points), yielding a recall rate of 74.45%. The Control Group, using the 12-point font
size, resulted in a recall rate of 68.75%. However, the smallest font size — 6-point
font — revealed only a 57.22% accuracy rate.
Note that the above text ONLY highlights WHAT the data reveals, and it does not
mention any conclusions, explanations, or reasons attached to the data. Save that for
the Discussion section:
DISCUSSION SECTION:
The main hypothesis is confirmed by this experimental design, shown easily by
examining the numbers in Table 1. The larger the font size, the higher the recall.
Experimental Group 1 used the largest font size (24 points), yielding a recall rate of
74.45%. The Control Group, using the 12-point font size, was challenged by this
difference, resulting in a recall rate of 68.75%. However, the smallest font size — 6point font — did not allow the subjects to recall the material as readily, pulling only
a 57.22% accuracy rate. Therefore, the font size can be a determining factor in a recall
activity. Although differences may have occurred due to subjects’ ages and ability
levels, the trend appears established that the larger text size becomes easier to read,
and therefore easier to recall.
The above Discussion paragraph sample shows all the opinions and commentary in
green. Everything in the blue remains factual. Notice the balance between fact and
opinion in this paragraph and the opinionated nature of the wording.
Further Issues with Data Analysis
Recall Table 2 that shows the breakdown of the subjects’ recall across the four
comprehension questions:
Table 2. The Effects of Text Size on Recall Per Question: Total % Recall
CONTROL
EXP 1
EXP 2
Q1
87.04%
86.54%
66.67%
Q2
69.23%
78.00%
54.35%
Q3
44.00%
37.50%
30.95%
Q4
73.08%
86.96%
75.00%
Having this data in front of us, we can begin to draw some initial conclusions, but be
sure to spend time poring over the tables to find hidden gems that may give you very
interesting nuggets to discuss. Begin with your research questions and hypotheses to
generate discussion from your data, but dig deeper by asking more probing questions:
1.
2.
3.
4.
5.
6.
7.
What trends or common traits do you recognize?
What values appear more consistent than others?
Was one question easier or harder than the others?
Why?
Did sequence or position of the questions matter?
Why did Control outperform Exp. 1 in Question #1?
Are these numbers truly different from each other or are they too close to call?
Statistical Significance
This last question is a really important one. The difference between two numbers may
be too close to establish a meaningful difference. Since the recall rates of 87.04% and
86.54% are relatively the same value, can we really say that the text size made any
difference? Well, for Questions #1 and #3, no, but for #2 and #4 it worked the way we
expected. So, what does this new revelation indicate about the hypothesis (which
claimed that the bigger text size would improve recall)? If half the questions confirm
the hypothesis, but the other half reject it, then how confident can we be with the
assessment of the hypothesis overall? Ahh, now that gives you something to write!
You have found that your research studies contained a lot of numerical coding and
mathematical formulas that seemed like a foreign language to many, perhaps
something like this:
p
0.05
This is something called the “p-value,” and it represents a measure of statistical
significance. We have seen these before during election years, when the polling results
are displayed. Typically, 1000 people are surveyed for the results to be considered to be
reliable, and even then, the margin or error is usually +/- 3.5% or so. That means that
Candidate A, who is polling 46%, may be leading Candidate B, who only has 43%, but
because the difference falls within the margin or error, this election is considered tied.
A real scientist would establish a standard for assessing close data, and these standards
are typically set at 0.05, but sometimes lower (0.01) or even 0.001. The smaller the
number, the more strict the standard for statistical significance. Read your research
sources and see what p-value your authors had set. Confident scientists or those who
absolutely must have extreme precision will use the smallest p-values.
A p-value does not correspond to a percentage, but for our purposes in this assignment,
we will simply say that any two percentages that are within 5% of each other be
considered statistically the same. In other words, a baseball player with a .275 average
could be considered to have relatively the same output as another player who bats a
.285 average. Had we used a finer standard (such as p 0.01), then even a .278 hitter
would differ significantly from the .275 hitter. Again, the scientist sets the standard as
to how fine the results need to be. Because of our sample size, we will go with 5%.
In addition, real scientists would employ a range of statistical treatments to the raw
data, such as a t-test, a Chi Square analysis, and an ANOVA treatment. We will not be
performing any of these since this class is not a science class. Instead, I ask that you
simply assess your data through comparison and contrast and use the 5% margin of
error for the sake of simplicity.
Common Mistakes
One common mistake that students make when comparing two percentages is that they
often add or subtract one percentage from another; however, this is not appropriate. In
order to determine the difference between two numbers, you should DIVIDE the larger
one by the smaller one. The difference will then be revealed in a percentage.
For example, If the control group shows a 20% rate of compliance, but the experimental
group shows a 30% compliance, then how much larger is the experimental value over
the control? The answer is 50% larger. The value “30%” divided by “20%” would yield
an answer of 1.5. This means that the larger number is 1.5 times larger than the smaller
value in the Control group. The rate or 1.5 times equals 50%.
Many students erroneously subtract 20% from 30%, resulting in the incorrect answer
(10%). The value “30%” is not 10% larger than “20%.” It’s 1.5 times bigger.
Here’s another example. Once again, please recall Table 1:
CONT (12 pt)
143
208
EXP 1 (24 pt)
142
196
EXP 2 (6 pt)
103
180
% Recall
68.75%
74.45%
57.22%
How much easier was reading a font when it was twice the size (or half the size)? This
can be answered by looking for the difference ratio between two values. The two tables
below show the incorrect and correct ways of assessing the extent of the difference
between these two values. The wrong way is listed in red and the correct way in green:
CONTROL vs. EXP 1:
CONTROL vs. EXP 2:
WRONG:
5.70% (74.45% - 68.75%)
WRONG:
17.23% (74.45% - 57.22%)
RIGHT:
8.29% (74.45% / 68.75%)
RIGHT:
30.11% (74.45% / 57.22%)
Are both size differences from the control significantly different? YES, because the
difference in the percentages exceeds 5%. The difference between Exp. 1 and the
Control is 8.29%, while the difference between the smaller font (Exp. 2) and the Control
was a whopping 30.11%.
Therefore, which size difference generates the greatest impact: doubling the font size or
reducing to 50% of its original size?
ANSWER: Shrinking the font impacts recall far more significantly than enlarging it:
Shrinking the font to 50% generates a 30.11% difference in recall, while doubling the
size generates only a 8.29% difference.
Therefore, the impact of shrinking a font 50% is 3.63 times greater than doubling its
size.
Good luck with your own interpretations!