TO: Consensus Standards Approval Committee FR: Ashley Morsell

Transcription

TO:
Consensus Standards Approval Committee
FR:
Ashley Morsell, Program Manager, Measure Maintenance
RE:
Review of testing for the Centers for Medicare & Medicaid Services measures with time-limited
endorsement
DA:
April 8, 2014
CSAC ACTION REQUIRED
Pursuant to the CDP, the CSAC may consider the removal of “time-limited” endorsement status or
alternatively, termination of endorsement. If the time-limited endorsement status is removed, the
measures will be expected to go through maintenance review on schedule (regardless of the timing of
removal of the time-limited status).
NQF staff conducted an initial review of the measures’ testing results on reliability and validity.
NQF#
Title
Description
Endorsement Measure
Date
Steward
1814
Counseling for
Women of
Childbearing
Potential with
Epilepsy
All female patients of
childbearing potential
(12–44 years old)
diagnosed with epilepsy
who were counseled
about epilepsy
and how its treatment
may affect contraception
and pregnancy
at least once a year.
3/6/2013
American
Academy of
Neurology
Maintenance
Year/Topic
Area
2015/Neurolog
y
BACKGROUND
Measures were granted time-limited endorsement status if they met the criteria except for “field
testing.” Developers must demonstrate reliability and validity of the measure as specified as well as
potential threats to validity (exclusions, risk adjustment), and any feasibility issues.
The measure stewards are asked to provide the testing results in a standard form consistent with the
evaluation criteria related to measure testing. The CSAC serves as the review body for determining the
adequacy of testing results and then determining whether to remove the time-limited status or remove
endorsement of the measures. The measure information submitted by the measure stewards is
provided in this package of materials.
NQF DRAFT – DO NOT CITE, QUOTE, REPRODUCE, OR CIRCULATE
1
EVALUATION OF MEASURE TESTING
NQF staff conducted an initial review of testing results for the ACR measure. The attached measure
submission form from ACR provides additional detailed data. ACR included results on reliability and
validity testing.
NQF# 1814 Counseling for Women of Childbearing Potential with Epilepsy.
Reliability Testing
• The AAN identified and recruited Neurology practices in Minnesota that have experience
treating patients with epilepsy. As part of the recruitment process MNCM and the AAN hosted
an informational webinar explaining the purpose of the measurement testing project for the
Counseling for Women with Epilepsy measure.
• Three Neurology practices volunteered to participate and submit retrospective data from the
2012 calendar year (i.e. dates of service 01/01/2012 – 12/31/2012). MNCM produced a data
collection guide, measure flow and detailed file specifications to educate and assist each
medical group in the data collection and submission process.
o As a requirement of participating in the measure testing each group had to submit a
denominator certifications form (see appendix A).
• There were a few corrections and clarifications that required MNCM to send a follow-up email
to the respective group; however, each issue was resolved in a timely manner. The list below
documents the issues that were identified and required additional follow-up based on the
information received on the denominator certification forms:
o Incorrect diagnosis codes included in data query
o Group did not indicate if they would be submitting a sample or full population for the
measure
o Incorrect encounter codes included in data query
Validity Testing
• MNCM completed validation of the data in a three-step process: 1) denominator certification, 2)
data file quality checks, and 3) validation audit.
• Denominator certification is an essential step in the process to obtaining valid and accurate
data. It requires each participant to attest that they will submit accurate data and follow the
measure specifications exactly how they are written. It also ensures that each participant is
querying the correct diagnosis codes, encounter codes, date of birth ranges and date of service
ranges.
• After each medical group submitted their data file to MNCM, quality checks of the files were
completed. Each column in the data file represented a field of data for each patient row; the
following checks were completed:
o Number of patients/rows submitted were reasonable/expected
o Necessary data fields (columns) were included and completed appropriately
o Patient date of birth spanned the expected range
o Zip codes were 5-digit and primarily within MN and other bordering states as expected
2
•
•
o Race field(s) were included and populated appropriately
o Provider NPI field was included and number of providers expected
o Insurance information was included and was reasonable
o Office visit dates and counseling dates spanned the expected range
o Diagnoses were included and spanned the entire list of expected codes
o Medical reasons for NOT counseling were applied correctly; were not misused
After the data file checks were completed, MNCM completed audits of the patient records to
verify the submitted clinical data. We also verified the diagnosis of epilepsy and other
demographic data (e.g., race). MNCM uses a validation process developed by the NCQA –
National Committee for Quality Assurance, known as the “8 and 30” process. In this process, the
first eight records are verified for accuracy and if no errors are identified, the data is considered
to be 100% compliant. If errors in the first eight records are identified, we continue reviewing
the total 30 records to identify any error patterns and or issues that may need correction.
The validation process was successful in identifying errors (with subsequent corrections) and
verifying the accuracy of the data submitted by medical groups A, B, and C. Finding no significant
flaws or errors with the data MNCM is confident the rate calculation and any additional data
analysis can be completed using validated and reliable data. Additionally, during a review of the
National Quality Forum’s feedback to the American Academy of Neurology for this measure, it
was noted that there was a concern that this may simply be a “check-the-box” measure. During
the validation audit, it was noted on several occasions that the practices provided excellent,
personalized progress notes about the counseling that was being provided, that were above and
beyond a “check the box”.
Table 1. Individual Medical Group Results
3
Table 2. Validation/Audit Conclusion Data
Exclusion Analysis
• The main limitation that MNCM identified during the testing of the Counseling for Women with
Epilepsy measure is related to the denominator of included and excluded patients. The measure
specifications offered two different options for excluding patients from the measure:
o Patient was surgically sterile (tubal ligation, hysterectomy)
o Patient has an intellectual disability as defined by ICD-9 codes
 318.0 moderate intellectual disabilities; IQ 35 to 48
 318.1 severe intellectual disabilities; IQ 20 to 34
 318.2 profound intellectual disabilities; IQ under 20
• Groups submitted these patients and indicated which reason applied. Additionally, if they felt
that there was another medical reason for not providing counseling, they indicated this by a
code and accompanying description.
4
Table 3. Reasons Provided by Medical Groups for Not Providing Counseling
•
During the analysis of this data and also as a byproduct of the validation audit in reviewing
medical records, MNCM staff has some concerns regarding the denominator and intent of the
measure. There may need to be a consideration for adding a component of indicating that the
patient is sexually active or has the potential to be sexually active, and not physically
handicapped.
5
#1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014
Measure Information
This document contains the information submitted by measure developers/stewards, but is organized according to NQF’s measure
evaluation criteria and process. The item numbers refer to those in the submission form but may be in a slightly different order here.
In general, the item numbers also reference the related criteria (e.g., item 1b.1 relates to subcriterion 1b).
Brief Measure Information
NQF #: 1814
De.2. Measure Title: Counseling for Women of Childbearing Potential with Epilepsy
Co.1.1. Measure Steward: American Academy of Neurology
De.3. Brief Description of Measure: All female patients of childbearing potential (12–44 years old)
diagnosed with epilepsy who were counseled about epilepsy
and how its treatment may affect contraception and pregnancy
at least once a year
1b.1. Developer Rationale: Educate women about epilepsy and how its treatments may affect contraception and pregnancy. This
will inform women of childbearing potential about the risks of epilepsy and AED therapy prior to pregnancy. It will provide an
opportunity to educate this population about folic acid supplementation, monotherapyy, medication alternatives and how to obtain
obstetrical, prenantal and pregnancy care. This measure will help them understand the risk and mitigate the risks which may prevent
fetal malformation, unplanned pregnancies and improve the patients´ quality of life.
S.4. Numerator Statement: Female patients counseled about epilepsy and how its treatment may affect contraception and
pregnancy and documented in the medical record at least once a year.
S.7. Denominator Statement: All females of childbearing potential (12-44 years old) with a diagnosis of epilepsy.
S.10. Denominator Exclusions: Patient is surgically sterile, Patient has an intellectual disability.
De.1. Measure Type: Process
S.23. Data Source: Administrative claims, Electronic Clinical Data : Electronic Health Record
S.26. Level of Analysis: Clinician : Individual
IF Endorsement Maintenance – Original Endorsement Date: Mar 06, 2013 Most Recent Endorsement Date: Mar 06, 2013
IF this measure is included in a composite, NQF Composite#/title:
IF this measure is paired/grouped, NQF#/title:
De.4. IF PAIRED/GROUPED, what is the reason this measure must be reported with other measures to appropriately interpret
results?
1. Evidence, Performance Gap, Priority – Importance to Measure and Report
Extent to which the specific measure focus is evidence-based, important to making significant gains in healthcare quality, and
improving health outcomes for a specific high-priority (high-impact) aspect of healthcare where there is variation in or overall lessthan-optimal performance. Measures must be judged to meet all subcriteria to pass this criterion and be evaluated against the
remaining criteria.
1a. Evidence to Support the Measure Focus – See attached Evidence Submission Form
1814_Evidence_MSF5.0_Data.doc
1b. Performance Gap
Demonstration of quality problems and opportunity for improvement, i.e., data demonstrating:
• considerable variation, or overall less-than-optimal performance, in the quality of care across providers; and/or
NATIONAL QUALITY FORUM Form version 6.5
1
•
disparities in care across population groups.
1b.1. Briefly explain the rationale for this measure (e.g., the benefits or improvements in quality envisioned by use of this measure)
Educate women about epilepsy and how its treatments may affect contraception and pregnancy. This will inform women of
childbearing potential about the risks of epilepsy and AED therapy prior to pregnancy. It will provide an opportunity to educate this
population about folic acid supplementation, monotherapyy, medication alternatives and how to obtain obstetrical, prenantal and
pregnancy care. This measure will help them understand the risk and mitigate the risks which may prevent fetal malformation,
unplanned pregnancies and improve the patients´ quality of life.
1b.2. Provide performance scores on the measure as specified (current and over time) at the specified level of analysis. (This is
required for endorsement maintenance. Include mean, std dev, min, max, interquartile range, scores by decile. Describe the data
source including number of measured entities; number of patients; dates of data; if a sample, characteristics of the entities included).
This information also will be used to address the subcriterion on improvement (4b.1) under Usability and Use.
The recent Institute of Medicine Report "Epilepsy Across the Spectrum: Promoting Health and Understanding" noted there are gaps
in care in responding to the specific needs of women with epilepsy.
"The specific knowledge needed by women with epilepsy, which may vary by age, has generally received insufficient attention.
Because sex hormones can affect seizure frequency, girls and women need information related to hormonal fluctuations and seizure
frequency. Further, women of reproductive age need to understand how their epilepsy and its treatment could affect pregnancy. In a
UK survey, adult women with epilepsy between the ages of 19 and 44 identified their most important information needs as relating
to risks of epilepsy and medication affecting the fetus (87 percent), the effect of pregnancy on seizure control (49 percent), and the
risk of their children developing epilepsy (42 percent) (Crawford and Hudson, 2003). For example, recent findings that show an
increased risk for congenital malformations and impaired cognition in children of women treated during pregnancy with valproate, a
commonly used seizure medication, suggest that all women of child-bearing age need to be kept apprised of the latest research in
this area (Harden et al., 2009). Women with epilepsy also have been found to have higher than expected rates of sexual dysfunction
(Pennell and Thompson, 2009). Among women over age 44, the most important information needs concerned epilepsy medication
and osteoporosis (63 percent), seizure medications and aging (57 percent), and seizure changes during menopause (44 percent)
(Crawford and Hudson, 2003)."
Studies have found higher-than-expected onset of seizures during the year of menarche; in girls with preexisting seizures, 29 percent
experienced more frequent seizures during perimenarche (Klein et al., 2003). Because of hormonal fluctuation, some women have a
cyclic pattern of seizure frequency associated with their menses that often is unrecognized (Pennell and Thompson, 2009).
Epidemiological research is needed in large, representative U.S. populations to monitor trends in epilepsy incidence and related
mortality and to track outcomes. Studies need to be conducted among the general population and in subpopulations at higher risk:
children, for whom prognosis is a major concern; older adults, who have greater mortality associated with epilepsy; women, to track
outcomes, including reproductive outcomes; as well as veterans and diverse racial or ethnic and socioeconomic groups, in order to
assess any disparities in incidence, prognosis, and mortality and to determine opportunities for intervention. Within these
subpopulations, sufficient numbers are needed to compare incidence by etiology, seizure type, syndrome, and the presence of
comorbid conditions. With respect to treatment, these surveillance data could be used to monitor the outcomes of epilepsy care and
provide feedback to health care providers (Box et al., 2010; Trevathan, 2011). As examples, specific populations for whom further
research is needed—older adults, veterans, children, and people with epilepsy and associated comorbidities—are described below.
Women
•
Susceptibility to changes in seizures during menstrual cycle or at other times of hormonal fluctuations (e.g., menopause)
•
Potential impact of seizures and/or medications on reproductive functioning, pregnancy, breastfeeding
•
Risk for malformations and impaired cognitive development of offspring of women taking seizure medications or suffering
seizures during pregnancy"
QUIET Indicator Study. Indicators #16 and #24 were used as support for this measure. The 2011 Pugh et al. study on quality of care
for adults with epilepsy highlights the gap in care for women’s issues with very low concordance rates and the study states that
women’s issues should be of concern for clinicians. The proportion of patients receiving quality indicator concordance care by setting
for all settings for quality indicator #16 (patients with epilepsy should receive an annual review of information including such topics
as…contraception, family planning, and how pregnancy and menopause may affect seizures) was 1.93% and for quality indicator #24
(If a woman with epilepsy is of childbearing potential and receives oral contraceptives in conjunction with an enzyme inducing AED,
2
THEN decreased effectiveness of oral contraception should be addressed.) quality indicator concordant care was provided for only
20% of the patients. Overall of the 111 women of childbearing potential with 128 opportunities for recommended care, only 36.72%
of the time quality indicator concordant care was provided.
Patient reported adherence of their physician to the quality measure (Strongly Agree, Agree; Disagree, & Strongly Disagree not noted
here)
Women with epilepsy (27% Strongly Agree, 19% Agree)
(birth control)
1b.3. If no or limited performance data on the measure as specified is reported in 1b2, then provide a summary of data from the
literature that indicates opportunity for improvement or overall less than optimal performance on the specific focus of
measurement.
Crawford, P., and S. Hudson. 2003. Understanding the information needs of women with epilepsy at different lifestages: Results of the
“Ideal World” survey. Seizure 12(7):502-507.
Wick P, Fountain N. Patient reported clinician adherence to Epilepsy Performance Measures of Quality of Care. (Before the
publication of the Quality Measures) Poster. Epilepsy Meeting Dec 2010
Gumnit R. We are Failing Our Patients: Guidelines and Quality Measures Epilepsia Accessed 09/24/12.
http://www.mincep.com/pdfs/publications/Epilepsia%20Editorial%20We%20Are%20Failing%20Our%20Patients.pdf
Fountain NB, Van Ness PC, Swain-Eng R, et al. Quality improvement in neurology: AAN epilepsy quality measures. Neurology
2011;76:94-99.
Wicks P, Massagli M, Frost J, et al. Sharing health data for better outcomes on PatientsLikeMe. J Med Internet Res 2010;12:e19.
Harden CL, Meador KJ, Pennell PB, et al., the American Academy of Neurology, and the American Epilepsy Society. 2009.
Management issues for women with epilepsy—Focus on pregnancy (an evidence-based review): II. Teratogenesis and perinatal
outcomes: Report of the Quality Standards Subcommittee and Therapeutics and Technology Subcommittee of the American
Academy of Neurology and the American Epilepsy Society. Epilepsia 50(5):1237-1246.
Pennell, P. B., and P. Thompson. 2009. Gender-specific psychosocial impact of living with epilepsy. Epilepsy and Behavior 15(Suppl.
1):S20-S25
Shafer, P. O. 1998. Counseling women with epilepsy. Epilepsia 39(Suppl. 8):S38-S44.
2009. Epilepsy self-management in clinical practice: What we do and know. Paper read at AES Annual
Meeting, Boston, MA: Hynes Conference Center.
Institute of Medicine Report "Epilepsy Across the Spectrum: Promoting Health and Understanding"
http://www.iom.edu/Reports/2012/Epilepsy-Across-the-Spectrum.aspx
Pugh MJ, Berlowitz DR, Rao JK, et al. The quality of care for adults with epilepsy: an initial glimpse using the QUIET measure. 2011;
BMC, Health Services Research; 11:1 www.biomedcentral.com/1472-6963/11/1
1b.4. Provide disparities data from the measure as specified (current and over time) by population group, e.g., by race/ethnicity,
gender, age, insurance status, socioeconomic status, and/or disability. (This is required for endorsement maintenance. Describe the
data source including number of measured entities; number of patients; dates of data; if a sample, characteristics of the entities
include.) This information also will be used to address the subcriterion on improvement (4b.1) under Usability and Use.
Racial/ethnic minorities also appear to have limited knowledge about epilepsy and its treatment, experience barriers to care, lack
social support, and seek alternative therapies for epilepsy. (Szaflarski)
Specific risks for women with epilepsy:
Susceptibility to changes in seizures during menstrual cycle or at other times of hormonal fluctuations (e.g., menopause) Potential
3
impact of seizures and/or medications on reproductive functioning, pregnancy, breastfeeding
Risk for malformations and impaired cognitive development of offspring of women taking seizure medications or suffering seizures
during pregnancy
A study in the Harlem neighborhood of New York City found epilepsy prevalence to be higher in Hispanics than in non-Hispanics and
a higher prevalence of active epilepsy3 in whites than in blacks, although the prevalence of lifetime epilepsy4 was higher in blacks
compared to whites (Kelvin et al., 2007). In this community, there were racial and ethnic disparities in care; blacks were more likely to
receive care in the emergency department compared to whites and Hispanics. Similarly, Hope and colleagues (2009) found that
blacks and Hispanics were more likely than whites to be diagnosed in an emergency department, and blacks were more likely to
receive a suboptimal seizure medication. Differences in care for prevalent epilepsy were also observed in residents of Alabama and
surrounding states, where blacks were 60 percent less likely than non-Hispanic whites to undergo epilepsy surgery after receiving
electroencephalograph (EEG) monitoring as part of a surgical evaluation, an association that persisted after controlling for factors
such as SES and medical insurance coverage (Burneo et al.,2005). The degree to which differences in epilepsy incidence and
prevalence in different racial and ethnic groups reflect differences in socioeconomic status is unknown. Also unknown is the degree
to which the treatment gap contributes to the higher epilepsy prevalence in some subgroups.
There are some patient reported differences in physician adherence to the quality measures by the type of clinician. N=221 overall.
Women of childbearing N=61
Women with epilepsy (birth control): 67% adherence (epileptologist); 74% adherence (neurologist); 75% adherence (PCP); 0
responses (Other clinician); p-value 0.862
1b.5. If no or limited data on disparities from the measure as specified is reported in 1b4, then provide a summary of data from
the literature that addresses disparities in care on the specific focus of measurement. Include citations.
Szaflarski M, Szaflarski JP, Privitera MD et al. Racial/ethnic disparities in the treatment of epilepsy: What do we know? What do we
need to know? Epilepsy & Behavior 2006; 9(2): 243-64.
Wick P, Fountain N. Patient reported clinician adherence to Epilepsy Performance Measures of Quality of Care. (Before the
publication of the Quality Measures) Poster. Epilepsy Meeting Dec 2010.
http://www.iom.edu/Reports/2012/Epilepsy-Across-the-Spectrum.aspx
1c. High Priority (previously referred to as High Impact)
The measure addresses:
• a specific national health goal/priority identified by DHHS or the National Priorities Partnership convened by NQF;
OR
• a demonstrated high-priority (high-impact) aspect of healthcare (e.g., affects large numbers of patients and/or has a
substantial impact for a smaller population; leading cause of morbidity/mortality; high resource use (current and/or
future); severity of illness; and severity of patient/societal consequences of poor quality).
1c.1. Demonstrated high priority aspect of healthcare
Affects large numbers, A leading cause of morbidity/mortality, Patient/societal consequences of poor quality, Severity of illness
1c.2. If Other:
1c.3. Provide epidemiologic or resource use data that demonstrates the measure addresses a high priority aspect of healthcare.
List citations in 1c.4.
Recent estimates of the US population and prevalence of epilepsy indicate that approximately one-half million women with epilepsy
(WWE) are of childbearing age. It has also been estimated that 3-5 births per thousand will be to WWE. Epilepsy is defined by the
presence of recurrent, unprovoked seizures, and the treatment is typically a daily, long-term antiepileptic drugs (AED) regiment. The
majority of people with epilepsy have well-controlled seizures, are otherwise healthy, and therefore expect to participate fully in life
experiences, including childbearing. Epilepsy is associated with sexual dysfunction, reduced fertility, increased pregnancy risks, and
risks for malformations in the infant. Seizures can transiently disrupt pituitary hormone secretion. Treatment of seizures with
antiepileptic drugs may alter hormone levels, render oral contraceptives less effective and may interfere with embryonic and fetal
development. Certain antiepileptic mediations may have specific malformation risks. Since unplanned pregnancy is common,
4
patients need to be informed about the risks of epilepsy and antiepileptic drug therapy prior to pregnancy. Folic acid
supplementation, monotherapy for epilepsy, using lower doses of medication when possible and proper obstetrical, prenatal and
pre-pregnancy care all should be discussed with the patient so they understand the risks involved and how to mitigate these risks.
The specific knowledge needed by women with epilepsy, which may vary by age, has generally received insufficient attention.
Because sex hormones can affect seizure frequency, girls and women need information related to hormonal fluctuations and seizure
frequency. Studies have found higher-than-expected onset of seizures during the year of menarche; in girls with preexisting seizures,
29 percent experienced more frequent seizures during perimenarche (Klein et al., 2003). Because of hormonal fluctuation, some
women have a cyclic pattern of seizure frequency associated with their menses that often is unrecognized (Pennell and Thompson,
2009).
"Our suspicions have been confirmed: epilepsy affects women differently. Their hormonal and menstrual cycles, pregnancy,
menopause—all of those life stages are affected by epilepsy," said Edna Kane-Williams, vice president of programs and services for
the Epilepsy Foundation. Furthermore, Ms. Kane-Williams said, many medical professionals seem to be in the dark. "We´ve done a
professional awareness survey that showed that the physicians these women were seeing weren´t aware of the differences," she
reported. When women with epilepsy have problems, they are often hormone-based, according to Dr. Mark Yerby, founder of North
Pacific Epilepsy Research in Portland, Oregon, and a nationally recognized authority on the subject.
Risks from seizures and from anti-epileptic drugs
Both seizures and medications are associated with some risks. The risk of seizures is associated with seizure type. Partial seizures
probably do not carry as much risk but they may become generalized seizures, and generalized tonic-clonic seizures are associated
with increased risk to both the mother and baby. These risks include trauma from falls or burns, increased risk of premature labor,
miscarriages, and fetal heart rate suppression. Seizure control is necessary because the risks from seizures are felt by epileptologists
to be greater than the risks from medications, which may be minimized by utilizing specific strategies.
Strategies to minimize risks
Most importantly, women should get accurate information prior to and during pregnancy. If anti-epileptic drugs are not needed,
multiple medications are being taken, or medications are given at high dosages, changes should be considered with a neurologist
prior to a planned pregnancy. The lowest possible anti-epileptic drug dose that will continue to maintain seizure control is
recommended. Being on a single drug, monotherapy, will decrease the risk of birth defects and result in fewer drug interactions,
fewer side effects, and improve compliance.
The 2012 IOM report “Epilepsy Across the Spectrum” explicitly stated the need for the development and implementation of a
national quality measurement and improvement strategy for epilepsy care. “An independent organization with expertise in quality
measurement and care should assist in the development of the national strategy, particularly the development of performance
metrics.” Specifically, the IOM report calls for the national quality strategy to include defining performance metrics for epilepsy with
specific attention to access to care for underserved populations, access to specialized care, co-management of care among
specialized epilepsy providers, and coordination of care with other health care providers and community services organizations.
The AAN is a non-profit professional association with extensive experience and expertise in developing quality measures for
neurological conditions and has developed eight quality measures for epilepsy care. The AAN has not yet completed testing of these
measures. Three of the epilepsy measures were chosen for inclusion in the 2012 PQRS program and thus are under consideration for
endorsement by the NQF at this time.
The incidence in females, at 41 cases per 100,000 person years, is less than that for males, at 49 cases per 100,000 person years.[2]
Approximately 1 million women of childbearing age in the United States have seizure disorders. Of these women, approximately
20,000 give birth each year. Concerns during these pregnancies include the risk of fetal malformation, miscarriage, perinatal death,
and increased seizure frequency.[1]
In women who are pregnant, the volume of distribution and the hepatic metabolism of AEDs are increased. This, along with
decreased compliance with AEDs because of concerns about their effects on the fetus, leads to an increase in seizure frequency,
which is observed in as many as 17-33% of pregnancies.
The use of antiepileptic drugs (AEDs) is associated with a greater baseline risk of fetal malformations during pregnancy. When
treating pregnant women who have epilepsy, the risks of increased seizure frequency versus the risks of AED use must be weighed
carefully.
A population-based study conducted in Norway found that pregnant women with epilepsy had a lower risk of complications but an
5
increased risk of induction, cesarean delivery, and postpartum hemorrhage.[2] However, whether this is a result of AEDs or severe
epilepsy is unclear.
1c.4. Citations for data demonstrating high priority provided in 1a.3
United States Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health
Statistics, Bridged-Race Population Estimates, United States. July 1st resident population by state, country, age, sex, bridged-race, and
Hispanic origin on CDC WONDER on-line Database. Available at: http://wonder/cdc.gov/ Accessed June 2012.
Hirtz D, Thurman DJ, Gwinn-Hardy K, et al. How common are the “common” neurological disorders? Neurology 2007; 68:326-337.
Yerby MS. Quality of life, epilepsy advances, and the evolving role of anticonvulsants in women with epilepsy. Neurology 2000;
55:S21-31.
Harden CL, Hopp J, Tin TY, et al. Practice parameter update: Management issues for women with epilepsy-Focus on pregnancy (an
evidence-based review): Obstetrical complications and change in seizure frequency: Report of the Quality Standards Subcommittee
and Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology and American Epilepsy Society.
Neurology 2009; 73:126-132.
Practice parameter: management issues for women with epilepsy (summary statement). Report of the Quality Standards
Subcommittee of the American Academy of Neurology. Neurology. 51(4):944-8, 1998
Pennell PB. The importance of monotherapy in pregnancy. Neurology. 60(11 Suppl 4):S31-8, 2003
Yerby MS. Management issues for women with epilepsy: neural tube defects and folic acid supplementation. Neurology. 61(6 Suppl
2):S23-6, 2003
Crawford, P., and S. Hudson. 2003. Understanding the information needs of women with epilepsy at different lifestages: Results of the
“Ideal World” survey. Seizure 12(7):502-507.
Pennell, P. B., and P. Thompson. 2009. Gender-specific psychosocial impact of living with epilepsy. Epilepsy and Behavior 15(Suppl.
1):S20-S25
Shafer, P. O. Counseling women with epilepsy. Epilepsia 1998; 39(Suppl. 8):S38-S44.
2009. Epilepsy self-management in clinical practice: What we do and know. Paper read at AES Annual Meeting, Boston, MA: Hynes
Conference Center.
Cramer JA, Gordon J, Schachter S, Devinsky O, and the Epilepsy Therapy Development Project Women’s Issues Work Group. Women
with Epilepsy: Hormonal Issues from Menarche through Menopause. Epilepsy Behav. 2007; 11: 160-178
Epilepsy Therapy Project
http://www.epilepsy.com/INFO/WOMEN_PREGNANCY
http:www.iom.edu/Reports/2012/Epilepsy-Across-the-Spectrum.aspx
1.
Katz O, Levy A, Wiznitzer A, Sheiner E. Pregnancy and perinatal outcome in epileptic women: a population-based study. J
Matern Fetal Neonatal Med. Jan 2006;19(1):21-5.
2.
Borthen I, Eide MG, Daltveit AK, Gilhus NE. Delivery outcome of women with epilepsy: a population-based cohort study.
BJOG. Nov 2010;117(12):1537-43
1c.5. If a PRO-PM (e.g. HRQoL/functional status, symptom/burden, experience with care, health-related behaviors), provide
evidence that the target population values the measured PRO and finds it meaningful. (Describe how and from whom their input
was obtained.)
6
2. Reliability and Validity—Scientific Acceptability of Measure Properties
Extent to which the measure, as specified, produces consistent (reliable) and credible (valid) results about the quality of care when
implemented. Measures must be judged to meet the subcriteria for both reliability and validity to pass this criterion and be
evaluated against the remaining criteria.
2a.1. Specifications The measure is well defined and precisely specified so it can be implemented consistently within and across
organizations and allows for comparability. eMeasures should be specified in the Health Quality Measures Format (HQMF) and the
Quality Data Model (QDM).
De.5. Subject/Topic Area (check all the areas that apply):
Neurology, Perinatal and Reproductive Health, Prevention
De.6. Cross Cutting Areas (check all the areas that apply):
Patient and Family Engagement, Prevention, Safety, Safety : Medication Safety
S.1. Measure-specific Web Page (Provide a URL link to a web page specific for this measure that contains current detailed
specifications including code lists, risk model details, and supplemental materials. Do not enter a URL linking to a home page or to
general information.)
http://www.aan.com/globals/axon/assets/9079.pdf
S.2a. If this is an eMeasure, HQMF specifications must be attached. Attach the output from the eMeasure authoring tool (MAT) - if
the MAT was not used, contact staff. (Use the specification fields in this online form for the plain-language description of the
specifications)
Attachment:
S.2b. Data Dictionary, Code Table, or Value Sets (and risk model codes and coefficients when applicable) must be attached. (Excel or
csv file in the suggested format preferred - if not, contact staff)
Attachment:
S.3. For endorsement maintenance, please briefly describe any changes to the measure specifications since last endorsement date
and explain the reasons.
S.4. Numerator Statement (Brief, narrative description of the measure focus or what is being measured about the target population,
i.e., cases from the target population with the target process, condition, event, or outcome)
IF an OUTCOME MEASURE, state the outcome being measured. Calculation of the risk-adjusted outcome should be described in the
calculation algorithm.
Female patients counseled about epilepsy and how its treatment may affect contraception and pregnancy and documented in the
medical record at least once a year.
S.5. Time Period for Data (What is the time period in which data will be aggregated for the measure, e.g., 12 mo, 3 years, look back
to August for flu vaccination? Note if there are different time periods for the numerator and denominator.)
Within a 12 month reporting period
S.6. Numerator Details (All information required to identify and calculate the cases from the target population with the target
process, condition, event, or outcome such as definitions, specific data collection items/responses, code/value sets – Note: lists of
individual codes with descriptors that exceed 1 page should be provided in an Excel or csv file in required format at S.2b)
IF an OUTCOME MEASURE, describe how the observed outcome is identified/counted. Calculation of the risk-adjusted outcome
should be described in the calculation algorithm.
CPT II 4340F: Counseling for women of childbearing
potential with epilepsy
S.7. Denominator Statement (Brief, narrative description of the target population being measured)
All females of childbearing potential (12-44 years old) with a diagnosis of epilepsy.
7
S.8. Target Population Category (Check all the populations for which the measure is specified and tested if any):
Senior Care
S.9. Denominator Details (All information required to identify and calculate the target population/denominator such as definitions,
specific data collection items/responses , code/value sets – Note: lists of individual codes with descriptors that exceed 1 page should
be provided in an Excel or csv file in required format at S.2b)
Epilepsy ICD-9-CM diagnosis codes
n 345.00, 345.01, 345.10, 345.11, 345.40. 345.41, 345.50, 345.51,
345.60, 345.61, 345.70, 345.71, 345.90, 345.91
AND
CPT codes
99201, 99202, 99203, 99204, 99205
99212, 99213, 99214, 99215
99304, 99305, 99306, 99307, 99308, 99309
S.10. Denominator Exclusions (Brief narrative description of exclusions from the target population)
Patient is surgically sterile, Patient has an intellectual disability.
S.11. Denominator Exclusion Details (All information required to identify and calculate exclusions from the denominator such as
definitions, specific data collection items/responses, code/value sets – Note: lists of individual codes with descriptors that exceed 1
page should be provided in an Excel or csv file in required format at S.2b)
CPT II 4340F–1P: Documentation of medical reason(s) why
counseling was not performed for women of childbearing
potential with epilepsy.
CPT II 4340F–8P: Counseling about epilepsy specific safety
issues provided to patient or caregiver was not performed,
reason not otherwise specified.
S.12. Stratification Details/Variables (All information required to stratify the measure results including the stratification variables,
definitions, specific data collection items/responses, code/value sets – Note: lists of individual codes with descriptors that exceed 1
page should be provided in an Excel or csv file in required format with at S.2b)
S.13. Risk Adjustment Type (Select type. Provide specifications for risk stratification in S.12 and for statistical model in S.14-15)
No risk adjustment or risk stratification
If other:
S.14. Identify the statistical risk model method and variables (Name the statistical method - e.g., logistic regression and list all the
risk factor variables. Note - risk model development and testing should be addressed with measure testing under Scientific
Acceptability)
N/A
S.15. Detailed risk model specifications (must be in attached data dictionary/code list Excel or csv file. Also indicate if available at
measure-specific URL identified in S.1.)
Note: Risk model details (including coefficients, equations, codes with descriptors, definitions), should be provided on a separate
worksheet in the suggested format in the Excel or csv file with data dictionary/code lists at S.2b.
S.15a. Detailed risk model specifications (if not provided in excel or csv file at S.2b)
N/A
S.16. Type of score:
Rate/proportion
If other:
S.17. Interpretation of Score (Classifies interpretation of score according to whether better quality is associated with a higher score,
8
a lower score, a score falling within a defined interval, or a passing score)
Better quality = Higher score
S.18. Calculation Algorithm/Measure Logic (Describe the calculation of the measure score as an ordered sequence of steps including
identifying the target population; exclusions; cases meeting the target process, condition, event, or outcome; aggregating data; risk
adjustment; etc.)
See Full AAN Women with Epilepsy of Childbearing Potential Measure Testing Report. MNCM completed validation of the data in a
three-step process: 1) denominator certification, 2) data file quality checks, and 3) validation audit. Details of this validation are
described in this report.
S.19. Calculation Algorithm/Measure Logic Diagram URL or Attachment (You also may provide a diagram of the Calculation
Algorithm/Measure Logic described above at measure-specific Web page URL identified in S.1 OR in attached appendix at A.1)
S.20. Sampling (If measure is based on a sample, provide instructions for obtaining the sample and guidance on minimum sample
size.)
IF a PRO-PM, identify whether (and how) proxy responses are allowed.
See Full AAN Women with Epilepsy of Childbearing Potential Measure Testing Report. To achieve the goal of testing the Counseling
for Women with Epilepsy measure the AAN contracted with Minnesota Community Measurement (MNCM), a non-profit
organization specializing in health care quality measurement and reporting, to collect data pertaining to this measure from
Neurology practices. In order to achieve a reliable sample of patients for this measure MNCM and AAN are sought a minimum of
1,000 combined patient records from the Neurology sites that agree to participate. The AAN identified and recruited Neurology
practices in Minnesota that have experience treating patients with epilepsy. Three Neurology practices volunteered to participate
and submit retrospective data from the 2012 calendar year (i.e., dates of service 01/01/2012 – 12/31/2012). MNCM produced a
data collection guide, measure flow and detailed file specifications to educate and assist each medical group in the data collection
and submission process. As a requirement of participating in the measure testing each group had to submit a denominator
certifications form (see appendix A of the final report). The denominator certification process helps ensure that each medical group
is using the appropriate measure parameters and collecting data in a standardized way. Once the denominator certifications were
complete each group submitted their data files using a secure FTP transfer process. Once the files were received MNCM performed
quality checks on each files using the methods outlined in Appendix B of the final report. Once the files passed the quality checks
MNCM calculated and sent the results back to each group for review. If the group did not find any issues with the measure results
then MNCM conducted an audit to validate the accuracy of data using the auditing principles outlined in Appendix C of the final
report. The final step in testing the Women with Epilepsy measure was a post-data submission survey that was sent to each of the
participating medical groups. The intent of the survey was to shed light on the amount of resources that were required to produce
the data as well as gauge the level of data collection burden that each data field presented to the group. The results of the survey
can be found in Appendix E of the final report. The following tables are excerpted from the AAN’s report.
S.21. Survey/Patient-reported data (If measure is based on a survey, provide instructions for conducting the survey and guidance on
minimum response rate.)
IF a PRO-PM, specify calculation of response rates to be reported with performance measure results.
S.22. Missing data (specify how missing data are handled, e.g., imputation, delete case.)
Required for Composites and PRO-PMs.
S.23. Data Source (Check ONLY the sources for which the measure is SPECIFIED AND TESTED).
If other, please describe in S.24.
Administrative claims, Electronic Clinical Data : Electronic Health Record
S.24. Data Source or Collection Instrument (Identify the specific data source/data collection instrument e.g. name of database,
clinical registry, collection instrument, etc.)
IF a PRO-PM, identify the specific PROM(s); and standard methods, modes, and languages of administration.
Counseling for Women of Childbearing Potential with Epilepsy Collection Sheet from the AMA PCPI www.physicianconsortium.org
NeuroPI Program: for maintenance of certification Performance in Practice module. www.aan.com/practice/pip
9
CECity PQRS Wizard and registry (in development)
Physician Quality Reporting System measurement set
Patients Like Me website: patient report of physician use of the quality measure
S.25. Data Source or Collection Instrument (available at measure-specific Web page URL identified in S.1 OR in attached appendix at
A.1)
URL
S.26. Level of Analysis (Check ONLY the levels of analysis for which the measure is SPECIFIED AND TESTED)
Clinician : Individual
S.27. Care Setting (Check ONLY the settings for which the measure is SPECIFIED AND TESTED)
Ambulatory Care : Clinician Office/Clinic
If other:
S.28. COMPOSITE Performance Measure - Additional Specifications (Use this section as needed for aggregation and weighting rules,
or calculation of individual performance measures if not individually endorsed.)
2a. Reliability – See attached Measure Testing Submission Form
2b. Validity – See attached Measure Testing Submission Form
1814_MeasSubm_MeasTesting_2014_2_27.docx
3. Feasibility
Extent to which the specifications including measure logic, require data that are readily available or could be captured without undue
burden and can be implemented for performance measurement.
3a. Byproduct of Care Processes
For clinical measures, the required data elements are routinely generated and used during care delivery (e.g., blood pressure, lab
test, diagnosis, medication order).
3a.1. Data Elements Generated as Byproduct of Care Processes.
generated by and used by healthcare personnel during the provision of care, e.g., blood pressure, lab value, medical condition, Coded
by someone other than person obtaining original information (e.g., DRG, ICD-9 codes on claims), Abstracted from a record by
someone other than person obtaining original information (e.g., chart abstraction for quality measure or registry)
If other:
3b. Electronic Sources
The required data elements are available in electronic health records or other electronic sources. If the required data are not in
electronic health records or existing electronic sources, a credible, near-term path to electronic collection is specified.
3b.1. To what extent are the specified data elements available electronically in defined fields? (i.e., data elements that are needed
to compute the performance measure score are in defined, computer-readable fields)
No data elements are in defined fields in electronic sources
3b.2. If ALL the data elements needed to compute the performance measure score are not from electronic sources, specify a
credible, near-term path to electronic capture, OR provide a rationale for using other than electronic sources.
Currently, this measure has been specified for administrative claims. The AAN has contracted with two separate consultants to learn
the process to develop eSpecifications, code value sets, logic, and develop eMeasures. The training was complete as of 9/25/12 and
the measures will be full specified for eMeasures by December 2012
3b.3. If this is an eMeasure, provide a summary of the feasibility assessment in an attached file or make available at a measurespecific URL.
Attachment:
10
3c. Data Collection Strategy
Demonstration that the data collection strategy (e.g., source, timing, frequency, sampling, patient confidentiality, costs
associated with fees/licensing of proprietary measures) can be implemented (e.g., already in operational use, or testing
demonstrates that it is ready to put into operational use). For eMeasures, a feasibility assessment addresses the data elements
and measure logic and demonstrates the eMeasure can be implemented or feasibility concerns can be adequately addressed.
3c.1. Describe what you have learned/modified as a result of testing and/or operational use of the measure regarding data
collection, availability of data, missing data, timing and frequency of data collection, sampling, patient confidentiality, time and
cost of data collection, other feasibility/implementation issues.
IF a PRO-PM, consider implications for both individuals providing PROM data (patients, service recipients, respondents) and those
whose performance is being measured.
During the analysis of this data and also as a byproduct of the validation audit in reviewing medical records, MNCM staff has some
concerns regarding the denominator and intent of the measure. There may need to be a consideration for adding a component of
indicating that the patient is sexually active or has the potential to be sexually active, and not physically handicapped. AAN could
refer to the NCQA specifications for the Chlamydia Screening in Women measure (NQF# 0033/ CMS 153v1) for reference tables
indicating how to identify potentially sexually active women via pharmacy codes, CPTs, ICD-9, UB Revenue and LOINC codes. Rather
than trying to capture/ code every possible exclusion; this may be an option. MNCM would not recommend having a general type
exclusions code, like one that is stated as “any documented medical reason”, because providers will use this to their advantage and
exclude patients that are at risk for pregnancy and truly belong in the denominator. Having this type of exclusion weakens the
measure, and can impact the validity and reliability of the results.
3c.2. Describe any fees, licensing, or other requirements to use any aspect of the measure as specified (e.g., value/code set, risk
model, programming code, algorithm).
4. Usability and Use
Extent to which potential audiences (e.g., consumers, purchasers, providers, policy makers) are using or could use performance
results for both accountability and performance improvement to achieve the goal of high-quality, efficient healthcare for individuals
or populations.
4a. Accountability and Transparency
Performance results are used in at least one accountability application within three years after initial endorsement and are
publicly reported within six years after initial endorsement (or the data on performance results are available). If not in use at the
time of initial endorsement, then a credible plan for implementation within the specified timeframes is provided.
4.1. Current and Planned Use
NQF-endorsed measures are expected to be used in at least one accountability application within 3 years and publicly reported within
6 years of initial endorsement in addition to performance improvement.
Planned
Current Use (for current use provide URL)
Public Reporting
Payment Program
Regulatory and Accreditation Programs
Professional Certification or Recognition
Program
Quality Improvement (Internal to the
specific organization)
4a.1. For each CURRENT use, checked above, provide:
11
• Name of program and sponsor
• Purpose
• Geographic area and number and percentage of accountable entities and patients included
NeuroPI MOC program http://tools.aan.com/practice/pip/
4a.2. If not currently publicly reported OR used in at least one other accountability application (e.g., payment program,
certification, licensing) what are the reasons? (e.g., Do policies or actions of the developer/steward or accountable entities restrict
access to performance results or impede implementation?)
4a.3. If not currently publicly reported OR used in at least one other accountability application, provide a credible plan for
implementation within the expected timeframes -- any accountability application within 3 years and publicly reported within 6
years of initial endorsement. (Credible plan includes the specific program, purpose, intended audience, and timeline for
implementing the measure within the specified timeframes. A plan for accountability applications addresses mechanisms for data
aggregation and reporting.)
4b. Improvement
Progress toward achieving the goal of high-quality, efficient healthcare for individuals or populations is demonstrated. If not in
use for performance improvement at the time of initial endorsement, then a credible rationale describes how the performance
results could be used to further the goal of high-quality, efficient healthcare for individuals or populations.
4b.1. Progress on Improvement. (Not required for initial endorsement unless available.)
Performance results on this measure (current and over time) should be provided in 1b.2 and 1b.4. Discuss:
• Progress (trends in performance results, number and percentage of people receiving high-quality healthcare)
• Geographic area and number and percentage of accountable entities and patients included
We have yet to demonstrate improvement until it can be implemented in programming. The primary reason this measure has not
been integrated into more programming is because it isn’t NQF endorsed.
4b.2. If no improvement was demonstrated, what are the reasons? If not in use for performance improvement at the time of
initial endorsement, provide a credible rationale that describes how the performance results could be used to further the goal of
high-quality, efficient healthcare for individuals or populations.
4c. Unintended Consequences
The benefits of the performance measure in facilitating progress toward achieving high-quality, efficient healthcare for
individuals or populations outweigh evidence of unintended negative consequences to individuals or populations (if such
evidence exists).
4c.1. Were any unintended negative consequences to individuals or populations identified during testing; OR has evidence of
unintended negative consequences to individuals or populations been reported since implementation? If so, identify the negative
unintended consequences and describe how benefits outweigh them or actions taken to mitigate them.
Testing has not begun yet but will be completed by January 2014. Strategies to prevent, minimize or detect unintended
consequences will be identified during testing in 2013. Operational use of this measure has not identified any inaccuracies, errors or
unintended consequences of measurement
5. Comparison to Related or Competing Measures
If a measure meets the above criteria and there are endorsed or new related measures (either the same measure focus or the same
target population) or competing measures (both the same measure focus and the same target population), the measures are
compared to address harmonization and/or selection of the best measure.
5. Relation to Other NQF-endorsed Measures
Are there related measures (conceptually, either same measure focus or target population) or competing measures (conceptually
both the same measure focus and same target population)? If yes, list the NQF # and title of all related and/or competing measures.
12
5.1a. List of related or competing measures (selected from NQF-endorsed measures)
5.1b. If related or competing measures are not NQF endorsed please indicate measure title and steward.
5a. Harmonization
The measure specifications are harmonized with related measures;
OR
The differences in specifications are justified
5a.1. If this measure conceptually addresses EITHER the same measure focus OR the same target population as NQF-endorsed
measure(s):
Are the measure specifications completely harmonized?
5a.2. If the measure specifications are not completely harmonized, identify the differences, rationale, and impact on
interpretability and data collection burden.
5b. Competing Measures
The measure is superior to competing measures (e.g., is a more valid or efficient way to measure);
OR
Multiple measures are justified.
5b.1. If this measure conceptually addresses both the same measure focus and the same target population as NQF-endorsed
measure(s):
Describe why this measure is superior to competing measures (e.g., a more valid or efficient way to measure quality); OR provide
a rationale for the additive value of endorsing an additional measure. (Provide analyses when possible.)
Appendix
A.1 Supplemental materials may be provided in an appendix. All supplemental materials (such as data collection instrument or
methodology reports) should be organized in one file with a table of contents or bookmarks. If material pertains to a specific
submission form number, that should be indicated. Requested information should be provided in the submission form and required
attachments. There is no guarantee that supplemental materials will be reviewed.
Attachment:
Contact Information
Co.1 Measure Steward (Intellectual Property Owner): American Academy of Neurology
Co.2 Point of Contact: Rebecca, Swain-Eng, [email protected], 612-928-6121Co.3 Measure Developer if different from Measure Steward: American Academy of Neurology
Co.4 Point of Contact: Rebecca, Swain-Eng, [email protected], 612-928-6121-
Additional Information
Ad.1 Workgroup/Expert Panel involved in measure development
Provide a list of sponsoring organizations and workgroup/panel members’ names and organizations. Describe the members’ role
in measure development.
Co-chairs : Nathan Fountain, MD; Paul Van Ness, MD
American Academy of Neurology Facilitator
Christopher Bever Jr., MD
13
American Academy of Neurology
Jeffrey Buchhalter, MD
Andres Kanner, MD
K. Babu Krishnamurthy, MD
Susan Naselli, MD
Piotr Olejniczak, MD
Rita Richardson, MD
Joseph Sirven, MD
Michael Sperling, MD
John Stern, MD
American Epilepsy Society
Allan Krumholz, MD
Paul Levisohn, MD
Epilepsy Foundation of America
Gregory L. Barkley, MD, FAAN
Michael C. Smith, MD
National Association of Epilepsy Centers
David Labiner, MD
Thaddeus Walczak, MD
American Academy of Family Physicians
Mark Potter, MD
American Academy of Pediatrics
Dennis Dlugos, MD
American Academy of Neurological Surgeons/Congress of Neurological Surgeons
Joshua Rosenow, MD
American Clinical Neurophysiology Society
William Tatum IV, DO
American College of Emergency Physicians
Andrew Jagoda, MD
American College of Radiology
Eric Russell, MD
American Psychological Association
Bruce Hermann, PhD
American Society of Neuroimaging
Ruben Kuzniecky, MD
Child Neurology Society
Kevin Chapman, MD
National Academy of Neuropsychology
Gregory Lee, PhD
National Organization of Rare Disorders
Suki Bagal, MD, MPH
14
Society of Nuclear Medicine
James M. Mountz, MD, PhD
American Academy of Neurology Staff
Rebecca Swain-Eng, MS
Sarah Tonn, MPH
Gina Gjorvad
American Medical Association Convened-Physician Consortium for Performance Improvement
Mark Antman, DDS, MBA
Consultants
Rebecca Kresowik
Timothy Kresowik, MD
Insurance Representatives
Kay Schwebke, MD, UnitedHealth Care
Wesley Wong, MD, Wellpoint
Thomas James, MD, Humana
Robert Kropp, MD, Aetna
Measure Developer/Steward Updates and Ongoing Maintenance
Ad.2 Year the measure was first released: 2009
Ad.3 Month and Year of most recent revision: 01, 2010
Ad.4 What is your frequency for review/update of this measure? Every 3 years
Ad.5 When is the next scheduled review/update for this measure? 01, 2013
Ad.6 Copyright statement: ©2009 American Academy of Neurology. All rights reserved.
Physician Performance Measures (measures) and related data specifications developed by the American Academy of Neurology
(AAN) are intended to facilitate quality improvement activities by physicians.
These measures are intended to assist physicians in enhancing quality of care. Measures are designed for use by any physician who
manages the care of a patient for a specific condition or for prevention. These measures are not clinical guidelines and do not
establish a standard of medical care, and have not been tested for all potential applications.
Measures are subject to review and may be revised or rescinded at any time by the AAN. The measures may not be altered without
prior written approval from the AAN. The measures, while copyrighted, can be reproduced and distributed, without modification, for
noncommercial purposes (e.g. use by health care providers in connection with their practices). Commercial use is defined as the
sale, license, or distribution of the measures for commercial gain, or incorporation of the measures into a product or service that is
sold, licensed, or distributed for commercial gain. Commercial uses of the measures require a license agreement between the user
and the AAN. Neither the AAN nor its members shall be responsible for any use of the measures.
Ad.7 Disclaimers: THESE MEASURES AND SPECIFICATIONS ARE PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND.
Limited proprietary coding is contained in the measure specifications for convenience. Users of the proprietary coding sets should
obtain all necessary licenses from the owners of these code sets. The AAN and its members disclaim all liability for use or accuracy of
any Current Procedural Terminology (CPT®) or other coding contained in the specifications.
Ad.8 Additional Information/Comments: CPT ® is a registered trademark of the American Medical Association.
15
NQF #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated Date: Mar 20, 2013
NATIONAL QUALITY FORUM
Measure missing data in MSF 6.5 from MSF 5.0
NQF #: 1814
NQF Project: Neurology Project
1. IMPACT, OPPORTUITY, EVIDENCE - IMPORTANCE TO MEASURE AND REPORT
Importance to Measure and Report is a threshold criterion that must be met in order to recommend a measure for endorsement. All
three subcriteria must be met to pass this criterion. See guidance on evidence.
Measures must be judged to be important to measure and report in order to be evaluated against the remaining criteria.
(evaluation criteria)
1c.1 Structure-Process-Outcome Relationship (Briefly state the measure focus, e.g., health outcome, intermediate clinical
outcome, process, structure; then identify the appropriate links, e.g., structure-process-health outcome; process- health outcome;
intermediate clinical outcome-health outcome):
By counseling female patients about epilepsy and how its treatment may affect contraception the expected patient outcomes
include decreased contraception failures due to AEDs use that result in an unplanned pregnancy, decreased embryonic or fetal
malformation, improved patient quality of life, better seizure management, appropriate medication and treatments prescribed, and
the patient will understand the risks involved with AEDs, contraception and pregnancy. In addition the patient will know how to
mitigate these risks.
The American Academy of Neurology has preliminary data from the implementation of these measures into the Maintenance of
Certification Performance in Practice (NeuroPI) Epilepsy Module. There have been 291 physicians to date who have enrolled in
Epilepsy module. However, the extrapolation of data from this module is not yet appropriate as the sample size is believed to be
too small to be able to provide generalizable data. However, by the time this measure comes back to the NQF for the end of the
Temporary Endorsement period (estimated by 1/2014) there were will be additional data available to support the link of this
measure to the desired patient outcomes. In addition, we will have some data back from the CECity registry database, which just
went live in August 2012, by 1/2014 to add additional support to this measure.
1c.2-3 Type of Evidence (Check all that apply):
Clinical Practice Guideline
Other
Quality Indicator Paper
1c.4 Directness of Evidence to the Specified Measure (State the central topic, population, and outcomes addressed in the body
of evidence and identify any differences from the measure focus and measure target population):
Topic: Epilepsy
Population: Women with epilepsy of childbearing age (identified as 12-44 years old)
Outcomes addressed in the body of evidence: counseling that leads to appropriate education of women with epilepsy of
childbearing potential about anti-epileptic drugs (AEDs) and their effect on contraception, pregnancy and fetal malformations.
Expected outcome would be empowered female patients who are well educated and able to make appropriate decisions on
epilepsy and its treatment. This would indirectly relate to decreased fetal malformations and appropriate seizure management.
Differences from the measure focus and the measure target population: not applicable
1c.5 Quantity of Studies in the Body of Evidence (Total number of studies, not articles): 5 recommendation
statements/indicators were used as the basis for this quality measure.
1c.6 Quality of Body of Evidence (Summarize the certainty or confidence in the estimates of benefits and harms to patients
across studies in the body of evidence resulting from study factors. Please address: a) study design/flaws; b)
See Guidance for Definitions of Rating Scale: H=High; M=Moderate; L=Low; I=Insufficient; NA=Not Applicable
Created on: 06/18/2013 at 02:57 AM
1
directness/indirectness of the evidence to this measure (e.g., interventions, comparisons, outcomes assessed, population included
in the evidence); and c) imprecision/wide confidence intervals due to few patients or events): The guideline/indicator authors did
not provide an explicit process or documentation of a process like GRADE whereby precision, directness, etc were detailed in a
systematic review to demonstrate the quality of the body of evidence for this measure. The available information from the
guideline/indicator paper is provided below.
Recommendation: Women (and, if appropriate, their family and/or caregivers or others closely involved) should be given information
about contraception, conception, pregnancy and breastfeeding. Information should be given in advance of sexual activity or
pregnancy. (Level C) NICE 2004
1.
NICE National Collaborating Centre for Primary Care. The diagnosis and management of the epilepsies in adults and
children in primary and secondary care. London (UK): Royal College of General Practitioners; 2004 Oct.
From the Guideline with Direct Evidence for this recommendation Statement:
No systematic reviews of RCTs of information provision for women with epilepsy were identified.
One systematic review of other evidence was found. Couldridge and colleagues reviewed the primary evidence (including non-RCT
studies) on the information and counseling needs of people with epilepsy, the preferred format, timing, and delivery of information
and counseling, and the outcomes of information giving and counselling.216
None of the 40 included studies reported the role or effects of information or counseling in women with epilepsy as a group,
although some studies did have women in the study population.
Primary evidence
No RCTs on the effectiveness of information giving or counseling were identified. Since the publication of the systematic review
described above216, two large surveys of women with epilepsy were found. Crawford 1999241
Crawford and Lee reported the results of a questionnaire survey of female members of the British Epilepsy Association. 1855
questionnaires (from a total of 6000) were included in the results (response rate 31%).
47% (n=89) of women taking oral contraception felt they had not been given enough information about the oral contraception pill
and their AED(s). 43% (n=637) reported receiving no information about pregnancy, and 25% (n=459) had discussed pregnancy with
no-one. Many women intending to have children in the subsequent two years felt they still had unanswered questions (see Table
17).
Table 17 Concerns about pregnancy241 Modified from Seizure, 8, Crawford P and Lee P, Gender difference in management of
epilepsy - What women are hearing, pages 135-9, Copyright (1999) with permission from BEA Trading Ltd.
Overall, women felt there was a need for more information about epilepsy and pregnancy. The survey concluded that women with
epilepsy wanted, and needed, more information and counseling about issues relating to contraception, pregnancy, and the
menopause.241
Crawford 20032 42
In 2001, the Ideal World survey aimed to assess the quality of current treatment information provision to women with epilepsy at
different life stages, and to identify the information needs and wants with a view to ensuring that all women with epilepsy are
counseled appropriately, in a timely manner, and are able to make informed choices about their treatment.
Approximately 12,000 female members of Epilepsy Action were surveyed, and the questionnaire was also posted on the Epilepsy
Action website. 2,600 questionnaires and 90 web responses were completed, and 2000 responses randomly selected for analysis.
The most important issues for women aged 19 to 44 years who were considering having children were:
1. risk of epilepsy/medication affecting the unborn child (87%)
2. effect of pregnancy on seizure control (49%)
3. risk of a child developing epilepsy (42%)
Most women (84%) wanted to be better informed about treatment decisions, and 41% wanted to take a more proactive role in
discussions around treatment. 43% wanted more information so they could ask for a review of their medication. 57% wanted the
Created on: 06/18/2013 at 02:57 AM
2
latest information on epilepsy treatment and the risk of birth defects on an ongoing basis, even if the data were incomplete. The
preferred timing of receiving information can be seen in Table 18.
Table 18 Preferred time to receive information242 Modified from Seizure, 12, Crawford P and Hudson
S, Understanding the information needs of women with epilepsy at different lifestages: results of the ´Ideal World´ survey, pages
502-7, Copyright (2003) with permission from BEA Trading Ltd.
The survey showed consistently that information is preferred before the time it is needed. 59% wanted information in a written
format, and 28% through conversation with a healthcare professional.242
Recommendations:
If AEDs are to be used in pregnancy the relative risks of seizures and fetal malformation should be discussed with the woman.
(Level C) SIGN(April 2003) 23
Whenever possible, a woman should conceive on the lowest effective dose of one AED appropriate for her epilepsy syndrome. If
she has good seizure control and presents already pregnant, there is probably little to be gained by altering her AEDs. (Level C)
SIGN(April 2003) 23
SIGN. Scottish Intercollegiate Guidelines Network (SIGN): SIGN 70: (1) Diagnosis and management of epilepsy in adults. A
national clinical guideline. (2) Diagnosis and management of epilepsy in adults. Update to printed guideline. Scottish Intercollegiate
Guidelines Network - National Government Agency [Non-U.S.]. 2003 Apr (addendum released 2004 Jun 7). Original guideline: 49
pages; Addendum: 3 pages. NGC:003832
4.2.2 Risks to the fetus from antiepileptic drugs
Major and minor fetal malformations occur more commonly in infants exposed to AEDs during pregnancy.209, 230-2 The overall
risk of major fetal malformation in any pregnancy is approximately 2%. This increases two to three fold in women taking a single
AED. Current Data suggest that the risk with valproate may be higher than with carbamazepine or lamotrigine.233 Polytherapy,
particularly with certain combinations of drugs, carriers a much higher risk (up to 24% in women taking four AEDs).
The most common major malformation associated with established AEDs are: neural tube defects, (valproate 3%, carbamazepine
1%), orofacial defects, congenital heart abnormalities and hypospadias.230, 234 The risk of minor malformations including
hypertelorism, epicanthic folds and digital hypoplasia is increased with AED therapy in pregnancy.231
“Fetal anticonvulsant syndromes” comprising typical dysmophic craniofacial appearances and a variety of musculoskeletal
abnormalities have been described in associated with AED treatment in pregnancy.235,236 Although individual drugs have been
associated with specific patterns, there is overlap between them and genetic factors may influence susceptibility.237
Whether AEDs taken during pregnancy can affect the child’s intellectual development is uncertain but concern about the effects of
valproate on infant development has recently been raised.238,239
At present there is insufficient evidence on which to base advise about the risks of most of the new AEDS (gabapentin,
levetiracetam, tiagabine, topiramate, vigabatrin) in pregnancy. Current data on lamotrigine show a malformation rate of 3% (95%
confidence interval 1.5-5.7)233
Additional studies not initially used to support the measure but new guidelines that have come out since this project was completed.
Update: Management Issues for Women with Epilepsy-Focus on Pregnancy: Obstetrical Complications and Change in Seizure
Frequency Neurology® 2009;73:126–132
Do WWE have an increased risk of pregnancy-related complications?
Twenty-five articles met inclusion criteria for pregnancy-related complications in WWE. Several articles included information
pertinent to more than one question. Of these 25 articles, 9 were graded Class III or higher (table e-1).
Women with epilepsy (WWE) should be counseled that seizure freedom for at least 9 months prior to pregnancy is probably
Created on: 06/18/2013 at 02:57 AM
3
associated with a high rate (84%–92%) of remaining seizure-free during pregnancy (Level B).
Evidence to support this recommendation:
Seizure recurrence in previously seizure-free WWE.
Two Class II articles16,17 showed that for WWE who were seizure-free for 9 months prior to pregnancy, 84%–92% remained
seizure-free during pregnancy (table e-4). In one study, 38 of 45 (84%; CI 0.71– 0.92) pregnant WWE remained seizure-free,16 and
in the other study, 47 of 51 (92%; CI 0.82– 0.97) pregnant WWE remained seizure-free.17 One larger Class III article22 showed
that 80% of a group of WWE (n _ 450) who were seizure-free at least 1 year prior to pregnancy remained seizure-free during
pregnancy (exact number not provided). One Class III article showed that of 72 WWE who were seizure-free for 10 months, 74%
(95% CI 0.62– 0.82) remained seizure-free during pregnancy.18 A second Class III article showed that of 54 WWE who were
seizure-free for 9 months, 94% (95% CI 0.85– 0.98) remained seizure-free during pregnancy, and of 48 WWE who were seizurefree for 1 year, 92% (95% CI 0.80–0.98) remained seizure-free during pregnancy.19 These results are all fairly consistent across
the class of evidence and sample size of the studies.
Conclusion. Two Class II articles show the rate of remaining seizure-free during pregnancy if WWE are seizure-free for at least 9
months to 1 year prior to pregnancy is probably 84%–92%.
16. Gjerde IO, Strandjord RE, Ulstein M. The course of epilepsy during pregnancy: a study of 78 cases. Acta Neurol
Scand 1988;78:198–205.
17. Tomson T, Lindbom U, Ekqvist B, Sundqvist A. Epilepsy and pregnancy: a prospective study of seizure control in relation to free
and total plasma concentrations of carbamazepine and phenytoin. Epilepsia 1994;35:122–130.
18. Otani K. Risk factors for the increased seizure frequency during pregnancy and puerperium. Folia Psychiatr Neurol
Jpn 1985;39:33–41.
19. Tanganelli P, Regesta G. Epilepsy, pregnancy, and major birth anomalies: an Italian prospective, controlled study. Neurology
1992;42(4 suppl 5):89–93.
1c.7 Consistency of Results across Studies (Summarize the consistency of the magnitude and direction of the effect): The
studies are consistent that women of childbearing potential should be counseled about epilepsy and how its treatment may affect
contraception and pregnancy.
1c.8 Net Benefit (Provide estimates of effect for benefit/outcome; identify harms addressed and estimates of effect; and net benefit
- benefit over harms):
See quality of body of evidence question
1c.9 Grading of Strength/Quality of the Body of Evidence. Has the body of evidence been graded? Yes
1c.10 If body of evidence graded, identify the entity that graded the evidence including balance of representation and any
disclosures regarding bias: Pugh et al. work group
NICE guideline work group
SIGN guideline work group
1c.11 System Used for Grading the Body of Evidence: Other
1c.12 If other, identify and describe the grading scale with definitions: See 1c.22
1c.13 Grade Assigned to the Body of Evidence: see 1c.15 or 1c.16
1c.14 Summary of Controversy/Contradictory Evidence: Not applicable
1c.15 Citations for Evidence other than Guidelines(Guidelines addressed below):
IF a woman with epilepsy is of childbearing potential and receives oral contraceptives in conjunction with an enzyme inducing AED,
THEN decreased effectiveness of oral contraception should be addressed. (higher doses of the oral contraceptive, alternative birth
control methods, or change AED). (Level A 2++/Primary) Pugh (2007)17
Created on: 06/18/2013 at 02:57 AM
4
Patients with epilepsy should receive an annual review of information including topics such as:
- Chronic effects of epilepsy and its treatment including drug side-effects, drug-drug interactions, effect on bone health;Contraception, family planning, and how pregnancy and menopause may affect seizures (EVIDENCE GRADE C);- Screening for
mood disorders;- Triggers and lifestyle issues that may affect seizures;- Impact of epilepsy on other chronic and acute diseases;Driving and safety issues (Level D/Secondary) Pugh (2007) 17
PUGH Pugh MJ, Berlowitz DR, Montouris G, et al. What constitutes high quality of care for adults with epilepsy? Neurology. 2007
20;69(21):2020-7.
This is not a guideline. It is a quality indicator panel that developed the measures using an evidence and consensus based
process. There is not a lot of data available on the methodology or specific studies used to support the development of the
indicators. However, their conclusions and recommendations concur with our existing guideline recommendation statements.
“We developed explicit quality indicators using the modified Delphi process (RAND Appropriateness Method) that has been
successfully employed in the development of quality indicators for over 30 different preventive health, acute, or chronic
diseases.3,18,19 This method incorporated a systematic review of the literature and guidelines to assure that selected process of
care
criteria are linked to relevant patient outcomes in clinical trials or expert clinical opinion (best practices)16 and an expert rating
panel.”
3. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med
2003;348:2635–2645.
18. McGlynn EA, Kerr EA, Asch SM. New approach to
assessing clinical quality of care for women: the QA Tool system.
Womens Health Iss 1999;9:184–192.
19. Shekelle PG, MacLean CH, Morton SC, Wenger NS. Acove quality indicators. Ann Intern Med 2001;135: 653–667.
1c.16 Quote verbatim, the specific guideline recommendation (Including guideline # and/or page #):
Women (and, if appropriate, their family and/or caregivers or others closely involved) should be given information about
contraception, conception, pregnancy and breastfeeding. Information should be given in advance of sexual activity or pregnancy.
(Level C) NICE 2004
If AEDs are to be used in pregnancy the relative risks of seizures and fetal malformation should be discussed with the woman.
(Level C) SIGN(April 2003)
Whenever possible, a woman should conceive on the lowest effective dose of one AED appropriate for her epilepsy syndrome. If
she has good seizure control and presents already pregnant, there is probably little to be gained by altering her AEDs. (Level C)
SIGN(April 2003)
1c.17 Clinical Practice Guideline Citation: 23.
SIGN. Scottish Intercollegiate Guidelines Network (SIGN): SIGN 70: (1)
Diagnosis and management of epilepsy in adults. A national clinical guideline. (2) Diagnosis and management of epilepsy in adults.
Update to printed guideline. Scottish Intercollegiate Guidelines Network - National Government Agency [Non-U.S.]. 2003 Apr
(addendum released 2004 Jun 7). Original guideline: 49 pages; Addendum: 3 pages. NGC:003832
22.
NICE National Collaborating Centre for Primary Care. The diagnosis and management of the epilepsies in adults and
children in primary and secondary care. (Uses information from Reference 20 and 21) London (UK): Royal College of General
Practitioners; 2004 Oct
1c.18 National Guideline Clearinghouse or other URL: Yes http://guideline.gov/content.aspx?id=5694&search=epilepsy ;
http://www.sign.ac.uk/methodology/index.html; http://publications.nice.org.uk/the-epilepsies-the-diagnosis-and-management-of-theepilepsies-in-adults-and-children-in-primary-and-cg137
1c.19 Grading of Strength of Guideline Recommendation. Has the recommendation been graded? Yes
Created on: 06/18/2013 at 02:57 AM
5
1c.20 If guideline recommendation graded, identify the entity that graded the evidence including balance of representation
and any disclosures regarding bias: See individual guideline/indicator citations in 1c.15 or 1c.16
1c.21 System Used for Grading the Strength of Guideline Recommendation: Other
1c.22 If other, identify and describe the grading scale with definitions: NICE National Collaborating Centre for Primary Care.
The diagnosis and management of the epilepsies in adults and children in primary and secondary care. London (UK): Royal
College of General Practitioners; 2004 Oct.
Rating Scheme for Strength of the Evidence
Ia-Systematic review or meta-analysis of randomized controlled trials
Ib-At least one randomized controlled trial
IIa-At least one well-designed controlled stud without randomization
IIb-At least one well-designed quasi-experimental descriptive studies, such as a cohort study
III-Well-designed non-experimental descriptive studies, case-control studies, and case studies
IV-Expert committee reports, opinions and/or clinical experience of respected authorities
Rating Recommendations
A*
Directly based on category I evidence (meta-analysis of randomized controlled trials (RCTs) or at least one RCT)
B*
Directly based on category II evidence (at least one controlled study without randomization or at least one other quasiexperimental study) or extrapolated from category I evidence
C*
Directly based on category III evidence (non-experimental descriptive studies) or extrapolated from category I or II
evidence
D*
Directly based on category III evidence (expert committee reports or opinions and/or clinical experience of respected
authorities) or extrapolated from category I, II or III evidence
N
Recommendation taken from NICE guideline or technology appraisal guidance
SIGN (1): SIGN 70: Diagnosis and Management of Epilepsy in Adults. A National Clinical Guideline. Edinburgh (Scotland) 2003
April p.49. Under revision as of June 2008.
Grading of Recommendations (Note: Only measures graded as A, B, or C were included in the table)
A: At least one meta-analysis, systematic review of randomized controlled trials (RCTs), or randomized controlled trial rated as 1++
and directly applicable to the target population; or A body of evidence consisting principally of studies rated as 1+, directly
applicable to the target population, and demonstrating overall consistency of results
B: A body of evidence including studies rated as 2++, directly applicable to the target population and demonstrating overall
consistency of results; or Extrapolated evidence from studies rated as 1++ or 1+
C: A body of evidence including studies rated as 2+, directly applicable to the target population and demonstrating overall
consistency of results; or Extrapolated evidence from studies rate as 2++
D: Evidence level 3 or 4; or
Extrapolated evidence from studies rated as 2+
Levels of Evidence
1++: High quality meta-analyses, systematic reviews of randomized controlled trials (RCTs), or RCTs with a very low risk of bias
1+: Well-conducted meta-analyses, systematic reviews of RCTs, or RCTs with a low risk of bias
1-: Meta-analyses, systematic reviews of RCTs, or RCTs with a high risk of bias
2++: High quality systematic reviews of case control or cohort studies. High quality case control or cohort studies with a very low
risk of confounding or bias and a high probability that the relationship is causal
2+: Well-conducted case control or cohort studies with a low risk of confounding or bias and a moderate probability that the
relationship is causal
2-: Case control or cohort studies with a high risk of confounding or bias and a significant risk that the relationship is not causal
3: Non-analytic studies, e.g. case reports, case series
4: Expert opinion
Created on: 06/18/2013 at 02:57 AM
6
Pugh Paper: Epilepsy Measures Work Group Grading of Evidence and Indicators
Pugh MJ, Berlowitz DR, Montouris G, Bokhour B, Cramer JA, Bohm V, Bollinger M, Helmers S, Ettinger A, Meador KJ, Fountain N,
Boggs J, Tatum WO 4th, Knoefel J, Harden C, Mattson RH, Kazis L. What constitutes high quality of care for adults with epilepsy?
Neurology. 2007 Nov 20;69(21):2020-7.
A: Rated as appropriate
F: Rated as feasible
N: Rated as necessary
N/A: Not Rated
Ratings
1-3 clearly appropriate/ reliable/ necessary
4-6 uncertain or equivocal
7-10 appropriate/ reliable/ necessary
1c.23 Grade Assigned to the Recommendation: See individual guideline/indicator citations in 1c.15 or 1c.16
1c.24 Rationale for Using this Guideline Over Others: These recommendaiton statements and guidelines were chosen because
of the high impact on care and gap in care for women with epilepsy. These guidelines and indicator papers were chosen over
others because of their applicabilility to meet the gap in care and improve the quality of care of women with epilepsy of childbearing
potential.
Based on the NQF descriptions for rating the evidence, what was the developer’s assessment of the quantity, quality, and
consistency of the body of evidence?
1c.25 Quantity: Moderate 1c.26 Quality: Moderate1c.27 Consistency: High
Created on: 06/18/2013 at 02:57 AM
7
NATIONAL QUALITY FORUM—Measure Testing (subcriteria 2a2, 2b2-2b7)
Measure Number (if previously endorsed): 1814
Measure Title: Counseling for Women of Childbearing Potential with Epilepsy
Date of Submission: 2/27/2014
Type of Measure:
☐ Composite – STOP – use composite testing form
☐ Cost/resource
☐ Efficiency
☐ Outcome (including PRO-PM)
☒ Process
☐ Structure
Instructions
• Measures must be tested for all the data sources and levels of analyses that are specified. If there is more than
one set of data specifications or more than one level of analysis, contact NQF staff about how to present all
the testing information in one form.
• For all measures, sections 1, 2a2, 2b2, 2b3, and 2b5 must be completed.
• For outcome and resource use measures, section 2b4 also must be completed.
• If specified for multiple data sources/sets of specificaitons (e.g., claims and EHRs), section 2b6 also must be
completed.
• Respond to all questions as instructed with answers immediately following the question. All information on
testing to demonstrate meeting the subcriteria for reliability (2a2) and validity (2b2-2b6) must be in this form.
An appendix for supplemental materials may be submitted, but there is no guarantee it will be reviewed.
• If you are unable to check a box, please highlight or shade the box for your response.
• Maximum of 20 pages (incuding questions/instructions; minimum font size 11 pt; do not change margins).
Contact NQF staff if more pages are needed.
• Contact NQF staff regarding questions. Check for resources at Submitting Standards webpage.
Note: The information provided in this form is intended to aid the Steering Committee and other stakeholders in
understanding to what degree the testing results for this measure meet NQF’s evaluation criteria for testing.
2a2. Reliability testing 10 demonstrates the measure data elements are repeatable, producing the same results a
high proportion of the time when assessed in the same population in the same time period and/or that the
measure score is precise. For PRO-PMs and composite performance measures, reliability should be demonstrated
for the computed performance score.
2b2. Validity testing 11 demonstrates that the measure data elements are correct and/or the measure score
correctly reflects the quality of care provided, adequately identifying differences in quality. For PRO-PMs and
composite performance measures, validity should be demonstrated for the computed performance score.
2b3. Exclusions are supported by the clinical evidence; otherwise, they are supported by evidence of sufficient
frequency of occurrence so that results are distorted without the exclusion; 12
AND
If patient preference (e.g., informed decisionmaking) is a basis for exclusion, there must be evidence that the
exclusion impacts performance on the measure; in such cases, the measure must be specified so that the
information about patient preference and the effect on the measure is transparent (e.g., numerator category
computed separately, denominator exclusion category computed separately). 13
Version 6.5 08/20/13
1
2b4. For outcome measures and other measures when indicated (e.g., resource use):
• an evidence-based risk-adjustment strategy (e.g., risk models, risk stratification) is specified; is based on patient
factors that influence the measured outcome (but not factors related to disparities in care or the quality of care)
and are present at start of care; 14,15 and has demonstrated adequate discrimination and calibration
OR
• rationale/data support no risk adjustment/ stratification.
2b5. Data analysis of computed measure scores demonstrates that methods for scoring and analysis of the
specified measure allow for identification of statistically significant and practically/clinically meaningful 16
differences in performance;
OR
there is evidence of overall less-than-optimal performance.
2b6. If multiple data sources/methods are specified, there is demonstration they produce comparable results.
2b7. For eMeasures, composites, and PRO-PMs (or other measures susceptible to missing data), analyses identify
the extent and distribution of missing data (or nonresponse) and demonstrate that performance results are not
biased due to systematic missing data (or differences between responders and nonresponders) and how the
specified handling of missing data minimizes bias.
Notes
10. Reliability testing applies to both the data elements and computed measure score. Examples of reliability testing for data
elements include, but are not limited to: inter-rater/abstractor or intra-rater/abstractor studies; internal consistency for multiitem scales; test-retest for survey items. Reliability testing of the measure score addresses precision of measurement (e.g.,
signal-to-noise).
11. Validity testing applies to both the data elements and computed measure score. Validity testing of data elements typically
analyzes agreement with another authoritative source of the same information. Examples of validity testing of the measure
score include, but are not limited to: testing hypotheses that the measures scores indicate quality of care, e.g., measure scores
are different for groups known to have differences in quality assessed by another valid quality measure or method; correlation
of measure scores with another valid indicator of quality for the specific topic; or relationship to conceptually related
measures (e.g., scores on process measures to scores on outcome measures). Face validity of the measure score as a quality
indicator may be adequate if accomplished through a systematic and transparent process, by identified experts, and explicitly
addresses whether performance scores resulting from the measure as specified can be used to distinguish good from poor
quality.
12. Examples of evidence that an exclusion distorts measure results include, but are not limited to: frequency of occurrence,
variability of exclusions across providers, and sensitivity analyses with and without the exclusion.
13. Patient preference is not a clinical exception to eligibility and can be influenced by provider interventions.
14. Risk factors that influence outcomes should not be specified as exclusions.
15. Risk models should not obscure disparities in care for populations by including factors that are associated with
differences/inequalities in care, such as race, socioeconomic status, or gender (e.g., poorer treatment outcomes of African
American men with prostate cancer or inequalities in treatment for CVD risk factors between men and women). It is
preferable to stratify measures by race and socioeconomic status rather than to adjust out the differences.
16. With large enough sample sizes, small differences that are statistically significant may or may not be practically or clinically
meaningful. The substantive question may be, for example, whether a statistically significant difference of one percentage
point in the percentage of patients who received smoking cessation counseling (e.g., 74 percent v. 75 percent) is clinically
meaningful; or whether a statistically significant difference of $25 in cost for an episode of care (e.g., $5,000 v. $5,025) is
practically meaningful. Measures with overall less-than-optimal performance may not demonstrate much variability across
providers.
Version 6.5 08/20/13
2
1. DATA/SAMPLE USED FOR ALL TESTING OF THIS MEASURE
Often the same data are used for all aspects of measure testing. In an effort to eliminate duplication, the
first five questions apply to all measure testing. If there are differences by aspect of testing,(e.g.,
reliability vs. validity) be sure to indicate the specific differences in question 1.7.
1.1. What type of data was used for testing? (Check all the sources of data identified in the measure
specifications and data used for testing the measure. Testing must be provided for all the sources of data
specified and intended for measure implementation. If different data sources are used for the
numerator and denominator, indicate N [numerator] or D [denominator] after the checkbox.)
Measure Specified to Use Data From:
Measure Tested with Data From:
(must be consistent with data sources entered in S.23)
☒ abstracted from paper record
☒ abstracted from paper record
☒ administrative claims
☐ administrative claims
☐ clinical database/registry
☐ clinical database/registry
☒ abstracted from electronic health record
☒ abstracted from electronic health record
☐ eMeasure (HQMF) implemented in EHRs
☐ eMeasure (HQMF) implemented in EHRs
☐ other: Click here to describe
1.2. If an existing dataset was used, identify the specific dataset (the dataset used for testing must be
consistent with the measure specifications for target population and healthcare entities being measured;
e.g., Medicare Part A claims, Medicaid claims, other commercial insurance, nursing home MDS, home
health OASIS, clinical registry).
The AAN identified and recruited Neurology practices in Minnesota that have experience treating
patients with epilepsy. Three Neurology practices volunteered to participate and submit retrospective
data from the 2012 calendar year (i.e. dates of service 01/01/2012 – 12/31/2012). Denominator
certification is an essential step in the process to obtaining valid and accurate data. It requires each
participant to attest that they will submit accurate data and follow the measure specifications exactly
how they are written. It also ensures that each participant is querying the correct:
• Diagnosis codes (i.e. 345.00, 345.01, 345.10, 345.11, 345.40, 345.41, 345.50, 345.51, 345.70, 345.71,
345.90, 345.91)
• Encounter codes (i.e. 99201, 99202, 99203, 99204, 99205, 99212, 99213, 99214, 99215, 99241,
99242, 99243, 99244, 99245)
• Date of birth ranges (i.e. 01/01/1968-01/01/2000)
• Date of service ranges (i.e. 01/01/2012- 12/31/2012)
1.3. What are the dates of the data used in testing? 01/01/2012-12/31/2012
1.4. What levels of analysis were tested? (testing must be provided for all the levels specified and
intended for measure implementation, e.g., individual clinician, hospital, health plan)
Measure Specified to Measure Performance of:
Measure Tested at Level of:
(must be consistent with levels entered in item S.26)
☒ individual clinician
☐ individual clinician
☒ group/practice
☒ group/practice
☐ hospital/facility/agency
☐ hospital/facility/agency
☐ health plan
☐ health plan
Version 6.5 08/20/13
3
1.5. How many and which measured entities were included in the testing and analysis (by level of
analysis and data source)? (identify the number and descriptive characteristics of measured entities
included in the analysis (e.g., size, location, type); if a sample was used, describe how entities were
selected for inclusion in the sample)
Two of the practices were independent, physician owned practices located in the Twin Cities and the
third practice was a large integrated delivery system in southeast Minnesota.
1.6. How many and which patients were included in the testing and analysis (by level of analysis and
data source)? (identify the number and descriptive characteristics of patients included in the analysis
(e.g., age, sex, race, diagnosis); if a sample was used, describe how patients were selected for inclusion in
the sample) In order to achieve a reliable sample of patients for this measure MNCM and AAN sought a
minimum of 1,000 combined patient records from the three Neurology sites that agree to participate.
See below.
Version 6.5 08/20/13
4
1.7. If there are differences in the data or sample used for different aspects of testing (e.g., reliability,
validity, exclusions, risk adjustment), identify how the data or sample are different for each aspect of
testing reported below.
________________________________
2a2. RELIABILITY TESTING
Note: If accuracy/correctness (validity) of data elements was empirically tested, separate reliability
testing of data elements is not required – in 2a2.1 check critical data elements; in 2a2.2 enter “see
section 2b2 for validity testing of data elements”; and skip 2a2.3 and 2a2.4.
2a2.1. What level of reliability testing was conducted? (may be one or both levels)
☒ Critical data elements used in the measure (e.g., inter-abstractor reliability; data element reliability
must address ALL critical data elements)
☒ Performance measure score (e.g., signal-to-noise analysis)
2a2.2. For each level checked above, describe the method of reliability testing and what it tests
(describe the steps―do not just name a method; what type of error does it test; what statistical analysis
was used)
The AAN identified and recruited Neurology practices in Minnesota that have experience treating
patients with epilepsy. As part of the recruitment process MNCM and the AAN hosted an informational
webinar explaining the purpose of the measurement testing project for the Counseling for Women with
Epilepsy measure. Three Neurology practices volunteered to participate and submit retrospective data
from the 2012 calendar year (i.e. dates of service 01/01/2012 – 12/31/2012). MNCM produced a data
collection guide, measure flow and detailed file specifications to educate and assist each medical group
in the data collection and submission process. As a requirement of participating in the measure testing
each group had to submit a denominator certifications form (see appendix A). The denominator
certification process helps ensure that each medical group is using the appropriate measure parameters
and collecting data in a standardized way.
2a2.3. For each level of testing checked above, what were the statistical results from reliability
testing? (e.g., percent agreement and kappa for the critical data elements; distribution of reliability
statistics from a signal-to-noise analysis)
MNCM did not identify any major flaws or issues during the review of each medical group’s
denominator forms and therefore each medical group passed denominator certification within the given
timeframe.
2a2.4 What is your interpretation of the results in terms of demonstrating
reliability? (i.e., what do the results mean and what are the norms for the test conducted?)
There were a few corrections and clarifications that required MNCM to send a follow-up email to the
respective group; however, each issue was resolved in a timely manner. The list below documents the
Version 6.5 08/20/13
5
issues that were identified and required additional follow-up based on the information received on the
denominator certification forms:
• Incorrect diagnosis codes included in data query
• Group did not indicate if they would be submitting a sample or full population for the measure
• Incorrect encounter codes included in data query
_________________________________
2b2. VALIDITY TESTING
2b2.1. What level of validity testing was conducted? (may be one or both levels)
☒ Critical data elements (data element validity must address ALL critical data elements)
☒ Performance measure score
☐ Empirical validity testing
☐ Systematic assessment of face validity of performance measure score as an indicator of quality
or resource use (i.e., is an accurate reflection of performance on quality or resource use and can
distinguish good from poor performance)
2b2.2. For each level of testing checked above, describe the method of validity testing
and what it tests (describe the steps―do not just name a method; what was tested, e.g.,
accuracy of data elements compared to authoritative source, relationship to another measure
as expected; what statistical analysis was used)
MNCM completed validation of the data in a three-step process: 1) denominator certification, 2) data
file quality checks, and 3) validation audit. Details of this validation are described in this report.
Denominator Certification
Denominator certification is an essential step in the process to obtaining valid and accurate data. It
requires each participant to attest that they will submit accurate data and follow the measure
specifications exactly how they are written. It also ensures that each participant is querying the correct:
• Diagnosis codes (i.e. 345.00, 345.01, 345.10, 345.11, 345.40, 345.41, 345.50, 345.51, 345.70, 345.71,
345.90, 345.91)
• Encounter codes (i.e. 99201, 99202, 99203, 99204, 99205, 99212, 99213, 99214, 99215, 99241,
99242, 99243, 99244, 99245)
• Date of birth ranges (i.e. 01/01/1968-01/01/2000)
• Date of service ranges (i.e. 01/01/2012- 12/31/2012)
timeframe. There were a few corrections and clarifications that required MNCM to send a follow-up
email to the respective group; however, each issue was resolved in a timely manner. The list below
documents the issues that were identified and required additional follow-up based on the information
received on the denominator certification forms:
Data File Quality Checks
After each medical group submitted their data file to MNCM, quality checks of the files were completed.
Each column in the data file represented a field of data for each patient row; the following checks were
completed:
Version 6.5 08/20/13
6
• Number of patients/rows submitted were reasonable/expected
• Necessary data fields (columns) were included and completed appropriately
• Patient date of birth spanned the expected range
• Zip codes were 5-digit and primarily within MN and other bordering states as expected
• Race field(s) were included and populated appropriately
• Provider NPI field was included and number of providers expected
• Insurance information was included and was reasonable
• Office visit dates and counseling dates spanned the expected range
• Diagnoses were included and spanned the entire list of expected codes
• Medical reasons for NOT counseling were applied correctly; were not misused
Issues identified through the data file quality checks were generally minor, requiring no corrections.
Other mentionable items include:
1. All three groups did not have patients with diagnosis codes 345.70 (Epilepsia partialis continua,
without mention of intractable epilepsy) or 345.71 (Epilepsia partialis continua, with intractable
epilepsy). These are rare diagnoses and did not come up in the population.
2. Medical groups B and C listed many neurological or congenital conditions as reasons for the patient to
NOT receive counseling. These were verified during audit.
3. Medical group C did not include their entire population in first submission, excluding patients whose
date of birth was between January thru June 1968. They queried their system again, this time using the
specific dates of birth (rather than “age” values) and included the additional patients in their
denominator.
Validation Audit
After the data file checks were completed, MNCM completed audits of the patient records to verify the
submitted clinical data. We also verified the diagnosis of epilepsy and other demographic data (e.g.,
race). MNCM uses a validation process developed by the NCQA – National Committee for Quality
Assurance, known as the “8 and 30” process. In this process, the first eight records are verified for
accuracy and if no errors are identified, the data is considered to be 100% compliant. If errors in the first
eight records are identified, we continue reviewing the total 30 records to identify any error patterns
and or issues that may need correction. The audits revealed some data errors, requiring one medical
group to make corrections and resubmit data. Individual medical group results were as follows:
Version 6.5 08/20/13
7
2b2.3. What were the statistical results from validity testing? (e.g., correlation; t-test)
Validation/ Audit Conclusion
2b2.4. What is your interpretation of the results in terms of demonstrating validity? (i.e., what do the
results mean and what are the norms for the test conducted?)
The validation process was successful in identifying errors (with subsequent corrections) and verifying
the accuracy of the data submitted by medical groups A, B, and C. Finding no significant flaws or errors
with the data MNCM is confident the rate calculation and any additional data analysis can be completed
using validated and reliable data. Additionally, during a review of the National Quality Forum’s feedback
to the American Academy of Neurology for this measure, it was noted that there was a concern that this
Version 6.5 08/20/13
8
may simply be a “check-the-box” measure. During the validation audit, it was noted on several occasions
that the practices provided excellent, personalized progress notes about the counseling that was being
provided, that were above and beyond a “check the box”.
_________________________
2b3. EXCLUSIONS ANALYSIS
NA ☐ no exclusions — skip to section 2b4
2b3.1. Describe the method of testing exclusions and what it tests (describe the
steps―do not just name a method; what was tested, e.g., whether exclusions affect overall
performance scores; what statistical analysis was used)
The main limitation that MNCM identified during the testing of the Counseling for Women with Epilepsy
measure is related to the denominator of included and excluded patients. The measure specifications
offered two different options for excluding patients from the measure:
1. Patient was surgically sterile (tubal ligation, hysterectomy)
2. Patient has an intellectual disability as defined by ICD-9 codes
a. 318.0 moderate intellectual disabilities; IQ 35 to 48
b. 318.1 severe intellectual disabilities; IQ 20 to 34
c. 318.2 profound intellectual disabilities; IQ under 20
Groups submitted these patients and indicated which reason applied. Additionally, if they felt that there
was another medical reason for not providing counseling, they indicated this by a code and
accompanying description. These reasons were not used to exclude patients from the measure; rather
the purpose was to provide additional information about the population of patients included in the
measure. Reasons Provided by Medical Groups for Not Providing Counseling:
2b3.2. What were the statistical results from testing exclusions? (include overall number and
percentage of individuals excluded, frequency distribution of exclusions across measured entities, and
Version 6.5 08/20/13
9
impact on performance measure scores)
2b3.3. What is your interpretation of the results in terms of demonstrating that
exclusions are needed to prevent unfair distortion of performance results? (i.e., the
value outweighs the burden of increased data collection and analysis. Note: If patient
preference is an exclusion, the measure must be specified so that the effect on the
performance score is transparent, e.g., scores with and without exclusion)
During the analysis of this data and also as a byproduct of the validation audit in reviewing medical
records, MNCM staff has some concerns regarding the denominator and intent of the measure. There
may need to be a consideration for adding a component of indicating that the patient is sexually active
or has the potential to be sexually active, and not physically handicapped. AAN could refer to the
NCQA specifications for the Chlamydia Screening in Women measure (NQF# 0033/ CMS 153v1)
for reference tables indicating how to identify potentially sexually active women via pharmacy
codes, CPTs, ICD-9, UB Revenue and LOINC codes. Rather than trying to capture/ code every
possible exclusion; this may be an option. MNCM would not recommend having a general type
exclusions code, like one that is stated as “any documented medical reason, because providers
will use this to their advantage and exclude patients that are at risk for pregnancy and truly
belong in the denominator. Having this type of exclusion weakens the measure, and can impact
the validity and reliability of the results.
____________________________
2b4. RISK ADJUSTMENT/STRATIFICATION FOR OUTCOME OR RESOURCE USE MEASURES
If not an intermediate or health outcome, or PRO-PM, or resource use measure, skip to section 2b5.
2b4.1. What method of controlling for differences in case mix is used?
☒ No risk adjustment or stratification
☐ Statistical risk model with Click here to enter number of factors risk factors
☐ Stratification by Click here to enter number of categories risk categories
Version 6.5 08/20/13
10
☐ Other, Click here to enter description
2b4.2. If an outcome or resource use measure is not risk adjusted or stratified, provide rationale and
analyses to demonstrate that controlling for differences in patient characteristics (case mix) is not
needed to achieve fair comparisons across measured entities.
2b4.3. Describe the conceptual/clinical and statistical methods and criteria used to select patient
factors used in the statistical risk model or for stratification by risk (e.g., potential factors identified in
the literature and/or expert panel; regression analysis; statistical significance of p<0.10; correlation of x
or higher; patient factors should be present at the start of care and not related to disparities)
2b4.4. What were the statistical results of the analyses used to select risk factors?
2b4.5. Describe the method of testing/analysis used to develop and validate the adequacy of the
statistical model or stratification approach (describe the steps―do not just name a method; what
statistical analysis was used)
Provide the statistical results from testing the approach to controlling for differences in patient
characteristics (case mix) below.
If stratified, skip to 2b4.9
2b4.6. Statistical Risk Model Discrimination Statistics (e.g., c-statistic, R-squared):
2b4.7. Statistical Risk Model Calibration Statistics (e.g., Hosmer-Lemeshow statistic):
2b4.8. Statistical Risk Model Calibration – Risk decile plots or calibration curves:
2b4.9. Results of Risk Stratification Analysis:
2b4.10. What is your interpretation of the results in terms of demonstrating adequacy of controlling
for differences in patient characteristics (case mix)? (i.e., what do the results mean and what are the
norms for the test conducted)
2b4.11. Optional Additional Testing for Risk Adjustment (not required, but would provide additional
support of adequacy of risk model, e.g., testing of risk model in another data set; sensitivity analysis for
missing data; other methods that were assessed)
_______________________
2b5. IDENTIFICATION OF STATISTICALLY SIGNIFICANT & MEANINGFUL DIFFERENCES IN PERFORMANCE
2b5.1. Describe the method for determining if statistically significant and clinically/practically
meaningful differences in performance measure scores among the measured entities can be identified
Version 6.5 08/20/13
11
(describe the steps―do not just name a method; what statistical analysis was used? Do not just repeat
the information provided related to performance gap in 1b)
2b5.2. What were the statistical results from testing the ability to identify statistically significant
and/or clinically/practically meaningful differences in performance measure scores across measured
entities? (e.g., number and percentage of entities with scores that were statistically significantly
different from mean or some benchmark, different from expected; how was meaningful difference
defined)
2b5.3. What is your interpretation of the results in terms of demonstrating the ability to identify
statistically significant and/or clinically/practically meaningful differences in performance across
measured entities? (i.e., what do the results mean in terms of statistical and meaningful differences?)
_______________________________________
2b6. COMPARABILITY OF PERFORMANCE SCORES WHEN MORE THAN ONE SET OF SPECIFICATIONS
If only one set of specifications, this section can be skipped.
Note: This criterion is directed to measures with more than one set of specifications/instructions (e.g.,
one set of specifications for how to identify and compute the measure from medical record abstraction
and a different set of specifications for claims or eMeasures). It does not apply to measures that use
more than one source of data in one set of specifications/instructions (e.g., claims data to identify the
denominator and medical record abstraction for the numerator). If comparability is not demonstrated,
the different specifications should be submitted as separate measures.
2b6.1. Describe the method of testing conducted to demonstrate comparability of performance scores
for the same entities across the different data sources/specifications (describe the steps―do not just
name a method; what statistical analysis was used)
2b6.2. What were the statistical results from testing comparability of performance scores for the
same entities when using different data sources/specifications? (e.g., correlation, rank order)
2b6.3. What is your interpretation of the results in terms of demonstrating comparability of
performance measure scores for the same entities across the different data sources/specifications?
(i.e., what do the results mean and what are the norms for the test conducted)
_______________________________________
2b7. MISSING DATA ANALYSIS AND MINIMIZING BIAS
2b7.1. Describe the method of testing conducted to identify the extent and distribution
of missing data (or nonresponse) and demonstrate that performance results are not biased
due to systematic missing data (or differences between responders and nonresponders) and
how the specified handling of missing data minimizes bias (describe the steps―do not just
name a method; what statistical analysis was used)
Version 6.5 08/20/13
12
See Full AAN Women with Epilepsy of Childbearing Potential Measure Testing Report. The validation
process is conducted to verify that the submitted data matches the source data in the medical record.
After the clinical data file is successfully transferred to MNCM and passes the initial quality checks,
MNCM will contact the medical group about the validation process. MNCM utilizes the National
Committee for Quality Assurance (NCQA) “8 and 30” process for validation audits. If the first clinic site is
in high compliance and the data collection process for all clinic sites within the medical group is
identical, further review may be abbreviated at the discretion of the MNCM auditor. If clinic sites are not
in high compliance after review of the first eight records, the MNCM auditor will continue to review the
remaining 22 records. If after review of all 30 records the clinic site is not in high compliance on all
factors (less than 90%), the MNCM auditor will review the results with the clinic representative and
communicate the results with MNCM. MNCM will then contact the medical group to develop a mutually
agreed upon re-submission plan. (Re-submission plans will only be allowed for errors in the numerator
portion.) Clinic sites that are not in high compliance or have not been in high compliance in a previous
MNCM audit may be held to a more rigorous denominator certification and validation audit.
2b7.2. What is the overall frequency of missing data, the distribution of missing data across providers,
and the results from testing related to missing data? (e.g., results of sensitivity analysis of the effect of
various rules for missing data/nonresponse; if no empirical sensitivity analysis, identify the approaches
for handling missing data that were considered and pros and cons of each)
2b7.3. What is your interpretation of the results in terms of demonstrating that performance results
are not biased due to systematic missing data (or differences between responders and nonresponders)
and how the specified handling of missing data minimizes bias? (i.e., what do the results mean in terms
of supporting the selected approach for missing data and what are the norms for the test conducted; if
no empirical analysis, provide rationale for the selected approach for missing data)
timeframe. There were a few corrections and clarifications that required MNCM to send a follow-up
email to the respective group; however, each issue was resolved in a timely manner. The list below
documents the issues that were identified and required additional follow-up based on the information
received on the denominator certification forms:
Version 6.5 08/20/13
13
MINNESOTA COMMUNITY MEASUREMENT
Measure Testing Summary Report: Counseling
for Women of Childbearing Potential with
Epilepsy
Prepared for: American Academy of Neurology
12/18/2013
Table of Contents
Project Background ................................................................................................................................................................. 2
Purpose ............................................................................................................................................................................... 2
Methods .............................................................................................................................................................................. 2
Results & Findings ................................................................................................................................................................... 3
Descriptive Measure Statistics ............................................................................................................................................ 3
Table 1: Patient Payer Information ................................................................................................................................. 3
Table 2: Patient Place of Residence (based on zip code) ................................................................................................ 3
Table 3: Patient Race and Ethnicity Information ............................................................................................................ 3
Table 4: Patient Age Breakdown ..................................................................................................................................... 4
Measure Results.................................................................................................................................................................. 4
Validation Results / Audit Results ....................................................................................................................................... 5
Denominator Certification .............................................................................................................................................. 5
Data File Quality Checks.................................................................................................................................................. 5
Validation Audit .............................................................................................................................................................. 6
Validation/ Audit Conclusion .......................................................................................................................................... 6
Limitations........................................................................................................................................................................... 7
Future Measure Implementation ....................................................................................................................................... 7
Appendix A: Denominator Certification Form ........................................................................................................................ 9
Appendix B: File Quality Checks & Validation ....................................................................................................................... 12
Appendix C: MNCM Audit Strategy ....................................................................................................................................... 13
Appendix D: Questions and Answers Received During Measure Testing ............................................................................. 14
Appendix E: Post Measure Survey Results: Resource Use and Data Burden.................................................................... 17
Project Background
In 2012, the AAN submitted epilepsy quality measures to the National Quality Forum (NQF), a national, nonprofit
organization that reviews and endorses health care quality measures for use by public and private payers. The AAN
received conditional endorsement of its Counseling for Women with Epilepsy measure. To receive full endorsement the
AAN must field test the Counseling for Women with Epilepsy measure for feasibility, reliability, and validity prior to
NQF’s review of this measure in March 2014. To achieve the goal of testing the Counseling for Women with Epilepsy
measure the AAN contracted with Minnesota Community Measurement (MNCM), a non-profit organization specializing
in health care quality measurement and reporting, to collect data pertaining to this measure from Neurology practices.
Purpose
This project aims to successfully collect de-identified patient-level data from Neurology practices that have the capability
to report accurate denominator and numerator information for the Counseling for Women with Epilepsy measure. In
order to achieve a reliable sample of patients for this measure MNCM and AAN are seeking a minimum of 1,000
combined patient records from the Neurology sites that agree to participate.
Methods
The AAN identified and recruited Neurology practices in Minnesota that have experience treating patients with epilepsy.
As part of the recruitment process MNCM and the AAN hosted an informational webinar explaining the purpose of the
measurement testing project for the Counseling for Women with Epilepsy measure. Three Neurology practices
volunteered to participate and submit retrospective data from the 2012 calendar year (i.e. dates of service 01/01/2012 –
12/31/2012).
MNCM produced a data collection guide, measure flow and detailed file specifications to educate and assist each
medical group in the data collection and submission process. As a requirement of participating in the measure testing
each group had to submit a denominator certifications form (see appendix A). The denominator certification process
helps ensure that each medical group is using the appropriate measure parameters and collecting data in a standardized
way.
Once the denominator certifications were complete each group submitted their data files using a secure FTP transfer
process. Once the files were received MNCM performed quality checks on each files using the methods outlined in
Appendix B. Once the files passed the quality checks MNCM calculated and sent the results back to each group for
review. If the group did not find any issues with the measure results then MNCM conducted an audit to validate the
accuracy of data using the auditing principles outlined in Appendix C.
The final step in testing the Women with Epilepsy measure was a post-data submission survey that was sent to each of
the participating medical groups. The intent of the survey was to shed light on the amount of resources that were
required to produce the data as well as gauge the level of data collection burden that each data field presented to the
group. The results of the survey can be found in Appendix E.
Results & Findings
Descriptive Measure Statistics
Table 1: Patient Payer Information
Payer Type
Medicaid
Medicare
SelfPay/Uninsured
Commercial
Medicaid
Clinic A
105
9
461
176
Clinic B
76
72
Clinic C
11
422
2
45
5
Grand Total
76
184
7
22
928
181
Table 2: Patient Place of Residence (based on zip code)
State
Minnesota
Iowa
Wisconsin
North Dakota
South Dakota
Alaska
Arkansas
Colorado
Illinois
Maryland
Michigan
Nebraska
New York
North
Carolina
Oklahoma
Ohio
Pennsylvania
Blank
Group A
Patients
32
7
4
3
1
1
1
2
1
Group B
Patients
649
19
49
9
11
1
1
1
1
1
5
1
Group C
Patients
561
3
14
1
1
1
Total
1242
29
67
9
15
1
1
2
2
2
3
7
1
1
2
1
2
1
1
5
1
1
3
2
Table 3: Patient Race and Ethnicity Information
Race
American Indian/Alaska Native (Code 1)
Asian (Code 2)
Black/African American (Code 3)
Hispanic/Latino (Code 4)
Native Hawaiian/Other Pacific Islander (Code 5)
Group A
Patients
2
Group B
Patients
6
12
19
12
3
Group C
Patients
3
8
18
Total
9
22
37
12
3
White (Code 6)
Some Other Race (Code 7)
Unknown (Code 98)
Chose not to disclose (Code 97)
Blank
54
3
335
2
1
236
625
5
5
11
662
4
11
301
361
Table 4: Patient Age Breakdown
Total by clinic
Ages 12-17
Ages 18-25
Ages 26-30
Age 31-35
Ages 36-40
Ages 41-44
Group A
Patients
14
13
8
13
8
3
Group B
Patients
170
206
129
111
93
42
Group C
Patients
112
175
113
73
60
48
Total
296
394
250
197
161
93
Measure Results
Measure Results
Number of Providers (NPI)
Number of Patients Submitted
Clinic A
9
751
Clinic B
36
581
Clinic C
22
59
Total
67
1391
127
18
145
606
420
419
419
32
12
44
537
26
77
21
2
5
7
52
11
7
6
161
35
196
1195
457
503
446
Rate for Contraceptive Counseling
69.3%
4.8%
21.2%
38.2%
Rates for Pregnancy Counseling
69.1%
14.3%
13.5%
42.1%
Rates for Contraceptive and Pregnancy Counseling
69.1%
3.9%
11.5%
37.3%
Other Medical Reasons for Not Counseling Patients
Number of Patients Submitted
Number of Patients with “Other Medical Reason”
Percentage of Patients with “Other Medical Reason”
Clinic A
751
122
16.2%
Clinic B
581
156
26.9%
Clinic C
59
6
10.2%
Total
1391
446
20.4%
Number of Patients Excluded; intellectual disability codes
Number of Patients Excluded; surgically sterile
Number of Patients with valid exclusions
Denominator: Number of Patient Eligible for Counseling
Number of Patients with Counseling for Contraception
Number of Patients with Counseling for Pregnancy
Number of Patients with Contraception and Pregnancy
Please note: The following rate re-calculation is for analytical purposes only; removing all patients that had “Other
Medical Reason Documented”. MNCM does not recommend reporting this rate. Please see Limitations Section
Rates for Contraceptive and Pregnancy Counseling
if these patients are also removed from the denominator
86.6%
5.5%
13.0%
49.0%
Validation Results / Audit Results
MNCM completed validation of the data in a three-step process: 1) denominator certification, 2) data file quality
checks, and 3) validation audit. Details of this validation are described in this report.
Denominator Certification
Denominator certification is an essential step in the process to obtaining valid and accurate data. It requires each
participant to attest that they will submit accurate data and follow the measure specifications exactly how they are
written. It also ensures that each participant is querying the correct:




Diagnosis codes (i.e. 345.00, 345.01, 345.10, 345.11, 345.40, 345.41, 345.50, 345.51, 345.70, 345.71, 345.90,
345.91)
Encounter codes (i.e. 99201, 99202, 99203, 99204, 99205, 99212, 99213, 99214, 99215, 99241, 99242, 99243,
99244, 99245)
Date of birth ranges (i.e. 01/01/1968-01/01/2000)
Date of service ranges (i.e. 01/01/2012- 12/31/2012)
MNCM did not identify any major flaws or issues during the review of each medical group’s denominator forms and
therefore each medical group passed denominator certification within the given timeframe. There were a few
corrections and clarifications that required MNCM to send a follow-up email to the respective group; however, each
issue was resolved in a timely manner. The list below documents the issues that were identified and required additional
follow-up based on the information received on the denominator certification forms:
 Incorrect diagnosis codes included in data query
 Group did not indicate if they would be submitting a sample or full population for the measure
 Incorrect encounter codes included in data query
Data File Quality Checks
After each medical group submitted their data file to MNCM, quality checks of the files were completed. Each column in
the data file represented a field of data for each patient row; the following checks were completed:










Number of patients/rows submitted were reasonable/expected
Necessary data fields (columns) were included and completed appropriately
Patient date of birth spanned the expected range
Zip codes were 5-digit and primarily within MN and other bordering states as expected
Race field(s) were included and populated appropriately
Provider NPI field was included and number of providers expected
Insurance information was included and was reasonable
Office visit dates and counseling dates spanned the expected range
Diagnoses were included and spanned the entire list of expected codes
Medical reasons for NOT counseling were applied correctly; were not misused
Issues identified through the data file quality checks were generally minor, requiring no corrections. Other mentionable
items include:
1. All three groups did not have patients with diagnosis codes 345.70 (Epilepsia partialis continua, without mention
of intractable epilepsy) or 345.71 (Epilepsia partialis continua, with intractable epilepsy). These are rare
diagnoses and did not come up in the population.
2. Medical groups B and C listed many neurological or congenital conditions as reasons for the patient to NOT
receive counseling. These were verified during audit.
3. Medical group C did not include their entire population in first submission, excluding patients whose date of
birth was between January thru June 1968. They queried their system again, this time using the specific dates of
birth (rather than “age” values) and included the additional patients in their denominator.
Validation Audit
After the data file checks were completed, MNCM completed audits of the patient records to verify the submitted
clinical data. We also verified the diagnosis of epilepsy and other demographic data (e.g., race).
MNCM uses a validation process developed by the NCQA – National Committee for Quality Assurance,
known as the “8 and 30” process. In this process, the first eight records are verified for accuracy and if
no errors are identified, the data is considered to be 100% compliant. If errors in the first eight records
are identified, we continue reviewing the total 30 records to identify any error patterns and or issues
that may need correction.
The audits revealed some data errors, requiring one medical group to make corrections and resubmit data. Individual
medical group results were as follows:
Medical
group
A
B
C
Audit details
Follow-up action
8 records reviewed, 8 records compliant (100%)
No further action necessary
3 additional records were reviewed for “other” reason patient was not
counseled
o 2 records were compliant
o 1 record indicated patient had functional seizures and not epilepsy,
but should have been counted as no counseling provided
30 records reviewed, 26 compliant (87%)
o Errors: three records had code “2” for no counseling due to
intellectual disability, however, we could not verify the diagnosis in
the record; one record reported as “1” counseling given could not be
verified
Group verified patients they
submitted who were “surgically
sterile” or who had “intellectual
disability”; resubmitted data with
corrections
3 additional records were reviewed whose patients were listed as “cognitively
impaired” as a reason for not receiving counseling; verified that these patients
had mild retardation; medical group staff corroborated that all 115 patients
with this designation also had mild retardation
8 records reviewed, 8 records compliant (100%)
No further action necessary
We identified one record in the eight reviewed in which the patient could have
been flagged for a medical reason to NOT receive counseling (99 “other), but
because the reasons were not either type (surgically sterile, intellectual
disability), it was appropriate that these patients could have been counseled;
these were not counted as errors
Validation/ Audit Conclusion
The validation process was successful in identifying errors (with subsequent corrections) and verifying the accuracy of
the data submitted by medical groups A, B, and C. Finding no significant flaws or errors with the data MNCM is
confident the rate calculation and any additional data analysis can be completed using validated and reliable data.
Additionally, during a review of the National Quality Forum’s feedback to the American Academy of Neurology for this
measure, it was noted that there was a concern that this may simply be a “check-the-box” measure. During the
validation audit, it was noted on several occasions that the practices provided excellent, personalized progress notes
about the counseling that was being provided, that were above and beyond a “check the box”.
Limitations
The main limitation that MNCM identified during the testing of the Counseling for Women with Epilepsy measure is
related to the denominator of included and excluded patients. The measure specifications offered two different options
for excluding patients from the measure:
1. Patient was surgically sterile (tubal ligation, hysterectomy)
2. Patient has an intellectual disability as defined by ICD-9 codes
a. 318.0 moderate intellectual disabilities; IQ 35 to 48
b. 318.1 severe intellectual disabilities; IQ 20 to 34
c. 318.2 profound intellectual disabilities; IQ under 20
Groups submitted these patients and indicated which reason applied. Additionally, if they felt that there was another
medical reason for not providing counseling, they indicated this by a code and accompanying description. These reasons
were not used to exclude patients from the measure; rather the purpose was to provide additional information about
the population of patients included in the measure.
Reasons Provided by Medical Groups for Not Providing Counseling:
Reason by Frequency
Count
Valid
Thoughts
cognitive impairment/ deficit
138
Maybe
cerebral palsy
45
Yes
need to quantify by code
neurodevelopmental disorder
18
Yes
encephalopathy
15
Yes
developmental delay
14
No
may still be at risk
hydrocephalus
8
Yes
brain injury
8
Yes
pregnancy
7
No
still needs counseling
pre-menarche
7
No
autism
4
No
downs syndrome
3
Maybe
aspergers
2
No
spectrum of functioning; at risk
birth control- IUD
2
No
learning disability
2
No
spectrum of functioning; at risk
menopause
1
Yes
no menses
1
Maybe
multiple sclerosis
1
No
subjective and may still be at risk
Spectrum of functioning; at risk
Future Measure Implementation
During the analysis of this data and also as a byproduct of the validation audit in reviewing medical records, MNCM staff
has some concerns regarding the denominator and intent of the measure. There may need to be a consideration for
adding a component of indicating that the patient is sexually active or has the potential to be sexually active, and not
physically handicapped. AAN could refer to the NCQA specifications for the Chlamydia Screening in Women measure
(NQF# 0033/ CMS 153v1) for reference tables indicating how to identify potentially sexually active women via pharmacy
codes, CPTs, ICD-9, UB Revenue and LOINC codes. Rather than trying to capture/ code every possible exclusion; this may
be an option. MNCM would not recommend having a general type exclusions code, like one that is stated as “any
documented medical reason, because providers will use this to their advantage and exclude patients that are at risk for
pregnancy and truly belong in the denominator. Having this type of exclusion weakens the measure, and can impact the
validity and reliability of the results.
Appendix A: Denominator Certification Form
What information do I submit to MNCM?
The instructions in this document will guide you in creating the necessary documentation for Denominator Certification.
Using the following steps, you will construct a Word document. Your document must describe the process you use to
identify eligible patients, including source code and screen shots of your query. After you submit your document to the
MNCM, MNCM will review your document and respond within two business days. Please note: MNCM will review your
denominator method, however, you are ultimately responsible for interpreting and applying the measure specifications
correctly in your query.
Please submit your denominator document to [email protected]
**Please do not submit patient data along with your denominator document**
What is denominator certification?
Denominator Certification is the process by which medical groups submit a document that explains the process they use
to identify patients for the measure. MNCM then reviews the documentation to verify the measure specifications for
the denominator were followed.
What are the denominator criteria?
All females of childbearing potential (12 to 44 years old) with a diagnosis of epilepsy are in the denominator. Please
refer to the data collection guide for the complete measure specifications.
Complete this table and provide additional details
Denominator document
Your response
instructions
1. Medical group information
Measure: Enter measure name
Supply the following information: Medical group name: Enter medical group name
Your name: [Enter your name]
Your phone number: Enter your phone number
Your email address: Enter your email
Name of your medical director, administrator or lead: Enter name of
medical director, administrator or lead
2. Date of birth range:
We will use the following date range to identify patients age 12 to
44:
Enter the date range,
3. Date of service range:
We will use the following date range to identify patients with one or
more face-to-face office visits with a provider:
Enter the date range,
4. ICD-9 diagnosis codes
(epilepsy):
We will use the following ICD-9 diagnosis codes to identify patients
with epilepsy:
Denominator document
instructions
5. Exceptions:
Please indicate how you will
identify exceptions for the
denominator.
Your response
We will identify exceptions as follows:
Patients who are surgically sterile:
 we will identify through manual data abstraction
 other, please describe: Enter here
Patients with intellectual disabilities:
 we will remove upfront using ICD-9 codes 318.0, 318.1 and
318.2
 we will identify through manual data abstraction
6. Attestations
Read each attestation carefully.
You must agree to all
attestations before you submit
your denominator. By submitting
this document, you are indicating
that you agree with these
attestations.
Please contact MNCM if you
have any questions.
1. We agree to follow the denominator criteria outlined in the
measure specifications when searching for eligible patients,
and we are ultimately responsible for interpreting and
applying the measure specifications correctly in our query.
2. We agree to include patients who are not active patients if
they are eligible based on the measure criteria (i.e., we will
include patients whose status is “inactive” or patients who
transferred care).
3. We agree to identify exceptions for surgically sterile or
intellectual disability only. All other patients that meet the
denominator criteria will be included in the denominator.
4. Include one of the following attestations:
a. We agree to submit our full population of eligible
patients
b. We agree to use one of the sampling methods
described in the data collection guide to randomly
select patients
5. We agree to identify and remove any duplicate patients.
6. Our medical director, administrator or other lead attests
that the measure specifications will be followed and all
eligible patients included in the denominator.
Supply source code or screen shots of your query in a Word document
1. Generate a query of your record system (e.g., electronic medical record, practice management system, billing
system); maintain the source code and/or screen shots of the steps you take to search for eligible patients.
Copy and paste 1) the source code or 2) the screen shots of your steps into the Word document
2. Highlight details for the MNCM reviewer:
 Date of birth range
 Date of service range
 ICD-9 diagnosis codes (epilepsy)
 Exceptions (e.g., ICD-9 codes for intellectual disability)
Screen shots: If you cannot include data source code, you can instead copy and paste screen shots of your process in
your Word document. Please only include screen shots that will demonstrate to the MNCM reviewer that you have
used the correct criteria for querying your system for eligible patients. For example, the following criteria must be
clearly shown in the screen shot: correct date of birth range, correct dates of service, correct diagnosis codes if
applicable. Do not include screen shots with blank codes, dates, etc. (this will not demonstrate to the MNCM review
that the criteria used are correct).
**Please do not submit patient data along with your denominator document**
Appendix B: File Quality Checks & Validation
Counseling for Women with Epilepsy measure| File Quality Checks & Validation
File Quality Checks | to verify the data submitted is complete and in the correct format
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
Clinic name is entered consistently for each record
Patient IDs are not duplicated
Date of birth is between = 01/01/2000 to 01/01/1968; full range is covered in data set
Zip codes are 5 digits and primarily MN zip codes
Race was entered in one or more fields with valid codes
Provider NPI fields are all populated with 10-digit values
Insurance product name fields are all populated with valid entries
Primary payer type fields are all populated with valid codes
Office visit date is between 01/01/2012 and 12/31/2012; full range is covered in data set
Diagnosis code was entered with valid code; expect to see more than one type of code entered
Date of contraception counseling is between 01/01/2012 and 12/31/2012; full range is covered in data set
Contraception counseling received? fields are all populated with valid codes
Date of pregnancy counseling is between 01/01/2012 and 12/31/2012; full range is covered in data set
Pregnancy counseling received? fields are all populated with valid codes
Medical reason for not receiving counseling fields that are populated contain valid code
Medical reason for not receiving counseling “other” fields that are populated contain valid reasons (review
each reason given)
17. “Medical Reason” fields are populated minimally
18. “Medical Reason” fields (columns S and T) that are populated must not have counseling dates or values entered
19. Number of rows of patients matches what was expected
Appendix C: MNCM Audit Strategy
The validation process is conducted to verify that the submitted data matches the source data in the medical record.
After the clinical data file is successfully transferred to MNCM and passes the initial quality checks, MNCM will contact
the medical group about the validation process.
MNCM Audit Process
MNCM utilizes the NCQA (National Committee for Quality Assurance) “8 and 30” process for validation audits. The
following method is used for each measure:






MNCM randomly selects 33 records for each clinic site for validation. At most, 30 records for each clinic site will
be reviewed. The additional three records requested are oversamples to ensure there will be 30 records
available on the day of the review.
MNCM auditor reviews the first eight records of the clinic site’s selected sample to verify that the submitted
data matches the source data in the medical record.
If all of the first eight records reviewed are in perfect compliance (100%), the clinic site is determined to be in
high compliance, and the MNCM auditor may determine that no further record review for that site is necessary.
If the first clinic site is in high compliance and the data collection process for all clinic sites within the medical
group is identical, further review may be abbreviated at the discretion of the MNCM auditor.
If clinic sites are not in high compliance after review of the first eight records, the MNCM auditor will continue to
review the remaining 22 records. If after review of all 30 records the clinic site is not in high compliance on all
factors (less than 90%), the MNCM auditor will review the results with the clinic representative and
communicate the results with MNCM. MNCM will then contact the medical group to develop a mutually agreed
upon re-submission plan. (Re-submission plans will only be allowed for errors in the numerator portion.)
Clinic sites that are not in high compliance or have not been in high compliance in a previous MNCM audit may
be held to a more rigorous denominator certification and validation audit.
Data Fields for Audit: Counseling for Women of Childbearing Potential with Epilepsy












Patient ID
Patient DOB
Patient is female
Race 1-5
Office visit date ?
Patient has epilepsy
Date of contraception counseling
Contraception counseling received
Date of pregnancy counseling
Pregnancy counseling received
Medical reason code – no counseling
Medical reason “other” – no counseling
Appendix D: Questions and Answers Received During Measure Testing
Q: Does the counseling need to take place during the measurement period (2012). If patient was seen, for
example in 2011, and counseling was done then, I would assume that would not count towards meeting the
measure. Or can the counseling occur prior to the measurement period .
A: Counseling does need to take place during the measurement period for it to count towards meeting
the measure criteria.
Q: What about patients who have the diagnoses of epilepsy but who may not be on anti-epileptic medication?
Is the presence of anti-epileptic medication being prescribed considered an exclusion from the denominator?
Or should the counseling be provided whether they are on medication or not?
A: The counseling should be provided regardless of whether or not the patient is on medication. A
patient who has epilepsy, but is not on anti-epileptic medication still would be included in the
measure. Although medication is a common treatment, this is a measure to capture if a female patient with
epilepsy was counseled on the disease and treatment (regardless of what type of treatment) and how it may
affect contraception and pregnancy.
Q: We have some patient seen during the measurement period in our clinic for something completely
unrelated to their epilepsy diagnosis, example for sleep disturbance. Their epilepsy is not being addressed at
all. They carry the diagnosis of epilepsy so they are showing up in the denominator. Some may have also been
seen by their regular neurologist but some may not have been. Should these patients be included in the
denominator?
A: These patients should still be included in the denominator as the visits are opportunities to address
how epilepsy and its treatment may affect contraception and pregnancy.
Q: We have patients that are seen by more than one provider during the measurement period with epilepsy
diagnosis so we have two entries for them in the denominator. Should I take only the most recent encounter,
and just use the provider who saw the patient most recently?
A: The patient should only be counted one time in the denominator. For “Provider NPI”, refer to the
data field specs – If both providers saw the patient equally enter the provider NPI who saw the patient most
recently. If the patient received counseling from another provider, though, still submit the dates of the
counseling.
Q: Does patient use of birth control (part of medication list ie; birth control pills) provide an exception for
receiving contraception counseling?
A: No – counseling is still an expectation regardless if the patient is using birth control. This is an
opportunity for the provider to address how epilepsy treatment may affect contraception.
Q: Is tubal ligation considered surgical sterilization?
A: Yes, patients with tubal ligation are excluded because they are considered to be surgically sterile
Q: Is it appropriate to count Intrauterine Device (IUD) as “other” medical reason for not receiving counseling?
A: Actually, based on the intent of the measures this does not count as a reason for not receiving
counseling because having an IUD does not guarantee pregnancy will not occur for that patient. IUDs are not
a permanent procedure and thus it is still appropriate to counsel the about the risks associated with
pregnancy. The measure focuses on counseling about medications that may affect pregnancy or fetal
malformation which may determine the course of treatment for a patient who is not sterile.
Q: If a patient does not plan to get pregnant again and her husband had a vasectomy should she still receive
counseling?
A: While it may be true she does not plan on becoming pregnant at this time it is still appropriate for
the patient to be counseled about the risks associated with using some epilepsy medications that can
negatively affect a pregnancy or may limit the effectiveness of certain contraceptive medications. Unplanned
pregnancies are extremely common.
Q: We reviewed several patients who were on the young side of the age included who did not received counseling.
Yet
we wondered about the cultural appropriateness of discussing contraceptives and pregnancy with 12-14 year old
patients from conservative religious environments.
A: The measure specifications establishes 12 as the lowest age for receiving counseling so in the
example above these patients will not be excluded from the numerator. The measure development work group
discussed this issue quite extensively. The decision to use 12 as the minimum age came from the fact that the
average age that girls have their first experience menstruation is 12. This has been cited by numerous different
research studies and peer reviewed papers.
Q: We found that the ICD-9 codes for intellectual disability (318.0, 318.1, or 318.2) were not sufficiently inclusive.
One
of our patients with ICD-9 code of 315.9 Developmental Delay Mental, would definitely not have been able to
understand counseling. Another patient had Developmental Delay Global 783.42, lives at group home, Family Medicine
started Oral Contraceptives to help with menses, and father makes the decisions for patient. We coded both case as did
not receive counseling for “other” medical reasons.
A: AAN understands that this is a retrospective chart review and that most clinicians and clinics are not
familiar with the use of CPT-II exclusion codes or that they would need to document why something was not
done. The AAN is not looking for specific codes to justify a medical exception or exclusion. Rather we are
looking for something that is written in the medical record that would justify why the measure wasn’t done. So
my response is that if the clinician and/or abstractor looking at what is documented by the clinician feels that
the patient would have a medical exclusion/exception that was appropriate based upon what is written in the
medical record that we accept that. So in the first case where the patient couldn’t understand any counseling
it would be appropriate to exclude this patient with a medical exclusion. Given the details provided for the 2nd
individual (with the inappropriate description of the code 783.42) who lives at a group home, is on
contraception, etc. I would leave it up to the clinician/abstractor to make the decision whether or not she
should be excluded since we do not know all the details of her cognitive impairment. In short, exclusions are
primarily a judgment call by the physician; however, there should be documentation in the record explaining
why the counseling was not appropriate. However, the codes you referenced will not be added to the
exclusion list for the following reasons:
315.9 is actually “Unspecified delay in development” and the definition includes developmental disorder not
otherwise specified and learning disorder not otherwise specified. This code is way too general and could be
used for so many other things besides intellectual disabilities. Thus, MNCM recommends not including it.
783.42 is actually “Lack of normal physiological development in childhood; delayed milestones” and the
definition includes late talker and late walker. Again, we would not recommend adding this code to define
intellectual disability, it would be inappropriate.
Q: If an oral contraceptive is on the active medication list, does the patient need further counseling?
A: Yes, it is still appropriate to counsel the about the risks associated with pregnancy because the
patient may still become pregnant in the future.
Q: What should be done is cases with an initial diagnosis of epilepsy but found to have non-epileptic events upon
further investigation?
A: Since this patient met the denominator for the current measurement year they should be
included. However, in future reporting years this patient would not be included in the denominator according
to your example above. This issue will likely be a random and somewhat rare occurrence and thus
performance results would likely not be significantly affected. At the time/visit that the retrospective chart
review was being done if the patient had a diagnosis of epilepsy the measure should have been done. If the
diagnosis changes in the future they would not be eligible for the measure and shouldn’t have the counseling
at that future date.
Appendix E: Post Measure Survey Results: Resource Use and Data Burden
Q4: Respondent Comments on the Data Elements:
“Some items were extremely time consuming and required me to look through the majority of charts to find data. I
would expect that it will get easier with time as more information is put into our structured data fields that were created
mid-year in 2012, hence the need to manually go through charts in EMR”
Q8: Comments/Suggestions about your experience
“Difficult to put items in your excel spreadsheet format. For example, our insurance extraction pulls what the name of
the insurance is, it doesn't automatically group them into medicaid, medicare, commercial, etc. So it took some time as
some plans can be both. Pulling dates that education was received was difficult as patients may have been seen more
than once in a year, and I can tell by the data extraction that the patient was educated sometime in that year, but then I
have to go through the chart manually to find which visit it took place in.”

TO: Consensus Standards Approval Committee FR: Ashley Morsell

Transcription

Similar documents

Inaugural Epilepsy Symposium - Commonwealth Neurological Society

Seafood and Sandals

Studio della “Connettività Funzionale” con Risonanza Magnetica

Jolly Jumper Young Epilepsy

The manual dexterity subtests of the for Children, and handwriting

Eisenhower Park Parking Field 1, Dogwood Picnic Area East

our profile from the December 2015

The Largest Festival for Epilepsy in the World!

PNACAC Ad Draft 2.2 - Idaho School Counselor Association