TO: Consensus Standards Approval Committee FR: Ashley Morsell
Transcription
TO: Consensus Standards Approval Committee FR: Ashley Morsell
TO: Consensus Standards Approval Committee FR: Ashley Morsell, Program Manager, Measure Maintenance RE: Review of testing for the Centers for Medicare & Medicaid Services measures with time-limited endorsement DA: April 8, 2014 CSAC ACTION REQUIRED Pursuant to the CDP, the CSAC may consider the removal of “time-limited” endorsement status or alternatively, termination of endorsement. If the time-limited endorsement status is removed, the measures will be expected to go through maintenance review on schedule (regardless of the timing of removal of the time-limited status). NQF staff conducted an initial review of the measures’ testing results on reliability and validity. NQF# Title Description Endorsement Measure Date Steward 1814 Counseling for Women of Childbearing Potential with Epilepsy All female patients of childbearing potential (12–44 years old) diagnosed with epilepsy who were counseled about epilepsy and how its treatment may affect contraception and pregnancy at least once a year. 3/6/2013 American Academy of Neurology Maintenance Year/Topic Area 2015/Neurolog y BACKGROUND Measures were granted time-limited endorsement status if they met the criteria except for “field testing.” Developers must demonstrate reliability and validity of the measure as specified as well as potential threats to validity (exclusions, risk adjustment), and any feasibility issues. The measure stewards are asked to provide the testing results in a standard form consistent with the evaluation criteria related to measure testing. The CSAC serves as the review body for determining the adequacy of testing results and then determining whether to remove the time-limited status or remove endorsement of the measures. The measure information submitted by the measure stewards is provided in this package of materials. NQF DRAFT – DO NOT CITE, QUOTE, REPRODUCE, OR CIRCULATE 1 EVALUATION OF MEASURE TESTING NQF staff conducted an initial review of testing results for the ACR measure. The attached measure submission form from ACR provides additional detailed data. ACR included results on reliability and validity testing. NQF# 1814 Counseling for Women of Childbearing Potential with Epilepsy. Reliability Testing • The AAN identified and recruited Neurology practices in Minnesota that have experience treating patients with epilepsy. As part of the recruitment process MNCM and the AAN hosted an informational webinar explaining the purpose of the measurement testing project for the Counseling for Women with Epilepsy measure. • Three Neurology practices volunteered to participate and submit retrospective data from the 2012 calendar year (i.e. dates of service 01/01/2012 – 12/31/2012). MNCM produced a data collection guide, measure flow and detailed file specifications to educate and assist each medical group in the data collection and submission process. o As a requirement of participating in the measure testing each group had to submit a denominator certifications form (see appendix A). • There were a few corrections and clarifications that required MNCM to send a follow-up email to the respective group; however, each issue was resolved in a timely manner. The list below documents the issues that were identified and required additional follow-up based on the information received on the denominator certification forms: o Incorrect diagnosis codes included in data query o Group did not indicate if they would be submitting a sample or full population for the measure o Incorrect encounter codes included in data query Validity Testing • MNCM completed validation of the data in a three-step process: 1) denominator certification, 2) data file quality checks, and 3) validation audit. • Denominator certification is an essential step in the process to obtaining valid and accurate data. It requires each participant to attest that they will submit accurate data and follow the measure specifications exactly how they are written. It also ensures that each participant is querying the correct diagnosis codes, encounter codes, date of birth ranges and date of service ranges. • After each medical group submitted their data file to MNCM, quality checks of the files were completed. Each column in the data file represented a field of data for each patient row; the following checks were completed: o Number of patients/rows submitted were reasonable/expected o Necessary data fields (columns) were included and completed appropriately o Patient date of birth spanned the expected range o Zip codes were 5-digit and primarily within MN and other bordering states as expected NQF DRAFT – DO NOT CITE, QUOTE, REPRODUCE, OR CIRCULATE 2 • • o Race field(s) were included and populated appropriately o Provider NPI field was included and number of providers expected o Insurance information was included and was reasonable o Office visit dates and counseling dates spanned the expected range o Diagnoses were included and spanned the entire list of expected codes o Medical reasons for NOT counseling were applied correctly; were not misused After the data file checks were completed, MNCM completed audits of the patient records to verify the submitted clinical data. We also verified the diagnosis of epilepsy and other demographic data (e.g., race). MNCM uses a validation process developed by the NCQA – National Committee for Quality Assurance, known as the “8 and 30” process. In this process, the first eight records are verified for accuracy and if no errors are identified, the data is considered to be 100% compliant. If errors in the first eight records are identified, we continue reviewing the total 30 records to identify any error patterns and or issues that may need correction. The validation process was successful in identifying errors (with subsequent corrections) and verifying the accuracy of the data submitted by medical groups A, B, and C. Finding no significant flaws or errors with the data MNCM is confident the rate calculation and any additional data analysis can be completed using validated and reliable data. Additionally, during a review of the National Quality Forum’s feedback to the American Academy of Neurology for this measure, it was noted that there was a concern that this may simply be a “check-the-box” measure. During the validation audit, it was noted on several occasions that the practices provided excellent, personalized progress notes about the counseling that was being provided, that were above and beyond a “check the box”. Table 1. Individual Medical Group Results NQF DRAFT – DO NOT CITE, QUOTE, REPRODUCE, OR CIRCULATE 3 Table 2. Validation/Audit Conclusion Data Exclusion Analysis • The main limitation that MNCM identified during the testing of the Counseling for Women with Epilepsy measure is related to the denominator of included and excluded patients. The measure specifications offered two different options for excluding patients from the measure: o Patient was surgically sterile (tubal ligation, hysterectomy) o Patient has an intellectual disability as defined by ICD-9 codes 318.0 moderate intellectual disabilities; IQ 35 to 48 318.1 severe intellectual disabilities; IQ 20 to 34 318.2 profound intellectual disabilities; IQ under 20 • Groups submitted these patients and indicated which reason applied. Additionally, if they felt that there was another medical reason for not providing counseling, they indicated this by a code and accompanying description. NQF DRAFT – DO NOT CITE, QUOTE, REPRODUCE, OR CIRCULATE 4 Table 3. Reasons Provided by Medical Groups for Not Providing Counseling • During the analysis of this data and also as a byproduct of the validation audit in reviewing medical records, MNCM staff has some concerns regarding the denominator and intent of the measure. There may need to be a consideration for adding a component of indicating that the patient is sexually active or has the potential to be sexually active, and not physically handicapped. NQF DRAFT – DO NOT CITE, QUOTE, REPRODUCE, OR CIRCULATE 5 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 Measure Information This document contains the information submitted by measure developers/stewards, but is organized according to NQF’s measure evaluation criteria and process. The item numbers refer to those in the submission form but may be in a slightly different order here. In general, the item numbers also reference the related criteria (e.g., item 1b.1 relates to subcriterion 1b). Brief Measure Information NQF #: 1814 De.2. Measure Title: Counseling for Women of Childbearing Potential with Epilepsy Co.1.1. Measure Steward: American Academy of Neurology De.3. Brief Description of Measure: All female patients of childbearing potential (12–44 years old) diagnosed with epilepsy who were counseled about epilepsy and how its treatment may affect contraception and pregnancy at least once a year 1b.1. Developer Rationale: Educate women about epilepsy and how its treatments may affect contraception and pregnancy. This will inform women of childbearing potential about the risks of epilepsy and AED therapy prior to pregnancy. It will provide an opportunity to educate this population about folic acid supplementation, monotherapyy, medication alternatives and how to obtain obstetrical, prenantal and pregnancy care. This measure will help them understand the risk and mitigate the risks which may prevent fetal malformation, unplanned pregnancies and improve the patients´ quality of life. S.4. Numerator Statement: Female patients counseled about epilepsy and how its treatment may affect contraception and pregnancy and documented in the medical record at least once a year. S.7. Denominator Statement: All females of childbearing potential (12-44 years old) with a diagnosis of epilepsy. S.10. Denominator Exclusions: Patient is surgically sterile, Patient has an intellectual disability. De.1. Measure Type: Process S.23. Data Source: Administrative claims, Electronic Clinical Data : Electronic Health Record S.26. Level of Analysis: Clinician : Individual IF Endorsement Maintenance – Original Endorsement Date: Mar 06, 2013 Most Recent Endorsement Date: Mar 06, 2013 IF this measure is included in a composite, NQF Composite#/title: IF this measure is paired/grouped, NQF#/title: De.4. IF PAIRED/GROUPED, what is the reason this measure must be reported with other measures to appropriately interpret results? 1. Evidence, Performance Gap, Priority – Importance to Measure and Report Extent to which the specific measure focus is evidence-based, important to making significant gains in healthcare quality, and improving health outcomes for a specific high-priority (high-impact) aspect of healthcare where there is variation in or overall lessthan-optimal performance. Measures must be judged to meet all subcriteria to pass this criterion and be evaluated against the remaining criteria. 1a. Evidence to Support the Measure Focus – See attached Evidence Submission Form 1814_Evidence_MSF5.0_Data.doc 1b. Performance Gap Demonstration of quality problems and opportunity for improvement, i.e., data demonstrating: • considerable variation, or overall less-than-optimal performance, in the quality of care across providers; and/or NATIONAL QUALITY FORUM Form version 6.5 1 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 • disparities in care across population groups. 1b.1. Briefly explain the rationale for this measure (e.g., the benefits or improvements in quality envisioned by use of this measure) Educate women about epilepsy and how its treatments may affect contraception and pregnancy. This will inform women of childbearing potential about the risks of epilepsy and AED therapy prior to pregnancy. It will provide an opportunity to educate this population about folic acid supplementation, monotherapyy, medication alternatives and how to obtain obstetrical, prenantal and pregnancy care. This measure will help them understand the risk and mitigate the risks which may prevent fetal malformation, unplanned pregnancies and improve the patients´ quality of life. 1b.2. Provide performance scores on the measure as specified (current and over time) at the specified level of analysis. (This is required for endorsement maintenance. Include mean, std dev, min, max, interquartile range, scores by decile. Describe the data source including number of measured entities; number of patients; dates of data; if a sample, characteristics of the entities included). This information also will be used to address the subcriterion on improvement (4b.1) under Usability and Use. The recent Institute of Medicine Report "Epilepsy Across the Spectrum: Promoting Health and Understanding" noted there are gaps in care in responding to the specific needs of women with epilepsy. "The specific knowledge needed by women with epilepsy, which may vary by age, has generally received insufficient attention. Because sex hormones can affect seizure frequency, girls and women need information related to hormonal fluctuations and seizure frequency. Further, women of reproductive age need to understand how their epilepsy and its treatment could affect pregnancy. In a UK survey, adult women with epilepsy between the ages of 19 and 44 identified their most important information needs as relating to risks of epilepsy and medication affecting the fetus (87 percent), the effect of pregnancy on seizure control (49 percent), and the risk of their children developing epilepsy (42 percent) (Crawford and Hudson, 2003). For example, recent findings that show an increased risk for congenital malformations and impaired cognition in children of women treated during pregnancy with valproate, a commonly used seizure medication, suggest that all women of child-bearing age need to be kept apprised of the latest research in this area (Harden et al., 2009). Women with epilepsy also have been found to have higher than expected rates of sexual dysfunction (Pennell and Thompson, 2009). Among women over age 44, the most important information needs concerned epilepsy medication and osteoporosis (63 percent), seizure medications and aging (57 percent), and seizure changes during menopause (44 percent) (Crawford and Hudson, 2003)." Studies have found higher-than-expected onset of seizures during the year of menarche; in girls with preexisting seizures, 29 percent experienced more frequent seizures during perimenarche (Klein et al., 2003). Because of hormonal fluctuation, some women have a cyclic pattern of seizure frequency associated with their menses that often is unrecognized (Pennell and Thompson, 2009). Epidemiological research is needed in large, representative U.S. populations to monitor trends in epilepsy incidence and related mortality and to track outcomes. Studies need to be conducted among the general population and in subpopulations at higher risk: children, for whom prognosis is a major concern; older adults, who have greater mortality associated with epilepsy; women, to track outcomes, including reproductive outcomes; as well as veterans and diverse racial or ethnic and socioeconomic groups, in order to assess any disparities in incidence, prognosis, and mortality and to determine opportunities for intervention. Within these subpopulations, sufficient numbers are needed to compare incidence by etiology, seizure type, syndrome, and the presence of comorbid conditions. With respect to treatment, these surveillance data could be used to monitor the outcomes of epilepsy care and provide feedback to health care providers (Box et al., 2010; Trevathan, 2011). As examples, specific populations for whom further research is needed—older adults, veterans, children, and people with epilepsy and associated comorbidities—are described below. Women • Susceptibility to changes in seizures during menstrual cycle or at other times of hormonal fluctuations (e.g., menopause) • Potential impact of seizures and/or medications on reproductive functioning, pregnancy, breastfeeding • Risk for malformations and impaired cognitive development of offspring of women taking seizure medications or suffering seizures during pregnancy" QUIET Indicator Study. Indicators #16 and #24 were used as support for this measure. The 2011 Pugh et al. study on quality of care for adults with epilepsy highlights the gap in care for women’s issues with very low concordance rates and the study states that women’s issues should be of concern for clinicians. The proportion of patients receiving quality indicator concordance care by setting for all settings for quality indicator #16 (patients with epilepsy should receive an annual review of information including such topics as…contraception, family planning, and how pregnancy and menopause may affect seizures) was 1.93% and for quality indicator #24 (If a woman with epilepsy is of childbearing potential and receives oral contraceptives in conjunction with an enzyme inducing AED, NATIONAL QUALITY FORUM Form version 6.5 2 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 THEN decreased effectiveness of oral contraception should be addressed.) quality indicator concordant care was provided for only 20% of the patients. Overall of the 111 women of childbearing potential with 128 opportunities for recommended care, only 36.72% of the time quality indicator concordant care was provided. Patient reported adherence of their physician to the quality measure (Strongly Agree, Agree; Disagree, & Strongly Disagree not noted here) Women with epilepsy (27% Strongly Agree, 19% Agree) (birth control) 1b.3. If no or limited performance data on the measure as specified is reported in 1b2, then provide a summary of data from the literature that indicates opportunity for improvement or overall less than optimal performance on the specific focus of measurement. Crawford, P., and S. Hudson. 2003. Understanding the information needs of women with epilepsy at different lifestages: Results of the “Ideal World” survey. Seizure 12(7):502-507. Wick P, Fountain N. Patient reported clinician adherence to Epilepsy Performance Measures of Quality of Care. (Before the publication of the Quality Measures) Poster. Epilepsy Meeting Dec 2010 Gumnit R. We are Failing Our Patients: Guidelines and Quality Measures Epilepsia Accessed 09/24/12. http://www.mincep.com/pdfs/publications/Epilepsia%20Editorial%20We%20Are%20Failing%20Our%20Patients.pdf Fountain NB, Van Ness PC, Swain-Eng R, et al. Quality improvement in neurology: AAN epilepsy quality measures. Neurology 2011;76:94-99. Wicks P, Massagli M, Frost J, et al. Sharing health data for better outcomes on PatientsLikeMe. J Med Internet Res 2010;12:e19. Harden CL, Meador KJ, Pennell PB, et al., the American Academy of Neurology, and the American Epilepsy Society. 2009. Management issues for women with epilepsy—Focus on pregnancy (an evidence-based review): II. Teratogenesis and perinatal outcomes: Report of the Quality Standards Subcommittee and Therapeutics and Technology Subcommittee of the American Academy of Neurology and the American Epilepsy Society. Epilepsia 50(5):1237-1246. Pennell, P. B., and P. Thompson. 2009. Gender-specific psychosocial impact of living with epilepsy. Epilepsy and Behavior 15(Suppl. 1):S20-S25 Shafer, P. O. 1998. Counseling women with epilepsy. Epilepsia 39(Suppl. 8):S38-S44. 2009. Epilepsy self-management in clinical practice: What we do and know. Paper read at AES Annual Meeting, Boston, MA: Hynes Conference Center. Institute of Medicine Report "Epilepsy Across the Spectrum: Promoting Health and Understanding" http://www.iom.edu/Reports/2012/Epilepsy-Across-the-Spectrum.aspx Pugh MJ, Berlowitz DR, Rao JK, et al. The quality of care for adults with epilepsy: an initial glimpse using the QUIET measure. 2011; BMC, Health Services Research; 11:1 www.biomedcentral.com/1472-6963/11/1 1b.4. Provide disparities data from the measure as specified (current and over time) by population group, e.g., by race/ethnicity, gender, age, insurance status, socioeconomic status, and/or disability. (This is required for endorsement maintenance. Describe the data source including number of measured entities; number of patients; dates of data; if a sample, characteristics of the entities include.) This information also will be used to address the subcriterion on improvement (4b.1) under Usability and Use. Racial/ethnic minorities also appear to have limited knowledge about epilepsy and its treatment, experience barriers to care, lack social support, and seek alternative therapies for epilepsy. (Szaflarski) Specific risks for women with epilepsy: Susceptibility to changes in seizures during menstrual cycle or at other times of hormonal fluctuations (e.g., menopause) Potential NATIONAL QUALITY FORUM Form version 6.5 3 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 impact of seizures and/or medications on reproductive functioning, pregnancy, breastfeeding Risk for malformations and impaired cognitive development of offspring of women taking seizure medications or suffering seizures during pregnancy A study in the Harlem neighborhood of New York City found epilepsy prevalence to be higher in Hispanics than in non-Hispanics and a higher prevalence of active epilepsy3 in whites than in blacks, although the prevalence of lifetime epilepsy4 was higher in blacks compared to whites (Kelvin et al., 2007). In this community, there were racial and ethnic disparities in care; blacks were more likely to receive care in the emergency department compared to whites and Hispanics. Similarly, Hope and colleagues (2009) found that blacks and Hispanics were more likely than whites to be diagnosed in an emergency department, and blacks were more likely to receive a suboptimal seizure medication. Differences in care for prevalent epilepsy were also observed in residents of Alabama and surrounding states, where blacks were 60 percent less likely than non-Hispanic whites to undergo epilepsy surgery after receiving electroencephalograph (EEG) monitoring as part of a surgical evaluation, an association that persisted after controlling for factors such as SES and medical insurance coverage (Burneo et al.,2005). The degree to which differences in epilepsy incidence and prevalence in different racial and ethnic groups reflect differences in socioeconomic status is unknown. Also unknown is the degree to which the treatment gap contributes to the higher epilepsy prevalence in some subgroups. There are some patient reported differences in physician adherence to the quality measures by the type of clinician. N=221 overall. Women of childbearing N=61 Women with epilepsy (birth control): 67% adherence (epileptologist); 74% adherence (neurologist); 75% adherence (PCP); 0 responses (Other clinician); p-value 0.862 1b.5. If no or limited data on disparities from the measure as specified is reported in 1b4, then provide a summary of data from the literature that addresses disparities in care on the specific focus of measurement. Include citations. Szaflarski M, Szaflarski JP, Privitera MD et al. Racial/ethnic disparities in the treatment of epilepsy: What do we know? What do we need to know? Epilepsy & Behavior 2006; 9(2): 243-64. Wick P, Fountain N. Patient reported clinician adherence to Epilepsy Performance Measures of Quality of Care. (Before the publication of the Quality Measures) Poster. Epilepsy Meeting Dec 2010. Institute of Medicine Report "Epilepsy Across the Spectrum: Promoting Health and Understanding" http://www.iom.edu/Reports/2012/Epilepsy-Across-the-Spectrum.aspx 1c. High Priority (previously referred to as High Impact) The measure addresses: • a specific national health goal/priority identified by DHHS or the National Priorities Partnership convened by NQF; OR • a demonstrated high-priority (high-impact) aspect of healthcare (e.g., affects large numbers of patients and/or has a substantial impact for a smaller population; leading cause of morbidity/mortality; high resource use (current and/or future); severity of illness; and severity of patient/societal consequences of poor quality). 1c.1. Demonstrated high priority aspect of healthcare Affects large numbers, A leading cause of morbidity/mortality, Patient/societal consequences of poor quality, Severity of illness 1c.2. If Other: 1c.3. Provide epidemiologic or resource use data that demonstrates the measure addresses a high priority aspect of healthcare. List citations in 1c.4. Recent estimates of the US population and prevalence of epilepsy indicate that approximately one-half million women with epilepsy (WWE) are of childbearing age. It has also been estimated that 3-5 births per thousand will be to WWE. Epilepsy is defined by the presence of recurrent, unprovoked seizures, and the treatment is typically a daily, long-term antiepileptic drugs (AED) regiment. The majority of people with epilepsy have well-controlled seizures, are otherwise healthy, and therefore expect to participate fully in life experiences, including childbearing. Epilepsy is associated with sexual dysfunction, reduced fertility, increased pregnancy risks, and risks for malformations in the infant. Seizures can transiently disrupt pituitary hormone secretion. Treatment of seizures with antiepileptic drugs may alter hormone levels, render oral contraceptives less effective and may interfere with embryonic and fetal development. Certain antiepileptic mediations may have specific malformation risks. Since unplanned pregnancy is common, NATIONAL QUALITY FORUM Form version 6.5 4 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 patients need to be informed about the risks of epilepsy and antiepileptic drug therapy prior to pregnancy. Folic acid supplementation, monotherapy for epilepsy, using lower doses of medication when possible and proper obstetrical, prenatal and pre-pregnancy care all should be discussed with the patient so they understand the risks involved and how to mitigate these risks. The specific knowledge needed by women with epilepsy, which may vary by age, has generally received insufficient attention. Because sex hormones can affect seizure frequency, girls and women need information related to hormonal fluctuations and seizure frequency. Studies have found higher-than-expected onset of seizures during the year of menarche; in girls with preexisting seizures, 29 percent experienced more frequent seizures during perimenarche (Klein et al., 2003). Because of hormonal fluctuation, some women have a cyclic pattern of seizure frequency associated with their menses that often is unrecognized (Pennell and Thompson, 2009). "Our suspicions have been confirmed: epilepsy affects women differently. Their hormonal and menstrual cycles, pregnancy, menopause—all of those life stages are affected by epilepsy," said Edna Kane-Williams, vice president of programs and services for the Epilepsy Foundation. Furthermore, Ms. Kane-Williams said, many medical professionals seem to be in the dark. "We´ve done a professional awareness survey that showed that the physicians these women were seeing weren´t aware of the differences," she reported. When women with epilepsy have problems, they are often hormone-based, according to Dr. Mark Yerby, founder of North Pacific Epilepsy Research in Portland, Oregon, and a nationally recognized authority on the subject. Risks from seizures and from anti-epileptic drugs Both seizures and medications are associated with some risks. The risk of seizures is associated with seizure type. Partial seizures probably do not carry as much risk but they may become generalized seizures, and generalized tonic-clonic seizures are associated with increased risk to both the mother and baby. These risks include trauma from falls or burns, increased risk of premature labor, miscarriages, and fetal heart rate suppression. Seizure control is necessary because the risks from seizures are felt by epileptologists to be greater than the risks from medications, which may be minimized by utilizing specific strategies. Strategies to minimize risks Most importantly, women should get accurate information prior to and during pregnancy. If anti-epileptic drugs are not needed, multiple medications are being taken, or medications are given at high dosages, changes should be considered with a neurologist prior to a planned pregnancy. The lowest possible anti-epileptic drug dose that will continue to maintain seizure control is recommended. Being on a single drug, monotherapy, will decrease the risk of birth defects and result in fewer drug interactions, fewer side effects, and improve compliance. The 2012 IOM report “Epilepsy Across the Spectrum” explicitly stated the need for the development and implementation of a national quality measurement and improvement strategy for epilepsy care. “An independent organization with expertise in quality measurement and care should assist in the development of the national strategy, particularly the development of performance metrics.” Specifically, the IOM report calls for the national quality strategy to include defining performance metrics for epilepsy with specific attention to access to care for underserved populations, access to specialized care, co-management of care among specialized epilepsy providers, and coordination of care with other health care providers and community services organizations. The AAN is a non-profit professional association with extensive experience and expertise in developing quality measures for neurological conditions and has developed eight quality measures for epilepsy care. The AAN has not yet completed testing of these measures. Three of the epilepsy measures were chosen for inclusion in the 2012 PQRS program and thus are under consideration for endorsement by the NQF at this time. The incidence in females, at 41 cases per 100,000 person years, is less than that for males, at 49 cases per 100,000 person years.[2] Approximately 1 million women of childbearing age in the United States have seizure disorders. Of these women, approximately 20,000 give birth each year. Concerns during these pregnancies include the risk of fetal malformation, miscarriage, perinatal death, and increased seizure frequency.[1] In women who are pregnant, the volume of distribution and the hepatic metabolism of AEDs are increased. This, along with decreased compliance with AEDs because of concerns about their effects on the fetus, leads to an increase in seizure frequency, which is observed in as many as 17-33% of pregnancies. The use of antiepileptic drugs (AEDs) is associated with a greater baseline risk of fetal malformations during pregnancy. When treating pregnant women who have epilepsy, the risks of increased seizure frequency versus the risks of AED use must be weighed carefully. A population-based study conducted in Norway found that pregnant women with epilepsy had a lower risk of complications but an NATIONAL QUALITY FORUM Form version 6.5 5 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 increased risk of induction, cesarean delivery, and postpartum hemorrhage.[2] However, whether this is a result of AEDs or severe epilepsy is unclear. 1c.4. Citations for data demonstrating high priority provided in 1a.3 United States Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics, Bridged-Race Population Estimates, United States. July 1st resident population by state, country, age, sex, bridged-race, and Hispanic origin on CDC WONDER on-line Database. Available at: http://wonder/cdc.gov/ Accessed June 2012. Hirtz D, Thurman DJ, Gwinn-Hardy K, et al. How common are the “common” neurological disorders? Neurology 2007; 68:326-337. Yerby MS. Quality of life, epilepsy advances, and the evolving role of anticonvulsants in women with epilepsy. Neurology 2000; 55:S21-31. Harden CL, Hopp J, Tin TY, et al. Practice parameter update: Management issues for women with epilepsy-Focus on pregnancy (an evidence-based review): Obstetrical complications and change in seizure frequency: Report of the Quality Standards Subcommittee and Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology and American Epilepsy Society. Neurology 2009; 73:126-132. Practice parameter: management issues for women with epilepsy (summary statement). Report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology. 51(4):944-8, 1998 Pennell PB. The importance of monotherapy in pregnancy. Neurology. 60(11 Suppl 4):S31-8, 2003 Yerby MS. Management issues for women with epilepsy: neural tube defects and folic acid supplementation. Neurology. 61(6 Suppl 2):S23-6, 2003 Crawford, P., and S. Hudson. 2003. Understanding the information needs of women with epilepsy at different lifestages: Results of the “Ideal World” survey. Seizure 12(7):502-507. Pennell, P. B., and P. Thompson. 2009. Gender-specific psychosocial impact of living with epilepsy. Epilepsy and Behavior 15(Suppl. 1):S20-S25 Shafer, P. O. Counseling women with epilepsy. Epilepsia 1998; 39(Suppl. 8):S38-S44. 2009. Epilepsy self-management in clinical practice: What we do and know. Paper read at AES Annual Meeting, Boston, MA: Hynes Conference Center. Cramer JA, Gordon J, Schachter S, Devinsky O, and the Epilepsy Therapy Development Project Women’s Issues Work Group. Women with Epilepsy: Hormonal Issues from Menarche through Menopause. Epilepsy Behav. 2007; 11: 160-178 Epilepsy Therapy Project http://www.epilepsy.com/INFO/WOMEN_PREGNANCY Institute of Medicine Report "Epilepsy Across the Spectrum: Promoting Health and Understanding" http:www.iom.edu/Reports/2012/Epilepsy-Across-the-Spectrum.aspx 1. Katz O, Levy A, Wiznitzer A, Sheiner E. Pregnancy and perinatal outcome in epileptic women: a population-based study. J Matern Fetal Neonatal Med. Jan 2006;19(1):21-5. 2. Borthen I, Eide MG, Daltveit AK, Gilhus NE. Delivery outcome of women with epilepsy: a population-based cohort study. BJOG. Nov 2010;117(12):1537-43 1c.5. If a PRO-PM (e.g. HRQoL/functional status, symptom/burden, experience with care, health-related behaviors), provide evidence that the target population values the measured PRO and finds it meaningful. (Describe how and from whom their input was obtained.) NATIONAL QUALITY FORUM Form version 6.5 6 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 2. Reliability and Validity—Scientific Acceptability of Measure Properties Extent to which the measure, as specified, produces consistent (reliable) and credible (valid) results about the quality of care when implemented. Measures must be judged to meet the subcriteria for both reliability and validity to pass this criterion and be evaluated against the remaining criteria. 2a.1. Specifications The measure is well defined and precisely specified so it can be implemented consistently within and across organizations and allows for comparability. eMeasures should be specified in the Health Quality Measures Format (HQMF) and the Quality Data Model (QDM). De.5. Subject/Topic Area (check all the areas that apply): Neurology, Perinatal and Reproductive Health, Prevention De.6. Cross Cutting Areas (check all the areas that apply): Patient and Family Engagement, Prevention, Safety, Safety : Medication Safety S.1. Measure-specific Web Page (Provide a URL link to a web page specific for this measure that contains current detailed specifications including code lists, risk model details, and supplemental materials. Do not enter a URL linking to a home page or to general information.) http://www.aan.com/globals/axon/assets/9079.pdf S.2a. If this is an eMeasure, HQMF specifications must be attached. Attach the output from the eMeasure authoring tool (MAT) - if the MAT was not used, contact staff. (Use the specification fields in this online form for the plain-language description of the specifications) Attachment: S.2b. Data Dictionary, Code Table, or Value Sets (and risk model codes and coefficients when applicable) must be attached. (Excel or csv file in the suggested format preferred - if not, contact staff) Attachment: S.3. For endorsement maintenance, please briefly describe any changes to the measure specifications since last endorsement date and explain the reasons. S.4. Numerator Statement (Brief, narrative description of the measure focus or what is being measured about the target population, i.e., cases from the target population with the target process, condition, event, or outcome) IF an OUTCOME MEASURE, state the outcome being measured. Calculation of the risk-adjusted outcome should be described in the calculation algorithm. Female patients counseled about epilepsy and how its treatment may affect contraception and pregnancy and documented in the medical record at least once a year. S.5. Time Period for Data (What is the time period in which data will be aggregated for the measure, e.g., 12 mo, 3 years, look back to August for flu vaccination? Note if there are different time periods for the numerator and denominator.) Within a 12 month reporting period S.6. Numerator Details (All information required to identify and calculate the cases from the target population with the target process, condition, event, or outcome such as definitions, specific data collection items/responses, code/value sets – Note: lists of individual codes with descriptors that exceed 1 page should be provided in an Excel or csv file in required format at S.2b) IF an OUTCOME MEASURE, describe how the observed outcome is identified/counted. Calculation of the risk-adjusted outcome should be described in the calculation algorithm. CPT II 4340F: Counseling for women of childbearing potential with epilepsy S.7. Denominator Statement (Brief, narrative description of the target population being measured) All females of childbearing potential (12-44 years old) with a diagnosis of epilepsy. NATIONAL QUALITY FORUM Form version 6.5 7 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 S.8. Target Population Category (Check all the populations for which the measure is specified and tested if any): Senior Care S.9. Denominator Details (All information required to identify and calculate the target population/denominator such as definitions, specific data collection items/responses , code/value sets – Note: lists of individual codes with descriptors that exceed 1 page should be provided in an Excel or csv file in required format at S.2b) Epilepsy ICD-9-CM diagnosis codes n 345.00, 345.01, 345.10, 345.11, 345.40. 345.41, 345.50, 345.51, 345.60, 345.61, 345.70, 345.71, 345.90, 345.91 AND CPT codes 99201, 99202, 99203, 99204, 99205 99212, 99213, 99214, 99215 99304, 99305, 99306, 99307, 99308, 99309 S.10. Denominator Exclusions (Brief narrative description of exclusions from the target population) Patient is surgically sterile, Patient has an intellectual disability. S.11. Denominator Exclusion Details (All information required to identify and calculate exclusions from the denominator such as definitions, specific data collection items/responses, code/value sets – Note: lists of individual codes with descriptors that exceed 1 page should be provided in an Excel or csv file in required format at S.2b) CPT II 4340F–1P: Documentation of medical reason(s) why counseling was not performed for women of childbearing potential with epilepsy. CPT II 4340F–8P: Counseling about epilepsy specific safety issues provided to patient or caregiver was not performed, reason not otherwise specified. S.12. Stratification Details/Variables (All information required to stratify the measure results including the stratification variables, definitions, specific data collection items/responses, code/value sets – Note: lists of individual codes with descriptors that exceed 1 page should be provided in an Excel or csv file in required format with at S.2b) S.13. Risk Adjustment Type (Select type. Provide specifications for risk stratification in S.12 and for statistical model in S.14-15) No risk adjustment or risk stratification If other: S.14. Identify the statistical risk model method and variables (Name the statistical method - e.g., logistic regression and list all the risk factor variables. Note - risk model development and testing should be addressed with measure testing under Scientific Acceptability) N/A S.15. Detailed risk model specifications (must be in attached data dictionary/code list Excel or csv file. Also indicate if available at measure-specific URL identified in S.1.) Note: Risk model details (including coefficients, equations, codes with descriptors, definitions), should be provided on a separate worksheet in the suggested format in the Excel or csv file with data dictionary/code lists at S.2b. S.15a. Detailed risk model specifications (if not provided in excel or csv file at S.2b) N/A S.16. Type of score: Rate/proportion If other: S.17. Interpretation of Score (Classifies interpretation of score according to whether better quality is associated with a higher score, NATIONAL QUALITY FORUM Form version 6.5 8 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 a lower score, a score falling within a defined interval, or a passing score) Better quality = Higher score S.18. Calculation Algorithm/Measure Logic (Describe the calculation of the measure score as an ordered sequence of steps including identifying the target population; exclusions; cases meeting the target process, condition, event, or outcome; aggregating data; risk adjustment; etc.) See Full AAN Women with Epilepsy of Childbearing Potential Measure Testing Report. MNCM completed validation of the data in a three-step process: 1) denominator certification, 2) data file quality checks, and 3) validation audit. Details of this validation are described in this report. S.19. Calculation Algorithm/Measure Logic Diagram URL or Attachment (You also may provide a diagram of the Calculation Algorithm/Measure Logic described above at measure-specific Web page URL identified in S.1 OR in attached appendix at A.1) S.20. Sampling (If measure is based on a sample, provide instructions for obtaining the sample and guidance on minimum sample size.) IF a PRO-PM, identify whether (and how) proxy responses are allowed. See Full AAN Women with Epilepsy of Childbearing Potential Measure Testing Report. To achieve the goal of testing the Counseling for Women with Epilepsy measure the AAN contracted with Minnesota Community Measurement (MNCM), a non-profit organization specializing in health care quality measurement and reporting, to collect data pertaining to this measure from Neurology practices. In order to achieve a reliable sample of patients for this measure MNCM and AAN are sought a minimum of 1,000 combined patient records from the Neurology sites that agree to participate. The AAN identified and recruited Neurology practices in Minnesota that have experience treating patients with epilepsy. Three Neurology practices volunteered to participate and submit retrospective data from the 2012 calendar year (i.e., dates of service 01/01/2012 – 12/31/2012). MNCM produced a data collection guide, measure flow and detailed file specifications to educate and assist each medical group in the data collection and submission process. As a requirement of participating in the measure testing each group had to submit a denominator certifications form (see appendix A of the final report). The denominator certification process helps ensure that each medical group is using the appropriate measure parameters and collecting data in a standardized way. Once the denominator certifications were complete each group submitted their data files using a secure FTP transfer process. Once the files were received MNCM performed quality checks on each files using the methods outlined in Appendix B of the final report. Once the files passed the quality checks MNCM calculated and sent the results back to each group for review. If the group did not find any issues with the measure results then MNCM conducted an audit to validate the accuracy of data using the auditing principles outlined in Appendix C of the final report. The final step in testing the Women with Epilepsy measure was a post-data submission survey that was sent to each of the participating medical groups. The intent of the survey was to shed light on the amount of resources that were required to produce the data as well as gauge the level of data collection burden that each data field presented to the group. The results of the survey can be found in Appendix E of the final report. The following tables are excerpted from the AAN’s report. S.21. Survey/Patient-reported data (If measure is based on a survey, provide instructions for conducting the survey and guidance on minimum response rate.) IF a PRO-PM, specify calculation of response rates to be reported with performance measure results. S.22. Missing data (specify how missing data are handled, e.g., imputation, delete case.) Required for Composites and PRO-PMs. S.23. Data Source (Check ONLY the sources for which the measure is SPECIFIED AND TESTED). If other, please describe in S.24. Administrative claims, Electronic Clinical Data : Electronic Health Record S.24. Data Source or Collection Instrument (Identify the specific data source/data collection instrument e.g. name of database, clinical registry, collection instrument, etc.) IF a PRO-PM, identify the specific PROM(s); and standard methods, modes, and languages of administration. Counseling for Women of Childbearing Potential with Epilepsy Collection Sheet from the AMA PCPI www.physicianconsortium.org NeuroPI Program: for maintenance of certification Performance in Practice module. www.aan.com/practice/pip NATIONAL QUALITY FORUM Form version 6.5 9 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 CECity PQRS Wizard and registry (in development) Physician Quality Reporting System measurement set Patients Like Me website: patient report of physician use of the quality measure S.25. Data Source or Collection Instrument (available at measure-specific Web page URL identified in S.1 OR in attached appendix at A.1) URL S.26. Level of Analysis (Check ONLY the levels of analysis for which the measure is SPECIFIED AND TESTED) Clinician : Individual S.27. Care Setting (Check ONLY the settings for which the measure is SPECIFIED AND TESTED) Ambulatory Care : Clinician Office/Clinic If other: S.28. COMPOSITE Performance Measure - Additional Specifications (Use this section as needed for aggregation and weighting rules, or calculation of individual performance measures if not individually endorsed.) 2a. Reliability – See attached Measure Testing Submission Form 2b. Validity – See attached Measure Testing Submission Form 1814_MeasSubm_MeasTesting_2014_2_27.docx 3. Feasibility Extent to which the specifications including measure logic, require data that are readily available or could be captured without undue burden and can be implemented for performance measurement. 3a. Byproduct of Care Processes For clinical measures, the required data elements are routinely generated and used during care delivery (e.g., blood pressure, lab test, diagnosis, medication order). 3a.1. Data Elements Generated as Byproduct of Care Processes. generated by and used by healthcare personnel during the provision of care, e.g., blood pressure, lab value, medical condition, Coded by someone other than person obtaining original information (e.g., DRG, ICD-9 codes on claims), Abstracted from a record by someone other than person obtaining original information (e.g., chart abstraction for quality measure or registry) If other: 3b. Electronic Sources The required data elements are available in electronic health records or other electronic sources. If the required data are not in electronic health records or existing electronic sources, a credible, near-term path to electronic collection is specified. 3b.1. To what extent are the specified data elements available electronically in defined fields? (i.e., data elements that are needed to compute the performance measure score are in defined, computer-readable fields) No data elements are in defined fields in electronic sources 3b.2. If ALL the data elements needed to compute the performance measure score are not from electronic sources, specify a credible, near-term path to electronic capture, OR provide a rationale for using other than electronic sources. Currently, this measure has been specified for administrative claims. The AAN has contracted with two separate consultants to learn the process to develop eSpecifications, code value sets, logic, and develop eMeasures. The training was complete as of 9/25/12 and the measures will be full specified for eMeasures by December 2012 3b.3. If this is an eMeasure, provide a summary of the feasibility assessment in an attached file or make available at a measurespecific URL. Attachment: NATIONAL QUALITY FORUM Form version 6.5 10 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 3c. Data Collection Strategy Demonstration that the data collection strategy (e.g., source, timing, frequency, sampling, patient confidentiality, costs associated with fees/licensing of proprietary measures) can be implemented (e.g., already in operational use, or testing demonstrates that it is ready to put into operational use). For eMeasures, a feasibility assessment addresses the data elements and measure logic and demonstrates the eMeasure can be implemented or feasibility concerns can be adequately addressed. 3c.1. Describe what you have learned/modified as a result of testing and/or operational use of the measure regarding data collection, availability of data, missing data, timing and frequency of data collection, sampling, patient confidentiality, time and cost of data collection, other feasibility/implementation issues. IF a PRO-PM, consider implications for both individuals providing PROM data (patients, service recipients, respondents) and those whose performance is being measured. During the analysis of this data and also as a byproduct of the validation audit in reviewing medical records, MNCM staff has some concerns regarding the denominator and intent of the measure. There may need to be a consideration for adding a component of indicating that the patient is sexually active or has the potential to be sexually active, and not physically handicapped. AAN could refer to the NCQA specifications for the Chlamydia Screening in Women measure (NQF# 0033/ CMS 153v1) for reference tables indicating how to identify potentially sexually active women via pharmacy codes, CPTs, ICD-9, UB Revenue and LOINC codes. Rather than trying to capture/ code every possible exclusion; this may be an option. MNCM would not recommend having a general type exclusions code, like one that is stated as “any documented medical reason”, because providers will use this to their advantage and exclude patients that are at risk for pregnancy and truly belong in the denominator. Having this type of exclusion weakens the measure, and can impact the validity and reliability of the results. 3c.2. Describe any fees, licensing, or other requirements to use any aspect of the measure as specified (e.g., value/code set, risk model, programming code, algorithm). 4. Usability and Use Extent to which potential audiences (e.g., consumers, purchasers, providers, policy makers) are using or could use performance results for both accountability and performance improvement to achieve the goal of high-quality, efficient healthcare for individuals or populations. 4a. Accountability and Transparency Performance results are used in at least one accountability application within three years after initial endorsement and are publicly reported within six years after initial endorsement (or the data on performance results are available). If not in use at the time of initial endorsement, then a credible plan for implementation within the specified timeframes is provided. 4.1. Current and Planned Use NQF-endorsed measures are expected to be used in at least one accountability application within 3 years and publicly reported within 6 years of initial endorsement in addition to performance improvement. Planned Current Use (for current use provide URL) Public Reporting Payment Program Regulatory and Accreditation Programs Professional Certification or Recognition Program Quality Improvement (Internal to the specific organization) 4a.1. For each CURRENT use, checked above, provide: NATIONAL QUALITY FORUM Form version 6.5 11 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 • Name of program and sponsor • Purpose • Geographic area and number and percentage of accountable entities and patients included NeuroPI MOC program http://tools.aan.com/practice/pip/ 4a.2. If not currently publicly reported OR used in at least one other accountability application (e.g., payment program, certification, licensing) what are the reasons? (e.g., Do policies or actions of the developer/steward or accountable entities restrict access to performance results or impede implementation?) 4a.3. If not currently publicly reported OR used in at least one other accountability application, provide a credible plan for implementation within the expected timeframes -- any accountability application within 3 years and publicly reported within 6 years of initial endorsement. (Credible plan includes the specific program, purpose, intended audience, and timeline for implementing the measure within the specified timeframes. A plan for accountability applications addresses mechanisms for data aggregation and reporting.) 4b. Improvement Progress toward achieving the goal of high-quality, efficient healthcare for individuals or populations is demonstrated. If not in use for performance improvement at the time of initial endorsement, then a credible rationale describes how the performance results could be used to further the goal of high-quality, efficient healthcare for individuals or populations. 4b.1. Progress on Improvement. (Not required for initial endorsement unless available.) Performance results on this measure (current and over time) should be provided in 1b.2 and 1b.4. Discuss: • Progress (trends in performance results, number and percentage of people receiving high-quality healthcare) • Geographic area and number and percentage of accountable entities and patients included We have yet to demonstrate improvement until it can be implemented in programming. The primary reason this measure has not been integrated into more programming is because it isn’t NQF endorsed. 4b.2. If no improvement was demonstrated, what are the reasons? If not in use for performance improvement at the time of initial endorsement, provide a credible rationale that describes how the performance results could be used to further the goal of high-quality, efficient healthcare for individuals or populations. 4c. Unintended Consequences The benefits of the performance measure in facilitating progress toward achieving high-quality, efficient healthcare for individuals or populations outweigh evidence of unintended negative consequences to individuals or populations (if such evidence exists). 4c.1. Were any unintended negative consequences to individuals or populations identified during testing; OR has evidence of unintended negative consequences to individuals or populations been reported since implementation? If so, identify the negative unintended consequences and describe how benefits outweigh them or actions taken to mitigate them. Testing has not begun yet but will be completed by January 2014. Strategies to prevent, minimize or detect unintended consequences will be identified during testing in 2013. Operational use of this measure has not identified any inaccuracies, errors or unintended consequences of measurement 5. Comparison to Related or Competing Measures If a measure meets the above criteria and there are endorsed or new related measures (either the same measure focus or the same target population) or competing measures (both the same measure focus and the same target population), the measures are compared to address harmonization and/or selection of the best measure. 5. Relation to Other NQF-endorsed Measures Are there related measures (conceptually, either same measure focus or target population) or competing measures (conceptually both the same measure focus and same target population)? If yes, list the NQF # and title of all related and/or competing measures. NATIONAL QUALITY FORUM Form version 6.5 12 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 5.1a. List of related or competing measures (selected from NQF-endorsed measures) 5.1b. If related or competing measures are not NQF endorsed please indicate measure title and steward. 5a. Harmonization The measure specifications are harmonized with related measures; OR The differences in specifications are justified 5a.1. If this measure conceptually addresses EITHER the same measure focus OR the same target population as NQF-endorsed measure(s): Are the measure specifications completely harmonized? 5a.2. If the measure specifications are not completely harmonized, identify the differences, rationale, and impact on interpretability and data collection burden. 5b. Competing Measures The measure is superior to competing measures (e.g., is a more valid or efficient way to measure); OR Multiple measures are justified. 5b.1. If this measure conceptually addresses both the same measure focus and the same target population as NQF-endorsed measure(s): Describe why this measure is superior to competing measures (e.g., a more valid or efficient way to measure quality); OR provide a rationale for the additive value of endorsing an additional measure. (Provide analyses when possible.) Appendix A.1 Supplemental materials may be provided in an appendix. All supplemental materials (such as data collection instrument or methodology reports) should be organized in one file with a table of contents or bookmarks. If material pertains to a specific submission form number, that should be indicated. Requested information should be provided in the submission form and required attachments. There is no guarantee that supplemental materials will be reviewed. Attachment: Contact Information Co.1 Measure Steward (Intellectual Property Owner): American Academy of Neurology Co.2 Point of Contact: Rebecca, Swain-Eng, [email protected], 612-928-6121Co.3 Measure Developer if different from Measure Steward: American Academy of Neurology Co.4 Point of Contact: Rebecca, Swain-Eng, [email protected], 612-928-6121- Additional Information Ad.1 Workgroup/Expert Panel involved in measure development Provide a list of sponsoring organizations and workgroup/panel members’ names and organizations. Describe the members’ role in measure development. Co-chairs : Nathan Fountain, MD; Paul Van Ness, MD American Academy of Neurology Facilitator Christopher Bever Jr., MD NATIONAL QUALITY FORUM Form version 6.5 13 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 American Academy of Neurology Jeffrey Buchhalter, MD Andres Kanner, MD K. Babu Krishnamurthy, MD Susan Naselli, MD Piotr Olejniczak, MD Rita Richardson, MD Joseph Sirven, MD Michael Sperling, MD John Stern, MD American Epilepsy Society Allan Krumholz, MD Paul Levisohn, MD Epilepsy Foundation of America Gregory L. Barkley, MD, FAAN Michael C. Smith, MD National Association of Epilepsy Centers David Labiner, MD Thaddeus Walczak, MD American Academy of Family Physicians Mark Potter, MD American Academy of Pediatrics Dennis Dlugos, MD American Academy of Neurological Surgeons/Congress of Neurological Surgeons Joshua Rosenow, MD American Clinical Neurophysiology Society William Tatum IV, DO American College of Emergency Physicians Andrew Jagoda, MD American College of Radiology Eric Russell, MD American Psychological Association Bruce Hermann, PhD American Society of Neuroimaging Ruben Kuzniecky, MD Child Neurology Society Kevin Chapman, MD National Academy of Neuropsychology Gregory Lee, PhD National Organization of Rare Disorders Suki Bagal, MD, MPH NATIONAL QUALITY FORUM Form version 6.5 14 #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated: Feb 27, 2014 Society of Nuclear Medicine James M. Mountz, MD, PhD American Academy of Neurology Staff Rebecca Swain-Eng, MS Sarah Tonn, MPH Gina Gjorvad American Medical Association Convened-Physician Consortium for Performance Improvement Mark Antman, DDS, MBA Consultants Rebecca Kresowik Timothy Kresowik, MD Insurance Representatives Kay Schwebke, MD, UnitedHealth Care Wesley Wong, MD, Wellpoint Thomas James, MD, Humana Robert Kropp, MD, Aetna Measure Developer/Steward Updates and Ongoing Maintenance Ad.2 Year the measure was first released: 2009 Ad.3 Month and Year of most recent revision: 01, 2010 Ad.4 What is your frequency for review/update of this measure? Every 3 years Ad.5 When is the next scheduled review/update for this measure? 01, 2013 Ad.6 Copyright statement: ©2009 American Academy of Neurology. All rights reserved. Physician Performance Measures (measures) and related data specifications developed by the American Academy of Neurology (AAN) are intended to facilitate quality improvement activities by physicians. These measures are intended to assist physicians in enhancing quality of care. Measures are designed for use by any physician who manages the care of a patient for a specific condition or for prevention. These measures are not clinical guidelines and do not establish a standard of medical care, and have not been tested for all potential applications. Measures are subject to review and may be revised or rescinded at any time by the AAN. The measures may not be altered without prior written approval from the AAN. The measures, while copyrighted, can be reproduced and distributed, without modification, for noncommercial purposes (e.g. use by health care providers in connection with their practices). Commercial use is defined as the sale, license, or distribution of the measures for commercial gain, or incorporation of the measures into a product or service that is sold, licensed, or distributed for commercial gain. Commercial uses of the measures require a license agreement between the user and the AAN. Neither the AAN nor its members shall be responsible for any use of the measures. Ad.7 Disclaimers: THESE MEASURES AND SPECIFICATIONS ARE PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND. Limited proprietary coding is contained in the measure specifications for convenience. Users of the proprietary coding sets should obtain all necessary licenses from the owners of these code sets. The AAN and its members disclaim all liability for use or accuracy of any Current Procedural Terminology (CPT®) or other coding contained in the specifications. Ad.8 Additional Information/Comments: CPT ® is a registered trademark of the American Medical Association. NATIONAL QUALITY FORUM Form version 6.5 15 NQF #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated Date: Mar 20, 2013 NATIONAL QUALITY FORUM Measure missing data in MSF 6.5 from MSF 5.0 NQF #: 1814 NQF Project: Neurology Project 1. IMPACT, OPPORTUITY, EVIDENCE - IMPORTANCE TO MEASURE AND REPORT Importance to Measure and Report is a threshold criterion that must be met in order to recommend a measure for endorsement. All three subcriteria must be met to pass this criterion. See guidance on evidence. Measures must be judged to be important to measure and report in order to be evaluated against the remaining criteria. (evaluation criteria) 1c.1 Structure-Process-Outcome Relationship (Briefly state the measure focus, e.g., health outcome, intermediate clinical outcome, process, structure; then identify the appropriate links, e.g., structure-process-health outcome; process- health outcome; intermediate clinical outcome-health outcome): By counseling female patients about epilepsy and how its treatment may affect contraception the expected patient outcomes include decreased contraception failures due to AEDs use that result in an unplanned pregnancy, decreased embryonic or fetal malformation, improved patient quality of life, better seizure management, appropriate medication and treatments prescribed, and the patient will understand the risks involved with AEDs, contraception and pregnancy. In addition the patient will know how to mitigate these risks. The American Academy of Neurology has preliminary data from the implementation of these measures into the Maintenance of Certification Performance in Practice (NeuroPI) Epilepsy Module. There have been 291 physicians to date who have enrolled in Epilepsy module. However, the extrapolation of data from this module is not yet appropriate as the sample size is believed to be too small to be able to provide generalizable data. However, by the time this measure comes back to the NQF for the end of the Temporary Endorsement period (estimated by 1/2014) there were will be additional data available to support the link of this measure to the desired patient outcomes. In addition, we will have some data back from the CECity registry database, which just went live in August 2012, by 1/2014 to add additional support to this measure. 1c.2-3 Type of Evidence (Check all that apply): Clinical Practice Guideline Other Quality Indicator Paper 1c.4 Directness of Evidence to the Specified Measure (State the central topic, population, and outcomes addressed in the body of evidence and identify any differences from the measure focus and measure target population): Topic: Epilepsy Population: Women with epilepsy of childbearing age (identified as 12-44 years old) Outcomes addressed in the body of evidence: counseling that leads to appropriate education of women with epilepsy of childbearing potential about anti-epileptic drugs (AEDs) and their effect on contraception, pregnancy and fetal malformations. Expected outcome would be empowered female patients who are well educated and able to make appropriate decisions on epilepsy and its treatment. This would indirectly relate to decreased fetal malformations and appropriate seizure management. Differences from the measure focus and the measure target population: not applicable 1c.5 Quantity of Studies in the Body of Evidence (Total number of studies, not articles): 5 recommendation statements/indicators were used as the basis for this quality measure. 1c.6 Quality of Body of Evidence (Summarize the certainty or confidence in the estimates of benefits and harms to patients across studies in the body of evidence resulting from study factors. Please address: a) study design/flaws; b) See Guidance for Definitions of Rating Scale: H=High; M=Moderate; L=Low; I=Insufficient; NA=Not Applicable Created on: 06/18/2013 at 02:57 AM 1 NQF #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated Date: Mar 20, 2013 directness/indirectness of the evidence to this measure (e.g., interventions, comparisons, outcomes assessed, population included in the evidence); and c) imprecision/wide confidence intervals due to few patients or events): The guideline/indicator authors did not provide an explicit process or documentation of a process like GRADE whereby precision, directness, etc were detailed in a systematic review to demonstrate the quality of the body of evidence for this measure. The available information from the guideline/indicator paper is provided below. Recommendation: Women (and, if appropriate, their family and/or caregivers or others closely involved) should be given information about contraception, conception, pregnancy and breastfeeding. Information should be given in advance of sexual activity or pregnancy. (Level C) NICE 2004 1. NICE National Collaborating Centre for Primary Care. The diagnosis and management of the epilepsies in adults and children in primary and secondary care. London (UK): Royal College of General Practitioners; 2004 Oct. From the Guideline with Direct Evidence for this recommendation Statement: No systematic reviews of RCTs of information provision for women with epilepsy were identified. One systematic review of other evidence was found. Couldridge and colleagues reviewed the primary evidence (including non-RCT studies) on the information and counseling needs of people with epilepsy, the preferred format, timing, and delivery of information and counseling, and the outcomes of information giving and counselling.216 None of the 40 included studies reported the role or effects of information or counseling in women with epilepsy as a group, although some studies did have women in the study population. Primary evidence No RCTs on the effectiveness of information giving or counseling were identified. Since the publication of the systematic review described above216, two large surveys of women with epilepsy were found. Crawford 1999241 Crawford and Lee reported the results of a questionnaire survey of female members of the British Epilepsy Association. 1855 questionnaires (from a total of 6000) were included in the results (response rate 31%). 47% (n=89) of women taking oral contraception felt they had not been given enough information about the oral contraception pill and their AED(s). 43% (n=637) reported receiving no information about pregnancy, and 25% (n=459) had discussed pregnancy with no-one. Many women intending to have children in the subsequent two years felt they still had unanswered questions (see Table 17). Table 17 Concerns about pregnancy241 Modified from Seizure, 8, Crawford P and Lee P, Gender difference in management of epilepsy - What women are hearing, pages 135-9, Copyright (1999) with permission from BEA Trading Ltd. Overall, women felt there was a need for more information about epilepsy and pregnancy. The survey concluded that women with epilepsy wanted, and needed, more information and counseling about issues relating to contraception, pregnancy, and the menopause.241 Crawford 20032 42 In 2001, the Ideal World survey aimed to assess the quality of current treatment information provision to women with epilepsy at different life stages, and to identify the information needs and wants with a view to ensuring that all women with epilepsy are counseled appropriately, in a timely manner, and are able to make informed choices about their treatment. Approximately 12,000 female members of Epilepsy Action were surveyed, and the questionnaire was also posted on the Epilepsy Action website. 2,600 questionnaires and 90 web responses were completed, and 2000 responses randomly selected for analysis. The most important issues for women aged 19 to 44 years who were considering having children were: 1. risk of epilepsy/medication affecting the unborn child (87%) 2. effect of pregnancy on seizure control (49%) 3. risk of a child developing epilepsy (42%) Most women (84%) wanted to be better informed about treatment decisions, and 41% wanted to take a more proactive role in discussions around treatment. 43% wanted more information so they could ask for a review of their medication. 57% wanted the See Guidance for Definitions of Rating Scale: H=High; M=Moderate; L=Low; I=Insufficient; NA=Not Applicable Created on: 06/18/2013 at 02:57 AM 2 NQF #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated Date: Mar 20, 2013 latest information on epilepsy treatment and the risk of birth defects on an ongoing basis, even if the data were incomplete. The preferred timing of receiving information can be seen in Table 18. Table 18 Preferred time to receive information242 Modified from Seizure, 12, Crawford P and Hudson S, Understanding the information needs of women with epilepsy at different lifestages: results of the ´Ideal World´ survey, pages 502-7, Copyright (2003) with permission from BEA Trading Ltd. The survey showed consistently that information is preferred before the time it is needed. 59% wanted information in a written format, and 28% through conversation with a healthcare professional.242 Recommendations: If AEDs are to be used in pregnancy the relative risks of seizures and fetal malformation should be discussed with the woman. (Level C) SIGN(April 2003) 23 Whenever possible, a woman should conceive on the lowest effective dose of one AED appropriate for her epilepsy syndrome. If she has good seizure control and presents already pregnant, there is probably little to be gained by altering her AEDs. (Level C) SIGN(April 2003) 23 SIGN. Scottish Intercollegiate Guidelines Network (SIGN): SIGN 70: (1) Diagnosis and management of epilepsy in adults. A national clinical guideline. (2) Diagnosis and management of epilepsy in adults. Update to printed guideline. Scottish Intercollegiate Guidelines Network - National Government Agency [Non-U.S.]. 2003 Apr (addendum released 2004 Jun 7). Original guideline: 49 pages; Addendum: 3 pages. NGC:003832 4.2.2 Risks to the fetus from antiepileptic drugs Major and minor fetal malformations occur more commonly in infants exposed to AEDs during pregnancy.209, 230-2 The overall risk of major fetal malformation in any pregnancy is approximately 2%. This increases two to three fold in women taking a single AED. Current Data suggest that the risk with valproate may be higher than with carbamazepine or lamotrigine.233 Polytherapy, particularly with certain combinations of drugs, carriers a much higher risk (up to 24% in women taking four AEDs). The most common major malformation associated with established AEDs are: neural tube defects, (valproate 3%, carbamazepine 1%), orofacial defects, congenital heart abnormalities and hypospadias.230, 234 The risk of minor malformations including hypertelorism, epicanthic folds and digital hypoplasia is increased with AED therapy in pregnancy.231 “Fetal anticonvulsant syndromes” comprising typical dysmophic craniofacial appearances and a variety of musculoskeletal abnormalities have been described in associated with AED treatment in pregnancy.235,236 Although individual drugs have been associated with specific patterns, there is overlap between them and genetic factors may influence susceptibility.237 Whether AEDs taken during pregnancy can affect the child’s intellectual development is uncertain but concern about the effects of valproate on infant development has recently been raised.238,239 At present there is insufficient evidence on which to base advise about the risks of most of the new AEDS (gabapentin, levetiracetam, tiagabine, topiramate, vigabatrin) in pregnancy. Current data on lamotrigine show a malformation rate of 3% (95% confidence interval 1.5-5.7)233 Additional studies not initially used to support the measure but new guidelines that have come out since this project was completed. Update: Management Issues for Women with Epilepsy-Focus on Pregnancy: Obstetrical Complications and Change in Seizure Frequency Neurology® 2009;73:126–132 Do WWE have an increased risk of pregnancy-related complications? Twenty-five articles met inclusion criteria for pregnancy-related complications in WWE. Several articles included information pertinent to more than one question. Of these 25 articles, 9 were graded Class III or higher (table e-1). Women with epilepsy (WWE) should be counseled that seizure freedom for at least 9 months prior to pregnancy is probably See Guidance for Definitions of Rating Scale: H=High; M=Moderate; L=Low; I=Insufficient; NA=Not Applicable Created on: 06/18/2013 at 02:57 AM 3 NQF #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated Date: Mar 20, 2013 associated with a high rate (84%–92%) of remaining seizure-free during pregnancy (Level B). Evidence to support this recommendation: Seizure recurrence in previously seizure-free WWE. Two Class II articles16,17 showed that for WWE who were seizure-free for 9 months prior to pregnancy, 84%–92% remained seizure-free during pregnancy (table e-4). In one study, 38 of 45 (84%; CI 0.71– 0.92) pregnant WWE remained seizure-free,16 and in the other study, 47 of 51 (92%; CI 0.82– 0.97) pregnant WWE remained seizure-free.17 One larger Class III article22 showed that 80% of a group of WWE (n _ 450) who were seizure-free at least 1 year prior to pregnancy remained seizure-free during pregnancy (exact number not provided). One Class III article showed that of 72 WWE who were seizure-free for 10 months, 74% (95% CI 0.62– 0.82) remained seizure-free during pregnancy.18 A second Class III article showed that of 54 WWE who were seizure-free for 9 months, 94% (95% CI 0.85– 0.98) remained seizure-free during pregnancy, and of 48 WWE who were seizurefree for 1 year, 92% (95% CI 0.80–0.98) remained seizure-free during pregnancy.19 These results are all fairly consistent across the class of evidence and sample size of the studies. Conclusion. Two Class II articles show the rate of remaining seizure-free during pregnancy if WWE are seizure-free for at least 9 months to 1 year prior to pregnancy is probably 84%–92%. 16. Gjerde IO, Strandjord RE, Ulstein M. The course of epilepsy during pregnancy: a study of 78 cases. Acta Neurol Scand 1988;78:198–205. 17. Tomson T, Lindbom U, Ekqvist B, Sundqvist A. Epilepsy and pregnancy: a prospective study of seizure control in relation to free and total plasma concentrations of carbamazepine and phenytoin. Epilepsia 1994;35:122–130. 18. Otani K. Risk factors for the increased seizure frequency during pregnancy and puerperium. Folia Psychiatr Neurol Jpn 1985;39:33–41. 19. Tanganelli P, Regesta G. Epilepsy, pregnancy, and major birth anomalies: an Italian prospective, controlled study. Neurology 1992;42(4 suppl 5):89–93. 1c.7 Consistency of Results across Studies (Summarize the consistency of the magnitude and direction of the effect): The studies are consistent that women of childbearing potential should be counseled about epilepsy and how its treatment may affect contraception and pregnancy. 1c.8 Net Benefit (Provide estimates of effect for benefit/outcome; identify harms addressed and estimates of effect; and net benefit - benefit over harms): See quality of body of evidence question 1c.9 Grading of Strength/Quality of the Body of Evidence. Has the body of evidence been graded? Yes 1c.10 If body of evidence graded, identify the entity that graded the evidence including balance of representation and any disclosures regarding bias: Pugh et al. work group NICE guideline work group SIGN guideline work group 1c.11 System Used for Grading the Body of Evidence: Other 1c.12 If other, identify and describe the grading scale with definitions: See 1c.22 1c.13 Grade Assigned to the Body of Evidence: see 1c.15 or 1c.16 1c.14 Summary of Controversy/Contradictory Evidence: Not applicable 1c.15 Citations for Evidence other than Guidelines(Guidelines addressed below): IF a woman with epilepsy is of childbearing potential and receives oral contraceptives in conjunction with an enzyme inducing AED, THEN decreased effectiveness of oral contraception should be addressed. (higher doses of the oral contraceptive, alternative birth control methods, or change AED). (Level A 2++/Primary) Pugh (2007)17 See Guidance for Definitions of Rating Scale: H=High; M=Moderate; L=Low; I=Insufficient; NA=Not Applicable Created on: 06/18/2013 at 02:57 AM 4 NQF #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated Date: Mar 20, 2013 Patients with epilepsy should receive an annual review of information including topics such as: - Chronic effects of epilepsy and its treatment including drug side-effects, drug-drug interactions, effect on bone health;Contraception, family planning, and how pregnancy and menopause may affect seizures (EVIDENCE GRADE C);- Screening for mood disorders;- Triggers and lifestyle issues that may affect seizures;- Impact of epilepsy on other chronic and acute diseases;Driving and safety issues (Level D/Secondary) Pugh (2007) 17 PUGH Pugh MJ, Berlowitz DR, Montouris G, et al. What constitutes high quality of care for adults with epilepsy? Neurology. 2007 20;69(21):2020-7. This is not a guideline. It is a quality indicator panel that developed the measures using an evidence and consensus based process. There is not a lot of data available on the methodology or specific studies used to support the development of the indicators. However, their conclusions and recommendations concur with our existing guideline recommendation statements. “We developed explicit quality indicators using the modified Delphi process (RAND Appropriateness Method) that has been successfully employed in the development of quality indicators for over 30 different preventive health, acute, or chronic diseases.3,18,19 This method incorporated a systematic review of the literature and guidelines to assure that selected process of care criteria are linked to relevant patient outcomes in clinical trials or expert clinical opinion (best practices)16 and an expert rating panel.” 3. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med 2003;348:2635–2645. 18. McGlynn EA, Kerr EA, Asch SM. New approach to assessing clinical quality of care for women: the QA Tool system. Womens Health Iss 1999;9:184–192. 19. Shekelle PG, MacLean CH, Morton SC, Wenger NS. Acove quality indicators. Ann Intern Med 2001;135: 653–667. 1c.16 Quote verbatim, the specific guideline recommendation (Including guideline # and/or page #): Women (and, if appropriate, their family and/or caregivers or others closely involved) should be given information about contraception, conception, pregnancy and breastfeeding. Information should be given in advance of sexual activity or pregnancy. (Level C) NICE 2004 If AEDs are to be used in pregnancy the relative risks of seizures and fetal malformation should be discussed with the woman. (Level C) SIGN(April 2003) Whenever possible, a woman should conceive on the lowest effective dose of one AED appropriate for her epilepsy syndrome. If she has good seizure control and presents already pregnant, there is probably little to be gained by altering her AEDs. (Level C) SIGN(April 2003) 1c.17 Clinical Practice Guideline Citation: 23. SIGN. Scottish Intercollegiate Guidelines Network (SIGN): SIGN 70: (1) Diagnosis and management of epilepsy in adults. A national clinical guideline. (2) Diagnosis and management of epilepsy in adults. Update to printed guideline. Scottish Intercollegiate Guidelines Network - National Government Agency [Non-U.S.]. 2003 Apr (addendum released 2004 Jun 7). Original guideline: 49 pages; Addendum: 3 pages. NGC:003832 22. NICE National Collaborating Centre for Primary Care. The diagnosis and management of the epilepsies in adults and children in primary and secondary care. (Uses information from Reference 20 and 21) London (UK): Royal College of General Practitioners; 2004 Oct 1c.18 National Guideline Clearinghouse or other URL: Yes http://guideline.gov/content.aspx?id=5694&search=epilepsy ; http://www.sign.ac.uk/methodology/index.html; http://publications.nice.org.uk/the-epilepsies-the-diagnosis-and-management-of-theepilepsies-in-adults-and-children-in-primary-and-cg137 1c.19 Grading of Strength of Guideline Recommendation. Has the recommendation been graded? Yes See Guidance for Definitions of Rating Scale: H=High; M=Moderate; L=Low; I=Insufficient; NA=Not Applicable Created on: 06/18/2013 at 02:57 AM 5 NQF #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated Date: Mar 20, 2013 1c.20 If guideline recommendation graded, identify the entity that graded the evidence including balance of representation and any disclosures regarding bias: See individual guideline/indicator citations in 1c.15 or 1c.16 1c.21 System Used for Grading the Strength of Guideline Recommendation: Other 1c.22 If other, identify and describe the grading scale with definitions: NICE National Collaborating Centre for Primary Care. The diagnosis and management of the epilepsies in adults and children in primary and secondary care. London (UK): Royal College of General Practitioners; 2004 Oct. Rating Scheme for Strength of the Evidence Ia-Systematic review or meta-analysis of randomized controlled trials Ib-At least one randomized controlled trial IIa-At least one well-designed controlled stud without randomization IIb-At least one well-designed quasi-experimental descriptive studies, such as a cohort study III-Well-designed non-experimental descriptive studies, case-control studies, and case studies IV-Expert committee reports, opinions and/or clinical experience of respected authorities Rating Recommendations A* Directly based on category I evidence (meta-analysis of randomized controlled trials (RCTs) or at least one RCT) B* Directly based on category II evidence (at least one controlled study without randomization or at least one other quasiexperimental study) or extrapolated from category I evidence C* Directly based on category III evidence (non-experimental descriptive studies) or extrapolated from category I or II evidence D* Directly based on category III evidence (expert committee reports or opinions and/or clinical experience of respected authorities) or extrapolated from category I, II or III evidence N Recommendation taken from NICE guideline or technology appraisal guidance SIGN (1): SIGN 70: Diagnosis and Management of Epilepsy in Adults. A National Clinical Guideline. Edinburgh (Scotland) 2003 April p.49. Under revision as of June 2008. Grading of Recommendations (Note: Only measures graded as A, B, or C were included in the table) A: At least one meta-analysis, systematic review of randomized controlled trials (RCTs), or randomized controlled trial rated as 1++ and directly applicable to the target population; or A body of evidence consisting principally of studies rated as 1+, directly applicable to the target population, and demonstrating overall consistency of results B: A body of evidence including studies rated as 2++, directly applicable to the target population and demonstrating overall consistency of results; or Extrapolated evidence from studies rated as 1++ or 1+ C: A body of evidence including studies rated as 2+, directly applicable to the target population and demonstrating overall consistency of results; or Extrapolated evidence from studies rate as 2++ D: Evidence level 3 or 4; or Extrapolated evidence from studies rated as 2+ Levels of Evidence 1++: High quality meta-analyses, systematic reviews of randomized controlled trials (RCTs), or RCTs with a very low risk of bias 1+: Well-conducted meta-analyses, systematic reviews of RCTs, or RCTs with a low risk of bias 1-: Meta-analyses, systematic reviews of RCTs, or RCTs with a high risk of bias 2++: High quality systematic reviews of case control or cohort studies. High quality case control or cohort studies with a very low risk of confounding or bias and a high probability that the relationship is causal 2+: Well-conducted case control or cohort studies with a low risk of confounding or bias and a moderate probability that the relationship is causal 2-: Case control or cohort studies with a high risk of confounding or bias and a significant risk that the relationship is not causal 3: Non-analytic studies, e.g. case reports, case series 4: Expert opinion See Guidance for Definitions of Rating Scale: H=High; M=Moderate; L=Low; I=Insufficient; NA=Not Applicable Created on: 06/18/2013 at 02:57 AM 6 NQF #1814 Counseling for Women of Childbearing Potential with Epilepsy, Last Updated Date: Mar 20, 2013 Pugh Paper: Epilepsy Measures Work Group Grading of Evidence and Indicators Pugh MJ, Berlowitz DR, Montouris G, Bokhour B, Cramer JA, Bohm V, Bollinger M, Helmers S, Ettinger A, Meador KJ, Fountain N, Boggs J, Tatum WO 4th, Knoefel J, Harden C, Mattson RH, Kazis L. What constitutes high quality of care for adults with epilepsy? Neurology. 2007 Nov 20;69(21):2020-7. A: Rated as appropriate F: Rated as feasible N: Rated as necessary N/A: Not Rated Ratings 1-3 clearly appropriate/ reliable/ necessary 4-6 uncertain or equivocal 7-10 appropriate/ reliable/ necessary 1c.23 Grade Assigned to the Recommendation: See individual guideline/indicator citations in 1c.15 or 1c.16 1c.24 Rationale for Using this Guideline Over Others: These recommendaiton statements and guidelines were chosen because of the high impact on care and gap in care for women with epilepsy. These guidelines and indicator papers were chosen over others because of their applicabilility to meet the gap in care and improve the quality of care of women with epilepsy of childbearing potential. Based on the NQF descriptions for rating the evidence, what was the developer’s assessment of the quantity, quality, and consistency of the body of evidence? 1c.25 Quantity: Moderate 1c.26 Quality: Moderate1c.27 Consistency: High See Guidance for Definitions of Rating Scale: H=High; M=Moderate; L=Low; I=Insufficient; NA=Not Applicable Created on: 06/18/2013 at 02:57 AM 7 NATIONAL QUALITY FORUM—Measure Testing (subcriteria 2a2, 2b2-2b7) Measure Number (if previously endorsed): 1814 Measure Title: Counseling for Women of Childbearing Potential with Epilepsy Date of Submission: 2/27/2014 Type of Measure: ☐ Composite – STOP – use composite testing form ☐ Cost/resource ☐ Efficiency ☐ Outcome (including PRO-PM) ☒ Process ☐ Structure Instructions • Measures must be tested for all the data sources and levels of analyses that are specified. If there is more than one set of data specifications or more than one level of analysis, contact NQF staff about how to present all the testing information in one form. • For all measures, sections 1, 2a2, 2b2, 2b3, and 2b5 must be completed. • For outcome and resource use measures, section 2b4 also must be completed. • If specified for multiple data sources/sets of specificaitons (e.g., claims and EHRs), section 2b6 also must be completed. • Respond to all questions as instructed with answers immediately following the question. All information on testing to demonstrate meeting the subcriteria for reliability (2a2) and validity (2b2-2b6) must be in this form. An appendix for supplemental materials may be submitted, but there is no guarantee it will be reviewed. • If you are unable to check a box, please highlight or shade the box for your response. • Maximum of 20 pages (incuding questions/instructions; minimum font size 11 pt; do not change margins). Contact NQF staff if more pages are needed. • Contact NQF staff regarding questions. Check for resources at Submitting Standards webpage. Note: The information provided in this form is intended to aid the Steering Committee and other stakeholders in understanding to what degree the testing results for this measure meet NQF’s evaluation criteria for testing. 2a2. Reliability testing 10 demonstrates the measure data elements are repeatable, producing the same results a high proportion of the time when assessed in the same population in the same time period and/or that the measure score is precise. For PRO-PMs and composite performance measures, reliability should be demonstrated for the computed performance score. 2b2. Validity testing 11 demonstrates that the measure data elements are correct and/or the measure score correctly reflects the quality of care provided, adequately identifying differences in quality. For PRO-PMs and composite performance measures, validity should be demonstrated for the computed performance score. 2b3. Exclusions are supported by the clinical evidence; otherwise, they are supported by evidence of sufficient frequency of occurrence so that results are distorted without the exclusion; 12 AND If patient preference (e.g., informed decisionmaking) is a basis for exclusion, there must be evidence that the exclusion impacts performance on the measure; in such cases, the measure must be specified so that the information about patient preference and the effect on the measure is transparent (e.g., numerator category computed separately, denominator exclusion category computed separately). 13 Version 6.5 08/20/13 1 2b4. For outcome measures and other measures when indicated (e.g., resource use): • an evidence-based risk-adjustment strategy (e.g., risk models, risk stratification) is specified; is based on patient factors that influence the measured outcome (but not factors related to disparities in care or the quality of care) and are present at start of care; 14,15 and has demonstrated adequate discrimination and calibration OR • rationale/data support no risk adjustment/ stratification. 2b5. Data analysis of computed measure scores demonstrates that methods for scoring and analysis of the specified measure allow for identification of statistically significant and practically/clinically meaningful 16 differences in performance; OR there is evidence of overall less-than-optimal performance. 2b6. If multiple data sources/methods are specified, there is demonstration they produce comparable results. 2b7. For eMeasures, composites, and PRO-PMs (or other measures susceptible to missing data), analyses identify the extent and distribution of missing data (or nonresponse) and demonstrate that performance results are not biased due to systematic missing data (or differences between responders and nonresponders) and how the specified handling of missing data minimizes bias. Notes 10. Reliability testing applies to both the data elements and computed measure score. Examples of reliability testing for data elements include, but are not limited to: inter-rater/abstractor or intra-rater/abstractor studies; internal consistency for multiitem scales; test-retest for survey items. Reliability testing of the measure score addresses precision of measurement (e.g., signal-to-noise). 11. Validity testing applies to both the data elements and computed measure score. Validity testing of data elements typically analyzes agreement with another authoritative source of the same information. Examples of validity testing of the measure score include, but are not limited to: testing hypotheses that the measures scores indicate quality of care, e.g., measure scores are different for groups known to have differences in quality assessed by another valid quality measure or method; correlation of measure scores with another valid indicator of quality for the specific topic; or relationship to conceptually related measures (e.g., scores on process measures to scores on outcome measures). Face validity of the measure score as a quality indicator may be adequate if accomplished through a systematic and transparent process, by identified experts, and explicitly addresses whether performance scores resulting from the measure as specified can be used to distinguish good from poor quality. 12. Examples of evidence that an exclusion distorts measure results include, but are not limited to: frequency of occurrence, variability of exclusions across providers, and sensitivity analyses with and without the exclusion. 13. Patient preference is not a clinical exception to eligibility and can be influenced by provider interventions. 14. Risk factors that influence outcomes should not be specified as exclusions. 15. Risk models should not obscure disparities in care for populations by including factors that are associated with differences/inequalities in care, such as race, socioeconomic status, or gender (e.g., poorer treatment outcomes of African American men with prostate cancer or inequalities in treatment for CVD risk factors between men and women). It is preferable to stratify measures by race and socioeconomic status rather than to adjust out the differences. 16. With large enough sample sizes, small differences that are statistically significant may or may not be practically or clinically meaningful. The substantive question may be, for example, whether a statistically significant difference of one percentage point in the percentage of patients who received smoking cessation counseling (e.g., 74 percent v. 75 percent) is clinically meaningful; or whether a statistically significant difference of $25 in cost for an episode of care (e.g., $5,000 v. $5,025) is practically meaningful. Measures with overall less-than-optimal performance may not demonstrate much variability across providers. Version 6.5 08/20/13 2 1. DATA/SAMPLE USED FOR ALL TESTING OF THIS MEASURE Often the same data are used for all aspects of measure testing. In an effort to eliminate duplication, the first five questions apply to all measure testing. If there are differences by aspect of testing,(e.g., reliability vs. validity) be sure to indicate the specific differences in question 1.7. 1.1. What type of data was used for testing? (Check all the sources of data identified in the measure specifications and data used for testing the measure. Testing must be provided for all the sources of data specified and intended for measure implementation. If different data sources are used for the numerator and denominator, indicate N [numerator] or D [denominator] after the checkbox.) Measure Specified to Use Data From: Measure Tested with Data From: (must be consistent with data sources entered in S.23) ☒ abstracted from paper record ☒ abstracted from paper record ☒ administrative claims ☐ administrative claims ☐ clinical database/registry ☐ clinical database/registry ☒ abstracted from electronic health record ☒ abstracted from electronic health record ☐ eMeasure (HQMF) implemented in EHRs ☐ eMeasure (HQMF) implemented in EHRs ☐ other: Click here to describe ☐ other: Click here to describe 1.2. If an existing dataset was used, identify the specific dataset (the dataset used for testing must be consistent with the measure specifications for target population and healthcare entities being measured; e.g., Medicare Part A claims, Medicaid claims, other commercial insurance, nursing home MDS, home health OASIS, clinical registry). The AAN identified and recruited Neurology practices in Minnesota that have experience treating patients with epilepsy. Three Neurology practices volunteered to participate and submit retrospective data from the 2012 calendar year (i.e. dates of service 01/01/2012 – 12/31/2012). Denominator certification is an essential step in the process to obtaining valid and accurate data. It requires each participant to attest that they will submit accurate data and follow the measure specifications exactly how they are written. It also ensures that each participant is querying the correct: • Diagnosis codes (i.e. 345.00, 345.01, 345.10, 345.11, 345.40, 345.41, 345.50, 345.51, 345.70, 345.71, 345.90, 345.91) • Encounter codes (i.e. 99201, 99202, 99203, 99204, 99205, 99212, 99213, 99214, 99215, 99241, 99242, 99243, 99244, 99245) • Date of birth ranges (i.e. 01/01/1968-01/01/2000) • Date of service ranges (i.e. 01/01/2012- 12/31/2012) 1.3. What are the dates of the data used in testing? 01/01/2012-12/31/2012 1.4. What levels of analysis were tested? (testing must be provided for all the levels specified and intended for measure implementation, e.g., individual clinician, hospital, health plan) Measure Specified to Measure Performance of: Measure Tested at Level of: (must be consistent with levels entered in item S.26) ☒ individual clinician ☐ individual clinician ☒ group/practice ☒ group/practice ☐ hospital/facility/agency ☐ hospital/facility/agency ☐ health plan ☐ health plan ☐ other: Click here to describe ☐ other: Click here to describe Version 6.5 08/20/13 3 1.5. How many and which measured entities were included in the testing and analysis (by level of analysis and data source)? (identify the number and descriptive characteristics of measured entities included in the analysis (e.g., size, location, type); if a sample was used, describe how entities were selected for inclusion in the sample) Two of the practices were independent, physician owned practices located in the Twin Cities and the third practice was a large integrated delivery system in southeast Minnesota. 1.6. How many and which patients were included in the testing and analysis (by level of analysis and data source)? (identify the number and descriptive characteristics of patients included in the analysis (e.g., age, sex, race, diagnosis); if a sample was used, describe how patients were selected for inclusion in the sample) In order to achieve a reliable sample of patients for this measure MNCM and AAN sought a minimum of 1,000 combined patient records from the three Neurology sites that agree to participate. See below. Version 6.5 08/20/13 4 1.7. If there are differences in the data or sample used for different aspects of testing (e.g., reliability, validity, exclusions, risk adjustment), identify how the data or sample are different for each aspect of testing reported below. ________________________________ 2a2. RELIABILITY TESTING Note: If accuracy/correctness (validity) of data elements was empirically tested, separate reliability testing of data elements is not required – in 2a2.1 check critical data elements; in 2a2.2 enter “see section 2b2 for validity testing of data elements”; and skip 2a2.3 and 2a2.4. 2a2.1. What level of reliability testing was conducted? (may be one or both levels) ☒ Critical data elements used in the measure (e.g., inter-abstractor reliability; data element reliability must address ALL critical data elements) ☒ Performance measure score (e.g., signal-to-noise analysis) 2a2.2. For each level checked above, describe the method of reliability testing and what it tests (describe the steps―do not just name a method; what type of error does it test; what statistical analysis was used) The AAN identified and recruited Neurology practices in Minnesota that have experience treating patients with epilepsy. As part of the recruitment process MNCM and the AAN hosted an informational webinar explaining the purpose of the measurement testing project for the Counseling for Women with Epilepsy measure. Three Neurology practices volunteered to participate and submit retrospective data from the 2012 calendar year (i.e. dates of service 01/01/2012 – 12/31/2012). MNCM produced a data collection guide, measure flow and detailed file specifications to educate and assist each medical group in the data collection and submission process. As a requirement of participating in the measure testing each group had to submit a denominator certifications form (see appendix A). The denominator certification process helps ensure that each medical group is using the appropriate measure parameters and collecting data in a standardized way. 2a2.3. For each level of testing checked above, what were the statistical results from reliability testing? (e.g., percent agreement and kappa for the critical data elements; distribution of reliability statistics from a signal-to-noise analysis) MNCM did not identify any major flaws or issues during the review of each medical group’s denominator forms and therefore each medical group passed denominator certification within the given timeframe. 2a2.4 What is your interpretation of the results in terms of demonstrating reliability? (i.e., what do the results mean and what are the norms for the test conducted?) There were a few corrections and clarifications that required MNCM to send a follow-up email to the respective group; however, each issue was resolved in a timely manner. The list below documents the Version 6.5 08/20/13 5 issues that were identified and required additional follow-up based on the information received on the denominator certification forms: • Incorrect diagnosis codes included in data query • Group did not indicate if they would be submitting a sample or full population for the measure • Incorrect encounter codes included in data query _________________________________ 2b2. VALIDITY TESTING 2b2.1. What level of validity testing was conducted? (may be one or both levels) ☒ Critical data elements (data element validity must address ALL critical data elements) ☒ Performance measure score ☐ Empirical validity testing ☐ Systematic assessment of face validity of performance measure score as an indicator of quality or resource use (i.e., is an accurate reflection of performance on quality or resource use and can distinguish good from poor performance) 2b2.2. For each level of testing checked above, describe the method of validity testing and what it tests (describe the steps―do not just name a method; what was tested, e.g., accuracy of data elements compared to authoritative source, relationship to another measure as expected; what statistical analysis was used) MNCM completed validation of the data in a three-step process: 1) denominator certification, 2) data file quality checks, and 3) validation audit. Details of this validation are described in this report. Denominator Certification Denominator certification is an essential step in the process to obtaining valid and accurate data. It requires each participant to attest that they will submit accurate data and follow the measure specifications exactly how they are written. It also ensures that each participant is querying the correct: • Diagnosis codes (i.e. 345.00, 345.01, 345.10, 345.11, 345.40, 345.41, 345.50, 345.51, 345.70, 345.71, 345.90, 345.91) • Encounter codes (i.e. 99201, 99202, 99203, 99204, 99205, 99212, 99213, 99214, 99215, 99241, 99242, 99243, 99244, 99245) • Date of birth ranges (i.e. 01/01/1968-01/01/2000) • Date of service ranges (i.e. 01/01/2012- 12/31/2012) MNCM did not identify any major flaws or issues during the review of each medical group’s denominator forms and therefore each medical group passed denominator certification within the given timeframe. There were a few corrections and clarifications that required MNCM to send a follow-up email to the respective group; however, each issue was resolved in a timely manner. The list below documents the issues that were identified and required additional follow-up based on the information received on the denominator certification forms: • Incorrect diagnosis codes included in data query • Group did not indicate if they would be submitting a sample or full population for the measure • Incorrect encounter codes included in data query Data File Quality Checks After each medical group submitted their data file to MNCM, quality checks of the files were completed. Each column in the data file represented a field of data for each patient row; the following checks were completed: Version 6.5 08/20/13 6 • Number of patients/rows submitted were reasonable/expected • Necessary data fields (columns) were included and completed appropriately • Patient date of birth spanned the expected range • Zip codes were 5-digit and primarily within MN and other bordering states as expected • Race field(s) were included and populated appropriately • Provider NPI field was included and number of providers expected • Insurance information was included and was reasonable • Office visit dates and counseling dates spanned the expected range • Diagnoses were included and spanned the entire list of expected codes • Medical reasons for NOT counseling were applied correctly; were not misused Issues identified through the data file quality checks were generally minor, requiring no corrections. Other mentionable items include: 1. All three groups did not have patients with diagnosis codes 345.70 (Epilepsia partialis continua, without mention of intractable epilepsy) or 345.71 (Epilepsia partialis continua, with intractable epilepsy). These are rare diagnoses and did not come up in the population. 2. Medical groups B and C listed many neurological or congenital conditions as reasons for the patient to NOT receive counseling. These were verified during audit. 3. Medical group C did not include their entire population in first submission, excluding patients whose date of birth was between January thru June 1968. They queried their system again, this time using the specific dates of birth (rather than “age” values) and included the additional patients in their denominator. Validation Audit After the data file checks were completed, MNCM completed audits of the patient records to verify the submitted clinical data. We also verified the diagnosis of epilepsy and other demographic data (e.g., race). MNCM uses a validation process developed by the NCQA – National Committee for Quality Assurance, known as the “8 and 30” process. In this process, the first eight records are verified for accuracy and if no errors are identified, the data is considered to be 100% compliant. If errors in the first eight records are identified, we continue reviewing the total 30 records to identify any error patterns and or issues that may need correction. The audits revealed some data errors, requiring one medical group to make corrections and resubmit data. Individual medical group results were as follows: Version 6.5 08/20/13 7 2b2.3. What were the statistical results from validity testing? (e.g., correlation; t-test) Validation/ Audit Conclusion 2b2.4. What is your interpretation of the results in terms of demonstrating validity? (i.e., what do the results mean and what are the norms for the test conducted?) The validation process was successful in identifying errors (with subsequent corrections) and verifying the accuracy of the data submitted by medical groups A, B, and C. Finding no significant flaws or errors with the data MNCM is confident the rate calculation and any additional data analysis can be completed using validated and reliable data. Additionally, during a review of the National Quality Forum’s feedback to the American Academy of Neurology for this measure, it was noted that there was a concern that this Version 6.5 08/20/13 8 may simply be a “check-the-box” measure. During the validation audit, it was noted on several occasions that the practices provided excellent, personalized progress notes about the counseling that was being provided, that were above and beyond a “check the box”. _________________________ 2b3. EXCLUSIONS ANALYSIS NA ☐ no exclusions — skip to section 2b4 2b3.1. Describe the method of testing exclusions and what it tests (describe the steps―do not just name a method; what was tested, e.g., whether exclusions affect overall performance scores; what statistical analysis was used) The main limitation that MNCM identified during the testing of the Counseling for Women with Epilepsy measure is related to the denominator of included and excluded patients. The measure specifications offered two different options for excluding patients from the measure: 1. Patient was surgically sterile (tubal ligation, hysterectomy) 2. Patient has an intellectual disability as defined by ICD-9 codes a. 318.0 moderate intellectual disabilities; IQ 35 to 48 b. 318.1 severe intellectual disabilities; IQ 20 to 34 c. 318.2 profound intellectual disabilities; IQ under 20 Groups submitted these patients and indicated which reason applied. Additionally, if they felt that there was another medical reason for not providing counseling, they indicated this by a code and accompanying description. These reasons were not used to exclude patients from the measure; rather the purpose was to provide additional information about the population of patients included in the measure. Reasons Provided by Medical Groups for Not Providing Counseling: 2b3.2. What were the statistical results from testing exclusions? (include overall number and percentage of individuals excluded, frequency distribution of exclusions across measured entities, and Version 6.5 08/20/13 9 impact on performance measure scores) 2b3.3. What is your interpretation of the results in terms of demonstrating that exclusions are needed to prevent unfair distortion of performance results? (i.e., the value outweighs the burden of increased data collection and analysis. Note: If patient preference is an exclusion, the measure must be specified so that the effect on the performance score is transparent, e.g., scores with and without exclusion) During the analysis of this data and also as a byproduct of the validation audit in reviewing medical records, MNCM staff has some concerns regarding the denominator and intent of the measure. There may need to be a consideration for adding a component of indicating that the patient is sexually active or has the potential to be sexually active, and not physically handicapped. AAN could refer to the NCQA specifications for the Chlamydia Screening in Women measure (NQF# 0033/ CMS 153v1) for reference tables indicating how to identify potentially sexually active women via pharmacy codes, CPTs, ICD-9, UB Revenue and LOINC codes. Rather than trying to capture/ code every possible exclusion; this may be an option. MNCM would not recommend having a general type exclusions code, like one that is stated as “any documented medical reason, because providers will use this to their advantage and exclude patients that are at risk for pregnancy and truly belong in the denominator. Having this type of exclusion weakens the measure, and can impact the validity and reliability of the results. ____________________________ 2b4. RISK ADJUSTMENT/STRATIFICATION FOR OUTCOME OR RESOURCE USE MEASURES If not an intermediate or health outcome, or PRO-PM, or resource use measure, skip to section 2b5. 2b4.1. What method of controlling for differences in case mix is used? ☒ No risk adjustment or stratification ☐ Statistical risk model with Click here to enter number of factors risk factors ☐ Stratification by Click here to enter number of categories risk categories Version 6.5 08/20/13 10 ☐ Other, Click here to enter description 2b4.2. If an outcome or resource use measure is not risk adjusted or stratified, provide rationale and analyses to demonstrate that controlling for differences in patient characteristics (case mix) is not needed to achieve fair comparisons across measured entities. 2b4.3. Describe the conceptual/clinical and statistical methods and criteria used to select patient factors used in the statistical risk model or for stratification by risk (e.g., potential factors identified in the literature and/or expert panel; regression analysis; statistical significance of p<0.10; correlation of x or higher; patient factors should be present at the start of care and not related to disparities) 2b4.4. What were the statistical results of the analyses used to select risk factors? 2b4.5. Describe the method of testing/analysis used to develop and validate the adequacy of the statistical model or stratification approach (describe the steps―do not just name a method; what statistical analysis was used) Provide the statistical results from testing the approach to controlling for differences in patient characteristics (case mix) below. If stratified, skip to 2b4.9 2b4.6. Statistical Risk Model Discrimination Statistics (e.g., c-statistic, R-squared): 2b4.7. Statistical Risk Model Calibration Statistics (e.g., Hosmer-Lemeshow statistic): 2b4.8. Statistical Risk Model Calibration – Risk decile plots or calibration curves: 2b4.9. Results of Risk Stratification Analysis: 2b4.10. What is your interpretation of the results in terms of demonstrating adequacy of controlling for differences in patient characteristics (case mix)? (i.e., what do the results mean and what are the norms for the test conducted) 2b4.11. Optional Additional Testing for Risk Adjustment (not required, but would provide additional support of adequacy of risk model, e.g., testing of risk model in another data set; sensitivity analysis for missing data; other methods that were assessed) _______________________ 2b5. IDENTIFICATION OF STATISTICALLY SIGNIFICANT & MEANINGFUL DIFFERENCES IN PERFORMANCE 2b5.1. Describe the method for determining if statistically significant and clinically/practically meaningful differences in performance measure scores among the measured entities can be identified Version 6.5 08/20/13 11 (describe the steps―do not just name a method; what statistical analysis was used? Do not just repeat the information provided related to performance gap in 1b) 2b5.2. What were the statistical results from testing the ability to identify statistically significant and/or clinically/practically meaningful differences in performance measure scores across measured entities? (e.g., number and percentage of entities with scores that were statistically significantly different from mean or some benchmark, different from expected; how was meaningful difference defined) 2b5.3. What is your interpretation of the results in terms of demonstrating the ability to identify statistically significant and/or clinically/practically meaningful differences in performance across measured entities? (i.e., what do the results mean in terms of statistical and meaningful differences?) _______________________________________ 2b6. COMPARABILITY OF PERFORMANCE SCORES WHEN MORE THAN ONE SET OF SPECIFICATIONS If only one set of specifications, this section can be skipped. Note: This criterion is directed to measures with more than one set of specifications/instructions (e.g., one set of specifications for how to identify and compute the measure from medical record abstraction and a different set of specifications for claims or eMeasures). It does not apply to measures that use more than one source of data in one set of specifications/instructions (e.g., claims data to identify the denominator and medical record abstraction for the numerator). If comparability is not demonstrated, the different specifications should be submitted as separate measures. 2b6.1. Describe the method of testing conducted to demonstrate comparability of performance scores for the same entities across the different data sources/specifications (describe the steps―do not just name a method; what statistical analysis was used) 2b6.2. What were the statistical results from testing comparability of performance scores for the same entities when using different data sources/specifications? (e.g., correlation, rank order) 2b6.3. What is your interpretation of the results in terms of demonstrating comparability of performance measure scores for the same entities across the different data sources/specifications? (i.e., what do the results mean and what are the norms for the test conducted) _______________________________________ 2b7. MISSING DATA ANALYSIS AND MINIMIZING BIAS 2b7.1. Describe the method of testing conducted to identify the extent and distribution of missing data (or nonresponse) and demonstrate that performance results are not biased due to systematic missing data (or differences between responders and nonresponders) and how the specified handling of missing data minimizes bias (describe the steps―do not just name a method; what statistical analysis was used) Version 6.5 08/20/13 12 See Full AAN Women with Epilepsy of Childbearing Potential Measure Testing Report. The validation process is conducted to verify that the submitted data matches the source data in the medical record. After the clinical data file is successfully transferred to MNCM and passes the initial quality checks, MNCM will contact the medical group about the validation process. MNCM utilizes the National Committee for Quality Assurance (NCQA) “8 and 30” process for validation audits. If the first clinic site is in high compliance and the data collection process for all clinic sites within the medical group is identical, further review may be abbreviated at the discretion of the MNCM auditor. If clinic sites are not in high compliance after review of the first eight records, the MNCM auditor will continue to review the remaining 22 records. If after review of all 30 records the clinic site is not in high compliance on all factors (less than 90%), the MNCM auditor will review the results with the clinic representative and communicate the results with MNCM. MNCM will then contact the medical group to develop a mutually agreed upon re-submission plan. (Re-submission plans will only be allowed for errors in the numerator portion.) Clinic sites that are not in high compliance or have not been in high compliance in a previous MNCM audit may be held to a more rigorous denominator certification and validation audit. 2b7.2. What is the overall frequency of missing data, the distribution of missing data across providers, and the results from testing related to missing data? (e.g., results of sensitivity analysis of the effect of various rules for missing data/nonresponse; if no empirical sensitivity analysis, identify the approaches for handling missing data that were considered and pros and cons of each) 2b7.3. What is your interpretation of the results in terms of demonstrating that performance results are not biased due to systematic missing data (or differences between responders and nonresponders) and how the specified handling of missing data minimizes bias? (i.e., what do the results mean in terms of supporting the selected approach for missing data and what are the norms for the test conducted; if no empirical analysis, provide rationale for the selected approach for missing data) MNCM did not identify any major flaws or issues during the review of each medical group’s denominator forms and therefore each medical group passed denominator certification within the given timeframe. There were a few corrections and clarifications that required MNCM to send a follow-up email to the respective group; however, each issue was resolved in a timely manner. The list below documents the issues that were identified and required additional follow-up based on the information received on the denominator certification forms: • Incorrect diagnosis codes included in data query • Group did not indicate if they would be submitting a sample or full population for the measure • Incorrect encounter codes included in data query Version 6.5 08/20/13 13 MINNESOTA COMMUNITY MEASUREMENT Measure Testing Summary Report: Counseling for Women of Childbearing Potential with Epilepsy Prepared for: American Academy of Neurology 12/18/2013 Table of Contents Project Background ................................................................................................................................................................. 2 Purpose ............................................................................................................................................................................... 2 Methods .............................................................................................................................................................................. 2 Results & Findings ................................................................................................................................................................... 3 Descriptive Measure Statistics ............................................................................................................................................ 3 Table 1: Patient Payer Information ................................................................................................................................. 3 Table 2: Patient Place of Residence (based on zip code) ................................................................................................ 3 Table 3: Patient Race and Ethnicity Information ............................................................................................................ 3 Table 4: Patient Age Breakdown ..................................................................................................................................... 4 Measure Results.................................................................................................................................................................. 4 Validation Results / Audit Results ....................................................................................................................................... 5 Denominator Certification .............................................................................................................................................. 5 Data File Quality Checks.................................................................................................................................................. 5 Validation Audit .............................................................................................................................................................. 6 Validation/ Audit Conclusion .......................................................................................................................................... 6 Limitations........................................................................................................................................................................... 7 Future Measure Implementation ....................................................................................................................................... 7 Appendix A: Denominator Certification Form ........................................................................................................................ 9 Appendix B: File Quality Checks & Validation ....................................................................................................................... 12 Appendix C: MNCM Audit Strategy ....................................................................................................................................... 13 Appendix D: Questions and Answers Received During Measure Testing ............................................................................. 14 Appendix E: Post Measure Survey Results: Resource Use and Data Burden.................................................................... 17 Project Background In 2012, the AAN submitted epilepsy quality measures to the National Quality Forum (NQF), a national, nonprofit organization that reviews and endorses health care quality measures for use by public and private payers. The AAN received conditional endorsement of its Counseling for Women with Epilepsy measure. To receive full endorsement the AAN must field test the Counseling for Women with Epilepsy measure for feasibility, reliability, and validity prior to NQF’s review of this measure in March 2014. To achieve the goal of testing the Counseling for Women with Epilepsy measure the AAN contracted with Minnesota Community Measurement (MNCM), a non-profit organization specializing in health care quality measurement and reporting, to collect data pertaining to this measure from Neurology practices. Purpose This project aims to successfully collect de-identified patient-level data from Neurology practices that have the capability to report accurate denominator and numerator information for the Counseling for Women with Epilepsy measure. In order to achieve a reliable sample of patients for this measure MNCM and AAN are seeking a minimum of 1,000 combined patient records from the Neurology sites that agree to participate. Methods The AAN identified and recruited Neurology practices in Minnesota that have experience treating patients with epilepsy. As part of the recruitment process MNCM and the AAN hosted an informational webinar explaining the purpose of the measurement testing project for the Counseling for Women with Epilepsy measure. Three Neurology practices volunteered to participate and submit retrospective data from the 2012 calendar year (i.e. dates of service 01/01/2012 – 12/31/2012). MNCM produced a data collection guide, measure flow and detailed file specifications to educate and assist each medical group in the data collection and submission process. As a requirement of participating in the measure testing each group had to submit a denominator certifications form (see appendix A). The denominator certification process helps ensure that each medical group is using the appropriate measure parameters and collecting data in a standardized way. Once the denominator certifications were complete each group submitted their data files using a secure FTP transfer process. Once the files were received MNCM performed quality checks on each files using the methods outlined in Appendix B. Once the files passed the quality checks MNCM calculated and sent the results back to each group for review. If the group did not find any issues with the measure results then MNCM conducted an audit to validate the accuracy of data using the auditing principles outlined in Appendix C. The final step in testing the Women with Epilepsy measure was a post-data submission survey that was sent to each of the participating medical groups. The intent of the survey was to shed light on the amount of resources that were required to produce the data as well as gauge the level of data collection burden that each data field presented to the group. The results of the survey can be found in Appendix E. Results & Findings Descriptive Measure Statistics Table 1: Patient Payer Information Payer Type Medicaid Medicare SelfPay/Uninsured Commercial Medicaid Clinic A 105 9 461 176 Clinic B 76 72 Clinic C 11 422 2 45 5 Grand Total 76 184 7 22 928 181 Table 2: Patient Place of Residence (based on zip code) State Minnesota Iowa Wisconsin North Dakota South Dakota Alaska Arkansas Colorado Illinois Maryland Michigan Nebraska New York North Carolina Oklahoma Ohio Pennsylvania Blank Group A Patients 32 7 4 3 1 1 1 2 1 Group B Patients 649 19 49 9 11 1 1 1 1 1 5 1 Group C Patients 561 3 14 1 1 1 Total 1242 29 67 9 15 1 1 2 2 2 3 7 1 1 2 1 2 1 1 5 1 1 3 2 Table 3: Patient Race and Ethnicity Information Race American Indian/Alaska Native (Code 1) Asian (Code 2) Black/African American (Code 3) Hispanic/Latino (Code 4) Native Hawaiian/Other Pacific Islander (Code 5) Group A Patients 2 Group B Patients 6 12 19 12 3 Group C Patients 3 8 18 Total 9 22 37 12 3 White (Code 6) Some Other Race (Code 7) Unknown (Code 98) Chose not to disclose (Code 97) Blank 54 3 335 2 1 236 625 5 5 11 662 4 11 301 361 Table 4: Patient Age Breakdown Total by clinic Ages 12-17 Ages 18-25 Ages 26-30 Age 31-35 Ages 36-40 Ages 41-44 Group A Patients 14 13 8 13 8 3 Group B Patients 170 206 129 111 93 42 Group C Patients 112 175 113 73 60 48 Total 296 394 250 197 161 93 Measure Results Measure Results Number of Providers (NPI) Number of Patients Submitted Clinic A 9 751 Clinic B 36 581 Clinic C 22 59 Total 67 1391 127 18 145 606 420 419 419 32 12 44 537 26 77 21 2 5 7 52 11 7 6 161 35 196 1195 457 503 446 Rate for Contraceptive Counseling 69.3% 4.8% 21.2% 38.2% Rates for Pregnancy Counseling 69.1% 14.3% 13.5% 42.1% Rates for Contraceptive and Pregnancy Counseling 69.1% 3.9% 11.5% 37.3% Other Medical Reasons for Not Counseling Patients Number of Patients Submitted Number of Patients with “Other Medical Reason” Percentage of Patients with “Other Medical Reason” Clinic A 751 122 16.2% Clinic B 581 156 26.9% Clinic C 59 6 10.2% Total 1391 446 20.4% Number of Patients Excluded; intellectual disability codes Number of Patients Excluded; surgically sterile Number of Patients with valid exclusions Denominator: Number of Patient Eligible for Counseling Number of Patients with Counseling for Contraception Number of Patients with Counseling for Pregnancy Number of Patients with Contraception and Pregnancy Please note: The following rate re-calculation is for analytical purposes only; removing all patients that had “Other Medical Reason Documented”. MNCM does not recommend reporting this rate. Please see Limitations Section Rates for Contraceptive and Pregnancy Counseling if these patients are also removed from the denominator 86.6% 5.5% 13.0% 49.0% Validation Results / Audit Results MNCM completed validation of the data in a three-step process: 1) denominator certification, 2) data file quality checks, and 3) validation audit. Details of this validation are described in this report. Denominator Certification Denominator certification is an essential step in the process to obtaining valid and accurate data. It requires each participant to attest that they will submit accurate data and follow the measure specifications exactly how they are written. It also ensures that each participant is querying the correct: Diagnosis codes (i.e. 345.00, 345.01, 345.10, 345.11, 345.40, 345.41, 345.50, 345.51, 345.70, 345.71, 345.90, 345.91) Encounter codes (i.e. 99201, 99202, 99203, 99204, 99205, 99212, 99213, 99214, 99215, 99241, 99242, 99243, 99244, 99245) Date of birth ranges (i.e. 01/01/1968-01/01/2000) Date of service ranges (i.e. 01/01/2012- 12/31/2012) MNCM did not identify any major flaws or issues during the review of each medical group’s denominator forms and therefore each medical group passed denominator certification within the given timeframe. There were a few corrections and clarifications that required MNCM to send a follow-up email to the respective group; however, each issue was resolved in a timely manner. The list below documents the issues that were identified and required additional follow-up based on the information received on the denominator certification forms: Incorrect diagnosis codes included in data query Group did not indicate if they would be submitting a sample or full population for the measure Incorrect encounter codes included in data query Data File Quality Checks After each medical group submitted their data file to MNCM, quality checks of the files were completed. Each column in the data file represented a field of data for each patient row; the following checks were completed: Number of patients/rows submitted were reasonable/expected Necessary data fields (columns) were included and completed appropriately Patient date of birth spanned the expected range Zip codes were 5-digit and primarily within MN and other bordering states as expected Race field(s) were included and populated appropriately Provider NPI field was included and number of providers expected Insurance information was included and was reasonable Office visit dates and counseling dates spanned the expected range Diagnoses were included and spanned the entire list of expected codes Medical reasons for NOT counseling were applied correctly; were not misused Issues identified through the data file quality checks were generally minor, requiring no corrections. Other mentionable items include: 1. All three groups did not have patients with diagnosis codes 345.70 (Epilepsia partialis continua, without mention of intractable epilepsy) or 345.71 (Epilepsia partialis continua, with intractable epilepsy). These are rare diagnoses and did not come up in the population. 2. Medical groups B and C listed many neurological or congenital conditions as reasons for the patient to NOT receive counseling. These were verified during audit. 3. Medical group C did not include their entire population in first submission, excluding patients whose date of birth was between January thru June 1968. They queried their system again, this time using the specific dates of birth (rather than “age” values) and included the additional patients in their denominator. Validation Audit After the data file checks were completed, MNCM completed audits of the patient records to verify the submitted clinical data. We also verified the diagnosis of epilepsy and other demographic data (e.g., race). MNCM uses a validation process developed by the NCQA – National Committee for Quality Assurance, known as the “8 and 30” process. In this process, the first eight records are verified for accuracy and if no errors are identified, the data is considered to be 100% compliant. If errors in the first eight records are identified, we continue reviewing the total 30 records to identify any error patterns and or issues that may need correction. The audits revealed some data errors, requiring one medical group to make corrections and resubmit data. Individual medical group results were as follows: Medical group A B C Audit details Follow-up action 8 records reviewed, 8 records compliant (100%) No further action necessary 3 additional records were reviewed for “other” reason patient was not counseled o 2 records were compliant o 1 record indicated patient had functional seizures and not epilepsy, but should have been counted as no counseling provided 30 records reviewed, 26 compliant (87%) o Errors: three records had code “2” for no counseling due to intellectual disability, however, we could not verify the diagnosis in the record; one record reported as “1” counseling given could not be verified Group verified patients they submitted who were “surgically sterile” or who had “intellectual disability”; resubmitted data with corrections 3 additional records were reviewed whose patients were listed as “cognitively impaired” as a reason for not receiving counseling; verified that these patients had mild retardation; medical group staff corroborated that all 115 patients with this designation also had mild retardation 8 records reviewed, 8 records compliant (100%) No further action necessary We identified one record in the eight reviewed in which the patient could have been flagged for a medical reason to NOT receive counseling (99 “other), but because the reasons were not either type (surgically sterile, intellectual disability), it was appropriate that these patients could have been counseled; these were not counted as errors Validation/ Audit Conclusion The validation process was successful in identifying errors (with subsequent corrections) and verifying the accuracy of the data submitted by medical groups A, B, and C. Finding no significant flaws or errors with the data MNCM is confident the rate calculation and any additional data analysis can be completed using validated and reliable data. Additionally, during a review of the National Quality Forum’s feedback to the American Academy of Neurology for this measure, it was noted that there was a concern that this may simply be a “check-the-box” measure. During the validation audit, it was noted on several occasions that the practices provided excellent, personalized progress notes about the counseling that was being provided, that were above and beyond a “check the box”. Limitations The main limitation that MNCM identified during the testing of the Counseling for Women with Epilepsy measure is related to the denominator of included and excluded patients. The measure specifications offered two different options for excluding patients from the measure: 1. Patient was surgically sterile (tubal ligation, hysterectomy) 2. Patient has an intellectual disability as defined by ICD-9 codes a. 318.0 moderate intellectual disabilities; IQ 35 to 48 b. 318.1 severe intellectual disabilities; IQ 20 to 34 c. 318.2 profound intellectual disabilities; IQ under 20 Groups submitted these patients and indicated which reason applied. Additionally, if they felt that there was another medical reason for not providing counseling, they indicated this by a code and accompanying description. These reasons were not used to exclude patients from the measure; rather the purpose was to provide additional information about the population of patients included in the measure. Reasons Provided by Medical Groups for Not Providing Counseling: Reason by Frequency Count Valid Thoughts cognitive impairment/ deficit 138 Maybe cerebral palsy 45 Yes need to quantify by code neurodevelopmental disorder 18 Yes need to quantify by code encephalopathy 15 Yes need to quantify by code developmental delay 14 No may still be at risk hydrocephalus 8 Yes need to quantify by code brain injury 8 Yes need to quantify by code pregnancy 7 No still needs counseling pre-menarche 7 No may still be at risk autism 4 No may still be at risk downs syndrome 3 Maybe may still be at risk aspergers 2 No spectrum of functioning; at risk birth control- IUD 2 No may still be at risk learning disability 2 No spectrum of functioning; at risk menopause 1 Yes need to quantify by code no menses 1 Maybe multiple sclerosis 1 No subjective and may still be at risk Spectrum of functioning; at risk Future Measure Implementation During the analysis of this data and also as a byproduct of the validation audit in reviewing medical records, MNCM staff has some concerns regarding the denominator and intent of the measure. There may need to be a consideration for adding a component of indicating that the patient is sexually active or has the potential to be sexually active, and not physically handicapped. AAN could refer to the NCQA specifications for the Chlamydia Screening in Women measure (NQF# 0033/ CMS 153v1) for reference tables indicating how to identify potentially sexually active women via pharmacy codes, CPTs, ICD-9, UB Revenue and LOINC codes. Rather than trying to capture/ code every possible exclusion; this may be an option. MNCM would not recommend having a general type exclusions code, like one that is stated as “any documented medical reason, because providers will use this to their advantage and exclude patients that are at risk for pregnancy and truly belong in the denominator. Having this type of exclusion weakens the measure, and can impact the validity and reliability of the results. Appendix A: Denominator Certification Form What information do I submit to MNCM? The instructions in this document will guide you in creating the necessary documentation for Denominator Certification. Using the following steps, you will construct a Word document. Your document must describe the process you use to identify eligible patients, including source code and screen shots of your query. After you submit your document to the MNCM, MNCM will review your document and respond within two business days. Please note: MNCM will review your denominator method, however, you are ultimately responsible for interpreting and applying the measure specifications correctly in your query. Please submit your denominator document to [email protected] **Please do not submit patient data along with your denominator document** What is denominator certification? Denominator Certification is the process by which medical groups submit a document that explains the process they use to identify patients for the measure. MNCM then reviews the documentation to verify the measure specifications for the denominator were followed. What are the denominator criteria? All females of childbearing potential (12 to 44 years old) with a diagnosis of epilepsy are in the denominator. Please refer to the data collection guide for the complete measure specifications. Complete this table and provide additional details Denominator document Your response instructions 1. Medical group information Measure: Enter measure name Supply the following information: Medical group name: Enter medical group name Your name: [Enter your name] Your phone number: Enter your phone number Your email address: Enter your email Name of your medical director, administrator or lead: Enter name of medical director, administrator or lead 2. Date of birth range: We will use the following date range to identify patients age 12 to 44: Enter the date range, 3. Date of service range: We will use the following date range to identify patients with one or more face-to-face office visits with a provider: Enter the date range, 4. ICD-9 diagnosis codes (epilepsy): We will use the following ICD-9 diagnosis codes to identify patients with epilepsy: Denominator document instructions 5. Exceptions: Please indicate how you will identify exceptions for the denominator. Your response We will identify exceptions as follows: Patients who are surgically sterile: we will identify through manual data abstraction other, please describe: Enter here Patients with intellectual disabilities: we will remove upfront using ICD-9 codes 318.0, 318.1 and 318.2 we will identify through manual data abstraction 6. Attestations Read each attestation carefully. You must agree to all attestations before you submit your denominator. By submitting this document, you are indicating that you agree with these attestations. Please contact MNCM if you have any questions. 1. We agree to follow the denominator criteria outlined in the measure specifications when searching for eligible patients, and we are ultimately responsible for interpreting and applying the measure specifications correctly in our query. 2. We agree to include patients who are not active patients if they are eligible based on the measure criteria (i.e., we will include patients whose status is “inactive” or patients who transferred care). 3. We agree to identify exceptions for surgically sterile or intellectual disability only. All other patients that meet the denominator criteria will be included in the denominator. 4. Include one of the following attestations: a. We agree to submit our full population of eligible patients b. We agree to use one of the sampling methods described in the data collection guide to randomly select patients 5. We agree to identify and remove any duplicate patients. 6. Our medical director, administrator or other lead attests that the measure specifications will be followed and all eligible patients included in the denominator. Supply source code or screen shots of your query in a Word document 1. Generate a query of your record system (e.g., electronic medical record, practice management system, billing system); maintain the source code and/or screen shots of the steps you take to search for eligible patients. Copy and paste 1) the source code or 2) the screen shots of your steps into the Word document 2. Highlight details for the MNCM reviewer: Date of birth range Date of service range ICD-9 diagnosis codes (epilepsy) Exceptions (e.g., ICD-9 codes for intellectual disability) Screen shots: If you cannot include data source code, you can instead copy and paste screen shots of your process in your Word document. Please only include screen shots that will demonstrate to the MNCM reviewer that you have used the correct criteria for querying your system for eligible patients. For example, the following criteria must be clearly shown in the screen shot: correct date of birth range, correct dates of service, correct diagnosis codes if applicable. Do not include screen shots with blank codes, dates, etc. (this will not demonstrate to the MNCM review that the criteria used are correct). **Please do not submit patient data along with your denominator document** Appendix B: File Quality Checks & Validation Counseling for Women with Epilepsy measure| File Quality Checks & Validation File Quality Checks | to verify the data submitted is complete and in the correct format 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. Clinic name is entered consistently for each record Patient IDs are not duplicated Date of birth is between = 01/01/2000 to 01/01/1968; full range is covered in data set Zip codes are 5 digits and primarily MN zip codes Race was entered in one or more fields with valid codes Provider NPI fields are all populated with 10-digit values Insurance product name fields are all populated with valid entries Primary payer type fields are all populated with valid codes Office visit date is between 01/01/2012 and 12/31/2012; full range is covered in data set Diagnosis code was entered with valid code; expect to see more than one type of code entered Date of contraception counseling is between 01/01/2012 and 12/31/2012; full range is covered in data set Contraception counseling received? fields are all populated with valid codes Date of pregnancy counseling is between 01/01/2012 and 12/31/2012; full range is covered in data set Pregnancy counseling received? fields are all populated with valid codes Medical reason for not receiving counseling fields that are populated contain valid code Medical reason for not receiving counseling “other” fields that are populated contain valid reasons (review each reason given) 17. “Medical Reason” fields are populated minimally 18. “Medical Reason” fields (columns S and T) that are populated must not have counseling dates or values entered 19. Number of rows of patients matches what was expected Appendix C: MNCM Audit Strategy The validation process is conducted to verify that the submitted data matches the source data in the medical record. After the clinical data file is successfully transferred to MNCM and passes the initial quality checks, MNCM will contact the medical group about the validation process. MNCM Audit Process MNCM utilizes the NCQA (National Committee for Quality Assurance) “8 and 30” process for validation audits. The following method is used for each measure: MNCM randomly selects 33 records for each clinic site for validation. At most, 30 records for each clinic site will be reviewed. The additional three records requested are oversamples to ensure there will be 30 records available on the day of the review. MNCM auditor reviews the first eight records of the clinic site’s selected sample to verify that the submitted data matches the source data in the medical record. If all of the first eight records reviewed are in perfect compliance (100%), the clinic site is determined to be in high compliance, and the MNCM auditor may determine that no further record review for that site is necessary. If the first clinic site is in high compliance and the data collection process for all clinic sites within the medical group is identical, further review may be abbreviated at the discretion of the MNCM auditor. If clinic sites are not in high compliance after review of the first eight records, the MNCM auditor will continue to review the remaining 22 records. If after review of all 30 records the clinic site is not in high compliance on all factors (less than 90%), the MNCM auditor will review the results with the clinic representative and communicate the results with MNCM. MNCM will then contact the medical group to develop a mutually agreed upon re-submission plan. (Re-submission plans will only be allowed for errors in the numerator portion.) Clinic sites that are not in high compliance or have not been in high compliance in a previous MNCM audit may be held to a more rigorous denominator certification and validation audit. Data Fields for Audit: Counseling for Women of Childbearing Potential with Epilepsy Patient ID Patient DOB Patient is female Race 1-5 Office visit date ? Patient has epilepsy Date of contraception counseling Contraception counseling received Date of pregnancy counseling Pregnancy counseling received Medical reason code – no counseling Medical reason “other” – no counseling Appendix D: Questions and Answers Received During Measure Testing Q: Does the counseling need to take place during the measurement period (2012). If patient was seen, for example in 2011, and counseling was done then, I would assume that would not count towards meeting the measure. Or can the counseling occur prior to the measurement period . A: Counseling does need to take place during the measurement period for it to count towards meeting the measure criteria. Q: What about patients who have the diagnoses of epilepsy but who may not be on anti-epileptic medication? Is the presence of anti-epileptic medication being prescribed considered an exclusion from the denominator? Or should the counseling be provided whether they are on medication or not? A: The counseling should be provided regardless of whether or not the patient is on medication. A patient who has epilepsy, but is not on anti-epileptic medication still would be included in the measure. Although medication is a common treatment, this is a measure to capture if a female patient with epilepsy was counseled on the disease and treatment (regardless of what type of treatment) and how it may affect contraception and pregnancy. Q: We have some patient seen during the measurement period in our clinic for something completely unrelated to their epilepsy diagnosis, example for sleep disturbance. Their epilepsy is not being addressed at all. They carry the diagnosis of epilepsy so they are showing up in the denominator. Some may have also been seen by their regular neurologist but some may not have been. Should these patients be included in the denominator? A: These patients should still be included in the denominator as the visits are opportunities to address how epilepsy and its treatment may affect contraception and pregnancy. Q: We have patients that are seen by more than one provider during the measurement period with epilepsy diagnosis so we have two entries for them in the denominator. Should I take only the most recent encounter, and just use the provider who saw the patient most recently? A: The patient should only be counted one time in the denominator. For “Provider NPI”, refer to the data field specs – If both providers saw the patient equally enter the provider NPI who saw the patient most recently. If the patient received counseling from another provider, though, still submit the dates of the counseling. Q: Does patient use of birth control (part of medication list ie; birth control pills) provide an exception for receiving contraception counseling? A: No – counseling is still an expectation regardless if the patient is using birth control. This is an opportunity for the provider to address how epilepsy treatment may affect contraception. Q: Is tubal ligation considered surgical sterilization? A: Yes, patients with tubal ligation are excluded because they are considered to be surgically sterile Q: Is it appropriate to count Intrauterine Device (IUD) as “other” medical reason for not receiving counseling? A: Actually, based on the intent of the measures this does not count as a reason for not receiving counseling because having an IUD does not guarantee pregnancy will not occur for that patient. IUDs are not a permanent procedure and thus it is still appropriate to counsel the about the risks associated with pregnancy. The measure focuses on counseling about medications that may affect pregnancy or fetal malformation which may determine the course of treatment for a patient who is not sterile. Q: If a patient does not plan to get pregnant again and her husband had a vasectomy should she still receive counseling? A: While it may be true she does not plan on becoming pregnant at this time it is still appropriate for the patient to be counseled about the risks associated with using some epilepsy medications that can negatively affect a pregnancy or may limit the effectiveness of certain contraceptive medications. Unplanned pregnancies are extremely common. Q: We reviewed several patients who were on the young side of the age included who did not received counseling. Yet we wondered about the cultural appropriateness of discussing contraceptives and pregnancy with 12-14 year old patients from conservative religious environments. A: The measure specifications establishes 12 as the lowest age for receiving counseling so in the example above these patients will not be excluded from the numerator. The measure development work group discussed this issue quite extensively. The decision to use 12 as the minimum age came from the fact that the average age that girls have their first experience menstruation is 12. This has been cited by numerous different research studies and peer reviewed papers. Q: We found that the ICD-9 codes for intellectual disability (318.0, 318.1, or 318.2) were not sufficiently inclusive. One of our patients with ICD-9 code of 315.9 Developmental Delay Mental, would definitely not have been able to understand counseling. Another patient had Developmental Delay Global 783.42, lives at group home, Family Medicine started Oral Contraceptives to help with menses, and father makes the decisions for patient. We coded both case as did not receive counseling for “other” medical reasons. A: AAN understands that this is a retrospective chart review and that most clinicians and clinics are not familiar with the use of CPT-II exclusion codes or that they would need to document why something was not done. The AAN is not looking for specific codes to justify a medical exception or exclusion. Rather we are looking for something that is written in the medical record that would justify why the measure wasn’t done. So my response is that if the clinician and/or abstractor looking at what is documented by the clinician feels that the patient would have a medical exclusion/exception that was appropriate based upon what is written in the medical record that we accept that. So in the first case where the patient couldn’t understand any counseling it would be appropriate to exclude this patient with a medical exclusion. Given the details provided for the 2nd individual (with the inappropriate description of the code 783.42) who lives at a group home, is on contraception, etc. I would leave it up to the clinician/abstractor to make the decision whether or not she should be excluded since we do not know all the details of her cognitive impairment. In short, exclusions are primarily a judgment call by the physician; however, there should be documentation in the record explaining why the counseling was not appropriate. However, the codes you referenced will not be added to the exclusion list for the following reasons: 315.9 is actually “Unspecified delay in development” and the definition includes developmental disorder not otherwise specified and learning disorder not otherwise specified. This code is way too general and could be used for so many other things besides intellectual disabilities. Thus, MNCM recommends not including it. 783.42 is actually “Lack of normal physiological development in childhood; delayed milestones” and the definition includes late talker and late walker. Again, we would not recommend adding this code to define intellectual disability, it would be inappropriate. Q: If an oral contraceptive is on the active medication list, does the patient need further counseling? A: Yes, it is still appropriate to counsel the about the risks associated with pregnancy because the patient may still become pregnant in the future. Q: What should be done is cases with an initial diagnosis of epilepsy but found to have non-epileptic events upon further investigation? A: Since this patient met the denominator for the current measurement year they should be included. However, in future reporting years this patient would not be included in the denominator according to your example above. This issue will likely be a random and somewhat rare occurrence and thus performance results would likely not be significantly affected. At the time/visit that the retrospective chart review was being done if the patient had a diagnosis of epilepsy the measure should have been done. If the diagnosis changes in the future they would not be eligible for the measure and shouldn’t have the counseling at that future date. Appendix E: Post Measure Survey Results: Resource Use and Data Burden Q4: Respondent Comments on the Data Elements: “Some items were extremely time consuming and required me to look through the majority of charts to find data. I would expect that it will get easier with time as more information is put into our structured data fields that were created mid-year in 2012, hence the need to manually go through charts in EMR” Q8: Comments/Suggestions about your experience “Difficult to put items in your excel spreadsheet format. For example, our insurance extraction pulls what the name of the insurance is, it doesn't automatically group them into medicaid, medicare, commercial, etc. So it took some time as some plans can be both. Pulling dates that education was received was difficult as patients may have been seen more than once in a year, and I can tell by the data extraction that the patient was educated sometime in that year, but then I have to go through the chart manually to find which visit it took place in.”