Data for Evaluation - Northwest Center for Public Health Practice

Comments

Transcription

Data for Evaluation - Northwest Center for Public Health Practice
Informatics, Data and Analysis
Ian Painter, Ph.D.
[email protected]
Part 1: Public Health Informatics Overview
Public Health Informatics
• Definition: The systematic application of information
technology to public health practice, research, and
learning
• Population focused rather than clinically focused
• Source of information sometimes clinical health
informatics systems
Public Health Informatics
• Technology (Hardware and software)
– Obtaining (and transferring) data
– Storing and accessing data
– Analyzing and using data
– Disseminating results
• Data
- What data can be collected?
- What data will be useful?
- How should data be used?
• Policies and procedures
- Accessing and sharing data
- Standards
• People
Big Data
• Big data means any data large enough to call big data
for purposes of funding…
• However there are some other definitions
- Data too big to analyze using traditional methods
- Data used mainly for predictions rather than inference
- NY City housing inspections “Big Data in the Big Apple”
Part 2: Data Sources
Existing data sources
• Data repositories
– wonder.cdc.gov
– Health Youth Survey
– http://www.countyhealthrankings.org/
• Health information technology (HIT) data
– Health care providers
– Health Information Exchanges (HIE)
– Surveillance data
HIT data
• Health care providers and Health Information Exchanges
(HIE)
– Transaction data vs. medical record data
• Surveillance data
HIT data
Clinical data may require considerable processing to be usable
MSH|^~\&|SOURCE|383018129|PRIORITY
HEALTH|382715520|2007100914484648||ORU^R01|0129938170710091448|P|2.3|
PID|1|1034157|012993817||LASTNAME^FIRSTNAME||19520101|M|||1234 MAIN^^DEARBORN
HEIGHT^MI^48127||||||||
PID|1||94000000000^^^Priority Health||LASTNAME^FIRSTNAME||19400101|F|
PD1|1|||1234567890^DOCLAST^DOCFIRST^M^^^^^NPI|
OBR|1|||80061^LIPID PROFILE^CPT-4||20070911||||||||||
OBX|1|NM|13457-7^LDL (CALCULATED)^LOINC|49.000|MG/DL| 0.000 - 100.000|N|||F|
OBX|2|NM|2093-3^CHOLESTEROL^LOINC|138.000|MG/DL|100.000 - 200.000|N|||F|
OBX|3|NM|2086-7^HDL^LOINC|24.000|MG/DL|45.000 - 150.000|L|||F|
OBX|4|NM|2571-8^TRIGLYCERIDES^LOINC|324.000|MG/DL| 0.000 - 150.000|H|||F|
Quality may not be as high as one would think
-
data use patterns may differ between nurses, providers, clinics
Representative only of people who utilize health care
Surveillance data
ED visit data
Date
2012-08-02
2012-08-02
2012-08-02
2012-08-02
2012-08-02
2012-08-02
2012-08-02
2012-08-02
2012-08-02
2012-08-02
2012-08-02
2012-08-02
2012-08-02
2012-08-02
Age Gender
15
M
54
F
2
M
8
M
54
M
91
F
5
F
35
F
56
F
2
F
39
M
33
M
26
F
19
M
Chief Complaint
FACE LACERATION
NECKBACK PAIN
ABD CRAMPING
POSSIBLE A-FIB; CHEST PAIN
6 WKS PG/ABD PN
WITHDRAWL SYMPTOMS
VISUAL DISTURBANCE
FACE LAC, TV FELL ON HIM
FLU LIKE SYMPTOMS HEADACHE
DIABETIC PROBLEMS
COUGH/NAUSEA/VOMIT/DIARRHEA
PELVIC/BLADDER PAIN/SPASMS
L PINKY FING JAMMED
CONGESTIONS
Collect your own data
• Process data
- Data collected as part of running program
- Data collected during the program for use in
evaluation
• Survey data
- Can always collect qualitative data
- Can be resource intensive to collect from public
Strategies – deciding what to collect
1. What data would be ideal to have for the evaluation
- Driven by the evaluation questions
2. What data will be available
- From program
- From outside sources
3. What can I reasonably collect?
Strategy tips
1. Directly link data into logic model / theory of change
1. Create blank results tables
- Will these answer the evaluation questions?
- What will different results look like?
Resources/Input
•
•
Mixture of paid and
volunteer staff Active
participation of Real
Change Board
Outputs
Activities
•
•
•
Training (vendor
orientation, volunteer
orientation, sales training)
Data collection (internal
blog, reader survey, vendor
demographic survey,
annual report)
Community hours at Real
Change
Communication with
vendors for service referrals
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Diverse group of
vendors
•
•
•
•
•
•
•
Engaged community
of readers
Generous individual
donors
Successful annual Real
Change breakfast
fundraiser
Grant funding
•
•
Listening Circles
Real Change Reads
Occupy CEHKC
Vendor referral services
Vendor advocacy groups
and initiatives
Planning of future cashless
transactions
Weekly distribution of Real
Change newspaper to
vendors
Selling of Real Change
newspaper to readers
•
•
•
•
•
•
•
•
•
•
•
Newspapers sold by
vendors
Total vendors
Vendors in 300 club
Vendors in 600 club
Vendors involved in
grassroots lobbying
Vendors attending
Advocacy group
Vendors attending vendor
meeting
Community referrals
Reader & Vendor surveys
Internal blog
•
•
•
•
•
•
•
•
•
Annual Real Change
breakfast fundraiser
Solicitation of donations
from community
•
•
Money raised by individual
donors
Money raised at annual
Real Change breakfast
Real Change newspapers
purchased
•
Impact
Change in number of
papers sold to vendors
Expansion of Real Change
to other counties
Adjust newspaper price to
support a living wage for
vendors
Increase in vendors meet
personal goals
Change in number of
vendors
Change in number of
vendors in 300 club
Change in number of
vendors in 600 club
Change in number of
papers sold to readers
Increase job skills and
employment opportunities
for vendors
Purchasing Real Change
newspaper
Individual donations
•
•
Newspapers sold to
venders
Satellite offices
Price adjustment of
newspaper to vendors
Price adjustment of
newspaper to readers
Attendees at trainings
Referrals to community
services
Effect of sales training on
vendor sales
Participation in surveys
Outcomes
Revenue generated from
fundraising
•
•
Build cross-class movement
for economic justice
Defend civil liberties of very
poor
Improved quality of life for
vendors
Part 3: Collecting and managing data
AKA 80% of the work….
Data collection tips
• Create overall data collection system that meets the
needs of multiple programs
• Use the same data collection system for all evaluation
purposes
• Keep data collection tools simple and “client” focused
Data collection tips
• Don’t collect data unless you have a use for it
• Become a sophisticated watch-dog
– Are the data useful? Are they used? If not, stop collecting them
or rethink approach.
• Evaluation report is a fringe benefit
– Largest benefit is being able to access information on the
problem(s) you are trying to solve
• Recycle
Data Collection Strategies 101
Review data and means used to collect that data:
?
“Would this same information answer questions we
currently have regarding program effectiveness?”
?
“Would this same information answer questions we
currently have regarding continuing or new needs in
our community for tobacco-use prevention activities?”
No
If no, get new tools
Yes If yes or sometimes, pull useful items that can
be reused and build new items into existing
resources.
Data Collection Strategies 101
• Use same items to collect longitudinal information.
• Only change items significantly if they are proven to
generate misleading information or are confusing to the
intended audience.
• Never discard existing data (this is your only resource
available to determine trends).
Data management tips
• Use a database for anything non trivial
(Excel is not a data base)
• Have an explicit data management process
• Expect to spend at least five times as much time
managing data as analyzing data
Data management tips
Have an explicit data management process
- Data retention
- Data storage
- Local or Central
- Redundant (backups)
- Technology obsolescence
- Change management
Part 4: Analyzing data
AKA 20% of the work….
Data analysis strategy
• Have a strategy!
• Plan analysis before starting
• Keep things simple
• Focus on meaning
– what do these numbers mean (and what do they measure?)
General analysis steps
1. Exploratory – understanding the data, are things as we
expect?
- Tables and/or graphs of all data elements
2. Generate descriptive summary tables and/or graphs
3. Generate outcome summary tables and/or graphs
4. Run formal analyses (IF ANY)
Software for analysis and visualization
- Excel
- Crude, but works for simple analyses
- SPSS
- Has nice scripting features
- STATA, SAS, R
- Generally only used through scripting
- Much more powerful than Excel or SPSS
- Steeper learning curves
- STATA is useful for complex samples
- R has best graphics
- Tableau
- Qualitative analysis: ATLAS.ti; NVivo
Wet feet
- Hands on BRFSS data with Excel
- SPSS demo
- R demo

Similar documents