Star Gazing – Why Bother?

“Star Gazing – Why Bother?”
Ken Diefenbach
Position Title : Project Manager, Business Intelligence @ CQU
Information Technology Division
Abuse, Arguments, Questions, etc: k.diefenbachATcqu.edu.au
11/11/2007
1
Presentation Sequence
Audience Poll
Setting the Scene
DW 101
What’s
Wh t’ progressed
d since
i
2006
Current Situation
Future Directions
Questions / Comments
22/09/2007
2
Setting the Scene - BIG
CQU
– COGNOS site since early 90’s
90 s for Business Intelligence
– PowerHouse site since before I started there in 1989
Business Intelligence Group
– ((“
Think BIG”)
BIG )
(“Think
BIG”
– Initially 3 members, 90+ years IT experience
As of Friday last, 5.7 EFT
And we lost 30+ of those years of experience
Moi
– 15 years + in Business Intelligence
– Chair of HEUG Reporting & Analysis Product Advisory
Group
Group
- Prone to talk too much and too fast
- Please tell me if it’s happening again!
•NB This makes it YOUR FAULT if you let me rabbit on!!!
22/09/2007
3
Data Warehouse 101
NOT a copy
py of the transaction data
base on a separate server
A repository where information from
multiple sources is restructured and
stored in a format more suitable for
analysis and reporting
CQU is
i still
till adopting
d ti th
the W
Warehouse
h
Gospel a la Ralph Kimball
– Operational
p
Data Stores
– Star Schemas
Sometimes the
we disagree
team disagrees
on
on
interpretation or emphasis – but being
the boss helps ☺
22/09/2007
4
DW 101 or – Why Stargaze?
Stargaze?…
Not because it’s
it s fun
Not because almost everyone is doing it
Hang it on some business needs!
Faster reporting from the data warehouse
Faster access to the originating
applications
Better quality data
22/09/2007
5
Why Stargaze Part 2?
Value added data – information that does
not exist in the source applications
Ability to report across applications
Provide a single version of the truth
Have the ability to import additional data
Provide better data structures for ad hoc
reporting
22/09/2007
6
Limitations
Not necessarily the latest view of the data
Development will be evolutionary, not big
bang
Data must already exist or be derivable
– Within ERP
– Within Excel / Access CSV / text
22/09/2007
7
Some appropriate music please
Maestro….
Maestro
http--www.wdisneyw.co.uk-photos-wish.mid.url
22/09/2007
8
Kimball According to Disney
Base your report on a star
Makes no difference who you are
Fact tables and dimensions
Will work for you…
Demands from Chancellery or Dean?
No request is too extreme!
Anything their hearts desire
Can come from you….
22/09/2007
9
Dimensions,
Dimensions Facts
Student
Term
Admissions reporting
Admissions
22/09/2007
Leave
type
yp
Job
Action
Time
Career
Application
Staff
Leave reporting
Faculty
Leave
10
What do you need to get a
?
warehouse?
A location to store it
– “Pontious”
“P ti
” was our pilot
il t
– Living proof that SysAdmins can’t spell
– And / or never went to Sunday School
A way to get the data in
– An ETL tool is one way:
EXTRACT the data from multiple sources
TRANSFORM this data into a different form to optimise it for reporting
LOAD it into the Data Warehouse
– And you need a server to run this on
“Morph” is the server that we use to massage and reshape the data
A window to run the “refreshes” in according to need
– Refresh our complete student data early each morning except Sunday
(ODS and Warehouse)
– Takes pretty much 5 1/2 hours from 2am – 7:30 am counting PowerPlay
cubes
– Combination of Delta and full refreshes
– Expect more deltas as time becomes an issue
22/09/2007
11
We had the Politics
Business case from hell
Executive turnover
– A champion, a champion – my kingdom for a champion
Budget blues
– Project drawn up & commenced in a year of SARS
SARS--induced
reversal on international income
Business Case
RFI
ITO
Delays due to (un)due process and staff unavailability
Manyy selfself-appointed
pp
((and opinionated)
p
) Data Warehouse
Experts
22/09/2007
12
Expect Complications
Everyone wants to dump their old data in the
warehouse
– it’s a WAREHOUSE not an OUTHOUSE!!!!
ETL tools are powerful, complex and challenging
beasts – especially when in unfamiliar territory
(but that’s part of the fun)
Executive turnover + University review +
Reorganisation + … meant
Ever moving interview target
22/09/2007
13
Golden (Midas) Moments
First query on GPA at Melbourne Campus
on prepre-release
Unfortunate clarity on drop in student
numbers
Look on various faces when they see data
on maps with drillthrough
But for every Midas moment there’ll
there ll be
months of mud
– Strangely
g y many
y occur in lead up
p to Project
j
Board meetings ????
22/09/2007
14
What have we achieved so far?
Student Operational Data Store
Student Data Marts
–
–
–
–
Student Course
Student Program
Student Applications
Student Prospects
HR ODS now available – Meta Layer plus other tables
– Security of access based on org structure security extracted from
Alesco application
Other data feeds
– AIC Staff
St ff portal
t l
– Faculty restructure
– Axapta (used to manage external student mailouts and
packages)
22/09/2007
15
Current CQU IS Environment
PS Financials 8
8.4
4
Upgrading student from 7.6 to 8.9
HR system
t
(not
( t PS) upgraded
d d tto web
b
based architecture November 06
Cognos 7 reporting tools – at least the
majority we’ve deployed – are not
So we had to find a new mechanism for
the many
y reports
p
we have available
22/09/2007
16
So in June 2006
Decision to look for web based reporting
tools
Selected Cognos 8 after doing research
research,
leveraging existing work with some
migration aspects covered,
covered and some not
Training in late October 06
22/09/2007
17
Approach taken
Pause work in warehouse due to resource
limitations
A bit of training never hurts, neither
leveraging off a pretty useful consultant
Migrate Ye Olde Ancient Application
infrastructure and their reports first
– A lowlow-risk familiarityy with the new technology
gy
– Doesn’t hurt to do a few simpler things first
– Legacy
g y student and finance data complete
22/09/2007
18
There will always be gremlins…
Impromptu
p
p catalogs
g do migrate
g
to COGNOS 8
framework packages…
… but …
Migrations do not in and of themselves make
industrial strength infrastructure …
… particularly
ti l l if th
there’s
’ a very significant
i ifi
t
paradigm shift
Migrated infrastructure a starting point
No ROI on “tarting up” legacy system Impromptu
catalogs
g
22/09/2007
19
Sometimes
Sometimes, serendipity
serendipity…
Cognos 8 Framework Manager is very clever in
dealing with star schemas
Designate
g
dimensions,, fact tables,, and scope
p
Auto aggregation and other good gear follows
… which only goes to show Star Schemas make
infrastructure as well as reporting easy
… which
c makes
a es Ken
e a very
e y happy
appy ca
camper
pe
22/09/2007
20
New Boss Arrives
Emphasis
p
to shift to exec interface - dashboards
and high value adds
adds
“Strategic” is my mantra
BUT
– Need to convert existing report suite to COGNOS 8
as well
– New reports for HR & Student Upgrade
– Staff resourcing issues
Trips to two other unis in Oz to have a look and
share ideas, problems and solutions
22/09/2007
21
Reporting Pyramid
We had successfully used our reporting
tools in the operational and management
areas with some “bleed”
areas,
bleed into strategic
Not helping executives who need
information “no
no further than a click away”
away
and exposure to “games” some sections of
the University play
Dashboarding now a major focus
22/09/2007
22
The Fatal Flaws (and CQU’s Score
on some off them))
COGNOS has a white paper on these
Available from their web site
B aware that
Be
th t the
th 7 d
deadly
dl d
dashboard
hb d
don’ts (TM Ken D 2007) will vary
according
di tto which
hi h company produce
d
them
Briefly cover each and how well CQU has
done / is going
22/09/2007
23
1
1. “If
If we build itit, they will come”
come
Gartner says
y a successful initiative combines
business relevance with strong architecture
Ours works depending on a favourable definition
off “they”
“h ”
Having the data is part of it but it will remain
untapped without buy
buy--in and appropriate
delivery mechanisms
Change
C
a ge & co
communication
u cat o management
a age e t is
s key
ey
CQU rating (/10): 3
In summary,
y, g
guilty
y as charged
g
22/09/2007
24
2. “Managers need to dance with
the numbers””
Spreadsheet proliferation – everybody has
their own version of the numbers
Cf “One
Cf.
One version of the truth”
truth
Very important for data to be presented
f
free
off nuance and
d misinterpretation,
misinterpretation
i i t
t ti ,
and secure from “creative flair”
CQU Score – N/A. Yet. Turf war may be
approaching
22/09/2007
25
3. “We don’t have a data quality
problem””
We know _most_ of the student data issues
Errors reported to users for correction at source
Error dimension rows created so that data is visible
where valid and identified where not
We are becoming painfully aware of HR issues
Finance will be “interesting”
Data cleansing major byby-product of exposing
“information”
Diefenbach’s Doctrines of Data Daintiness
– Data will appear to all intents and purposes clean and clear until
you use it
– Do not for one moment consider you will find all the holes first
– Nothing “airs one’s dirty linen” or “crap detects” like 3rd party
tools
CQU score 66-7/10
22/09/2007
26
4. “Our business application vendor
will deliver the best solution””
Not going there
Have high opinions of and more than
significant relationships with both
COGNOS and Oracle and they’ll give you
different interpretations
Remember, it has to work for your
I tit ti
Institution
22/09/2007
27
5. “Darwin was wrong. There’s no
need ffor BI applications to evolve””
In which COGNOS p
pushes a single
g tool set to
cover all BI
Certainly makes maintenance easier, and from
experience
i
we kknow “f
“feral”
l” BI apps can breed
b d
like rabbits without the rigour & governance that
a single toolset can provide
Points of pain particularly occurs when one exec
discovers data delivered differently to another
exec and it’s data that they want as well
CQU 6/10
22/09/2007
28
6. “We can outsource the whole
thing””
Depends on how you define “the
the whole
thing”
Not touching this one in print either ☺
22/09/2007
29
7
7. “Just
Just give me a dashboard”
dashboard
Dashboards often win over balanced scorecard
because they’re seen to be easier and faster to
deliver
M
Must
reflect
fl
b
business
i
d
drivers
i
and
d allow
ll
d
drill
drillilldown to supporting detail
More effective if based on strategy maps which
show cause and effect
Easier
as e to do with
t a warehouse
a e ouse u
underpinning
de p
g
And we haven’t done enough in this space for
me to comment
22/09/2007
30
Diving in for
a Closer
Look
22/09/2007
31
Stars in Our Eyes
We chose Kimball Star
Schema approach as it
was our preferred of 2
approaches available cff
Bill Inmon’s approach
I Geekspeek:
G k
k
In
– (Conformed) Dimensional
Tables shared across all
Fact Tables
22/09/2007
32
St d t Program
Student
P
Data
D t Mart
M t – Every
E
Student Program
g
(since
(
1989 anyhow)
y
)
–
–
–
–
–
–
Total Courses Registered
Total Courses Dropped
T t l Courses
Total
C
E
Enrolled
ll d
Total Courses Passed
Total Courses Failed
Total Courses Outstanding
– Total Credit Points Attempted
– Total Credit Points Passed
– Total Credit Points
Outstanding
22/09/2007
– Weighted GPA
– Total Grade Points
– Total
T l GP Credits
C di T
Taken
k
– Total Workload Attempted
– Total Workload Passed
(EFTSL)
– Total Workload
Outstanding (EFTSL)
– CQU Transfer Credits
– Other Transfer Credits
– Total
T t l Transfer
T
f Credits
C dit
33
Student Program Dimensions
Student Personal Details
– Personal
P
l iinformation
f
ti about
b t each
h student,
t d t iincluding
l di names and
d
addresses, biodemographic data, citizenship and presence at the
University.
Program Details
– Information about each academic program.
Latest Student Program Details
– The latest information about each student program.
Expected Final Term Details
– The term the student expects to finish the program.
Completion Term Details
– The actual term the student completed
p
the p
program.
g
First Student Program Details
– The earliest information about each student program.
Application Details
– Some
S
information
i f
ti about
b t th
the student
t d t program application.
li ti
22/09/2007
34
Latest Stuff / Coming (Very)
S
Soon…
oon…
Access by new Faculty Structure
– SCD’s
SCD’ weren’t
’t meantt tto cope with
ith “Bi
“Big Bang”
B
”
Which teaching week every course was added or dropped during
semester – from 6 weeks or more prior to Week 1 to 10 or more
weeks into term
– NB vacations not considered teaching weeks
OP Score / Rank on entry but it isn’t quite where it should be yet
Cohort Sequencing for Dummies
–
–
–
–
–
–
Nth program commenced
db
by student
t d t
M awards from this program
This is the Pth award received
This student has commenced X p
programs
g
overall
This student has Y awards across all programs
Probably by early 08
Any significant requests
22/09/2007
35
Gut Clenching Finger Crossing
Demonstration Time
Remember Diefenbach’s
Diefenbach s 2 axioms of
demonstrations:
– “The
The chances of a demonstration working is
inversely proportional to the square of the
organisational
g
p
power and influence of the
audience”
– “The typing
yp g mistake rate increases
proportionally to (n+1)2 where n is the number
of observers”
22/09/2007
36
Questions?
22/09/2007
37