0610DublinSQLDesign

When Bad Things Happen
to Good Databases
(Or something like that… )
Dublin SSUG: June 2010
Paul S. Randal
Kimberly L. Tripp
Author/Instructor:
Paul S. Randal
Consultant/Trainer/Speaker/Author
CEO, SQLskills.com
Email: [email protected]
Blog: http://www.SQLskills.com/blogs/Paul
Twitter: @PaulRandal
Microsoft Regional Director and SQL Server MVP
Contributing Editor of TechNet and SQL Server Magazines, author of the SQL Q&A
columns, bi-monthly articles on DBA topics
Author of multiple 2008 whitepapers, book chapters, articles
5 years at DEC responsible for the VMS file-system and check/repair
Almost 9 years as developer/manager in the SQL Storage Engine team through
August 2007, ultimately responsible for Core Storage Engine
Wrote DBCC component, other Storage Engine code, DBCC CHECKDB/repair for SQL
2005
Regular presenter at worldwide TechEds, SQL PASS and other conferences on
disaster recovery, HA, maintenance, and internals
Wrote and presented SQL Server 2008 training for Microsoft
Course author/instructor for Microsoft Certified Master qualifications
Co-Chair of the SQL Connections conference
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Author/Instructor:
Kimberly L. Tripp
Consultant/Trainer/Speaker/Writer
Founder, SQLskills.com
e-mail: [email protected]
blog: http://www.SQLskills.com/blogs/Kimberly
Microsoft Regional Director and SQL Server MVP
Writer/Editor for SQL Magazine www.sqlmag.com
Author/Instructor for SQL Server courses: Designing for Performance,
Indexing for Performance, Maintenance, Disaster Recovery/HA
Author/Manager of SQL Server 2005 Launch Content, Co-author/Manager
for SQL Server 2008 Jumpstart Content, Author/Speaker at TechEd, SQL
Connections, SQLPASS, ITForum, Conference Co-chair for SQL Connections
Author of several SQL Server Whitepapers on MSDN/TechNet:
Partitioning, Snapshot Isolation, Manageability and SQLCLR for DBAs
Author/Presenter for more than 25 online webcasts on MSDN and
TechNet (two series and other individual webcasts)
Co-author MSPress Title: SQL Server 2000 High Availability,
SQL Server 2008 Internals, and the SQL Server MVP Project
Presenter/Technical Manager for SQL Server 2000
High Availability DVD and various 2005/2008 HOL DVDs
I still love this stuff… don’t hesitate to ask questions!
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
The Development – Production Gap
Development occurs in a vacuum
Sometimes finding excitement in a feature/option but not knowing
the management or operational implications (is it even supported in
production)
Development and production don’t talk – usually until too late!
Developers don’t appreciate effect on SQL Server
See this over and over when working with clients
ORMs/LINQ
Easier application development
Not always the most optimal Transact-SQL
Often a lot of ad-hoc or forced statements
Key point – neither is great for everything… need to find balance!
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Communication Barriers
Development doesn’t
understand logging,
manageability, online ops…
Database
Administrator
Database
Increase communication *
Developer
Constructive collaboration *
Cross-training/lessons learned *
Database Administration
doesn’t understand our
development priorities or
timelines. We just need to have
something to show – ASAP.
* Reduce complexity
* Clear release management
* Define requirements early
Pockets of information within disciplines
Unclear delineation of responsibilities – sometimes there’s no
defined “database architect/developer” only server administration
and client application development… whose job is it?
Conflicting strategic goals and objectives
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
How Design Affects Production
Seemingly simple design choices can *hugely* affect
production performance and availability
I often hear – let’s just get this working and we’ll deal with
performance later (and then it can be very difficult to undo the
affects of bad design)
Many different areas on which to focus:
Database Structures – Capacity planning
Table design – consistent, effective, efficient choices
Indexing strategies – designed, not ad-hoc
Data access options – ad-hoc, sp_executesql, procedures
Naming conventions – (not critical, per se but saves time later in coding
and usage)
Programming standards – (again, not critical, per se but saves time
later in troubleshooting and analysis)
Manageability – Does this architecture work under load? Can it be
maintained?
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 1: Nested Transactions
Nested transactions do not exist in SQL Server but many
developers believe the syntax and think they do
Trying to use nested transactions leads to transaction log
problems:
Committing a nested transaction does nothing, so the transaction log
cannot clear properly
Rolling-back a nested transaction rolls-back everything
The rollback behavior can be achieved with save-points, but
not the commit behavior
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 2: Bad Naming Schemes
Column/index/table names should:
Be descriptive and understandable
Be consistent
Use fully-spelled-out word
Column/index/table names should NOT:
Use abbreviations
What does dscrPrttps mean?
Use spaces or invalid characters
Include data types
E.g. colVrChrMxLstKnwnPtntAddr
Single characters (maybe with a number appended)
What’s stored in the table t406ab1?
Same applies to table/view/DMV aliases
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 3: Parameterization (1 of 2)
Compilation (CPU) vs. bad plans (usually IO)
How do you access SQL Server?
Statements built on the client -> ad-hoc
Simple parameterization (default)
Safe
Very simple statements
When safe their plan gets reused
Unsafe (typically unsafe unless very straightforward)
Can waste cache (single-use plan cache “bloat”)
Can cost you in compilation (and therefore CPU)
Forced parameterization (database-level setting)
Works harder to parameterize
Reduces CPU
May lead to bad plans
Statements built on the client -> sp_executesql
Forced parameterization -> plan defined by “first” execution
Better plan reuse
Again, may lead to bad plans
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 3: Parameterization (2 of 2)
Stored procedures
Centralized logic
Easier to maintain
More secure (options to control/limit data manipulation)
Don’t need to grant DELETE permissions on a table for a procedure
to do the delete if the ownership chain is not broken
Can control/limit how many rows are processed (e.g. always
require a WHERE clause)
Know how they work (they’re not perfect but they are VERY powerful!)
Know your options (they’re also VERY flexible!)
Procedure-level recompilation (CREATE … WITH RECOMPILE)
Good for small procedures with highly volatile code
Statement-level recompilation
Much better for more complex procedures where only a small
amount of code needs to be recompiled
OPTION (RECOMPILE)
OPTION (OPTIMIZE (@variable FOR value))
OPTION (OPTIMIZE FOR UNKNOWN)
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 4: Physical DB Layout
Single data file is fine for a small database
What about when the database becomes 100s of GBs?
Backups take a lot longer
Restores take a lot longer
No piecemeal or partial restores are possible
Migrating to new storage becomes more problematic
Database maintenance becomes problematic
I/O performance becomes harder to maintain
As databases become larger, partitioning them into multiple
filegroups becomes really necessary
Beware of:
One filegroup per table
I/O subsystem contention
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 5: Primary Key Choice
If a clustered index is not defined already, a PRIMARY KEY
constraint will use a clustered index to enforce uniqueness…
This can be good:
If narrow (as few bytes as possible)
Unique (by definition)
Static (little to no changes in value)
Less prone to fragmentation (ever-increasing pattern)
This can be horribly bad
Wide, natural keys used as the clustering key
Prone to significant fragmentation
And, it amazing how many things this affects and how many
things must be done to fix it…
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 6: Indexing Strategy
“SQL Server does all of that for you.”
Effective indexing strategies are an absolute requirement for
success
SQL Server does NOT do any of this for you…
SQL Server doesn’t automatically add necessary indexes
SQL Server doesn’t automatically drop unnecessary indexes
SQL Server will let you create as many useless indexes as you’d like… 
Scalability != add more indexes
Common mistakes:
Randomly adding indexes (one for each column of the table)
Narrow indexes have fewer uses
Trying to cover everything without re-evaluating existing
indexes
Different developers creating the same indexes with different
names resulting in redundant indexes
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 7: Database Settings
Database settings cause problems if not set correctly:
Data and log file sizes and auto-growth
Statistics auto-creation and updating
Auto-shrink and auto-close
Recovery model
Instant file initialization
No database maintenance causes problems:
Index fragmentation
Out-of-date statistics
No proactive corruption detection, no backups
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 8: Misuse of FILLFACTOR
FILLFACTOR allows delaying time before a page split
occurs (which causes fragmentation)
Can be set at instance level or individually
Beware of:
Setting it to a non-default (100%) value for everything
Leaving it at 100% value for everything
How to set?
As high as possible for unchanging data
‘It depends’ for OLTP
Start at 70%, monitor, tweak
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 9: Unnecessary Updates
Example: real-estate client
Several million house listings, from around 70 listing
agencies
Updates happen almost daily from each agency
Terrible update logic:
Multiple passes over the data, specifying all columns
values even though only 1 or 2 change
Also updates unchanged data
Enormous amounts of churn in tables and indexes
All logged
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 10: LOB Storage Choice
So many choices of data-type:
Fixed-length vs. variable length
8000-byte limit vs. 2-GB limit
Legacy 2-GB limit vs. new 2-GB limit
In-database vs. out-of-database (FILESTREAM)
Choices of where to store the values:
In-row vs. out-of-row
Same table vs. another table
Optimal choice depends on size and usage pattern
But, choice affects:
Query performance, database maintenance, HA
technologies
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 11: Documentation
The following should be documented:
Design choices, plus justifications
Schema
Database settings
Interfaces and APIs
SPs and functions
Tests
Changes to any of the above, with a date and name
Especially important that changes are tracked so they
can be correlated with perf/functionality changes and
rolled-back
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Example 12: Feature Use
SQL Server features should not be used just because
they’re there in the product
Examples we see that have caused problems:
Data compression (on everything in the database)
INCLUDEd columns (trying to cover all queries)
Fulltext (not compatible with partitioning)
Triggers
Partitioning
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Key Point: Real Testing
We see so many problems from *really* inadequate testing
If you test with:
2 or 3 concurrent connections
Thousands of data records
No indexes
Developer Edition on a dual-core laptop
Then it’s unlikely to work well when you run with:
Hundreds of concurrent connections
Billions of data records
Lots of indexes
Enterprise Edition on a 32-core NUMA server
Synchronous database mirroring
Always try to test as close to production conditions as possible
Don’t forget overhead from HA technologies and increasing load/size
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Summary: Bridging the Gap
Key points for success
Avoiding known design anti-patterns
Prototype and test design strategies
Workload analysis, baselines, and future projections – capacity
planning
Troubleshooting to identify problems rather then throwing H/W
Build trust and partnership between teams – success is not onessided both look bad when an application/project fails
Work to cross-train and educate both teams for best practices
Consistent post-mortems and “lessons-learned” from problems and
mistakes rather than laying blame
Ensure that consistent and automated maintenance is handling
common problems
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
For more info
Check out our website (no advertising! It’s only free content and
links to more!)
●
www.SQLskills.com
Check out our links to whitepapers:
●
www.SQLskills.com/whitepapers.asp
Check out our blogs:
●
●
www.SQLskills.com/blogs/Kimberly
www.SQLskills.com/blogs/Paul
Check out our online webcasts (6 individual webcasts and 2 multipart series for a total of 27 webcasts – over 50 hours of FREE
online/downloadable content with more coming…):
●
www.SQLskills.com/webcasts.asp
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases
Thank You!
Questions?
SQLskills.com
Dublin UG: When Bad Things Happen to Good Databases