Collecting Index Usage Statistics

calendar Jan 26, 2012 · Indexing SQL Server  ·
Share on: linkedin copy

Knowing when and how often your indexes are used can really come in handy. Indexes that are never utilized aren't worth keeping around, and knowing their usage patterns can be a big help when making decisions regarding things such as filegroup placement and compression settings. SQL Server 2005 brought some great advancements in the form of the sys.dm_db_index_usage_stats DMV, which returns statistics on index usage that are automatically being kept by the server. While it's a step forward, I feel it still leaves a few things to be desired:

  • The values are cumulative since the server was last restarted
  • Once the server is shut down or restarted, there's no way to get the previous values back

Collectable With this in mind, I came up with my own system for archiving non-cumulative index usage stats on a daily basis. Microsoft introduced Management Data Warehouse in SQL Server 2008 which provides a way to keep track of these statistics over time (along with many others), but I've been using my own method longer than that. This will also work in SQL Server Express Edition, which doesn't support MDW. I'll give descriptions of each part below, but if you just want the code right now download it here.

Tables

IndexUsageStats_LastCumulative
As I mentioned before, the output from sys.dm_db_index_usage_stats is cumulative since the last time the server was restarted, and I was looking for a way to find daily values that weren't cumulative. Restarting the server immediately after collecting the values would have accomplished this, but users probably wouldn't be very happy :). Instead I just keep track of the previous cumulative values and subtract them from the current ones. If the output shows an index was used 7 times on Tuesday at Midnight, and the same number was 12 on Wednesday at Midnight, then it must have been used 5 times in that 24 hour period. IndexUsageStats_LastCumulative holds the previous cumulative values for all indexes to make this calculation possible.

Names
The only true way to identify an object in SQL Server is by its name, but DMVs typically return object IDs instead of names. It's easy to translate between names and IDs, but over time names and IDs can change:
– If an object is dropped and re-created with the same name, chances of it having the same ID are virtually zero.
– If an object is renamed, its ID will not change, but of course its name will.
Using either IDs or Names exclusively can make historical values become useless over time – you might have an ID that's no longer valid or a name that changed a while back. To give myself more options for tracking changes over time, I store both. Since storing each object's name in a character data type would consume a lot of space and be repetitive, I prefer to map object names to numeric values, and this table facilitates that.

IndexUsageStats
This table does the heavy lifting storing for our system – it contains all the historical index usage statistics by both object ID and name. I've found it to be a great candidate for compression if you're running enterprise edition, but it's still pretty compact even when uncompressed thanks to the Names table.

Stored Procedures

CollectIndexUsageStats
The stored procedure that drives statistics collection. You'll want to set up a job to run this – mine runs at midnight. Basically it iterates through each online database in the instance collecting usage stats and then compares them to the previous cumulative values and writes the differences to the IndexUsageStats table. You'll find comments in-line.

Views

vw_IndexUsageStats
Since the data contained in the IndexUsageStats table is all numeric, it's not the easiest to browse. This view makes some joins back to the Names table so things are a little more user-friendly. Most of the queries I write are against this view. Object IDs aren't returned here as they're typically not needed. When they are necessary, I'll write more detailed queries against IndexUsageStats.

A few notes before the code:

  • As always, this code is offered as-is with no warranty whatsoever. Do not deploy this in a production environment before you have tested it and understand exactly what it does.
  • I recommend putting all these objects in their own database. If you already have a DBA database for management information that should work fine too. My database is called "iDBA" because I inherited it with that name.
  • This was written and tested on SQL Server 2008 and 2008R2. It can probably be tweaked to run in 2005 by removing the datetime2 datatype and writing substitute queries for the MERGE statement.