Integration Services in a SQL Server 2K8 Clustered Environment

Today I was setting up a new SQL Server 2008 Server in a cluster and I discovered that the Integration Services service is not cluster aware.  I also discovered that Microsoft advises AGAINST installing the service as a clustered resource.  Why on earth would Microsoft neglect to make the Integration Services service cluster aware?  That doesn’t seem very good as it leaves a single point of failure, right?  Well after careful consideration and researching the following two articles from Microsoft I decided that it wasn’t going to be as big of a deal as I thought. 

Both of these Microsoft articles give you the information you need to decide for yourself

Definition of the Integration Services Service and some alternatives to use when used in clustered Environments  -  The recommended ways.

To configure Integration Services as a cluster resource  -  The not so recommended way

Installing SQL Server 2008 on Windows 2008 R2

Today I was performing a new SQL Server 2008 install on a Windows 2008 R2 server.  I got 3 errors while performing the install and thought I would save everyone the trouble of searching the internet for all the workarounds so I included them all here.  The first error came when I tried installing the .Net framework via the SQL Setup.  It was not letting me install via the SQL install and I had to install the .Net framework in the features of the server management console.  You can do that by doing the following steps from this article

Setting up a SQL Server 2008 Cluster on Windows Server 2008

This week I was installing a new failover cluster at a client site.  Boy have there been a lot of changes regarding clustering between Windows Server 2003 and Windows Server 2008.  Microsoft has redefined clustering with Windows Server 2008, supposedly making it simpler and easier to implement, but I found it still quite an effort since a lot of the steps have changed.  I found this great article on the web that helped me to set everything up correctly so I thought I would share it here:

installing SQL Server on a Windows Server 2008 cluster

What features are supported by the various Editions of SQL Server 2008?

Have you ever wondered what edition of SQL Server you really needed for your specific project needs but you could never really remember where you can find the proper documentation to help you decide.  I was looking on the web today and it took me a while to find it myself so I thought I would share it with you all now.   Take a look at the following link and you can see for yourself. 

Features Supported by the Editions of SQL Server 2008

If you are interested in the costs of the various editions check out the following:

SQL Server 2008 Pricing

Auto-Magically visualize your Perfmon and Profiler results

Troubleshooting SQL Server issues usually involves setting up a perfmon and profiler trace and then trying to identify the issues based on particular times and high reads or writes etc.  These have been industry standard ways to diagnose issues, but did you know you can integrate the two to get the best of bost worlds in a pretty little interface?  In this post I would like to go over how you can integrate the two together and dynamically see what is causing the various spikes to occur.

 

Setting up Perfmon

Open up PerfMon by going to Control Panel -> Administrative tools -> Performance.  Next go to the Performance Logs and Alerts section. Right click Counter Logs and select New Log Settings. Type a name for the log. 

 

 Add all the counters you want to monitor with the Add Counters button.  I typically like to include the following:

MemoryAvailable MBytes
 MemoryCache Bytes
 MemoryPages/sec
 MemoryPool Paged Resident Bytes
 MemorySystem Cache Resident Bytes
 MemorySystem Code Resident Bytes
 MemorySystem Driver Resident Bytes
 PhysicalDisk(_Total) Disk Write Time
 PhysicalDisk(_Total)Avg. Disk Bytes/Read
 PhysicalDisk(_Total)Avg. Disk Bytes/Write
 PhysicalDisk(_Total)Avg. Disk Queue Length
 PhysicalDisk(_Total)Avg. Disk Read Queue Length
 PhysicalDisk(_Total)Avg. Disk Write Queue Length
 PhysicalDisk(_Total)Disk Read Bytes/sec
 PhysicalDisk(_Total)Disk Write Bytes/sec
 Processor(_Total)Access MethodsCount Lob Readahead
 SQLServer:Access MethodsFull Scans/sec
 SQLServer:Access MethodsPage Splits/sec
 SQLServer:Access MethodsWorktables Created/sec
 SQLServer:Buffer ManagerBuffer cache hit ratio
 SQLServer:Cursor Manager by Type(_Total)Cursor memory usage
 SQLServer:Cursor Manager by Type(_Total)Cursor worktable usage
 SQLServer:Databases(_Total)Active Transactions
 SQLServer:Databases(_Total)Log Flushes/sec
 SQLServer:Databases(_Total)Percent Log Used
 SQLServer:Databases(_Total)Transactions/sec
 SQLServer:General StatisticsActive Temp Tables
 SQLServer:General StatisticsProcesses blocked
 SQLServer:General StatisticsUser Connections
 SQLServer:LatchesLatch Waits/sec
 SQLServer:LatchesTotal Latch Wait Time (ms)
 SQLServer:Locks(_Total)Lock Timeouts/sec
 SQLServer:Locks(_Total)Lock Wait Time (ms)
 SQLServer:Locks(_Total)Lock Waits/sec
 SQLServer:Locks(_Total)Number of Deadlocks/sec
 SQLServer:Memory ManagerConnection Memory (KB)
 SQLServer:Memory ManagerLock Memory (KB)
 SQLServer:Memory ManagerSQL Cache Memory (KB)
 SQLServer:Memory ManagerTarget Server Memory (KB)
 SQLServer:Memory ManagerTotal Server Memory (KB)
 SQLServer:SQL StatisticsSQL Compilations/sec
 SQLServer:SQL StatisticsSQL Re-Compilations/sec

 

Go to the Log Files tab and you can choose the log file type I typically use the Binary File but you can use any means you would like.  Hit Configure button and specify the filename and location

 

On the third tab you can schedule the start time etc.

When you click OK your log starts gathering data immediately. The status is shown with the green icon in the list next to log name in the list.

Set up our SQL Profiler Trace

Next we start our profiler trace.  You can do that by clicking  Start–> Programs–>Microsoft SQL Server–> Performance Tools–>SQL Profiler.

When it opens up go to File -> New Trace. Choose the server you wish to monitor. A window to set the Trace properties will open. 

In the General tab check the save to file box and select the file and location.

For the events you can choose all the ones you are interested in.  There are so many things you can profile for but I typically use just 4 and I include all their columns (You must make sure to include both start and end times or else you will not be able to correlate):

 

Once you have all your data collected you need to stop both the profiler trace and the perfmon trace. 

 Now its time to merge the two together.

To do so make sure you have closed all your profiler windows and then reopen your profiler trace.  (If you do not close Profiler and reload the trace some options will be greyed out.)

Next click File–>Import Performance Data–> and then select your Perfmon trace file.

 

 

Once you do so the two will automagically be linked together and you can select the counters you want to include.

 

 

Conclusion

When you integrate the PerfMon stats with the SQL Profiler Trace data together you can easilly determine the exact SQL Statements that were running when you experiences certain bottlenecks.  Try it out yourself and see how helpful it is. 

Reduce your code by 75% and an alternative to cursors

Ok this is more just a little trick with Dynamic sql, but it saves you tons of code and you avoid having to use cursors or single row iterations.  You can use it anywhere you would a cursor.  More set based operations…Horray!!!!

For this example we will do backups 

declare @sql nvarchar(max) -- used for dynamic sql
declare @fileDate varchar(20) -- used for file name

select @fileDate = convert(varchar(20),getdate(),112) -- used for file name

set @sql = 'declare @fileName varchar(256)' + char(13)-- we need a declare statement for filename

select @sql = @sql + char(13)+
'set @fileName = ''c:/backup/'+ name + '_' + @fileDate + '.BAK''' + char(13)+
'backup database '+ [name] + ' to disk = @fileName '+ char(13)
from sys.sysdatabases
where name not in ('master','model','msdb','tempdb') 

exec sp_executesql @sql

Here is a snippit from http://www.mssqltips.com/tip.asp?tip=1070 that illustrates a simple cursor alternative

DECLARE @name VARCHAR(50) -- database name
DECLARE @path VARCHAR(256) -- path for backup files
DECLARE @fileName VARCHAR(256) -- filename for backup
DECLARE @fileDate VARCHAR(20) -- used for file name

SET @path = 'C:Backup' 

SELECT @fileDate = CONVERT(VARCHAR(20),GETDATE(),112)

DECLARE db_cursor CURSOR FOR
SELECT name
FROM master.dbo.sysdatabases
WHERE name NOT IN ('master','model','msdb','tempdb') 

OPEN db_cursor
FETCH NEXT FROM db_cursor INTO @name  

WHILE @@FETCH_STATUS = 0
BEGIN
       SET @fileName = @path + @name + '_' + @fileDate + '.BAK'
       BACKUP DATABASE @name TO DISK = @fileName 

       FETCH NEXT FROM db_cursor INTO @name
END  

CLOSE db_cursor
DEALLOCATE db_cursor

When you compare the two and look at the profiler results it is an extreme performance increase as well

dbcc dropcleanbuffers
dbcc freeproccache
go

declare @sql nvarchar(max) -- used for dynamic sql
declare @fileDate varchar(20) -- used for file name

select @fileDate = convert(varchar(20),getdate(),112) -- used for file name

set @sql = 'declare @fileName varchar(256)' + char(13)-- we need a declare statement for filename

select @sql = @sql + char(13)+
'set @fileName = ''c:/backup/'+ name + '_' + @fileDate + '.BAK''' + char(13)+
'backup database '+ [name] + ' to disk = @fileName '+ char(13)
from sys.sysdatabases
where name not in ('master','model','msdb','tempdb') 

print( @sql )
go

dbcc dropcleanbuffers
dbcc freeproccache
go

DECLARE @name VARCHAR(50) -- database name
DECLARE @path VARCHAR(256) -- path for backup files
DECLARE @fileName VARCHAR(256) -- filename for backup
DECLARE @fileDate VARCHAR(20) -- used for file name

SET @path = 'C:Backup' 

SELECT @fileDate = CONVERT(VARCHAR(20),GETDATE(),112)

DECLARE db_cursor CURSOR FOR
SELECT name
FROM master.dbo.sysdatabases
WHERE name NOT IN ('master','model','msdb','tempdb') 

OPEN db_cursor
FETCH NEXT FROM db_cursor INTO @name  

WHILE @@FETCH_STATUS = 0
BEGIN
       SET @fileName = @path + @name + '_' + @fileDate + '.BAK'
      Print ('BACKUP DATABASE @name TO DISK = ' + @fileName) 

       FETCH NEXT FROM db_cursor INTO @name
END  

CLOSE db_cursor
DEALLOCATE db_cursor

When you run this and profile the results you can see the duration, reads and cpu use are dramatically less.

profiler