Detecting poor cursor usage in SQL Server

Without a shadow of a doubt, poor cursor usage issue is an essential aspect to review so as to reduce the possibility of having CPU bottleneck issues, and determine whether cursors are the most appropriate means to accomplish the processing or whether a set-based operation is possible. It is well know that set-based operation is generally more efficient, but if you make the decision of using cursors, you should ensure that it does not represent an issue for the database in question. Thus, detecting poor cursor usage and taking certain measures to ease the problem is crucial, and in SQL Server there are some techniques available to be used to diagnose the issue. To begin with, by using performance counters we are able to analyse the issue and find out the extent to which poor cursor usage is adversely affecting on the performance of specific workloads or the whole database. For instance, here is the counter SQL Server: Cursor Manager By Type: Cursor Requests/Sec which retrieves information about the number of SQL cursor requests received by the server. On the top of that, it is possible to filter out by the cursor manager instance such as API Cursor (only the API cursor information), TSQL Global Cursor and TSQL Local Cursor. Have a look at SQL Server, Cursor Manager by Type Object to get more info about the counter.
On the other hand, using SQL Trace is also helpful, for example, use a trace that includes the RPC:Completed event class search for 'sp_cursorfetch' statements. The value of the fourth parameter is the number of rows returned by the fetch. It is worth noting that the maximum number of rows that are requested to be returned is specified as an input parameter in the corresponding RPC:Starting event class. Finally, by using the DMV 'sys.dm_exec_cursors' we can also determine whether poor cursor usage exists in the database server as Transact-SQL cursors always have a fetch buffer of 1 and for API cursors it should be higher.

select c.* 
from sys.dm_exec_sessions s
    cross apply sys.dm_exec_cursors(con.session_id) as c
where cur.fetch_buffer_size = 1 
    and cur.properties LIKE 'API%'

Consequently, if it is seen that API Cursors have a fetch buffer size of 1 then consider enabling multiple active results (MARS) when connecting to SQL Server and consult the appropriate documentation for your specific API to determine how to specify a higher fetch buffer size for the cursor either ODBC (SQL_ATTR_ROW_ARRAY_SIZE) or OLE DB (IRowset::GetNextRows, IRowsetLocate::GetRowsAt). After that, we can retrieve more details about the session and connection of the users associated to the harmful cursors so as to decide what to do.

select s.session_id, cn.client_net_address, s.login_name, s.status,s.client_interface_name, s.program_name, 
       c.cursor_id, c.name, c.properties, c.plan_generation_num, c.creation_time, c.is_open, c.fetch_status, 
       c.fetch_buffer_size, c.worker_time, c.reads, c.writes, c.dormant_duration
from sys.dm_exec_connections cn
inner join  sys.dm_exec_sessions s on cn.session_id = s.session_id
cross apply sys.dm_exec_cursors(s.session_id) as c
where c.fetch_buffer_size = 1 
     and c.properties LIKE 'API%'

That is all for now, thanks for reading. Let me know any remarks you may have. Stay tuned.

Getting information about memory used by SQL Server

Naturally, every DBA is asked to report information about the memory used by SQL Server. Common questions are related to memory reserved and memory used currently, and sometimes we might not know how to complete this task. How many times did we take a look at the Windows Manager Task to find out that info? Many of us might have ended up quite frustrated time and time again because it did not help much. Over time Microsoft decided to provide Administrators with more useful tools and released Resource Manager tool whereby useful memory information per process is available, nevertheless, whether we want to get that info from SQL Server it might be an uphill battle as it would need that we write certain complex code at windows level.  For the time being, thinking about this situation I made the snap decision of sharing with you some helpful scripts that will alleviate the pain.

In the likely event that you might need to get the total buffer pool memory used by all databases at SQL instance level, this script is for it.

SELECT cast( cast( COUNT(*) /128.0/1024.0 as decimal(10,2)) as varchar(10)) + 'GB'  AS TotalUsageBufferPool
FROM sys.dm_os_buffer_descriptors

And if you wanted to know the buffer pool memory used by each database, you can use this:

SELECT CASE database_id 
        WHEN 32767 THEN 'ResourceDb'  ELSE db_name(database_id)         END AS DatabaseName,
        cast( COUNT(*) /128.0 as decimal(10,2)) AS [BufferPool(MB)]
FROM sys.dm_os_buffer_descriptors
GROUP BY DB_NAME(database_id) ,database_id
ORDER BY [BufferPool(MB)] DESC

Finally, and more importantly, answering the question about the total memory used by the whole SQL instance:

-- SQL2012/2014/2016/2017
select cast(cast(physical_memory_kb /1024.0/1024.0  as decimal(10,2))  as varchar(10)) + 'GB' TotalPhysicalRAM, 
cast(cast(visible_target_kb /1024.0/1024.0  as decimal(10,2)) as varchar(10)) + 'GB' MaxRAM, -- max memory configure at sql server level
cast(cast(committed_target_kb /1024.0/1024.0  as decimal(10,2)) as  varchar(10)) + 'GB' ReservedRAM,  --memory reserved
cast(cast(committed_kb /1024.0/1024.0  as decimal(10,2)) as  varchar(10)) + 'GB' UsedRAM --memory used currently
FROM sys.dm_os_sys_info

-- for SQL2005/2008/2008R2
SELECT cast(cast(physical_memory_in_bytes /1024.0/1024.0  as decimal(10,2))  as varchar(10)) + 'GB' TotalPhysicalRAM, 
cast(cast(bpool_visible /128.0/1024.0  as decimal(10,2)) as varchar(10)) + 'GB' MaxRAM, -- max memory configure at sql server level
cast(cast(bpool_commit_target /128.0/1024.0  as decimal(10,2)) as  varchar(10)) + 'GB' ReservedRAM,  --memory reserved
cast(cast(bpool_committed /128.0/1024.0  as decimal(10,2)) as  varchar(10)) + 'GB' UsedRAM --memory used currently
FROM sys.dm_os_sys_info

That is all for now. I hope you find these scripts helpful. Let me know any remarks you may have.

Updating so-called 'column statistics' for the whole SQL Server database

Definitely, statistics in SQL Server are vital to ensure stable database performance as a whole. Every DBA must know the importance of keeping statistics up to date, however, not everyone who is in charge of a database knows much about it, especially, those who do not have this role or are DBA by accident. So, I will just mention that statistics are objects that contain data distribution, density, selectivity, cardinality, etc. for all columns that were at least once queried or linked to indexes. This information helps the SQL optimizer make the right decision of what indexes and resources should use so as to execute and process the queries as fast as possible. Put differently, statistics allow SQL optimizer to create the best possible execution plan to run queries efficiently, otherwise, when statistics are not up to date bad execution plans may be created which will make inefficient use of resources bringing down the performance of the whole database and, even worse the whole server. So, it is a basic task to routinely update statistics at least once a month as part of your database maintenance tasks.
It is worth noting that statistics that linked to indexes (so-called 'index statistics') are full updated automatically when indexes are rebuilt. So, those statistics do not need explicit update. Nevertheless, statistics linked to columns (so-called 'column statistics') still need of an explicit update and this is what we need to do after rebuilding all the indexes. Thinking about it, today I am going to share a script to only run the update of columns statistics:

SET NOCOUNT ON     
DECLARE @schema_name varchar(max)      
DECLARE @table_name varchar(max)     
DECLARE @stat_name varchar(max)        
DECLARE @update_stat_cmd varchar(max)        
DECLARE @update_stat_msg_header   varchar(max)      
DECLARE update_stat_cursor CURSOR FOR    
          select  schema_name(o.[schema_id]),  object_name(s1.[object_id]) , s1.name   
          from (  
            select s.[object_id], s.name from sys.stats s  
            left join sys.indexes i on s.name=i.name  
            where i.name is null) s1  
          inner join sys.objects o on o.[object_id]=s1.[object_id]  
          where  o.type='U'  
          order by schema_name(o.[schema_id]),  object_name(s1.[object_id]) , s1.name   
  
       
OPEN update_stat_cursor        
FETCH NEXT FROM update_stat_cursor INTO  @schema_name, @table_name,  @stat_name         
WHILE (@@fetch_status = 0)        
  BEGIN    
       DECLARE @ini DATETIME, @fin DATETIME     
       SET @update_stat_msg_header =  '->Updating ['+ RTRIM(@schema_name)  +'].[' + RTRIM(@table_name) + ']([' +@stat_name + '])'      
       PRINT @update_stat_msg_header    
       SET @update_stat_cmd ='UPDATE STATISTICS ['+ RTRIM(@schema_name)  +'].[' + RTRIM(@table_name) + ']([' +@stat_name + '])'    
       SET @ini=GETDATE()    
       EXEC (@update_stat_cmd)      
       SET @fin=GETDATE()    
      
       FETCH NEXT FROM update_stat_cursor INTO @schema_name, @table_name,  @stat_name            
  END        
 
PRINT ' '       
PRINT '----------------------------------------------------------------------------- '        
SET @update_stat_msg_header = '*************  THERE ARE NO MORE STATISTICS TO BE UPDATED **************'         
PRINT @update_stat_msg_header        
PRINT ' '        
PRINT 'All statistics not linked to any index were rebuilt!'        
  
CLOSE update_stat_cursor        
DEALLOCATE update_stat_cursor    
SET NOCOUNT OFF

That is all for now. Thanks for reading. Let me know any remarks you may have. Stay tuned.

Dealing with physical database corruptions

Beyond all doubt, not every single database in the world has a DBA dedicated to monitoring it 24x7 hours. Moreover, many database environments are unwittingly implemented/installed where there was no consideration to meet the basic software and hardware minimum requirements. This situation is compounded by the fact that many companies do not put much attention on databases from the beginning, maybe because at first the database are quite small and serving not many transactions per second. Unsurprisingly, all databases are becoming bigger and bigger with the passing of time so it is not rocket science to foresee that everything will get worse in terms of performance and physical integrity. In this new context, it is much more critical to have everything in place so as to prevent databases from getting damaged. However, many times it is too late when one realises that the database got damaged because of poor implementation. The overwhelmingly majority of physical corruption issues are not sparked by SQL Server on its own, but poor hardware implementation. Thus, if you find yourself working with databases struggling this problem, I wholeheartedly recommend reinstalling the whole server from scratch taking minimum requirements on board.
After having a good implementation of a database server, it is of paramount importance to carry out maintenance tasks at least once a month, and it should include executing full DBCC CHECKDB, and if possible DBCC CHECKDB WITH PHYSICAL_ONLY option once a week. It is also understandable that DBCC CHECKDB makes intensive use of resources, especially CPU and Disk, nevertheless it is possible to use MAXDOP option with DBCC CHECKDB to limit CPU usage and ease the pain. So, nowadays there is no feasible excuse to avoid executing it regularly.

Having said that, I would like to share with you some steps to follow in order to try to fix a physical database corruption:

1. To begin with, try to back your database up.
2. Set your database to SINGLE_USER mode, and then execute DBCC CHECKDB WITH PHYSICAL_ONLY to determine whether physical corruption exists or not.

ALTER DATABASE MyEnterpriseDB SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
GO
DBCC CHECKDB(MyEnterpriseDB) WITH PHYSICAL_ONLY 

If there is physical corruption, SQL Server will display some errors like these:

Msg 8909, Level 16, State 1, Line 1
Table error: Object ID 0, index ID -1, partition ID 0, alloc unit ID 0 (type Unknown),
page ID (1:219356) contains an incorrect page ID in its page header.
The PageId in the page header = (0:0).
Msg 8928, Level 16, State 1, Line 1
Object ID 501576825, index ID 1, partition ID 72057594040549376,
alloc unit ID 72057594044940288 (type In-row data): Page (1:1408252) could not be processed.

3. Sometimes only non-clustered indexes are damaged, and luckily in this case, you just need to recreate those indexes. If you want to figure out which indexes are damaged so that you can recreate them, you can have a look at this article I wrote many years ago.
4. However, if the clustered index or heap structure is damaged then you may need set the database to EMERGENCY mode and execute DBCC CHECKDB with REPAIR_ALLOW_DATA_LOSS option which may not always be the best option for bringing a database to a physically consistent state but when the clustered index or heap structure of a table is heavily damaged there is no other option, just do it at the cost of some data loss. It is also worth noting that if you want to find out which objects are damaged so that only execute DBCC CHECKDB repair process on them then you can also check out the same article. Here is this example, we are going to execute on the whole database.

ALTER DATABASE MyEnterpriseDB SET EMERGENCY;
GO
DBCC CHECKDB (MyEnterpriseDB, REPAIR_ALLOW_DATA_LOSS) WITH PHYSICAL_ONLY 
GO
ALTER DATABASE MyEnterpriseDB SET ONLINE WITH ROLLBACK IMMEDIATE;
GO
ALTER DATABASE MyEnterpriseDB SET MULTI_USER WITH ROLLBACK IMMEDIATE;

5. Having executed DBCC CHECKDB with REPAIR_ALLOW_DATA_LOSS option does not always ensure that the database will be repaired, however, many times it is enough. So, after getting your database repaired, try to go to bottom of the problem, and fix the problem at hardware level because it is most likely that disk storage is not working properly. Bearing in mind that database corruption issue is just the tip of the iceberg so consider DBCC CHECKDB as a quick fix, not a final solution.
6. Finally, you should take a full database backup.

That is all for now, I hope you find this post helpful. Let me know any remarks you may have. Stay tuned.

Altering SQL Jobs without granting SysAdmin privilege

What would you do if you were asked to grant a few users the permission for altering SQL Jobs? It is a tricky task to carry out although many of us would think that granting SQLAgentOperatorRole role might be enough, however, many years ago no sooner had I done it than I realised it did not work as expected, and now I think most of us faced up this issue at first.
The SQLAgentOperatorRole role allows user to alter SQL jobs as long as the user is the owner of the SQL job, otherwise, the user need to be SysAdmin at SQL instance level. What’s more, one of the best security practices says that nobody but DBA must be SysAdmin, and we should use Windows Authentication. Nevertheless, when it comes to owners for SQL jobs they should use 'sa' as owner which does not mean having enable that account, it should be disable. In this sense, it is highly advisable to have the disabled account “sa” as the owner of all SQL jobs and avoid granting SysAdmin privilege. So, it is of paramount importance not to use a windows user as owner of a SQL job because SQL Server will always validate windows users against the Active Directory and it is likely to get unforeseen errors during that process. 
Today I am going to share with you a stored procedure to enable users to alter SQL Jobs without the need of granting SysAdmin privilege. This stored procedure consists of a logic that will allow a specific user to take the ownership of a SQL Job so that the user can be able to alter it and after making the changes the user can change the ownership to ‘sa’ (or the original owner). This is the stored procedure that I mentioned above.

USE [msdb]
GO
CREATE PROC[dbo].[usp_change_owner_job] @jobname varchar(max), @newowner varchar(max)
with execute as owner
as
begin
    declare @old_owner varchar(max)
    select @old_owner= s.name from msdb.dbo.sysjobs j inner join sys.server_principals s
    on j.owner_sid= s.sid where j.name=@jobname

    --declare @newowner varchar(max)
    --set @newowner=ORIGINAL_LOGIN()
    EXEC msdb.dbo.sp_update_job @job_name=@jobname, @owner_login_name=@newowner

    --print @newowner
    declare @msg varchar(max)
    set @msg= 'The owner of ' +@jobname + ' job was changed from ''' + @old_owner + ''' to ''' + @newowner + ''''
    print @msg    
end

So, for instance, if you have a SQL job whose owner is 'sa' and your user is 'User2', you will not be able to alter the job until you take the ownership of it temporarily. Here are all the steps you must follow:

1. Create the stored procedure 'usp_change_owner_job' based on the code above.
2. Grant EXECUTE permission on 'usp_change_owner_job' to 'User2'.
3. Grant SQLAgentOperatorRole role to 'User2'.
4. Change the ownership of the SQL Job to 'User2' by using the stored procedure 'usp_change_owner_job'.

EXEC msdb.dbo.[usp_change_owner_job] 'BusinessJob01', 'User2'

The owner of BusinessJob01 job was changed from 'sa' to 'User2'

5. Now the User2 has the ownership of the SQL job and is now able to alter it.
6. After making the changes on the SQL Job, the 'User2' must change the ownership to 'sa'.

The owner of BusinessJob01 job was changed from 'User2' to 'sa'

The owner of BusinessJob01 job was changed from 'User2' to 'sa'

Having successfully completed all the steps will you be able to alter any SQL Job without granting SysAdmin privilege to users. That is all for now. Let me know any remarks you may have.