How to refresh Test, Dev Systems with Prod Data?

When I attended PASS Summit last October, there were many people whom I met during breakfast and lunch and sometimes in the lobby and we discussed a variety of topics on  SQL Server. During one of those conversation, I was asked to give my suggestion on how to refresh their Development and Test environments with the Production data. So I thought it would be nice to write the same, so that it might help some of those who are looking for the same info. 

Backup & Restore

One of the most common mechanism of refreshing environments is using database backups on the production server and restore on the DEV, QA server to refresh the existing one. The advantage of using backup / restore is it is easy to do it. You don’t have to perform any special steps in this procedure other than restoring, coz your most recent full backup of production database should be good. However, the downside of this is that your DEV / QA database would be the same size of the production and your DEV / QA machine might not be anywhere near the size of your PROD box. The disk space too might be a factor to consider when using this procedure.

SQL Server Integration Services (SSIS)

If you are aware and comfortable using SSIS packages, you can utilize this platform to refresh and  make other customization after your DEV / QA database is refreshed. With SSIS, you can automate the database refresh and setup a SQL Agent job and when you have to refresh you can run the job manually (or on a pre-determined schedule).. Some of the benefits using SSIS is the manual tasks that might be needed after you refresh the data, such as scrubbing off critical data (such as SSNs or Credit Card data etc), adding additional Developer / Test users etc.. You might also not want all of the data from your production databases, only the most recent year’s worth of data is good to test, in such cases backup restore will not solve the purpose, the data that is not necessary has to be dropped after restoring the database backup…

Import Export Wizard

Import Export Wizard would be another handy tool that can be used for data refresh, using this tool does not need any other program (such BIDS or SSDT as in the case of SSIS). You can pick only those table(s) that needed in the DEV / QA environment.. The disadvantage of this tool is that you cannot customize beyond a certain level and scheduling is not an option unless you save the SSIS package at the end of this wizard.. Also, you cannot push data simultaneously to 2 different databases such as DEV and QA at the same time (where as in SSIS you can do so using Multi-Cast Data Transformation).. 

Other options

Finally there are other options such as Scripting the database objects and transferring the data using stored procedures at either the source or destination (which involves a linked server between the source and destination servers) and / or command line utility called Bcp. 

Overall, I would prefer SSIS as my choice to refresh QA / Dev databases..

Do you like this site? Like our FB page @ Facebook.com\LearnSQLWithBru so that, you know when there is a new blog post.

–Bru Medishetty

How to Install Sample Databases in SQL Server 2008 R2

In this blog, we shall learn “How to install sample databases in SQL Server 2008 R2”. Sample databases are really helpful,  when you want to practice what you have learnt in SQL Server be it from a text book or an online article, blog etc sample databases are readily available to test. Knowing that it is easy to start from scratch on these sample databases makes you play with and even try some things that you would not want to do on other databases. Database and Log file shrinking (which is not suggested on user databases) but in order to learn such topics, your best choice is a sample database. With that lets focus on our goal (Installing the sample databases, if you have already forgotten…)

The first step is downloading the sample database. Visit the Microsoft Database Product Samples website and choose the sample database for the version that you want. Since we are dealing with SQL Server 2008 R2 he is the link. Once you have downloaded the file (approx 80 MB). Go to the location where the downloaded file is residing. Right click on the file AdventureWorks2008R2_SR1.exe (if you have not renamed it) and select Open; you can also double click on the file.

The exe file starts extracting to a temporary location on your local drive and once the extraction is done, the SQL Server 2008R2 SR1 Database Installer is displayed. The below screen is the first screen in which you have to accept to the license terms. Choose the check box and click Next.

In the next screen, it will displays on which database Instance the Sample databases will be added. If you have multiple instances on the machine, you can choose the Instance name from the drop down list.

Note that for some databases there are warning (yellow triangle with exclamation) icon being displayed, those databases cannot be installed through this installed but they can be added manually using Business Intelligence Developer Studio. You can click more information to find details on how to perform that task.

By default all sample databases that can be added, will be selected, you can un-check those databases that you would not want to install. Once you are done, click Install

The installation of sample databases progress is displayed, you can click Show Details button to view the detailed information in text format..

Once the installation is done, the progress bar indicates that installation is complete.. Click Finish to close the Installer.

Do you like this site? Like our FB page @ Facebook.com\LearnSQLWithBru so that, you know when there is a new blog post.

— Bru Medishetty

List of all Data and Log Files of a SQL Instance

Ever faced a sitaution where you had to find out all the database files (Data and Log) of a SQL Instance?

I came across this situaion many times, especially when I am working with a new SQL Instance which was taken over by my team OR when the previous admin of that box has left the company and its now my turn to maintain / administer that instance.

In order to list the files, we just need to query the system table sysaltfiles.  The picture below is a screen show that I ran earlier today on one of my SQL Instances.

SELECT * FROM SYSALTFILES

This  query can be further tweaked to get only the data files or only the log files by adding where clause and filter by griupid column. Groupid = 1 indicates it is a data file and groupid = 0 indicates it is a log file.

Do you like this site? Like our FB page @ Facebook.com\LearnSQLWithBru so that, you know when there is a new blog post.

– Bru Medishetty