Woops!! I filled up my tempdb!


sunset roller coaster

Being the best adhoc T-SQL developer this side of my keyboard, I can whip up a report for a manager in no time.

So, sit down and let me tell you a story…

The request came in one day, management needed information…
I had to call functions that call functions that loop through tables. Yah!! Kinda like a really cool roller coaster.

What was that noise? I think my database administrator just passed out. Poor guy.

By the way: I rewrote the query.. it’s fine now. I may run the little guy again next month.

But ANYWAY…my POINT is I filled up the tempDB… panic set in… I posted to twitter… I googled… I asked questions and felt like a total noob. How could this be? MY query is finished…shouldn’t the temp data have poofed when it died?

Nope.

I found this at Microsoft support

“When SQL Server starts, the tempdb is re-created by using a copy of the model database and is reset to its last configured size.”

But come on, you cant stop and start a production SQL Server, and really, I needed more coffee to deal with that. I wasn’t happy….

On a whim (get it? yeah I’m here all week folks) I decided that it had to be tied to my spid… I could disconnect my session or kill my spid and any temp objects I was working with in tempdb should go away. It worked!

I reconnected and yes… the tempdb was back down to a reasonable size.

Whew!
Lesson learned… if I blow up tempdb, disconnect all sessions and hide… er I mean reconnect and see if that fixed it before stopping/starting sql server.
I’m not saying this is the right way, but it’s what worked for me.

Now go have a good Monday. :)



Removing ‘USE databasename’ from generated scripts in SQL Server Management Studio 2008


Just recently we decided to start working on our own local sandboxes and then use
RedGate’s SQL Source Control tool to commit those changes to SVN and our main development server.
I right clicked one of my stored procedures, hit modify, and poof right there at the top was
“USE [Database_Sandbox_AGRAY]”
… rut ro raggy, this wasn’t going to work when the script was checked in.
I figured there had to be a way to prevent this from happening, and because I have the memory of a goldfish I’m putting it here:


Tools–> Options–> SQL Server Object Explorer –> Scripting

 

Set “Script USE <database>” to FALSE.


You’re welcome :)



An update from the cobwebs!


So I’ve been gone from blog land for a while, but dry your tears I’m going to remedy that.
Motivation always comes the week after the SQL PASS Summit to gather the troops and get some learning joo joo happening!

I got home from an amazing trip in Seattle on Saturday, check out Summit details here:

http://www.sqlpass.org/summit/na2010/

As far as the Knoxville PASS SQL Server Users group here are our past meetings, more to come in January!

Friday, November 05, 2010

Topic: SQL Server Consolidation and Virtualization – Considerations and Best Practices

Speaker: Frank Cicalese Consulting Systems Engineer UCS Application Architect – Microsoft Solutions Cisco Systems, Inc.

Friday, September 24th 2010

Topic:TSQL performance tuning and troubleshooting tips.

Speaker: Shelb Upchurch MCITP 2008 SQL DBA is the senior database administrator at Regal Entertainment Group. Shelb has been a database administrator since 2000 and a SQL report writer since 1997.


Skipped August meeting however there was SQL Saturday on August 21st in Nashville,TN


Wednesday, June 23rd 2010

Topic: SQL Server Internals and Architecture

Speaker: Kevin Kline is the Technical Strategy Manager for SQL Server Solutions at Quest Software.


Friday, May 21st 2010

Topic: SQL Server Reporting Services – Dressing Up Data with Report Builder

Speaker: Jesse Wimberley an Instructor at New Horizon


Wednesday, March 10th 2010

Topic: Self Service Business Intelligence

Speaker: Jesse Wimberley an Instructor at New Horizons



XML say wha?


I am working on a project where I need to export XML from my database, in the process of asking for help I decided I would document here (a similar version of what I am working on) what I was stuck on and ask for help in my comments since twitter #sqlhelp needed more info.

I am currently trying to use FOR XML EXPLICIT to pull this data but have realized that I might  be using the wrong tool. We have other feeds written in C#, I was just trying to get around the script task in SSIS.

Here you go:

xml_blog.txt



KnoxSQL an official SQL Server User Group


Watch out Knoxville! I have officially started a PASS Chapter in Knoxville,TN called KnoxSQL.

Stay tuned for meeting information.

As of right now my plan is to do a lunch time meeting at New Horizons off Cedar Bluff once a month or so, and maybe throw in a SQL Saturday if I see enough interest.

If you would like to come and hear the latest topic, or even volunteer to help out, let me know!



querying xml…


I need my query to return:

bar1
bar2
foo3
bar4

Here is what I have so far… I also tried to use a while statement to loop through… no luck(any ideas?):

DECLARE @xmlString XML;

SET
@xmlString = N'

<a promotion="PL10ONL">
<product-id>bar1</product-id>
<product-id>bar2</product-id>
<product-id>foo3</product-id>
<product-id>bar4</product-id>
</a>
'

SELECT ISNULL(T.lala.value('./product-id[1]','varchar(20)'),'found nothing')
FROM @xmlString.nodes('a') AS T(lala);

SELECT ISNULL(T.lala.value('./product-id[2]','varchar(20)'),'found nothing')
FROM @xmlString.nodes('a') AS T(lala);

/*====================================*/

edit post:

Got it!!! Joy!


DECLARE @xmlString XML;

SET @xmlString = N'
<a promotion="PL10ONL">
<product-id>bar1</product-id>
<product-id>bar2</product-id>
<product-id>foo3</product-id>
<product-id>bar4</product-id>
</a>'

SELECT s.lala.value('@promotion', 'varchar(20)'),T.lala.value('.', 'varchar(20)')
FROM @xmlString.nodes('a/product-id') AS T(lala)
OUTER
apply @xmlString.nodes('a') AS S(lala)



All good things…


The ETL database my friend and I wrote is going to launch into production Tuesday. It’s like our child’s first day of kindergarten.

There are at least 20 jobs that were not required for launch, those will need to be done in the upcoming months. It’s been almost a year since this project began. A giant endeavor which ultimately ends in a new shiny website with better analytics, promotional control, and user experience.

I’ve learned so much, granted there were, and still will be, moments where I will want to pull my hair out. But what an awesome thing to create from the bottom up?!



SSIS Configuration notes


The way I understood it, if you put a value in “set values” within your job step (calling an SSIS Package) it would overwrite what was in the SSIS Configuration table… apparently I was wrong. I tested this today. SSIS Configuration table is the winner.



How I became a Database Programmer


It all began in March of 2001. I survived 3 rounds of layoffs at Earthlink, an Internet service provider, but didn’t make the final round. A few weeks later I was hired by a small company (at the time) named Marketlinx.

Marketlinx writes Multiple Listing Service software for Realtors. I generated reports from listing data, and helped with cosmetic ASP/Javascript type work, which required me to query the database from time to time.

Over the next five years I became very familiar with the software, but felt that I wanted to focus more on the inner workings of the database. In April 2006 I was moved into a database programmer position. I was already familiar with the data and SQL Server 2000/2005. Anything I didn’t know I asked or researched on the web. I took many self paid for classes, I was totally amazed at what I could do.

After almost 3 years as a DBA, my database career took on a new direction, I was hired to join a team that would write a brand new data warehouse with SQL Server 2008, using SSIS, for a large Ecommerce company. This is something a DBA gets to do what, once or twice in a career?

Now, our warehouse is almost complete. I’ve learned so much and evolved with the software. I’ve made new friends in the database industry, and try to keep up with the not so local Nashville SQL Server User group. I have hopes of starting a Knoxville group later this year. I also registered for the annual Microsoft SQL PASS convention in Seattle.

I love my job and I’m excited about what I do, I don’t claim to be an expert, but every day I learn something new.

So there you go, that’s how I got to be a DBA, and I love it. :)



random thoughts


You know you’re a nerd when you lay around at night wondering…

Right now our SSIS configurations are stored in a control table in our primary database. However when installing an SSIS package if you do not choose an xml config you do not get prompted to change the connection string for the database (or any value you selected to be configurable). The config on the QA/Stage box just automagically has the correct connection string. But, I’m wondering if this is a feature or a flaw. If I’m not the one installing the package this might be a bad thing… hmmmm.

Also, right now we have a slowly changing dimension checking every attribute of a product for changes, the table is is checking has at least a million records… after going to the Nashville SQL Server User group I was told that this isn’t performant, but one product has many many attributes, so using multiple lookups is not an option. I’m thinking we are going to have to somehow do a large join on my staging table, or find another solution. But really right now the process doesn’t take THAT long, so in reality, we will probably leave it alone.



I upgraded my blog…


to a newer version of WordPress, and It seems to have lost some functionality. Getting it fix asap.



so I have this little job…


it’s small and very funny*

Okay it’s not so small but it’s called by a web service which updates a table that fires a trigger that starts the job(not my design)which updates an activity table and also starts my SSIS packages spinning. But I also want the job to fire, should the web service fail to call it.

All in all it seems really messy. I need to make it cleaner.

Also, SQL server 2008 offers a beautiful job status API however helpdesk has no access to my production server to use it to see what happened to a failed job at 3am. So I need it in a format that someone who has very little access to the database to see.

I once wrote an ASP web application that would take all job status and show a green, yellow, or red light on the page depending on the health of the job. I may have to do something like that again. Unless there’s something already out there that would save me the work.

So those are my two big questions for now. Input is welcome. :)



Pivot Data Flow Transformation in SSIS


I’ve been working strictly with SSIS in SQL Server 2008 for a a little over month now. It’s an amazing tool.

One thing I encounter when generating horizontal tables from adata warehouse is the factthat I have to pivot tables. Last week, when trying to pivot data using the “Pivot Task” I got the error:

“Cannot convert the pivot key value_ to the data type of the pivot key column.” (The underscorereplacesthe name of the output column I had created. )

Thing is, I was trying to be fancy, In my OLE DB Source I had decoded mycolumn names from numbers to text.

This bit me in the end. I couldn’t figure out why I was getting an error. But here was the solution.

Pass the numbers in and name them under your PivotKeyValue.

Correct:

SELECT[1AnchorColumn], [3PivotValue], [2PivotKey]
FROM [AdventureWorksDW].[dbo].[Table]

output column 1 “Blankets” PivotKeyValue=125 Name=Blankets SourceColumn=LineageID from Input

output column2 “HomeOffice” PivotKeyValue=126 Name=HomeOffice SourceColumn=LineageID fromInput

output column3 “Kitchen” PivotKeyValue=300 Name=Kitchen SourceColumn=LineageID fromInput

Incorrect:


SELECT
[1AnchorColumn], [3PivotValue],[2PivotKey] =
CASE [2PivotKey]
WHEN 125THEN 'Blankets'
WHEN 126 THEN 'HomeOffice'
WHEN 300 THEN 'Kitchen'
ELSE 'Other'END
FROM
[AdventureWorksDW].[dbo].[Table]

output column 1 “Blankets” PivotKeyValue=Blankets Name=Blankets SourceColumn=LineageID from Input

output column2 “HomeOffice” PivotKeyValue=HomeOffice Name=HomeOffice SourceColumn=LineageID fromInput

output column3 “Kitchen” PivotKeyValue=Kitchen Name=Kitchen SourceColumn=LineageID fromInput

The logic was that characters were coming from a field that was an integer. Thus the error:

“Cannot convert the pivot key value _ to the data type of the pivot key column”

Hope this helps you in your pivoting woes. I figured it myself! Brian Knight would be proud. ;)

Toodles!

(I really will try to post more often!)



SSIS and SFTP Task/XML Destination


So far Ive found some serious lack of tools I need in SSIS. Don’t get me wrong, I LOVE SSIS. So we’ve had to either write in house custom components or purchase them from third parties.

The two big ones are SFTP Task and XML Destination.

Here is what I could find form Microsoft about the XML Destination:
http://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=303815
I need to output some seriously complex XML… writing this component in house was the only option.

The other Component was a SFTP Task, now FTP is available, but I needed SFTP, my coworkers did a lot of research and came up with a nice plug-in from cozyroc.com to handle this.



That whole sql migration process…


I wanted to blog about whatever I had learned in the past week but really what came to mind was not answers, but more questions. Questions are good, they have a side effect of helping me learn. So here you go…

Migration:

What is the ideal way to push changes from your development server, tostage, and then production SQL servers with Microsoft products?

The way I am familiar with is using a third party tool, Apex Diff, to script data/schema differences add scripts to VSS, then having a custom script for any data cleanups… and push.

But Microsoft has built this allinto Team system I think, I’ve never used it personally.

Also, versioning… say I make changes to a table, “alter table add column blah blah”, what is the best way for VSS to pick up this change? I’ve always checked out the table creation script, manually added my alter script to the bottom with comments about the workorder where the new column was requested by the client, then checked in the script,hit”f5″ to push just that portion to my databaseandcalled it a day.

But is there a way for VSS to automagically pick up these changes, and who made them… it wouldsave me so much timeto just hit “f5″ and be done.

The same goes for SSIS. How do I know what changes were made to SSIS packages before I got to them? I suppose I could compare the XML files… but I need more, I need to know why the change was made and who did it…

I guess what I’m saying is, if I only had Microsoft tools to work with, what are the best practices for pushing these changes to my databases.

All thesequestions are things I’ll most likely be researching and blogging about in the next few weeks.

This seems like a vital part of having a database, code change andmigration process per Microsoft should be clear as a bell right?