SqlShare.com Logo
 
Skip Navigation Links
Login / Register
Subscribe RSS Feed 

Confio Ignite

Image of Ira W Whiteside
Author:

Author Bio:
Ira Whiteside and Pragmatic Works will partner with you or your company to make your DTS migration project a success. They have a number of approaches based on how much experience and time you have to complete your project. They will help you formulate a low-risk plan to make your project a success by blending our unique experience with your team.
Data Profiling in SSIS

Data Profiling in SSIS

In this presentation, Ira shows you how to data profile in SSIS using a Script transform and some creative methods like RegEx. He shows you how to look for patterns in your data to find bad data that wouldn't be obvious to the eye or casual viewer.

Video Information
Publish Date:
May 01, 2007
Length:
9:23
Skill Level:
400
Rating:
3.92 out of 5

Watch Video Now!  Watch it later!  

Bookmark and Share

  Rate This Video:  
 
1=Poor, 3=Good, 5=Excellent
Comments:
shank on 10/24/2008
good

mpadigela on 1/2/2009
content is good but voice quality is very poor.

Ira W Whiteside on 1/3/2009
Mahipay, Sorry to hear that, I do not seem to have that problem. If you have any questions you can contact me iwhiteside@iwhiteside.com.

Ira W Whiteside on 3/14/2009
I would like to share the new Melissa Data SSIS Total Data Quality Toolkit TDQ-IT, including SSIS Data Profiling 2005/2008. Melissa Data SSIS Total Data Quality Toolkit TDQ-IT offers a wide range of data transformation and cleansing functionality including data profiling, parsing, cleansing, matching and monitoring functionality built right in to SSIS. And, TDQ-IT leverages SSIS to provide a flexible, effective solution for your organization’s data quality and master data management (MDM) initiatives. Request a free trial today. http://www.melissadata.com/dqt/total-data-quality-integration.htm

shaun on 7/22/2009
can you please post the code for download?

Marsaline on 8/25/2009
Good!! I did not have any prob with the audio

Alok on 9/11/2009
Great video.. but I could not download the "Code"..Can you please make it available

Dean Gross on 9/15/2009
very informative

Joe Qian on 10/13/2009
Can't download source code

E2A1214606 on 11/23/2009
How do I download the sourcecode? The code download link takes me to an invalid page.

Stephen on 12/24/2009
You mentioned several sites for help with script code, and stated you would post them.

nick on 2/6/2010
where is the code man?

absher786 on 2/9/2010
Please upload the code soon.

Jay on 3/6/2010
Good job!

Magnas Hilaire on 3/16/2010
Where can I get the source code for this video?

Leonard on 3/24/2010
no code available

Arun on 4/3/2010
good

a.v.settle on 6/23/2010
I found the CODE!!! I know its late, but I just started working with data profiling via 2005: Microsoft SQL Server Integration Services user script component ' This is your new script component in Microsoft Visual Basic .NET ' ScriptMain is the entrypoint class for script components Imports System Imports System.Data Imports System.Math Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper Imports Microsoft.SqlServer.Dts.Runtime.Wrapper Public Class ScriptMain Inherits UserComponent Dim patAlphaUpper As String = "[^a-z0-9 ,@%&/#'.-]" Dim patAlphaLower As String = "[^A-Z0-9 ,@%&/#'.-]" Dim patNum As String = "[^A-Za-z ,@%&/#'.-]" Dim patSpecial As String = "[^A-Za-z0-9]" Dim patEmail As String = "\b[A-Za-z0-9._%-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,4}\b" Dim patUSPhone As String = "^(?:(?<1>[(])?(?<AreaCode>[1-9]\d{2})(?(1)[)])(?(1)(?<2>[ ])|(?:(?<3>[-])|(?<4>[ ])))?)?(?<Prefix>[1-9]\d{2})(?(AreaCode)(?:(?(1)(?(2)[- ]|[-]?))|(?(3)[-])|(?(4)[- ]))|[- ]?)(?<Suffix>\d{4})$" Dim rgxAlphaUpper As New Text.RegularExpressions.Regex(patAlphaUpper) Dim rgxAlphalower As New Text.RegularExpressions.Regex(patAlphaLower) Dim rgxSpecial As New Text.RegularExpressions.Regex(patSpecial) Dim rgxNum As New Text.RegularExpressions.Regex(patNum) Dim rgxEmail As New Text.RegularExpressions.Regex(patEmail) Dim rgxPhone As New Text.RegularExpressions.Regex(patUSPhone) Dim tagAlphaUpper As String Dim tagAlphaLower As String Dim tagNumber As String Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer) ' ' Add your code here ' Row.EmailLength = Len(Row.EmailAddress) tagAlphaUpper = rgxAlphaUpper.Replace(Row.EmailAddress, "A") tagAlphaLower = rgxAlphalower.Replace(tagAlphaUpper, "a") Row.EmailPattern = rgxNum.Replace(tagAlphaLower, "9") Row.EmailFlag = rgxEmail.Match(Row.EmailAddress).Success Row.PhoneLength = Len(Row.Phone) Row.PhoneFlag = rgxPhone.Match(Row.Phone).Success tagAlphaUpper = rgxAlphaUpper.Replace(Row.Phone, "A") tagAlphaLower = rgxAlphalower.Replace(tagAlphaUpper, "a") Row.PhonePattern = rgxNum.Replace(tagAlphaLower, "9") Row.Count = 1 End Sub End Class

a.v.settle on 6/23/2010
Sorry, the copy and paste didn't work like I thought it would. here's the link of the script code from the video: http://www.sqlservercentral.com/articles/Extract%2c+Transform%2c+Load+(ETL)/63792/ If you have a SQLServerCentral.com account, in the search bar copy and paste: Kimball ETL Part 1 – Data Profiling via SSIS Data Flow the first article will be the entire article and video of this method. Hope this helps!



Must Be Logged In

References
Brian's Newest SSIS Book

4 Day SSIS class taught by Brian.

Download the code example from this webcast.

Sponsored Ads
 

How Do I Become a Video Author? |  Newsletter History

Copyright © Fourdeuce, Inc., 2005-2009. All Rights Reserved | Privacy Policy | Terms & Conditions