Random thoughts

I came across many time with this question “How do I process data with some wrong records?”. In usual case if any error would be happened your ETL process will be stopped and if you are implementing transaction on that case whether it will be rolled back or partially committed in case if you have implemented save point but what if we want to process data and if any error record would be found skip and report that record and continue processing of remaining records. Let’s take an example we have to process a CSV file that contains 4 records where record number 3 has some problem. Length of one field is higher than the size defined on source & target metadata. It’s clear truncation error and by default it will interrupt the process. See the prototype of the problem and solution below: Here we are using one flat file that has some error record (Meta Mismatch) . So this mete mismatch could be possible one source component it self or it might be it is only at target side. So the Id...

It’s very interesting to see small small changes on development interface of SQL Server Data Tool in SQL server 2012. Now BIDS is gone it’s data tool that introduced in SQL Server 2012 version for the development of BI projects. Here is the first impression after using this interface. Very first that you can notice re-organization of component in tool box with new icons that gives you more glossy look. I’m using SSIS interface where you can notice four changes. 1. Project Level Parameter: It’s a new implementation to avoid using SQL level configuration that could be shared across multiple projects. Similar functionality you can achieve by suing this project level parameter. 2. Re-Organized toolbox: I like this very much, they organized component based on their usability and nature like putting all container tasks in one bucket. 3. Zoom Slider: Zoom slider is bigger and better. you can easily notice and easily use it.   Pic: Control Flow User Interface In case of data flow yo...