SSIS on Cathrine Wilhelmsen

Biml for OData Source and Connection Manager in SSIS

Tue, 24 Jul 2018 11:39:48 +0200

As of July 2018, there is no built-in Biml support for OData. To work with OData in Biml, you have to create a custom source and connection manager. This requires more Biml code than built-in functions like OleDbSource and may look a little overwhelming at first. But don’t worry! You don’t have to start from scratch.

In this blog post, we will first look at the properties of the OData Connection Manager and how to script it in Biml. Then, we will do the same for the OData Source. Finally, we will tie it all together and create a complete SSIS package that you can use a starting point for your own projects.

Azure Data Factory v2 with Mike Flasko

Thu, 05 Oct 2017 04:38:38 +0200

One of the sessions I was most looking forward to at Microsoft Ignite 2017 was New capabilities for data integration in the cloud with Mike Flasko. In that session, he talks about Azure Data Factory (ADF) v2 and its new first-class SSIS support.

After the session, I convinced Mike Flasko and Sanjay Krishnamurthi to have a chat with me 🤓 We talked about what’s new in Azure Data Factory v2, including the updated pipeline application model with a new visual design canvas, new Software Development Kits (SDKs) for working with Azure Data Factory, the new Integration Runtime, and the ability to run SSIS packages inside Azure Data Factory v2.

Azure Data Factory v2 with Mike Flasko

Why can't I create SSIS Project Parameters from Biml?

Fri, 27 May 2016 13:01:45 +0200

BIDS Helper and BimlExpress do not support creating SSIS project parameters from Biml out of the box. There are workarounds (and I have previously blogged about my solution for creating project parameters from Biml), but why is this not a standard feature in BIDS Helper or BimlExpress? Many people have asked about this, so I sat down with Biml creator Scott Currie (@ScottCurrie) to get the full story.

Why doesn’t BIDS Helper or BimlExpress emit SSIS project parameters from Biml?

Well, technically it could, but it shouldn’t. The user experience would have serious issues, leading to confusion, frequent errors, and the potential for data loss. How can that be?

Create SSIS Project Parameters from Biml

Wed, 25 Nov 2015 15:16:24 +0100

If you are using BIDS Helper or BimlExpress to generate SSIS packages in the Project Deployment model, you have probably noticed that it is not possible to create project parameters from Biml. You can write Biml for the project and project parameters, but BIDS Helper / BimlExpress will only generate the SSIS packages for you and not the SSIS project parameters. The recommended solution is that you create the project parameters manually before you generate your SSIS packages from Biml.

(Want to know why? Read Scott Currie’s explanation in my blog post Why can’t I create SSIS Project Parameters from Biml?)

However, if you are a lazy developer like me, you probably don’t want to create and update project parameters manually. Perhaps you want to automatically create or update project parameters based on some metadata? You can do that!

Let’s take a look at a (semi-hardcoded, semi-hack) solution for creating SSIS project parameters from Biml in BIDS Helper / BimlExpress 🤓

Batch update properties in SSIS packages (T-SQL Tuesday #68)

Tue, 14 Jul 2015 13:00:20 +0200

T-SQL Tuesday #68 is hosted by Andy Yun (@SQLBek). Many SQL Server defaults are not ideal, and most of us have a list of defaults we always change. Andy wants us to Just Say No to Defaults and blog about what, why or how we change defaults.

If you are an SSIS developer like me, there is a big chance that the ProtectionLevel in SSIS Packages is on top of your list of defaults to change. The default ProtectionLevel is EncryptSensitiveWithUserKey (ugh), but most of the time it is not the best option. Raise your hand if you have ever asked your favorite search engine for advice on issues like “SSIS package fails in SQL Server Agent job” or if you have ever heard someone exclaim “but it works on my machine!?” 😅

There are many great blog posts about the different ProtectionLevels, why you probably want to change to DontSaveSensitive as your default, and how to use configurations and parameters instead of encrypted SSIS packages. I will not go into details about any of that in this post, but I will use ProtectionLevel as an example default property you want to change in many SSIS packages at the same time.

How do you batch update properties in existing SSIS packages? You probably don’t want to open up every single package and change them manually?

Package Configurations and Connection Managers in Biml

Tue, 17 Jun 2014 19:40:56 +0200

At work we wanted to start using Biml to speed up development in our existing projects without making too many changes at once. I decided to start by writing Biml files that generates SSIS packages exactly like the ones we already have so we can implement changes faster, and step one was to figure out how to create Package Configurations and Connection Managers in Biml:

Create an XML configuration file Package Configuration
Create the Connection Manager specified in the XML configuration file
Create SQL Server Package Configurations that use the Connection Manager specified in the XML configuration file
Create Connection Managers specified in the [SSIS Configurations] table in SQL Server
Create a Parent package variable

I recommend reading Understanding Integration Services Package Configurations by Rafael Salas (@RafSalas) and Getting Started with BimlScript by Paul S. Waters (@pswaters) if you’re not already familiar with the basics of SSIS Package Configurations and Biml.

SQL Server, SSIS and Biml Data Types

Tue, 27 May 2014 18:33:04 +0200

The table below is a simplified mapping between SQL Server, SSIS and Biml Data Types. It does not include all possible mappings or all data types, but is meant as a quick reference while developing and learning Biml. I mainly created it as a cheat sheet for myself, but I hope other Biml developers will find it useful!

Can't remember this post's URL? Neither can I 😅 You can use this short URL instead: cathrinew.net/BimlDataTypes

Cheat Sheet for SQL Server, SSIS and Biml Data Types

SQL Server	SSIS	Biml
bigint	DT_I8	Int64
binary	DT_BYTES	Binary
bit	DT_BOOL	Boolean
char	DT_STR	AnsiStringFixedLength
date	DT_DBDATE	Date
datetime	DT_DBTIMESTAMP	DateTime
datetime2	DT_DBTIMESTAMP2	DateTime2
datetimeoffset	DT_DBTIMESTAMPOFFSET	DateTimeOffset
decimal	DT_NUMERIC	Decimal
float	DT_R8	Double
geography	DT_IMAGE	Object
geometry	DT_IMAGE	Object
hierarchyid	DT_BYTES	Object
image (*)	DT_IMAGE	Binary
int	DT_I4	Int32
money	DT_CY	Currency
nchar	DT_WSTR	StringFixedLength
ntext (*)	DT_NTEXT	String
numeric	DT_NUMERIC	Decimal
nvarchar	DT_WSTR	String
nvarchar(max)	DT_NTEXT	String
real	DT_R4	Single
rowversion	DT_BYTES	Binary
smalldatetime	DT_DBTIMESTAMP	DateTime
smallint	DT_I2	Int16
smallmoney	DT_CY	Currency
sql_variant	DT_WSTR	Object
text (*)	DT_TEXT	AnsiString
time	DT_DBTIME2	Time
timestamp (*)	DT_BYTES	Binary
tinyint	DT_UI1	Byte
uniqueidentifier	DT_GUID	Guid
varbinary	DT_BYTES	Binary
varbinary(max)	DT_IMAGE	Binary
varchar	DT_STR	AnsiString
varchar(max)	DT_TEXT	AnsiString
xml	DT_NTEXT	Xml

(* These data types will be removed in a future version of SQL Server. Avoid using these data types in new projects, and try to change them in current projects.)