Import Multiple Files to SQL Server Using DTS

March 24, 2004

In the typical IT environment, it is often necessary to import flat files to SQL Server tables. Sometimes it is necessary to import many files at the same time from same or different folders. In my previous article, I mentioned how to import such files using BULK Insert and BCP utility. In this article, I am going to discuss how to import all the files from a particular folder to SQL Server using a batch file and DTS package.

Simulation

Let's simulate the whole scenario for importing multiple files. First, let's create a folder C:\MyImport and create 3 files, a.csv, b.csv and c.csv with the below content. Also, create a table in SQL Server to hold the imported data.

Files

C:\MyImport\a.csv

1, MAK, A9411792711, 3400.25
2, Claire, A9411452711, 24000.33
3, Sam, A5611792711, 1200.34
4, Wright, A5611792711, 1200.34
5, Richard, G561d792755, 1223.34
6, Valarie, B5611792788, 1240.32

C:\MyImport\b.csv

11, Rubon, 9671792711, 400.14
22, Mike, 9418952711, 4000.56
39, Hsu, 75611792511, 1230.00

C:\MyImport\c.csv

69, Lucy, 8411992710, 305.11
45, Grace, 3413452713, 246.52
33, Saint, 5461795716, 1278.70

Table

Create Database Bank
Go
Use Bank
go
Create table Account([ID] int, Name Varchar(100), 
AccountNo varchar(100), Balance money)
Go
Create table logtable (id int identity(1,1), 
	Status varchar(500), 
	Importeddate datetime default getdate())
Go
use master
go
sp_addlogin 'importuser','import','Bank'
go
use Bank
go
sp_adduser 'importuser'
go
sp_addrolemember 'db_datareader','importuser'
go
sp_addrolemember 'db_datawriter','importuser'
go

Create DTS Package

Step1: Create Global variables FileName, ServerName and DatabaseName in the DTS package as shown below.

Step 2: Create connections and transformation.

Create the Text connection and SQL Server connection and add the transformation displayed below.

Step 3: Complete the mapping as shown below.

Step 4: Add dynamic tasks and link the "SQLServer" connection to the global variable "ServerName", link "inputfile" to the global variable "FileName" and link "initial catalog" in "SqlServer" connection to "DatabaseName".

Step 5: Add Success workflow between Dynamic Properties and InputFile Connection .

Step 6: Add a SQL Task to write to the logtable as shown below

SQL

Insert into LogTable (Status) values (?)