Database Journal
MS SQL Oracle DB2 Access MySQL PostgreSQL Sybase PHP SQL Etc SQL Scripts & Samples Tips Database Forum Rss Feed

» Database Journal Home
» Database Articles
» Database Tutorials
MS Access
SQL Scripts & Samples
» Database Forum
» Slideshows
Free Newsletters:

News Via RSS Feed

Rss Feed

Database Journal |DBA Support |SQLCourse |SQLCourse2

Featured Database Articles


Posted Feb 27, 2004

SQL Server 2000 DTS Part 10 - DTS Designer Tasks - The Data Driven Query Task

By Marcin Policht

In the arsenal of DTS Designer Tasks, the one that leads in terms of versatility (and complexity) is the Data Driven Query Task. Its most distinctive feature, distancing it from other tasks, is its ability to not only handle inserts of rows during transformation, but also process any other T-SQL statements, such as UPDATEs, DELETEs, or stored procedures. This flexibility comes, however, with a performance price tag, so you should avoid it unless its added functionality makes its use justified or necessary. Insert operations are much more efficiently handled with previously discussed Transform Data and Bulk Insert tasks.

Before we look into implementation details and examples, let's make sure you understand the basic idea on which this extra functionality is based. You will shortly recognize that the "Data Driven Query" name aptly reflects the mechanism employed for data transformation in this case.

Transformation of each row of data between its source and target is divided into two main stages. Initial row processing is done via a custom ActiveX script. Logic implemented by the script examines each row from data source (in addition to performing whatever other tasks might be required) and depending on conditions you specify (for example, in the simplest case, match on a specific value from a particular field), it calls one of TSQL statements you created. TSQL statements are typically parameterized, with parameter values derived from a table called, in Data Driven Query parlance, a binding table. Typically, the same table functions also as the data target, although this is not mandatory (since the target table is the one specified within the TSQL statement and can be any table available via a DTS connection used by the Data Driven Query task). Note that actual change to the target table happens only as the result of execution of the selected TSQL statement (which might change from one data source row to another), with values of parameters derived from columns in the binding table (which, in turn, are derived from ActiveX script transformation of rows in the source table).

Now, that the basic premise of the task has been presented, let's discuss each of its components in detail (you can find their graphical representation on the Data Driven Query Task Properties dialog box) and provide a simple example illustrating their functionality:

  • Source - points to the data source for the ActiveX script based transformation (fields from its records appear as DTSSource entries), which, in addition to serving as data modification origin, is also used to determine the type of TSQL statement to be executed. This might be simply a value indicating the type of action (update, insert, deletion) or a value from which such information can be derived (e.g. date field, which can be compared against another reference date to remove outdated records). In addition, each source row includes fields used as parameters in TSQL statements, since source columns typically map to destination columns (within ActiveX script transformation).

  • Bindings - points to a data table containing fields used in an ActiveX script based transformation as DTSDestination entries and provides values for parameters in TSQL statements. As we mentioned before, actual changes to a target table do not take place as the result of code executed as part of the transformation ActiveX script, but rather as the result of execution of TSQL statements. This means that in the case where a binding table is different from a target table (target table is specified within the TSQL statement), its data is not modified at all. Most commonly, though, binding table also functions as the target table.

  • Transformations - defines transformation between the source and the binding table. By default, this is an ActiveX script executing as Row Transform function (Data Driven Query has multi-phase capabilities similar to the Multi-Phase Data Pump task discussed in our most recent articles). Its interface (including the Transformation Options dialog box) is practically identical to the one described previously (when covering Transformations feature of the Multi-Phase Data Pump task), although the Binding Columns tab is used instead of the Target Columns tab. However, differences in the ActiveX script structure are more significant. In particular, besides already familiar mapping between DTSSource and DSTDestination columns, ActiveX script provides the ability to launch one of four different TSQL statements for each row of data being processed. The statement to be launched depends on the value returned from the Row Transform function, represented by one of the following four constants:

    • DTSTransformStat_InsertQuery - indicating that an insert statement will be executed,
    • DTSTransformStat_UpdateQuery - indicating that an update statement will be executed,
    • DTSTransformStat_DeleteQuery - indicating that a delete statement will be executed,
    • DTSTransformStat_UserQuery - intended for any arbitrarily chosen statement type (including updates, inserts and deletes).

    Note that despite the fact that names of these constants seem to indicate the type of statement to be executed, they function purely as labels identifying which one of up to four TSQL statements you created should be invoked. The respective TSQL statements can actually perform any type of action, not necessarily the one matching the label.

  • Queries - contains four placeholders for parameterized TSQL statements. The placeholders are labeled Insert, Update, Delete, and Select and correspond to the four different return values from the ActiveX script (listed above). By default, parameters are populated in the order they appear in the query, using the binding table columns in sequence. You can list the parameter-to-column mapping by clicking on the Parse/Show Parameters command button (at that point, the mapping will appear in the lower portion of the dialog box). To change the default arrangement, simply click on the entry in the Destination column in the lower portion of the dialog box and select the binding table column you want to map to a specific parameter.

  • Lookups - as already described earlier in this series, from here you can retrieve a value or set of values via a DTS connection and use them within the ActiveX script transformation. Even though you can use existing source or destination connections, you should keep in mind that a separate lookup connection offers the best performance. Each lookup entry consists of its unique name, the connection name, cache setting (number indicating how many lookup results are cached), and SQL Query that defines the lookup operation. The query is defined with DTS Query Designer and can include one or more parameters (which, as usual, are designated by question marks and substituted with values derived from DTS global variables, when transformation is executed). For more information on the lookups and referencing them in ActiveX script, refer to our recent article.

  • Options - contains subset of settings (allowing specifying exception file and its format, as well as data movement parameters, such as first and last row, maximum error count and fetch buffer size) which we described in our previous article, when discussing Options tab on the Properties dialog box of Transform Data task.

MS SQL Archives

Latest Forum Threads
MS SQL Forum
Topic By Replies Updated
SQL 2005: SSIS: Error using SQL Server credentials poverty 3 August 17th, 07:43 AM
Need help changing table contents nkawtg 1 August 17th, 03:02 AM
SQL Server Memory confifuration bhosalenarayan 2 August 14th, 05:33 AM
SQL Server Primary Key and a Unique Key katty.jonh 2 July 25th, 10:36 AM