How to Append two rows from two source in SSIS?2019 Community Moderator ElectionSSIS : Creating a flat file with different row formatsHow can I remove duplicate rows?How to return only the Date from a SQL Server DateTime datatypeHow to concatenate text from multiple rows into a single text string in SQL server?How do I UPDATE from a SELECT in SQL Server?SSIS - Source Error Output (No rows will be sent to error output(s)…)SSIS Flat File SourceSSIS Flat File Source Row Restructureredirect the records to different folder if the format is incorrect using SSISRedirect rows with errors from Flat File DestinationSSIS : Creating a flat file with different row formats
As a monk, can you make a melee attack roll using your Strength modifier, but roll damage with your Dexterity modifier?
Need some help with my first LaTeX drawing…
Why must traveling waves have the same amplitude to form a standing wave?
Ban on all campaign finance?
Time dilation for a moving electronic clock
Is it ok to include an epilogue dedicated to colleagues who passed away in the end of the manuscript?
How does Dispel Magic work against Stoneskin?
Does Linux have system calls to access all the features of the file systems it supports?
Why don't MCU characters ever seem to have language issues?
How could a female member of a species produce eggs unto death?
Is King K. Rool's down throw to up-special a true combo?
Do I need to leave some extra space available on the disk which my database log files reside, for log backup operations to successfully occur?
Excess Zinc in garden soil
Best approach to update all entries in a list that is paginated?
My adviser wants to be the first author
Want to switch to tankless, but can I use my existing wiring?
What is the difference between "shut" and "close"?
What Happens when Passenger Refuses to Fly Boeing 737 Max?
Making a sword in the stone, in a medieval world without magic
Sword in the Stone story where the sword was held in place by electromagnets
Why doesn't the EU now just force the UK to choose between referendum and no-deal?
Silly Sally's Movie
Playing ONE triplet (not three)
Can the druid cantrip Thorn Whip really defeat a water weird this easily?
How to Append two rows from two source in SSIS?
2019 Community Moderator ElectionSSIS : Creating a flat file with different row formatsHow can I remove duplicate rows?How to return only the Date from a SQL Server DateTime datatypeHow to concatenate text from multiple rows into a single text string in SQL server?How do I UPDATE from a SELECT in SQL Server?SSIS - Source Error Output (No rows will be sent to error output(s)…)SSIS Flat File SourceSSIS Flat File Source Row Restructureredirect the records to different folder if the format is incorrect using SSISRedirect rows with errors from Flat File DestinationSSIS : Creating a flat file with different row formats
I have to table in MySQL Server.
Header Table.
╔════════════╦════════╦═════════════╦═════════════════╦══════════╗
║ RecordType ║ CustID ║ DataGenDate ║ DataCreatedDate ║ SourceID ║
╠════════════╬════════╬═════════════╬═════════════════╬══════════╣
║ H ║ #1234 ║ 2018-01-05 ║ 2018-01-01 ║ V301 ║
╚════════════╩════════╩═════════════╩═════════════════╩══════════╝Transaction Table
╔════════════╦══════════╦══════════════╦══════════════╦════════════╗
║ RecordType ║ ProdCode ║ OpeningValue ║ ClosingValue ║ TranDate ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL001 ║ 95 ║ 90 ║ 2018-01-01 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL002 ║ 54 ║ 40 ║ 2018-01-01 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL003 ║ 63 ║ 43 ║ 2018-01-02 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL004 ║ 56 ║ 23 ║ 2018-01-01 ║
╚════════════╩══════════╩══════════════╩══════════════╩════════════╝
Header Table has Header Information and Transaction table have Transaction Data.
I want a text file (vertical pipe separated "|") to be generated through SSIS in below format.
H|#1234|2018-01-05|2018-01-01|V301
----------------------------------------
T|AL001|95 |90 |2018-01-01
T|AL002|54 |40 |2018-01-01
T|AL003|63 |43 |2018-01-02
T|AL004|56 |23 |2018-01-01
I tried it with ole DB source and flat file destination to export the file but wasn't successful. only I am getting either transaction or Header Rows.
sql-server ssis sql-server-2012 ssis-2012
add a comment |
I have to table in MySQL Server.
Header Table.
╔════════════╦════════╦═════════════╦═════════════════╦══════════╗
║ RecordType ║ CustID ║ DataGenDate ║ DataCreatedDate ║ SourceID ║
╠════════════╬════════╬═════════════╬═════════════════╬══════════╣
║ H ║ #1234 ║ 2018-01-05 ║ 2018-01-01 ║ V301 ║
╚════════════╩════════╩═════════════╩═════════════════╩══════════╝Transaction Table
╔════════════╦══════════╦══════════════╦══════════════╦════════════╗
║ RecordType ║ ProdCode ║ OpeningValue ║ ClosingValue ║ TranDate ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL001 ║ 95 ║ 90 ║ 2018-01-01 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL002 ║ 54 ║ 40 ║ 2018-01-01 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL003 ║ 63 ║ 43 ║ 2018-01-02 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL004 ║ 56 ║ 23 ║ 2018-01-01 ║
╚════════════╩══════════╩══════════════╩══════════════╩════════════╝
Header Table has Header Information and Transaction table have Transaction Data.
I want a text file (vertical pipe separated "|") to be generated through SSIS in below format.
H|#1234|2018-01-05|2018-01-01|V301
----------------------------------------
T|AL001|95 |90 |2018-01-01
T|AL002|54 |40 |2018-01-01
T|AL003|63 |43 |2018-01-02
T|AL004|56 |23 |2018-01-01
I tried it with ole DB source and flat file destination to export the file but wasn't successful. only I am getting either transaction or Header Rows.
sql-server ssis sql-server-2012 ssis-2012
Not sure if you want this from MySQL, or SQL Server, (i suspect the latter), however, what you are asking for here is a dynamic Pivot. SSIS, however, cannot handle a dynamic pivot for exported data. SSIS requires static definitions of data. If you want that type of data in an export I would suggest using SSRS and a matrix.
– Larnu
Mar 7 at 11:09
hi @Larnu I want this from SQL Server
– Shahab Haidar
Mar 7 at 11:21
That doesn't change my above comments in regards to dynamic data though.
– Larnu
Mar 7 at 11:33
You can use a C# script task to take a dataset and generate a .csv or Excel file dynamically. For Excel though you need to have the correct .dlls installed on the SSIS server.
– Brad
Mar 7 at 12:51
Possible duplicate of SSIS : Creating a flat file with different row formats
– Chris Albert
Mar 7 at 17:29
add a comment |
I have to table in MySQL Server.
Header Table.
╔════════════╦════════╦═════════════╦═════════════════╦══════════╗
║ RecordType ║ CustID ║ DataGenDate ║ DataCreatedDate ║ SourceID ║
╠════════════╬════════╬═════════════╬═════════════════╬══════════╣
║ H ║ #1234 ║ 2018-01-05 ║ 2018-01-01 ║ V301 ║
╚════════════╩════════╩═════════════╩═════════════════╩══════════╝Transaction Table
╔════════════╦══════════╦══════════════╦══════════════╦════════════╗
║ RecordType ║ ProdCode ║ OpeningValue ║ ClosingValue ║ TranDate ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL001 ║ 95 ║ 90 ║ 2018-01-01 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL002 ║ 54 ║ 40 ║ 2018-01-01 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL003 ║ 63 ║ 43 ║ 2018-01-02 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL004 ║ 56 ║ 23 ║ 2018-01-01 ║
╚════════════╩══════════╩══════════════╩══════════════╩════════════╝
Header Table has Header Information and Transaction table have Transaction Data.
I want a text file (vertical pipe separated "|") to be generated through SSIS in below format.
H|#1234|2018-01-05|2018-01-01|V301
----------------------------------------
T|AL001|95 |90 |2018-01-01
T|AL002|54 |40 |2018-01-01
T|AL003|63 |43 |2018-01-02
T|AL004|56 |23 |2018-01-01
I tried it with ole DB source and flat file destination to export the file but wasn't successful. only I am getting either transaction or Header Rows.
sql-server ssis sql-server-2012 ssis-2012
I have to table in MySQL Server.
Header Table.
╔════════════╦════════╦═════════════╦═════════════════╦══════════╗
║ RecordType ║ CustID ║ DataGenDate ║ DataCreatedDate ║ SourceID ║
╠════════════╬════════╬═════════════╬═════════════════╬══════════╣
║ H ║ #1234 ║ 2018-01-05 ║ 2018-01-01 ║ V301 ║
╚════════════╩════════╩═════════════╩═════════════════╩══════════╝Transaction Table
╔════════════╦══════════╦══════════════╦══════════════╦════════════╗
║ RecordType ║ ProdCode ║ OpeningValue ║ ClosingValue ║ TranDate ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL001 ║ 95 ║ 90 ║ 2018-01-01 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL002 ║ 54 ║ 40 ║ 2018-01-01 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL003 ║ 63 ║ 43 ║ 2018-01-02 ║
╠════════════╬══════════╬══════════════╬══════════════╬════════════╣
║ T ║ AL004 ║ 56 ║ 23 ║ 2018-01-01 ║
╚════════════╩══════════╩══════════════╩══════════════╩════════════╝
Header Table has Header Information and Transaction table have Transaction Data.
I want a text file (vertical pipe separated "|") to be generated through SSIS in below format.
H|#1234|2018-01-05|2018-01-01|V301
----------------------------------------
T|AL001|95 |90 |2018-01-01
T|AL002|54 |40 |2018-01-01
T|AL003|63 |43 |2018-01-02
T|AL004|56 |23 |2018-01-01
I tried it with ole DB source and flat file destination to export the file but wasn't successful. only I am getting either transaction or Header Rows.
sql-server ssis sql-server-2012 ssis-2012
sql-server ssis sql-server-2012 ssis-2012
edited Mar 7 at 11:07
Larnu
20.9k51733
20.9k51733
asked Mar 7 at 11:06
Shahab HaidarShahab Haidar
1509
1509
Not sure if you want this from MySQL, or SQL Server, (i suspect the latter), however, what you are asking for here is a dynamic Pivot. SSIS, however, cannot handle a dynamic pivot for exported data. SSIS requires static definitions of data. If you want that type of data in an export I would suggest using SSRS and a matrix.
– Larnu
Mar 7 at 11:09
hi @Larnu I want this from SQL Server
– Shahab Haidar
Mar 7 at 11:21
That doesn't change my above comments in regards to dynamic data though.
– Larnu
Mar 7 at 11:33
You can use a C# script task to take a dataset and generate a .csv or Excel file dynamically. For Excel though you need to have the correct .dlls installed on the SSIS server.
– Brad
Mar 7 at 12:51
Possible duplicate of SSIS : Creating a flat file with different row formats
– Chris Albert
Mar 7 at 17:29
add a comment |
Not sure if you want this from MySQL, or SQL Server, (i suspect the latter), however, what you are asking for here is a dynamic Pivot. SSIS, however, cannot handle a dynamic pivot for exported data. SSIS requires static definitions of data. If you want that type of data in an export I would suggest using SSRS and a matrix.
– Larnu
Mar 7 at 11:09
hi @Larnu I want this from SQL Server
– Shahab Haidar
Mar 7 at 11:21
That doesn't change my above comments in regards to dynamic data though.
– Larnu
Mar 7 at 11:33
You can use a C# script task to take a dataset and generate a .csv or Excel file dynamically. For Excel though you need to have the correct .dlls installed on the SSIS server.
– Brad
Mar 7 at 12:51
Possible duplicate of SSIS : Creating a flat file with different row formats
– Chris Albert
Mar 7 at 17:29
Not sure if you want this from MySQL, or SQL Server, (i suspect the latter), however, what you are asking for here is a dynamic Pivot. SSIS, however, cannot handle a dynamic pivot for exported data. SSIS requires static definitions of data. If you want that type of data in an export I would suggest using SSRS and a matrix.
– Larnu
Mar 7 at 11:09
Not sure if you want this from MySQL, or SQL Server, (i suspect the latter), however, what you are asking for here is a dynamic Pivot. SSIS, however, cannot handle a dynamic pivot for exported data. SSIS requires static definitions of data. If you want that type of data in an export I would suggest using SSRS and a matrix.
– Larnu
Mar 7 at 11:09
hi @Larnu I want this from SQL Server
– Shahab Haidar
Mar 7 at 11:21
hi @Larnu I want this from SQL Server
– Shahab Haidar
Mar 7 at 11:21
That doesn't change my above comments in regards to dynamic data though.
– Larnu
Mar 7 at 11:33
That doesn't change my above comments in regards to dynamic data though.
– Larnu
Mar 7 at 11:33
You can use a C# script task to take a dataset and generate a .csv or Excel file dynamically. For Excel though you need to have the correct .dlls installed on the SSIS server.
– Brad
Mar 7 at 12:51
You can use a C# script task to take a dataset and generate a .csv or Excel file dynamically. For Excel though you need to have the correct .dlls installed on the SSIS server.
– Brad
Mar 7 at 12:51
Possible duplicate of SSIS : Creating a flat file with different row formats
– Chris Albert
Mar 7 at 17:29
Possible duplicate of SSIS : Creating a flat file with different row formats
– Chris Albert
Mar 7 at 17:29
add a comment |
1 Answer
1
active
oldest
votes
This can be done using a Script Task as follows, with C# used in this case. This will create a CSV file with the pipe (|
) delimiter. Running a sample test with this I was able to import the output CSV file via an SSIS Flat File Connection Manager without any modifications to the file. This example assumes there's only a single row in the header table, otherwise you'll need to modify the SQL for this table to return the proper row.
using System.Data;
using System.Data.SqlClient;
using System.IO;
using System.Text;
//Windows Authentication (Integrated Security)
string connectionString = @"Data Source=ServerName;Initial Catalog=DatabaseName;Integrated Security=true";
string headerCmd = @"SELECT RecordType, CustID, DataGenDate, DataCreatedDate, SourceID FROM HeaderTable";
string rowCmd = @"SELECT RecordType, CustID, DataGenDate, DataCreatedDate, SourceID FROM TransactionTable";
string outputFile = Dts.Variables["User::FilePathVariable"].Value.ToString();
StringBuilder csvData = new StringBuilder();
int headerInt = 0;
DataTable headerDT = new DataTable();
DataTable rowDT = new DataTable();
using (SqlConnection conn = new SqlConnection(connectionString))
SqlCommand headerSQL = new SqlCommand(headerCmd, conn);
SqlCommand rowSQL = new SqlCommand(rowCmd, conn);
SqlDataAdapter da = new SqlDataAdapter();
conn.Open();
//get header row
da.SelectCommand = headerSQL;
da.Fill(headerDT);
//get data from Transaction table
da.SelectCommand = rowSQL;
da.Fill(rowDT);
//build header
foreach (DataRow hDR in headerDT.Rows)
foreach (DataColumn hDC in headerDT.Columns)
csvData.Append(hDR[headerInt].ToString() + "
//remove last pipe then start new line
csvData.Remove(csvData.Length - 1, 1);
csvData.Append(Environment.NewLine);
//add rows
foreach (DataRow rDR in rowDT.Rows)
for (int i = 0; i < headerInt; i++)
csvData.Append(rDR[i] + "
csvData.Remove(csvData.Length - 1, 1);
csvData.Append(Environment.NewLine);
//write to CSV
File.WriteAllText(outputFile, csvData.ToString());
Why not building the header with a script task and importing data using a data flow task?
– Hadi
Mar 7 at 18:19
1
@Hadi if the number of columns is static that would make sense. I wasn't sure if they were so I posted the answer to accommodate a varying number of columns. However if there's always 5 columns as indicated in the question using a script task to handle the columns and DFT for the import would simplify this.
– userfl89
Mar 7 at 18:24
1
I totally agree with that
– Hadi
Mar 7 at 18:35
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55042332%2fhow-to-append-two-rows-from-two-source-in-ssis%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
This can be done using a Script Task as follows, with C# used in this case. This will create a CSV file with the pipe (|
) delimiter. Running a sample test with this I was able to import the output CSV file via an SSIS Flat File Connection Manager without any modifications to the file. This example assumes there's only a single row in the header table, otherwise you'll need to modify the SQL for this table to return the proper row.
using System.Data;
using System.Data.SqlClient;
using System.IO;
using System.Text;
//Windows Authentication (Integrated Security)
string connectionString = @"Data Source=ServerName;Initial Catalog=DatabaseName;Integrated Security=true";
string headerCmd = @"SELECT RecordType, CustID, DataGenDate, DataCreatedDate, SourceID FROM HeaderTable";
string rowCmd = @"SELECT RecordType, CustID, DataGenDate, DataCreatedDate, SourceID FROM TransactionTable";
string outputFile = Dts.Variables["User::FilePathVariable"].Value.ToString();
StringBuilder csvData = new StringBuilder();
int headerInt = 0;
DataTable headerDT = new DataTable();
DataTable rowDT = new DataTable();
using (SqlConnection conn = new SqlConnection(connectionString))
SqlCommand headerSQL = new SqlCommand(headerCmd, conn);
SqlCommand rowSQL = new SqlCommand(rowCmd, conn);
SqlDataAdapter da = new SqlDataAdapter();
conn.Open();
//get header row
da.SelectCommand = headerSQL;
da.Fill(headerDT);
//get data from Transaction table
da.SelectCommand = rowSQL;
da.Fill(rowDT);
//build header
foreach (DataRow hDR in headerDT.Rows)
foreach (DataColumn hDC in headerDT.Columns)
csvData.Append(hDR[headerInt].ToString() + "
//remove last pipe then start new line
csvData.Remove(csvData.Length - 1, 1);
csvData.Append(Environment.NewLine);
//add rows
foreach (DataRow rDR in rowDT.Rows)
for (int i = 0; i < headerInt; i++)
csvData.Append(rDR[i] + "
csvData.Remove(csvData.Length - 1, 1);
csvData.Append(Environment.NewLine);
//write to CSV
File.WriteAllText(outputFile, csvData.ToString());
Why not building the header with a script task and importing data using a data flow task?
– Hadi
Mar 7 at 18:19
1
@Hadi if the number of columns is static that would make sense. I wasn't sure if they were so I posted the answer to accommodate a varying number of columns. However if there's always 5 columns as indicated in the question using a script task to handle the columns and DFT for the import would simplify this.
– userfl89
Mar 7 at 18:24
1
I totally agree with that
– Hadi
Mar 7 at 18:35
add a comment |
This can be done using a Script Task as follows, with C# used in this case. This will create a CSV file with the pipe (|
) delimiter. Running a sample test with this I was able to import the output CSV file via an SSIS Flat File Connection Manager without any modifications to the file. This example assumes there's only a single row in the header table, otherwise you'll need to modify the SQL for this table to return the proper row.
using System.Data;
using System.Data.SqlClient;
using System.IO;
using System.Text;
//Windows Authentication (Integrated Security)
string connectionString = @"Data Source=ServerName;Initial Catalog=DatabaseName;Integrated Security=true";
string headerCmd = @"SELECT RecordType, CustID, DataGenDate, DataCreatedDate, SourceID FROM HeaderTable";
string rowCmd = @"SELECT RecordType, CustID, DataGenDate, DataCreatedDate, SourceID FROM TransactionTable";
string outputFile = Dts.Variables["User::FilePathVariable"].Value.ToString();
StringBuilder csvData = new StringBuilder();
int headerInt = 0;
DataTable headerDT = new DataTable();
DataTable rowDT = new DataTable();
using (SqlConnection conn = new SqlConnection(connectionString))
SqlCommand headerSQL = new SqlCommand(headerCmd, conn);
SqlCommand rowSQL = new SqlCommand(rowCmd, conn);
SqlDataAdapter da = new SqlDataAdapter();
conn.Open();
//get header row
da.SelectCommand = headerSQL;
da.Fill(headerDT);
//get data from Transaction table
da.SelectCommand = rowSQL;
da.Fill(rowDT);
//build header
foreach (DataRow hDR in headerDT.Rows)
foreach (DataColumn hDC in headerDT.Columns)
csvData.Append(hDR[headerInt].ToString() + "
//remove last pipe then start new line
csvData.Remove(csvData.Length - 1, 1);
csvData.Append(Environment.NewLine);
//add rows
foreach (DataRow rDR in rowDT.Rows)
for (int i = 0; i < headerInt; i++)
csvData.Append(rDR[i] + "
csvData.Remove(csvData.Length - 1, 1);
csvData.Append(Environment.NewLine);
//write to CSV
File.WriteAllText(outputFile, csvData.ToString());
Why not building the header with a script task and importing data using a data flow task?
– Hadi
Mar 7 at 18:19
1
@Hadi if the number of columns is static that would make sense. I wasn't sure if they were so I posted the answer to accommodate a varying number of columns. However if there's always 5 columns as indicated in the question using a script task to handle the columns and DFT for the import would simplify this.
– userfl89
Mar 7 at 18:24
1
I totally agree with that
– Hadi
Mar 7 at 18:35
add a comment |
This can be done using a Script Task as follows, with C# used in this case. This will create a CSV file with the pipe (|
) delimiter. Running a sample test with this I was able to import the output CSV file via an SSIS Flat File Connection Manager without any modifications to the file. This example assumes there's only a single row in the header table, otherwise you'll need to modify the SQL for this table to return the proper row.
using System.Data;
using System.Data.SqlClient;
using System.IO;
using System.Text;
//Windows Authentication (Integrated Security)
string connectionString = @"Data Source=ServerName;Initial Catalog=DatabaseName;Integrated Security=true";
string headerCmd = @"SELECT RecordType, CustID, DataGenDate, DataCreatedDate, SourceID FROM HeaderTable";
string rowCmd = @"SELECT RecordType, CustID, DataGenDate, DataCreatedDate, SourceID FROM TransactionTable";
string outputFile = Dts.Variables["User::FilePathVariable"].Value.ToString();
StringBuilder csvData = new StringBuilder();
int headerInt = 0;
DataTable headerDT = new DataTable();
DataTable rowDT = new DataTable();
using (SqlConnection conn = new SqlConnection(connectionString))
SqlCommand headerSQL = new SqlCommand(headerCmd, conn);
SqlCommand rowSQL = new SqlCommand(rowCmd, conn);
SqlDataAdapter da = new SqlDataAdapter();
conn.Open();
//get header row
da.SelectCommand = headerSQL;
da.Fill(headerDT);
//get data from Transaction table
da.SelectCommand = rowSQL;
da.Fill(rowDT);
//build header
foreach (DataRow hDR in headerDT.Rows)
foreach (DataColumn hDC in headerDT.Columns)
csvData.Append(hDR[headerInt].ToString() + "
//remove last pipe then start new line
csvData.Remove(csvData.Length - 1, 1);
csvData.Append(Environment.NewLine);
//add rows
foreach (DataRow rDR in rowDT.Rows)
for (int i = 0; i < headerInt; i++)
csvData.Append(rDR[i] + "
csvData.Remove(csvData.Length - 1, 1);
csvData.Append(Environment.NewLine);
//write to CSV
File.WriteAllText(outputFile, csvData.ToString());
This can be done using a Script Task as follows, with C# used in this case. This will create a CSV file with the pipe (|
) delimiter. Running a sample test with this I was able to import the output CSV file via an SSIS Flat File Connection Manager without any modifications to the file. This example assumes there's only a single row in the header table, otherwise you'll need to modify the SQL for this table to return the proper row.
using System.Data;
using System.Data.SqlClient;
using System.IO;
using System.Text;
//Windows Authentication (Integrated Security)
string connectionString = @"Data Source=ServerName;Initial Catalog=DatabaseName;Integrated Security=true";
string headerCmd = @"SELECT RecordType, CustID, DataGenDate, DataCreatedDate, SourceID FROM HeaderTable";
string rowCmd = @"SELECT RecordType, CustID, DataGenDate, DataCreatedDate, SourceID FROM TransactionTable";
string outputFile = Dts.Variables["User::FilePathVariable"].Value.ToString();
StringBuilder csvData = new StringBuilder();
int headerInt = 0;
DataTable headerDT = new DataTable();
DataTable rowDT = new DataTable();
using (SqlConnection conn = new SqlConnection(connectionString))
SqlCommand headerSQL = new SqlCommand(headerCmd, conn);
SqlCommand rowSQL = new SqlCommand(rowCmd, conn);
SqlDataAdapter da = new SqlDataAdapter();
conn.Open();
//get header row
da.SelectCommand = headerSQL;
da.Fill(headerDT);
//get data from Transaction table
da.SelectCommand = rowSQL;
da.Fill(rowDT);
//build header
foreach (DataRow hDR in headerDT.Rows)
foreach (DataColumn hDC in headerDT.Columns)
csvData.Append(hDR[headerInt].ToString() + "
//remove last pipe then start new line
csvData.Remove(csvData.Length - 1, 1);
csvData.Append(Environment.NewLine);
//add rows
foreach (DataRow rDR in rowDT.Rows)
for (int i = 0; i < headerInt; i++)
csvData.Append(rDR[i] + "
csvData.Remove(csvData.Length - 1, 1);
csvData.Append(Environment.NewLine);
//write to CSV
File.WriteAllText(outputFile, csvData.ToString());
edited Mar 7 at 17:26
answered Mar 7 at 16:32
userfl89userfl89
2,9641513
2,9641513
Why not building the header with a script task and importing data using a data flow task?
– Hadi
Mar 7 at 18:19
1
@Hadi if the number of columns is static that would make sense. I wasn't sure if they were so I posted the answer to accommodate a varying number of columns. However if there's always 5 columns as indicated in the question using a script task to handle the columns and DFT for the import would simplify this.
– userfl89
Mar 7 at 18:24
1
I totally agree with that
– Hadi
Mar 7 at 18:35
add a comment |
Why not building the header with a script task and importing data using a data flow task?
– Hadi
Mar 7 at 18:19
1
@Hadi if the number of columns is static that would make sense. I wasn't sure if they were so I posted the answer to accommodate a varying number of columns. However if there's always 5 columns as indicated in the question using a script task to handle the columns and DFT for the import would simplify this.
– userfl89
Mar 7 at 18:24
1
I totally agree with that
– Hadi
Mar 7 at 18:35
Why not building the header with a script task and importing data using a data flow task?
– Hadi
Mar 7 at 18:19
Why not building the header with a script task and importing data using a data flow task?
– Hadi
Mar 7 at 18:19
1
1
@Hadi if the number of columns is static that would make sense. I wasn't sure if they were so I posted the answer to accommodate a varying number of columns. However if there's always 5 columns as indicated in the question using a script task to handle the columns and DFT for the import would simplify this.
– userfl89
Mar 7 at 18:24
@Hadi if the number of columns is static that would make sense. I wasn't sure if they were so I posted the answer to accommodate a varying number of columns. However if there's always 5 columns as indicated in the question using a script task to handle the columns and DFT for the import would simplify this.
– userfl89
Mar 7 at 18:24
1
1
I totally agree with that
– Hadi
Mar 7 at 18:35
I totally agree with that
– Hadi
Mar 7 at 18:35
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55042332%2fhow-to-append-two-rows-from-two-source-in-ssis%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Not sure if you want this from MySQL, or SQL Server, (i suspect the latter), however, what you are asking for here is a dynamic Pivot. SSIS, however, cannot handle a dynamic pivot for exported data. SSIS requires static definitions of data. If you want that type of data in an export I would suggest using SSRS and a matrix.
– Larnu
Mar 7 at 11:09
hi @Larnu I want this from SQL Server
– Shahab Haidar
Mar 7 at 11:21
That doesn't change my above comments in regards to dynamic data though.
– Larnu
Mar 7 at 11:33
You can use a C# script task to take a dataset and generate a .csv or Excel file dynamically. For Excel though you need to have the correct .dlls installed on the SSIS server.
– Brad
Mar 7 at 12:51
Possible duplicate of SSIS : Creating a flat file with different row formats
– Chris Albert
Mar 7 at 17:29