Project Description
Reactive ETL is a rewrite of Rhino ETL using the reactive extensions for .Net.
 
Rhino ETL is a great and easy to use ETL in C#. Rhino ETL rely on "IEnumerable" and "yield return" to provide ETL pipelines.
 
While Rhino ETL is great, the "Enumerable" pattern is a pulling technic, not really adapted to queues and pipelines. By using this pattern, you end writing too much code for simple operations.
For an ETL the Observable pattern is more appropriate because it's a push technique. You define your queues and pipelines by chaining methods, in the same way you write Linq expressions, and then you tell your datasources to push data to the pipelines.
 
Here is an example of a simple pipeline that reads data from a table, transform the data, and insert the result in another table.
  var result =
          Input.Query("input", "SELECT * FROM Users")
                .Transform(
                    row =>
                        {
                            string name = (string)row["name"];
                            row["FirstName"] = name.Split()[0];
                            row["LastName"] = name.Split()[1];
                            return row;
                        }
                )
                .DbCommand("output", (cmd, row) =>
                    {
                        cmd.CommandText = @"INSERT INTO People (UserId, FirstName, LastName, Email) VALUES (@UserId, @FirstName, @LastName, @Email)";
                        cmd.AddParameter("UserId", row["Id"]);
                        cmd.AddParameter("FirstName", row["FirstName"]);
                        cmd.AddParameter("LastName", row["LastName"]);
                        cmd.AddParameter("Email", row["Email"]);
                    })
                .Execute();


Acknowledgments
Reactive ETL is an adaptation of Rhino ETL, written by Ayende. Some parts of ReactiveETL's code came from Rhino ETL.
As well as Rhino ETL, Reactive ETL use FileHelpers library from (www.filehelpers.com), written by Marcos Meli to work with files.

Last edited Sep 18, 2010 at 2:40 PM by gleborgne, version 12