Never been to DZone Snippets before?

Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world

About this user

Fyodor Kupolov http://j2ee.by.ru

« Newer Snippets
Older Snippets »
Showing 1-5 of 5 total  RSS 

How to produce daily database table dumps in CSV format

The following code demonstrates how to produce CSV files with dynamic
file name pattern based on a current day. The produced files have the following naming format:
TABLE_NAME_MM_DD_YYYY.csv

   1  
   2  <!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
   3  <etl>
   4      <properties> <!-- Configure table name -->
   5          table_name=test
   6      </properties>
   7      <connection id="in" driver="auto" url="jdbc:oracle:thin:@localhost:1521:ORCL" 
   8        classpath="ojdbc14.jar" user="scott" password="tiger"/>
   9      <connection id="out" driver="csv" url="${table_name}_${etl.date.now('MM_dd_yyyy')}.csv" />
  10      <query connection-id="in"> <!-- Query table rows -->
  11          SELECT * FROM ${table_name}
  12          <script connection-id="out"> <!-- Export each row into a CSV -->
  13              $ID, $Name, $Surname <!-- Use column names from selected table -->
  14          </script>
  15      </query>
  16  </etl>

Use Scriptella ETL to run the example.

See How to execute an ETL file from command line, Ant or directly from Java .

Importing XML into a database with Scriptella ETL

The following Scriptella ETL simple usage example imports RSS file into a database table.
   1  
   2  <!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
   3  <etl>
   4      <connection id="in" driver="xpath" url="http://snippets.dzone.com/rss"/>
   5      <connection id="db" driver="hsqldb" url="jdbc:hsqldb:db/rss" user="sa" classpath="hsqldb.jar"/>
   6  classpath="hsqldb.jar"/>
   7      <query connection-id="in">
   8          /rss/channel/item
   9          <script connection-id="db">
  10              INSERT INTO Rss (ID, Title, Description, Link) 
  11              VALUES (?rownum, ?title, ?description, ?link);
  12          </script>
  13      </query>
  14  </etl>


Here is the full version of the example described above. It creates an RSS table, downloads rss file, inserts rss records into a database, converts rss.xml to a plain text file and saves it to rss.txt.
   1  
   2  <!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
   3  <etl>
   4      <connection id="in" driver="xpath" url="http://snippets.dzone.com/rss"/>
   5      <connection id="out" driver="text" url="rss.txt"/>
   6      <connection id="db" driver="hsqldb" url="jdbc:hsqldb:db/rss" user="sa" classpath="hsqldb.jar"/>
   7      <script connection-id="db">
   8         CREATE TABLE Rss (
   9             ID Integer,
  10             Title VARCHAR(255),
  11             Description VARCHAR(255),   
  12             Link VARCHAR(255)
  13  
  14         )
  15      </script>
  16      <query connection-id="in">
  17          /rss/channel/item
  18          <script connection-id="out">
  19              Title: $title
  20              Description: [
  21              ${description.substring(0, 20)}...
  22              ]
  23              Link: $link
  24              ----------------------------------
  25          </script>
  26          <script connection-id="db">
  27              INSERT INTO Rss (ID, Title, Description, Link) 
  28              VALUES (?rownum, ?title, ?description, ?link);
  29          </script>
  30      </query>
  31  </etl>

Script to insert BLOB from file into a database

Scriptella ETL allows inserting files into a database. This is achieved by a simple bind variables extension syntax ?{file ...}.
The following sample initializes table of music tracks. Each track has a DATA field containing a file loaded from an external location. File song1.mp3 is stored in the same directory as etl.xml and song2.mp3 is loaded from the web:
   1  
   2      <!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
   3      <etl>
   4          <connection driver="hsqldb" url="jdbc:hsqldb:file:tracks" user="sa" classpath="hsqldb.jar"/>
   5          <script>
   6              CREATE TABLE Track (
   7                ID INT,
   8                ALBUM_ID INT,
   9                NAME VARCHAR(100),
  10                DATA LONGVARBINARY
  11              );
  12              <!-- Inserts file with path relative to ETL script location -->
  13              INSERT INTO Track(id, album_id, name, data) VALUES
  14                     (1, 1, 'Song1.mp3', ?{file 'song1.mp3'});
  15              <!-- Inserts file from an external URL-->
  16              INSERT INTO Track(id, album_id, name, data) VALUES
  17                     (2, 2, 'Song2.mp3', ?{file 'http://musicstoresample.com/song2.mp3'});
  18          </script>
  19      </etl>

Copy table from one database to another

This Scriptella ETL script copies all rows from Src_Table to Dest_Table.
Src_Table contains the following columns: id, first_name, last_name
Dest_Table contains the following columns: id, name
The name column of the Dest_Table is produced by a concatenation of first_name and last_name from the Src_Table
This example demonstrates HSQLDB-To-Oracle copy procedure, although it works between virtually any databases.
   1  
   2  <!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
   3  <etl>
   4    <connection id="in" driver="hsqldb" url="jdbc:hsqldb:file:demo" 
   5                classpath="hsqldb.jar" user="sa"/>
   6    <connection id="out" driver="oracle" url="jdbc:oracle:thin:@localhost:1521:ORCL" 
   7                classpath="ojdbc14.jar" user="scott" password="tiger"/>
   8    <!-- Copy all table rows from one to another database -->
   9    <query connection-id="in">
  10        SELECT * FROM Src_Table --Selects all rows
  11        <!-- For each row executes insert -->  
  12        <script connection-id="out"> 
  13            INSERT INTO Dest_Table(ID, Name) 
  14            VALUES (?id,?{first_name+' '+last_name})
  15        </script>
  16    </query>
  17  </etl>

Load CSV data into a database (Scriptella ETL tool)

This example demonstrates usage of Scriptella ETL Tool to load CSV data into a database table.

Input CSV file data.csv:
   1  
   2  id,priority,summary,status
   3  1,Critical,NullPointerException in Main class,Open
   4  5,Low,"Checkstyle, PMD, Findbugs issues",Reopened
   5  7,Low,Maven integration,Open
   6  10,High,SPI API,Closed

The CSV loading script has the following content:
   1  
   2  <!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
   3  <etl>
   4    <connection id="in" driver="csv" url="data.csv"/>
   5    <connection id="out" driver="oracle" url="jdbc:oracle:thin:@localhost:1521:ORCL" 
   6        classpath="ojdbc14.jar" user="scott" password="tiger"/>
   7    <!-- Copy all CSV rows to a database table -->
   8    <query connection-id="in">
   9        <!-- Empty query means select all columns -->
  10        <script connection-id="out">
  11            INSERT INTO Table_Name VALUES (?id,?priority, ?summary, ?status)
  12        </script>
  13    </query>
  14  </etl>


Use RegEx to filter CSV data:
   1  
   2  <query connection-id="in">
   3      <!--Select bugs with status open or reopened.-->
   4      ,,,open|reopened
   5      <!--Inserts imported rows into a database-->
   6      <script connection-id="out">
   7         INSERT INTO Table_Name VALUES (?id, ?priority, ?summary, ?status);
   8      </script>
   9  </query>

« Newer Snippets
Older Snippets »
Showing 1-5 of 5 total  RSS