Never been to DZone Snippets before?

Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world

About this user

Fyodor Kupolov http://j2ee.by.ru

« Newer Snippets
Older Snippets »
Showing 1-6 of 6 total  RSS 

Using JavaScript for ETL transformations

Scriptella provides a simple way to perform various transformations in JavaScript (or other scripting language which have a corresponding driver).
Our example transformation consists of 3 steps:
1) Select rows from source table.
2) Transform a column value from number to text
3) Insert a transformed value into a destination table.


<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
    <connection id="db" driver="auto" url="jdbc:hsqldb:mem:tst" user="sa" password="" classpath="../lib/hsqldb.jar"/>
    <connection id="js" driver="script"/> 
    <connection id="log" driver="text"/> <!-- For printing debug information on the console -->

    <script connection-id="db">
        CREATE TABLE Table_In (
            Error_Code INT
        );
        CREATE TABLE Table_Out (
            Error VARCHAR(10)
        );
        
        INSERT INTO Table_IN VALUES (1);
        INSERT INTO Table_IN VALUES (7);
    </script>

    <query connection-id="db">
        SELECT * FROM Table_In
        <script connection-id="log">
            Transforming $Error_Code
        </script>
        <!-- Transformation is described as an enclosing query
         which is executed before nested elements -->
        <query connection-id="js"> 
            <![CDATA[
               if (Error_Code < 5) {
                    Error_Code='WARNING'; //Set a transformed value
               } else {
                    Error_Code='ERROR'; //Set a transformed value
               }
               query.next(); //Don't forget to trigger nested scripts execution
            ]]>
            <script connection-id="db">
                <!-- Insert transformed value -->
                INSERT INTO Table_Out VALUES (?Error_Code); 
            </script>
            <script connection-id="log">
                Transformed to $Error_Code
            </script>
        </query>
    </query>
</etl>

How to execute Scriptella ETL files

Scriptella ETL provides several ways to execute ETL files:

Invocation from Ant
<taskdef resource="antscriptella.properties" classpath="/path/to/scriptella.jar[;additional_drivers.jar]"/>
<etl file="path/to/etl/file/> <!-- Execute ETL file from specified location -->

Command-Line Execution
Just type scriptella to run the file named etl.xml in the current directory. Alternatively you can use Java launcher:
java -jar scriptella.jar [arguments]

Executing ETL Files from Java
It is extremely easy to run Scriptella ETL files from java code. Just make sure scriptella.jar is on classpath and use any of the following methods to execute an ETL file:
EtlExecutor.newExecutor(new File("etl.xml")).execute(); //Execute etl.xml file
EtlExecutor.newExecutor(getClass().getResource("etl.xml")).execute(); //Execute etl.xml file loaded from classpath
EtlExecutor.newExecutor(
    servletContext.getResource("/WEB-INF/etl.xml")).execute(); //Execute etl.xml file from web application WEB-INF dir

Integration with Spring Framework
<beans>
    <!-- Spring beans declarations -->

    <!-- Spring managed bean which executes etl.xml file -->
    <bean id="executor" class="scriptella.driver.spring.EtlExecutorBean">
        <property name="configLocation" value="etl.xml"/>
    </bean>
</beans>

The usage of executor is straightforward:
EtlExecutor exec = (EtlExecutor) beanFactory.getBean("executor");
exec.execute();

See Spring Driver JavaDoc for additional details.

How to produce daily database table dumps in CSV format

The following code demonstrates how to produce CSV files with dynamic
file name pattern based on a current day. The produced files have the following naming format:
TABLE_NAME_MM_DD_YYYY.csv

<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
    <properties> <!-- Configure table name -->
        table_name=test
    </properties>
    <connection id="in" driver="auto" url="jdbc:oracle:thin:@localhost:1521:ORCL" 
      classpath="ojdbc14.jar" user="scott" password="tiger"/>
    <connection id="out" driver="csv" url="${table_name}_${etl.date.now('MM_dd_yyyy')}.csv" />
    <query connection-id="in"> <!-- Query table rows -->
        SELECT * FROM ${table_name}
        <script connection-id="out"> <!-- Export each row into a CSV -->
            $ID, $Name, $Surname <!-- Use column names from selected table -->
        </script>
    </query>
</etl>

Use Scriptella ETL to run the example.

See How to execute an ETL file from command line, Ant or directly from Java .

Splitting large Scriptella ETL files

The following example demonstrates how to split a large Scriptella ETL file into several parts. This example is based on a traditional XML parsed entities approach:

<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd"
[
    <!-- Declaring the first external parsed entity to include -->
    <!ENTITY part1 SYSTEM "part1.xml">
    
    <!-- Declaring the second external parsed entity to include -->
    <!ENTITY part2 SYSTEM "part2.xml">
]>
<etl>
    <connection driver="text"/>

    <!-- Including file #1 -->
    &part1;

    <script>
        content of the script
    </script>
    
    <!-- Including file #2 -->
    &part2;

</etl>

List of Countries and ISO codes Supported by the JVM.

The following code displays country names and ISO-codes for the locales supported by the JVM.

        for (Locale locale : Locale.getAvailableLocales()) {
            final String contry = locale.getDisplayCountry();
            if (contry.length() > 0) {
                System.out.println("Country = " + contry + ". ISO code: " + locale.getISO3Country());
            }
        }

Package Name as a Constant

getClass().getPackage().getName() is not a reliable solution, because getPackage() can be null depending on a class loader.
static final String PACKAGE_NAME = MyClass.class.getName().substring(0, 
            MyClass.class.getName().lastIndexOf('.'));
« Newer Snippets
Older Snippets »
Showing 1-6 of 6 total  RSS