Automating ETL process with scripting technology
Vangipuram Radhakrishna, Vangipuram Sravan Kiran, K Raghavendra Ravikiran · Nirma University International Conference on Engineering · 2012
Improving ETL process system data flows can provide better return on your Business investment. An enterprise level scheduling solution that is easy to use and handles heterogeneous environments may just do what you need. In the past, the focus was mainly on business process design, modeling. Recently, enterprises have realized that they can benefit tremendously from automating the ETL process with the objective of optimizing or improving them. In our research, we address the problem of warehousing, show how to automate the ETL process for pulling data from source to target environment. Typically, the source data consists of event streams that indicate changes in the business process state i.e progression of the process. The idea is to automate ETL process using scripting technologies. This paper is written to give scope in future for coming up with effective ETL tools that support command based automation or script based automation. We throw light on the need for usage of scripting technologies to automate ETL tools processing end to end process which reduce manual headache of running ETL process handling and also lead to development of ETL tools in future that support command based interface.
Cited by 17 works. doi:10.1109/nuicone.2012.6493217
Authors
References
- The Data Warehouse Lifecycle Toolkit
- Data exchange: getting to the core
- Conceptual-model-based data extraction from multiple-record Web pages
- Arktos: towards the modeling, design, control and execution of ETL processes
- A Framework for the Design of ETL Scenarios
- On the Logical Modeling of ETL Processes
- Towards a Benchmark for ETL Workflows.
- Implementation of web-ETL transformation with pre-configured multi-source system connection and transformation mapping statistics report