ETLOps

 

Most of the existing DevOps frame works and technologies are developed for programming technologies like Java and .Net. But the ETL development is completely different from Java or .Net development. Developer cannot control how the underlying code or package was generated and stored. Developer cannot control how the code was released. Different ETL tools uses its own method to deploy the code. So when we try to use DevOps frame work developed for Java or .Net in ETL tools, it will not meet our objective. We end up in developing work around for each step and the effort will outweigh the benefits. When effort outweigh benefits, shareholders will not appreciate and eventually we will go back to manual tasks. It is better to develop a DevOps frame work that suits your ETL needs rather than simply following the practice in application projects.

 

Leave a comment