Seek and Ye Shall Find: Introducing the Structured ETL Engine for Qlik

TSEEQ (pronounced “seek”), The Structured ETL Engine for Qlik, implements centralized management of Extract, Transform and Load (ETL) operations that provide data to Qlik applications. TSEEQ is quite simply a rule-based ETL engine built specifically for QlikView and Qlik Sense.

For more information on the concept of rule-based ETL engines, please see the companion article on LinkedIn, “Thrones, Castles, and Catapults: The Value of Rule-Based ETL Engines in BI Projects“.

If you’re interested in implementing TSEEQ right now, without having to sift through my ramblings in the the rest of this blog post, please just click this GitHub link and download the Source code (zip) file for the most recent release.

Now, on to my ramblings…  A logical question is, “why did Qlik Consulting create TSEEQ?”  The answer is, we created  TSEEQ to accomplish the following goals:

  • Governance: ETL operations are defined in external (and therefore very manageable) rule sets.
  • Self-Service: Business users may easily define and modify ETL operations in sandbox environments.
  • Performance: A comprehensive profiler enables efficient ETL execution.
  • Migration:  ETL rule sets for QlikView can be used without modification for Qlik Sense (and vice versa).
  • Productivity: The core ETL engine and its surrounding utilities automate time-consuming tasks.

TSEEQ In Comparison to Traditional Embedded Scripting

The distinguishing characteristic of TSEEQ versus Traditional Embedding Scripting (TES) is that in TSEEQ, externalized rules (diagram above) provide a structured source of ETL control; in TES (diagram below), free-form textual ETL script is embedded within Qlik application files (QVWs in QlikView and QVFs in Qlik Sense).

Note: TSEEQ and TES are not mutually exclusive; a hybrid approach is useful in many cases. TSEEQ is available for download from GitHub.

In future blog posts, we will delve into configuration and use of TSEEQ; if you are anxious to get started right away, please do take a look at the documentation PDF for some pointers!