It was early 2005 when I first picked up a trial license for QlikView version 6 and I've been a true fan ever since. I spent a huge part of my career as a Qlik consultant, working with hundreds of companies to transform their understanding of their data and their businesses with Qlik's technology. I've not lost my love for Qlik, but dare I say, my passion for EasyMorph has grown to rival it.
"But I'm a highly skilled Qlik developer Matt, why would I need a no-code drag-and-drop tool to process my data?". Well, I'm glad you asked! Let's take a look at why Qlik and EasyMorph have a lot of similarities and why I've become an EasyMorph Evangelist as well as being a Qlik Addict.
My Journey from Qlik Addict to EasyMorph Evangelist
As context to this post, I should tell you a few things about my background and how I ended up at EasyMorph.
I started my career as a software engineer before getting into BI. As I touched on above, I've spent much of the last 20 years as a consultant, advising businesses of all sizes and across all industries. And a large part of that time has been working with QlikView and Qlik Sense. For many years I wrote a blog named by Qlik in their list of top blogs, I was a Qlik certified trainer, and I also consulted for Qlik on some of their biggest and most complex customer deployments across the US and Europe. Qlik script is basically my second language.
Whilst it's been a few years since I earned my living writing code, I still tinker on evenings and weekends to support my hobbies. Coding is my comfort zone!
You wouldn't have to travel back too many years therefore to find me very resistant to the idea of "no-code" tools for ETL or BI. There was just no way I could see such a tool could possibly be as capable and as flexible as a scripting language.
I've come to realise however that the true power of a technology isn't in the minute intricacies of what it can and can't do, but in its ability to impact people or a business. Even if a no-code tool was less capable, being able to put it in the hands of many more people; with all their knowledge, their ideas and their expertise; could be so much more impactful on a business.
I've known Dmitry, the founder and CEO of EasyMorph, for many years now; usually crossing paths at a Qlik or general BI event somewhere in the world. It was at one such chance encounter that he first told me about EasyMorph and what he wanted to achieve with it. I came away with a trial license key and I must admit, for a tool in its infancy, it showed a lot of potential.
Fast forward a few years and another chance encounter led to us discussing how EasyMorph had evolved and developed. The topic of it being a "no code" tool came up, one thing led to another, and Dmitry issued me a challenge - to find a problem I could solve in Qlik script that I couldn't solve in EasyMorph. Whilst I am no doubt competitive and rarely one to turn down a challenge, it was his confidence that really piqued my interest. Could this really be a no-code tool without compromises?
What ensued over the next few weeks was a cycle of me dreaming up a challenge from a past project and trying to solve them in EasyMorph. The outcome of which went one of two ways, either I'd realise it was indeed solvable and would move on to the next idea, or I'd email Dmitry thinking I'd finally found something, only for him to show me a feature I hadn't yet discovered for myself. Eventually I admitted defeat and so here I am, working at EasyMorph as a Technology Evangelist, helping customers to understand what EasyMorph is, cutting through some of the myths and assumptions, and helping them to use it to have the biggest possible impact on their businesses.
So let's take a look at some of the reasons as a Qlik developer you should consider EasyMorph also...
You don't have to write any script
This one might be the most contentious but bear keep reading. You might well be like me and your brain operates in a manner similar to the green scrolling text from The Matrix. But the reality is, we're the odd ones out. Most people who come to Qlik, find Qlik scripting to be very intimidating. Qlik Sense does have the Data Manager and Qlik Cloud also has the newly released "Data Flows" which allow you to load data into Qlik without having to write any script. But both are limited in what can be done in them and it doesn't take much effort to find things you want to do that aren't possible without reverting to Qlik Script (Qlik Cloud Data Flows even has a "Qlik script processor").
EasyMorph has more than 180 actions which are the building blocks with which you can build "Workflows" to process data. Compare that to the 20 processors currently available in Qlik Cloud Data Flows. Using these actions is as simple as dragging and dropping them, configuring their settings and checking it had the desired effect. And as you can guess from the above, I'm yet to find anything that's possible in Qlik script that can't just as easily be done in EasyMorph using one or more actions. EasyMorph actually goes well beyond traditional ETL and has actions for a wide range of business system automation also.
OK, but you're not new to Qlik and you are plugged into the Matrix, so why would EasyMorph workflows be better than scripting? Some of the reasons we'll look at below, but possibly the most important is "concurrent processing". Even when using the Data Manager or Data Flows, Qlik actually still writes Qlik script behind the scenes. And whilst this script is incredibly powerful, it has one critical limitation that all scripting languages have - it is linear and so is processed one step at a time from top to bottom. There are a few tricks used to make the processing of Qlik script as "concurrent" as possible, such as loading in the next set of data whilst manipulating the last, but there's a limit to what can be achieved with any linear scripting language. In contrast, EasyMorph workflows can be branched, allowing many simultaneous streams of data processing to be performed simultaneously. You can even bring these branches back together and EasyMorph's engine will automatically handle ensuring all necessary actions have been executed in the order you intended.
Converting Qlik scripts to EasyMorph workflows is actually very simple. There is an action in EasyMorph that matches very closely with every statement and function in Qlik script. And so working your way through your script logically step-by-step, adding the corresponding action will result in the same output in very little time. "In script I can write subroutines and call them repeatedly though", I hear you say. Don't worry, EasyMorph has "modules" which are subroutines of a workflow. You can pass in not just parameters, but entire data sets.
You don't have to wait for long scripts to run
Anyone who has worked with Qlik script has had the frustrating experience of waiting for a long reload to run, only to be met with an error message because you forgot a comma near the end. You fix it and wait all over again. I've lost entire days when trying to make a simple change to a very complex script that took several hours to run. Yes, you can limit the data load to a sample of rows to reduce your wait, but its not always possible and there can be issues in the data which didn't present in the limited number of rows.
One of my favourite features of EasyMorph helps to solve this problem. EasyMorph's engine caches the output of each action and so if I make a change to a workflow, it knows which actions are before and after my change and therefore which cached items are still valid or need to be recalculated. We can see this behaviour in action if we turn off the "Auto-run" feature from the "Main" toolbar.
With Auto-run turned off, if I make a change to my workflow, actions that appear after the change show a green "play" icon to indicate they need to be run. If I click on one of them I see an error message telling me that the action needs to be run and most importantly, I can't see the data output from the actions.
If I select an action appearing before my change, then I don't see this error and I can view the data output by that action because EasyMorph still has it cached.
I can also run each action one at a time by selecting it and clicking the run "Run action" button. This alone is helpful, but the real power is in being able to see the data output by each action just by simply clicking on it. In Qlik, I'd have to exit my script execution or write out a file after each step. The amount of time this could have saved me!
EasyMorph can read and write QVD files
If you are a Qlik developer, I shouldn't need to explain why this one is important. EasyMorph can read and write QVD files. Being able to output one or more QVD files from your EasyMorph project ready to load straight into your Qlik Sense or Qlik Cloud dashboard is probably the biggest reason they work so well together.
I've even used EasyMorph to load a QVD file quickly and explore the data in it as it was quicker than creating a Qlik Sense app, loading it in and creating a table and filters to achieve the same.
However, you aren't limited to QVD files for sharing data between EasyMorph and Qlik. Both share support many other file formats such as CSV files, Apache Parquet files, fixed width files, Excel spreadsheets and many more.
EasyMorph integrates with both Qlik Sense and Qlik Cloud
Not only can your share data between EasyMorph and Qlik, but they can interact in other ways also. For a start, there are EasyMorph actions for triggering reload tasks in Qlik Sense and Qlik Cloud. Place one of these at the end of your workflow after you've written out the data and you can reload the relevant Qlik dashboard immediately after.
As well as triggering task, using the Qlik Cloud command action also allows you to upload and delete files to Qlik Cloud also. Yep, you can send QVDs to Qlik Cloud and then trigger dashboards to reload.
EasyMorph Server also has open APIs and so if you really want you can even trigger EasyMorph Server tasks from within Qlik. There's even a Qlik extension object you can add to your dashboard to allow a user to trigger them on demand.
EasyMorph will write the documentation for you
As a Qlik developer, there was nothing I hated more than having to write documentation. It always took way longer than building the actual solution and I just wanted to get on with solving the next problem instead. Therefore this is possibly my favourite feature of EasyMorph - it writes the documentation for you!
Just click on the "Generate documentation" button on the Project toolbar, optionally add any custom HTML header or body you'd like to be included or to style the d
ocumentation to meet your business brand, and click "Generate". Yep, its really that easy.
You can even add annotations to each action, module or group as you build the workflow to explain the purpose of each step. These annotation end up in the documentation along with the all the data sources, transformation steps, business logic and outputs.
Want to make sure your documentation stays up to date if you change your workflow? No problem, just place a "Generate documentation" action at the end of it and each time it runs it will take care of it.
Conclusion
There are so many more similarities between Qlik and EasyMorph, so many more reasons they work great together, such as:
- EasyMorph expression syntax is very similar to Qlik script function syntax, both being derived from SQL.
- Action chains in EasyMorph are analogous to multiple proceeding loads, allowing you to layer transformations.
- Just like Qlik's engine, EasyMorph processes all data in-memory and compresses the data in a similar way. This makes them both fast and able to handle large volumes of data.
- Workflows have "groups" (soon to be renamed "stages") which allow you to organise your data processing in the same way as Qlik script tabs/sections.
- "Derived tables" in EasyMorph are equivalent to resident loads.
- EasyMorph can do a lot more than just ETL too, such as automating connections between disparate systems or between your business and your customers/suppliers/partners.
I could keep going, but if I do, this will end up more like a novella than a blog post. The main reason I fell in love with QlikView all those years ago was because of how it allowed me to both identify, solve and communicate a business problem so quickly. I've lost count of how many CEOs have told me that the insights they gained by having visibility of their data has revolutionised, or even saved their businesses. And its for this exact same reason I've fallen in love with EasyMorph also.
You don't have to take my word for it though, can download EasyMorph for free now and see for yourself. What have you got to lose?
