The ultimate skill of any solution that has anything to do with data would be the transformation of raw, unapologetic, written pieces of code into something that non-coders can understand. By non-coders we mean all other, although highly intelligent, not-necessarily-used-to-coding business managers who work with the tables that spit out the data on the other side.
And that is exactly what MANTA has always been determined to do. With all the buzzwords floating around like record level lineage (read here why MANTA does not have it), business level lineage, and all the others, people expect solution providers to give them an understanding of what is going on in their data environment. And we might have just succeeded in doing that.
As the first sentence of the article suggests, data transformation is the main topic here. To transform data into what you see in a report, you need lines and lines of code that use many transformations and algorithms to call upon the data to act and look like something more approachable. What usually happens is that no one needs to know what the transformations behind the data are, as long as the data in the tables are correct.
And data lineage shows you the whole “data journey” (which is the main reason why people need it so much). It displays from end to end exactly how the data flows: from where to where, what it does in the middle, and what table it lands in. For the average business user who does not speak the language of code, this is entirely satisfactory for them to understand and trust their data environment. It gives them insight if you will. It is also sufficient for regulatory institutions as well as for creating basic business reports like risk analyses and planning data migrations.
But sometimes, somewhere, you want to learn more.
Real-Life Case: So How Much Do We Pay Those Sales Guys?
Let’s take calculating commissions for sales guys as an example.
The calculation of these commissions is represented by 20 scripts of code in the data environment. These scripts do all the work behind the calculation – they take into account all the inputs from financial and accounting systems as well as a set of rules from the company’s commission policy. Based on this data, the scripts are able to calculate the individual commission for every sales guy in the company.
But what is the actual formula for calculating the commissions? Is there a way to reverse engineer it from the scripts? By deploying MANTA you can find out!
MANTA can identify the exact calculation of the commission and compare it with the ruleset from the company’s policy. Now we know exactly how much and how we pay those sales guys. 🙂
We have a little bit of information attached to each cell (where we are able to track it) that shows the user exactly which transformations were used to achieve that bit of information. It gives you a deeper level of understanding, while still keeping things easily visible and readable in MANTA’s data lineage.
Have any questions about how to get transformation logic in MANTA or how to use it in your projects? Write to us at firstname.lastname@example.org!