All Topics

For the third year in a row, we will be a sponsor at Collibra Data Citizens 2019 in New York! To warm you up a bit and give you some topics to discuss with us, we have prepared a special blog on how MANTA integrates with Collibra. Read more below!

How does MANTA work with Collibra?

There are two critical pieces to any data environment—structures (like files, database tables, columns, and associated business entities) and logic (programs, applications, and ETL workflows). MANTA does what others can’t—brings metadata from your business and data processing logic to Collibra.

Before, MANTA used Collibra Connect to connect to the Collibra Data Governance Center. But now that is a thing of the past. As of our newest release, MANTA integrates directly with Collibra via Collibra’s own API. We have been partnering with Collibra on the development of this synchronization API for quite some time now. So, we are pleased to inform you that we are now introducing the final version.

How is it different from the old integration?

1. Direct integration. We are so integrated that we are basically part of Collibra DGC. This makes your work with Collibra and MANTA so much faster and easier.
2. Automatic metadata update. Collibra can fully use this MANTA feature now.
3. Table synchronization. We are the first ones on the planet to be able to update your Collibra DGC with your database 1:1, meaning you can now get rid of non-existent tables in DGC and make room for new ones.
4. All in one. We are able to export all MANTA data lineage to Collibra, including the newly supported Microsoft SSRS.
5. Logical lineage. Since we support metadata extraction from E/R models and mappings between physical and logical layers, we can provide this information to Collibra so it can provide logical data lineage.
6. Installation. It is just so much easier now.

We are looking forward to seeing you this May in New York and talking all about MANTA’s tech bond with Collibra! 

Can’t wait till May and want answers to your questions now? Then feel free to write to us at manta@getmanta.com. 

After introducing “MANTA 4 Finance” as the pilot segment of our new “MANTA 4 Industries” series, we are moving on to another industry that MANTA is very familiar with. Read about what issues MANTA helps its healthcare customers solve in the article below.

Regulatory Compliance and GDPR

It can be said that the regulatory requirements in terms of data protection for this industry are even more strict than in other industries. The companies’ systems contain large amounts of sensitive personal data that, as we all know now, has to be safeguarded with extra care. The General Data Protection Regulation (GDPR) and Health Insurance Portability and Accountability Act (HIPAA) threaten astronomical penalties for noncompliance.

A big problem may also arise from an inability to accommodate the fundamental rights of data subjects (right to access, right to erasure, right to restrict data processing, etc.).  In these cases, the company has to show the supervisory authority exactly how it secures its data. 

In the worst-case scenario, if there would happen to be a data breach, the healthcare provider would have to prove that it did everything humanly possible to stop it! Data lineage is a way of showing that, by drawing out an end-to-end map of the data flows and all their movements within the BI and analytics environment.

Data Anonymization

Some of the consultancy companies that MANTA works with have recognized that the biggest struggle regarding GDPR compliance is most likely tracking customer data across multiple databases. The subject of data anonymization, personal identifiable information (PII), consists of data elements that alone or in combination can directly or indirectly lead to the identification of a specific individual.

Companies must identify the various locations where sensitive or noncompliant data is being stored as well as discover the relationships between this data. Not all records are equally sensitive; not all need to be anonymized. Sometimes, only parts of the data need to be re-written (e.g., a name and a country code in the same table will most likely not lead to the identification of a customer, but adding a city name could end up leading to quite precise identification).

Using MANTA you can construct and analyze metadata models that will identify PII in any component of your BI and analytics solution.

IoT and Healthcare

With all of the abovementioned regulations, healthcare is one of the most monitored industries in the world, and for good reason! It will only get harder to comply with these regulations, especially with the desire to digitalize all customer records and add IoT concepts that integrate data from wearable sensors. These wearables usually sync data in real time as well, giving doctors the ability to remotely monitor patients. This makes them a real-time security threat for medical experts, doctors, insurance providers, you name it! (Hence the possible data breach threats mentioned above.)

Conclusion?

The complexity of today’s BI and analytics environments makes it almost impossible to search for these relations manually. Luckily, MANTA can automatically analyze all database objects and data processing logic within your database, and if an additional description about the level of sensitivity of each record is provided, it can identify the locations of sensitive records. Then you can eliminate all potential threats to compliance as well as make sure you didn’t fail to find a complete record of customer data.

Do you have any questions for us, or do you want to read one of our case studies regarding this industry? Then hit us up at manta@getmanta.com and we will gladly reply! 

MANTA is a solution for companies that have loads of data: huge, complicated data warehouses. But are you wondering how exactly MANTA fits into the data warehouses of big financial institutions? Welcome to our series called MANTA for Industries: Finance.

Credit Risk Scoring

Financial institutions that offer consumer loans invest a lot of money, effort, and know-how from years of experience into the development of advanced credit scoring models. In most cases, only a few people in the organization know and have access to these algorithms.

One way to decode a credit scoring model is to collect a significant amount of customer data where credit scores are combined with variable credit factors and then use statistical methods to understand the model. These “dangerous” combinations of data are often present in BI solutions and data warehouses. Every BI solution should allow the separation of access to such data by properly categorizing data sensitivity and by enabling user entitlement setup. This is not easy to achieve, and it is especially difficult to verify if the setup is correct.

With MANTA, you can easily identify and visualize the components of BI solutions (for example, data marts) where unwanted combinations of such sensitive data are present, or you can analyze user data-access setup to see if there are direct or hidden and indirect ways to retrieve those data sets.

You can export lineage from MANTA that can then be used for BI security improvements. And you can even restrict the use of MANTA’s data lineage right in our native visualization. When you find such relations, you can then take them into account when setting up user access for different user groups and teams who have access to MANTA and monitor who has access to different parts of the lineage from different data marts.

Regulatory Compliance and GDPR

Another popular way to use MANTA in big banking institutions is to produce proof for internal auditors that your credit scoring models are well protected. With the enormous number of banking regulations as well as regulations such as GDPR, it is twice as important to have decent data lineage to show the auditor when he arrives. To learn more, read our GDPR article. (link)

GDPR has introduced many more threats pertaining to corporate internal data. For example, a customer may come and request that you honor one of his rights such as the right to be forgotten. In this case, you may need to use MANTA to find every single place in your company data marts where the customer’s data is being stored to make sure you delete every single one of those instances.

You may also need to anonymize data. And after using MANTA to find all the places where your data needs to be anonymized, you might want to use MANTA again to double check that there is really no way to identify that person.

Legal Threats

Being able to keep track of your environment and changes in your credit scoring algorithms is beneficial for many different reasons. One of our favorite client stories is about a customer who attempted to sue our client for not approving his loan the first time he applied, only to be approved a year later.

Using MANTA, the client was able to automatically find the changes made to the scoring algorithms over the last couple of years and identify the change in the algorithm for calculating credit scores for loans. The ability to show exactly what had changed and when allowed the financial institution to win the court case, saving them billions of dollars.

And how could you use MANTA in your financial institution?

Any comments or questions? Let us know at manta@getmanta.com, or go ahead and schedule a meeting using the form on the right.

Here in Prague, where MANTA’s engineering office is located, the snow has melted and sunny spring has arrived. As the first baby otters are born, we are delivering a new little baby of our own: MANTA 3.24. Read about it in our blog post below or check out the two-minute video where Jan Ulrych summarizes all the changes and updates.

What’s new this time? After finalizing our Microsoft SSRS connector, we have added two more new connectors. The first one is a scanner for Oracle Data Integrator (ODI); the second is for ER/Studio, which expands our influence in the realm of data modelling tools so that we can now create logical lineage automatically, making data lineage from MANTA more accessible for users who aren’t database tech pros.

However, the biggest success in this release is the direct integration with Collibra via API. We have been partnering with Collibra on the development of this synchronization API for quite some time now. So, we are pleased to inform you that we can now introduce the final version.

How is it different from the old integration?

1. Direct integration. We are so integrated that we are basically part of Collibra DGC. This makes your work with Collibra and MANTA so much faster and easier.
2. Automatic metadata update. Collibra can fully use this MANTA feature now.
3. Table synchronization. We are the first ones on the planet able to update your Collibra DGC with your database 1:1, meaning you can now get rid of non-existent tables in DGC and make room for new ones.
4. All in one. We are able to export all MANTA data lineage to Collibra, including the newly supported Microsoft SSRS.
5. Logical Lineage. Since we support metadata extraction from E/R models and mappings between physical and logical layers, we can provide this information to Collibra so it can provide logical data lineage.
6. Installation. It is just so much easier now.

Besides the hot stuff mentioned above, MANTA 3.24 finally offers transformation logic in Teradata and a long-awaited experimental Java version. We are currently doing closed beta testing with some of our customers, and from the next software release onward, we will be doing open testing.

Interested? Got questions? We are here for you. Throw a message into our trusty mailbox at manta@getmanta.com. We will reply!

Many companies seek to achieve an agile development strategy. Sometimes it is not exactly needed and might bring some difficulties (as our CEO Tomas Kratky already mentioned in one of his older articles here).  But sometimes an agile-type strategy naturally evolves in the development department. In the third part of our MANTA Cases series, we will take a look at how one of our customers uses MANTA in a (not agile) development team. 

One Too Many Patches

One of our customers, an international bank, was using a workshop-type software to release internal production software. Because this internal database software is so complex, they have 10 to 15 patches of code each day between releases. To make sure that the environment doesn’t crash, they have to deploy these patches of code all at once. Each patch of code is created by a different development team, and the individual teams are not aware of the dependencies between their parts of the code. Because of this, the patches were often deployed in the wrong order causing the entire environment to crash.

These crashes often made the deployment process last more than one day, with numerous other patches needing to be developed to fix the problems created while deploying the previous patches.

Understanding the Environment

The patches of code the development team wrote were all improvements to their internal database system (e.g., changing database logic, repairing certain objects, even adding new columns or creating more tables and views). Such patches resulted in changes in paths or even in values within the environment. And given the size of our customer’s database – which contained multiple banking systems, each having an average of 180 thousand scripts – and the amount of data being added to the database each day, it was necessary to perform the 10-15 patches daily.

The only way the customer could keep up with the (not) agile development pace and effectively find the dependencies between the patches was to apply automated software like MANTA.

Post-MANTA

Now that the customer has MANTA on their development team, they take all the patches that need to be deployed that day and let MANTA run them. MANTA draws out all the dependencies between the individual objects that the patches are related to. Then the customer exports them using MANTA’s API and has an automatically sorted order that it needs to deploy the patches in so that the environment will survive. This has cut the entire deployment down to a couple of minutes and has also made it a one-person job.

This one person now only needs to:

• Gather the patches
• Upload them into MANTA
• Use MANTA to determine the order of deployment
• Order the scripts accordingly
• Release them in that order
• Done!

Conclusion

Now, the customer can be sure that the patches will be deployed correctly and that it will be done in just a couple of minutes, making the daily-release strategy actually work out every day.

MANTA has been an enormous help to their situation. And well, what can we say: this case is proof that not every development team chooses to be agile. Some just have to adjust their development pace to the complexity of their environment.

Be sure to read more of our MANTA Cases! Any requests? Any questions? Send your comments to manta@getmanta.com.