All Topics

Quirks in Your Data Warehouse You Would Never Find

August 7, 2018 by

We all have a very good image of what our data warehouse should look like, and what roles and functions it should play in the company’s business intelligence environment. But that is rarely the reality. Our BI consists of more than just “perfect” structured data marts. It is usually filled with all kinds of quirks, the kind that only MANTA  can deal with. Read on to find out how.

We all have a very good image of what our data warehouse should look like, and what roles and functions it should play in the company’s business intelligence environment. But that is rarely the reality. Our BI consists of more than just “perfect” structured data marts. It is usually filled with all kinds of quirks, the kind that only MANTA  can deal with. Read on to find out how.

First there must be data, then there is a data warehouse

The usual reason why we don’t have a text-book perfect DWH in our company is that we simply never start from zero. Our developers don’t come to a blank (in this case empty) canvas where they can model and build an imposing BI environment from scratch. First there must be data, then there is a data warehouse. And when there is enough data in the company for someone to notice that it would be a good idea to start managing it, the developers are called to the rescue to build a DWH. They often get their hands on a load of unorganized data: non-integrated data marts that are usually just libraries scattered all over the enterprise.

It’s not like the data has a family tree

Since it’s enterprise data, not a fancy little dog, it probably won’t have a family tree  of its own. The metadata might not match across various libraries in different departments of the company, and even the code itself probably won’t be in the same style. There is little to no hope that the different implementers in the various departments process data identically, leaving the entire DWH a very inconsistent place. At this point, the amount of time and financial resources spent on each individual data mart are so high that many developers just “go with it” or try to do their best to standardize and migrate the data together. This process begs for errors and quirks. They may not occur at the moment, but they are sure to show up when it’s least convenient, for example, when the company needs to analyze the entire data warehouse and get end-to-end data lineage for further data migration or compliance projects.

Rely on software, not people

Luckily, no matter how many sins the company has collected over years of poor DWH development and no matter how many quirks are hiding in your BI environment waiting to be unleashed in an unpleasant BI malfunction crisis, MANTA can deal with it. Better yet, it can uncover quirks you didn’t even know you had.

MANTA scans all the metadata sources directly or pulls them out of your metadata management solution. Then it can automatically find all the hidden logic and dependencies in the environment. That would be hard, or maybe even impossible, to do manually, depending on the amount of data accumulated in the BI environment and the size of the team working on the project. But either way, the time of those employees could be spent in a far better way than manually going through metadata sources. Once MANTA is able to find all the weak spots, it is easy for the customer to mend them all at one time.

Does this sound like something you might be struggling with as well? Let us know at manta@getmanta.com. We’ll help you figure something out.

Updated Roadmap for MANTA: It’s Always About the Customer

July 25, 2018 by

It is MANTA’s second year as an independent company and startup; before we were just a tool (remember, Manta Tools?) in the box of our “big sister” Profinit. Since those times, we have been able to create an entire list of supported technologies, both scanners and integrations. But the story with each and every one remains the same. It is the same reason why MANTA was created in the first place: we were inspired by our customers.

It is MANTA’s second year as an independent company and startup; before we were just a tool (remember, Manta Tools?) in the box of our “big sister” Profinit. Since those times, we have been able to create an entire list of supported technologies, both scanners and integrations. But the story with each and every one remains the same. It is the same reason why MANTA was created in the first place: we were inspired by our customers.

The customer-centric approach, or whatever you want to call it, is a bit trendy now. There are a number of courses that offer to teach you how to think like your customers, how to find out what they need and what they lack, and how to use your findings to create a product, service, or even a startup. But the truth that has been proven over time by each and every sales guy at MANTA is: if you don’t talk to your customers about their issues, you will never know. The first deployment of MANTA was in a Czech retail bank. MANTA was born (okay, coded) to automatically map their specific BI environment, to help them with both compliance and data optimization issues.

And things haven’t changed since then. Our pre-sales team has had hours and hours of calls a week with dozens of potential customers that are looking to solve their issues within their BI environments. Even though MANTA has an A+ in connectivity, MANTA still often integrates with customer solutions through API or import. When this happens once, twice, or even a few more times, we ask ourselves if it isn’t time to add a new connector to the family. And that is usually the case! In the spirit of preparing our Q3 release, we would like to share a few plans that we have for upcoming releases. It’s a release roadmap!

Speaking of new technologies – we are always looking for pioneers to test them! If you are interested in becoming a MANTA pioneer, write to manta@getmanta.com and let us know which of these new technologies you want to probe. 

How to Solve Impact Analysis with MANTA

Our customers use MANTA for all kinds of projects, impact analysis being one of them. When you have a really complex BI environment and still want to perform a reliable impact analysis, using predicates (and MANTA) is one way! Keep on reading to learn how.

Our customers use MANTA for all kinds of projects, impact analysis being one of them. When you have a really complex BI environment and still want to perform a reliable impact analysis, using predicates (and MANTA) is one way! Keep on reading to learn how.

During our pilots and deployments, we often find data warehouse environments that use very general physical models including several big tables like PARTY, BALANCE, ORDER, and others.

These tables contain data obtained from various source systems, and there are a lot of data marts and reports built on top of them. These tables make things difficult during impact analysis because data lineage from almost every report goes through them and into all the sources, making the results hard to use, or even worthless.

Impact Analysis Is Easier Than Ever Before

Let’s take a closer look at an example to understand exactly what happens when you use MANTA for your impact analysis. The table PARTY contains all information about individuals and companies that are somehow related to the organization. Thus, in one table, it is possible to have records for clients, employees, suppliers, and the organization’s branch network. Each type of entity is identified by the unique attribute or source system from which the data is obtained – for example, clients are managed in a different system than employees.

Now, let’s assume that the data from the PARTY table goes into two separate reports – a report EMPL_REPORT that displays information about employees and another report BRANCH_REPORT that displays information about the branch network. If we use the standard data lineage analysis, we can get this picture:

Although only data from the EMPLOYEE source table is relevant for the report EMPL_REPORT, the impact analysis from that report also includes the CLIENT, BRANCH, and SUPPLIER source tables due to the PARTY table. The problem is the same for the report BRANCH_REPORT. From the other side, the impact analysis from the EMPLOYEE source table includes both the EMPL_REPORT and BRANCH_REPORT which is confusing.

The Advantage of Using Data Lineage

Luckily, there is a solution. When data is inserted into the PARTY table from different source systems, there is often a column like PARTY.source_system_id where the identification of the source system is stored as a constant. Similarly, when a report is created that consumes data only from specific source systems, there is a condition in the statement filtering data based on the PARTY.source_system_id column. Thus, it is possible to automatically analyze both the insertion and selection to/from the PARTY table and create predicates such as PARTY.source_system_id = 20 that are then stored together with data lineage in the metadata repository. Therefore, it is possible to include them in the computation during the impact analysis.

Thanks to that, if we perform an impact analysis from the report EMPL_REPORT, the predicate PARTY.source_system_id = 20 is gathered before the table PARTY. When the analysis continues towards source tables, the predicate for each path is selected and compared to what has already been gathered. Therefore, when the path to the source table CLIENT with the predicate PARTY.source_system_id = 10 is tested, the result is that both predicates cannot hold at once, so data for this report cannot come from this source table. Conversely, when the path to the source table EMPLOYEE with the predicate PARTY.souce_system_id = 20 is tested, the result is that data for this report can come from this source table, so it is included in the results of the impact analysis. We can get similar results if we perform an impact analysis for the BRANCH_REPORT and also from sources like the EMPLOYEE table.

The results of the advanced data lineage analysis can look like this (in reality, if we perform the impact analysis from the EMPL_REPORT, we will only see the EMPLOYEE and PARTY tables):

Surely, the situation can be far more complex. For example, the data from the PARTY table can be pre-computed for more source systems first, and then several reports can be created on top of them for only a specific source system, like in this picture:

If you have any questions or comments, feel free to contact Lukas at manta@mantatools.com. You can try these predicate-based impact analyses in our free trial!

 

 

Key Features of MANTA: Integration

July 12, 2018 by

If there is one thing I have noticed in my many years working for MANTA, it is that MANTA’s integration capabilities are highly appreciated. Customers worldwide notice that what MANTA has to offer in terms of connectivity is high above the average standard of other metadata management solutions.

If there is one thing I have noticed in my many years working for MANTA, it is that MANTA’s integration capabilities are highly appreciated. Customers worldwide notice that what MANTA has to offer in terms of connectivity is high above the average standard of other metadata management solutions.

There are several ways MANTA can connect to other pieces of software. MANTA has pre-built integrations for a number of specific data governance solutions, scanners for a variety of databases and data management platforms, and two types of APIs. Then, on top of that, there are a few other ways to push data into MANTA, if none of the above fit your needs. Here is a closer look at each of the options.

1. Integrations

MANTA has specific, pre-built integrations for complex data governance platforms that can use MANTA’s help when it comes to reading complex code structures and other features important for creating end-to-end data lineage. MANTA connects with them, scans their database resources, and then pushes the final data lineage into their own environments or visualizations. Some of MANTA’s integrations are Informatica EDC and IMM, Collibra DGC, IBM IGC, TopQuadrant EDG, and more. 

2. Scanners

MANTA also has its own visualization, where it can show you data lineage and allow you to work with it. MANTA can parse metadata from several databases and other tools, some of which are Teradata Database, Oracle Database, Microsoft SQL Server, SAP ASE, Hive, both IBM Netezza and IBM DB2, Impala, PostgreSQL, and many, many more. 

3. APIs

If you don’t use any of the solutions MANTA integrates with, there are other ways to get data lineage. MANTA can connect to basically any piece of software through its two APIs: the Service API and the Repository API. You can get detailed insight into both in our API blog post: here. 

4. Other ways

If you could not find “your” solution in any of the options mentioned above, no need to panic. MANTA always finds a way to create data lineage for our customers. MANTA can visualize data lineage based on imported files, Excel tables, data in CSV format, or just externally provided mapping and metadata from the database.

MANTA the mighty

To sum it up, MANTA can connect to basically any piece of software. If you need data lineage, MANTA can give it to you and provide the best quality, the most accurate lineage you can get on the market. Our developers are always adding more supported technologies to the list, and now you know that there are many other ways to push data besides just those. This makes MANTA the absolute leader in connectivity and integration capabilities.

Is there anything else you would like to know? More technical details perhaps? Then contact one of our tech guys at manta@getmanta.com

 

TopQuadrant and MANTA Partner to Further Automate the Discovery of Data

TopQuadrant™, a leading provider of knowledge graph-based data governance solutions, today announced that it has partnered with MANTA to help its customers automate the discovery of data lineage information.

TopQuadrant™, a leading provider of knowledge graph-based data governance solutions, today announced that it has partnered with MANTA to help its customers automate the discovery of data lineage information.

TopQuadrant’s flagship product, TopBraid Enterprise Data Governance (EDG), helps organizations succeed in data governance. It delivers easy and meaningful access for all data stakeholders to enterprise metadata, business terms, reference data, data and application catalogs, data lineage, data exchanges and pipelines, requirements, policies, and processes. MANTA helps enterprises all around the world to automate their data lineage across many different technologies. On top of its unique data lineage extraction capabilities, MANTA offers a broad variety of connectors to different data governance solutions, as well as API.

The technology partnership between TopQuadrant and MANTA delivers an integration between TopBraid EDG and MANTA. Customers will be able to take advantage of the combined capabilities for data governance and lineage insight. After the initial setup, MANTA automatically reads the customer’s metadata and provides the metadata to TopBraid EDG.

Our customers are looking to connect the business context and meaning of their diverse data to its technical metadata in order to understand and manage it as an enterprise asset. The integration of TopBraid EDG with MANTA lets them automate the discovery of technical data lineage embedded in the SQL code and seamlessly bring it into TopBraid EDG where it gets put in context and connected to other relevant information,” said Irene Polikoff, CEO of TopQuadrant.

The end result is faster delivery of comprehensive knowledge graphs that provide businesses with meaningful information inferred from connected catalogs of terms, data, technical and business assets. This enables our customers to comply with regulations, produce accurate reports, understand the impact of changes and meet other demands common in today’s data rich environments,” said Ralph Hodgson, CTO of TopQuadrant.

Tomas Kratky, CEO of MANTA, agrees that this technical partnership gives the customer outstanding features, that it would be hard to seek in any other one solution alone: “Building custom integrations for state-of-the-art data governance solutions such as TopBraid EDG gives MANTA the ability to serve a broad range of customers, allowing them to have a data governance solution with flawless data lineage extraction and visualization capabilities, which allows them to strengthen their trust in data.

Both MANTA and TopQuadrant have a very customer-centric approach and are aiming to target very specific customer needs as well as to become a one-stop-shop. Even though the partnership has formed only recently, MANTA’s and TopQuadrant’s technology partnership already hosts its first customer, a large financial services provider. The joint solution allows them to have complete trust in their data as well as be compliant with all global industry regulations.


About MANTA

MANTA automates its customers’ data lineage and enhances capabilities of data governance solutions. The essentiality of MANTA lies within its ability to process and understand custom programming code and the ability to describe its logic. The software uses metadata harvested from the customer’s systems to visualize data lineage throughout the Business Intelligence environment. The data lineage can be further used for data warehouse optimization, lowering development costs, impact analyses, data migration, data discovery, data security projects and for meeting regulatory compliance standards for CCAR, HIPAA, BASEL II/III, GDPR and other regulations. Some of MANTA’s customers include PayPal, Comcast, Vodafone and OBI. Visit getmanta.com for more.

About TopQuadrant

TopQuadrant helps organizations succeed in data governance with TopBraid Enterprise Data Governance (TopBraid EDG).  EDG is built on standards-based knowledge graph technology to seamlessly bring together enterprise information silos, connect enterprise metadata, ensure its quality and deliver easy and meaningful access for all data stakeholders. TopBraid EDG lets enterprises govern all relevant assets including business terms, reference data, enterprise metadata, data and application catalogs, data lineage, data exchanges and pipelines, requirements, policies, and processes. TopBraid EDG is the only solution built to support integrated governance across all types of assets and governance needs while, at the same time, offering a staged approach to data governance. TopQuadrant’s customer list includes organizations in financial services, pharma, healthcare, digital media, government and other sectors. www.topquadrant.com.

Any questions or comments about the partnership and/or its fruits? Ask MANTA here or TopQuadrant here.

MANTA Now Crunches (Almost) Everything from the Microsoft SQL Server Family

In Q2, MANTA released support for Microsoft SSAS, and it is no secret that our developers are working hard to bring you support for SSRS in Q4. This means MANTA will be fully on board with all Microsoft SQL technologies. Because these can be a little confusing, we have prepared a complete break-down below!

In Q2, MANTA released support for Microsoft SSAS, and it is no secret that our developers are working hard to bring you support for SSRS in Q4. This means MANTA will be fully on board with all Microsoft SQL technologies. Because these can be a little confusing, we have prepared a complete break-down below!

1. SSIS

SQL Server Integration Services (SSIS) is Microsoft’s visual tool for ETL (Extract, Transform, Load) processes. SQL Server generally uses it as a source or a destination.  It feeds sources like MS Office Excel tables and application backends into your data warehouse.

These sources are where MANTA finds all the most important information it uses to create data lineage for your Microsoft business intelligence environment.

2. SSAS

Microsoft SQL Server Analysis Services (SSAS) is a layer between the data mart and analytics platform that allows the customer to define the dimensions (master tables) and the facts (measurable details) in the data mart and is then able to measure and process them, helping the customer create reports.

So far, MANTA has only supported tabular models, but as of Q4, it will also be able to work with multidimensional models.

3. SSRS

This stands for Microsoft SQL Server Reporting Services (SSRS), and it is the highest layer of the Microsoft business intelligence environment. This layer is responsible for creating reports from data it extracts directly from the data mart as well as from the analytics center of your data warehouse.

So how does all of this affect MANTA’s lineage?

MANTA will be able to access all levels of the customer’s Microsoft business intelligence environment. If it runs completely on Microsoft technologies, MANTA can effortlessly access the sources, stage, and core using SSIS, where it can read all the SQL code in the customer’s internal database (including complicated custom code) and datamart. Then MANTA can access the analytical layers and reports through SSAS. When MANTA is able to read SSRS, it will be able to access every single report and provide the customer with complete coverage of its Microsoft data environment, creating the most precise end-to-end data lineage the Microsoft SQL Server family can get.

Do you have any questions about MANTA’s support for Microsoft or any other technologies or about the development of our software? We are happy to answer them all at manta@getmanta.com

Is Guessing Good Enough for Your GDPR Project?

June 28, 2018 by

I will tell you one thing — I am tired of GDPR buzz. Don’t get me wrong, I value privacy and data protection very much, but I hate the way how almost every vendor uses it to sell their goods and services, so much that the original idea is almost lost.

I will tell you one thing — I am tired of GDPR buzz. Don’t get me wrong, I value privacy and data protection very much, but I hate the way how almost every vendor uses it to sell their goods and services, so much that the original idea is almost lost.

It is similar to BCBS or other past data-oriented regulations. Consulting companies, legal firms, data governance/ security/ metadata vendors, we are all the same — buy our “thing” and you will be ok or at least safer with us! Every second book out there tells us that every change is an opportunity to improve, to evolve. So, what is the improvement here with GDPR? If I look around I see a lot of legal work being done, adding tons of words (very small letters, as always) to already long Terms & Conditions. And you know what? I don’t think there is any real improvement in it.

But things are not always so bad. There is also a lot of good stuff going on with one important goal—to better understand and govern data and its lifecycle in a company. And there is one challenging but critical part I want to discuss today—Data Lineage. That means how data is moved around in your organization. You must understand that a customer’s email address or their credit card number is not just in your CRM but is spread all over your company in tens or even hundreds of systems — your ERP, data warehouse, reporting, new data lake with analytics, customer portal, numerous Excel sheets and even external systems. The path of the data you collect can be very complex, and if you think about all possible ways you can move and transform data in your company, one thing should be clear — your data lineage has to be automated as much as possible.

Different Approaches to Data Lineage

That being said, I feel it is important to talk about different approaches to data lineage that are used by data governance vendors today. Because when you talk about metadata, you very often think about simple things — tables, columns, reports. But data lineage is more about logic — programming code in any form. It can be an SQL script, PL/SQL stored procedure, Java program or complex macro in your Excel sheet. It can literally be anything that somehow moves your data from one place to another, transforms it, modifies it. So, what are your options for understanding that logic?

This article is base on a presentation by Jan Ulrych at the DGIQ 2018 Conference. Click on any image to open the gallery.

Option 1: Ignore it! (aka data similarity lineage)

No, I am not crazy! There are products building lineage information without actually touching your code. They read metadata about tables, columns, reports, etc. They profile data in your tables too. And then they use all that information to create lineage based on similarities. Tables, columns with similar names and columns with very similar data values are examples of such similarities. And if you find a lot of them between two columns, you link them together in your data lineage diagram. And to make it even more cool, vendors usually call it AI (another buzzword I really hate). There is one great thing about this approach — if you watch data only, and not algorithms, you do not worry about technologies and it is no big deal if the customer uses Teradata, Oracle or MongoDB with Java on top of it. But on the other hand, this approach is not very accurate, performance impact can be significant (you work with data) and data privacy is at risk (you work with data). There are also a lot of details missing (like transformation logic for example, which is very often requested by customers) and lineage is limited to the database world, ignoring the application part of your environment.

Option 2: Do the “business” lineage manually

This approach usually starts from the top by mapping and documenting the knowledge in people’s heads. Talking to application owners, data stewards and data integration specialists should give you fair but often contradictory information about the movement of data in your organization. And if you miss talking to someone you simply don’t know about, a piece of the flow is missing! This often results in the dangerous situation where you have lineage but are unable to use it for real case scenarios — not only can you not trust your data, you cannot trust the lineage either.

Jan Ulrych presenting at DGIQ 2018.

Option 3: Do the technical lineage manually

I will get straight to the point here — trying to analyze technical flows manually is simply destined to fail. With the volume of code you have, the complexity of it and the rate of change, there’s no way to keep up with it. When you start considering the complexity of the code and especially the need to reverse engineer the existing code, this becomes extremely time consuming and sooner or later such manually managed lineage will fall out of sync with the actual data transfers within the environment and you will end up with the feeling that you have lineage you cannot actually trust.
Now that we know that automation is key, let’s take a look at some less labor intensive and error prone approaches.

Option 4: Trace it! (aka data tagging lineage)

Do you know the story of Theseus and the Minotaur? The Minotaur lives in a labyrinth and so does Ariadne who is in charge of the labyrinth. Ariadne gives Theseus a ball of thread to help him navigate the labyrinth by being able to retrace his path.

And this approach is a bit similar. The whole idea is that each piece of data that is being moved or transformed is tagged/labeled by a transformation engine which then tracks that label the whole way from start to finish. It is like Theseus. This approach looks great, but it only works well as long as the transformation engine controls the data’s every movement. A good example is a controlled environment like Cloudera. If anything happens outside its walls, the lineage is broken. It is also important to realize that the lineage is only there if the transformation logic is executed. But think about all the exceptions and rules that apply only once every couple of years. You will not see them in your lineage till they are executed. That is not exactly healthy for your data governance, especially if some of those pieces are critical to your organization.

Option 5: Control it! (aka self-lineage)

The whole idea here is that you have an all-in-one environment that gives you everything you need—you can define your logic there, track lineage, manage master data and metadata easily, etc. There are several tools like this, especially with the new big data/ data lake hype. If you have a software product of this kind, everything happens under its control — every data movement, every change in data. And so, it is easy for a such a tool to track lineage. But here you have the very same issue as in the previous case with data tagging. Everything that happens outside the controlled environment is invisible, especially when you consider long-term manageability. Over time, as new needs appear and new tools are acquired to address them, gaps in the lineage start to appear.

Option 6: Decode it! (aka decoded lineage)

Ok, so now we know that logic can be ignored, traced with tags and controlled. But all those approaches fall short in most real-life scenarios. Why? Simply because the world is complex, heterogeneous, wild and most importantly — it is constantly evolving. But there is still another way — to read all the logic, to understand it and to reverse engineer it. That literally means to understand every programming language used in your organization for data transformations and movements. And by programming language I mean really everything, including graphic and XML based languages used by ETL tools and reports. And that is the challenging part. It is not easy to develop sufficient support for one language, let alone the tens of them you need in most cases to cover the basics of your environment. Another challenging issue is when the code is dynamic, which means that you build your expressions on the fly based on program inputs, data in tables, environmental variables, etc. But there are ways to handle such situations. On the other hand, this approach is the most accurate and complete as every single piece of logic is processed. It also guarantees the most detailed lineage of all.

And that’s it. This was not meant to be a scientific article, but I wanted to show you the pros and cons of several popular data lineage approaches. Which leads me back to my first GDPR paragraph. I see enterprises investing a lot of money in data governance solutions with insufficient data lineage capabilities, offering tricks like data similarity, data tagging and even self-lineage. But that is just guesswork, nothing more. Guesswork with a lot of issues and manual labor to correct the lineage.

So, I am asking you once again — is guessing good enough for your GDPR project?

This article was also published on Tomas Kratky’s Linkedin Pulse.

To Migrate, or Not to Migrate, the Report?

When migrating data, you can get into some pretty tricky situations. It is always tough to decide what data to migrate and what not to. MANTA can help you decide as well as reveal some crucial relations within your environment. Read on to learn about the surprises you may find in your DWH.

When migrating data, you can get into some pretty tricky situations. It is always tough to decide what data to migrate and what not to. MANTA can help you decide as well as reveal some crucial relations within your environment. Read on to learn about the surprises you may find in your DWH.

The Case

While one of our customers was dealing with their data migration project, something unexpected came up. When MANTA helped the customer see the relations and detailed dependencies between all the reports, the customer came to realize that multiple reports in the environment were sharing tables with each other. But the reports were of different levels of importance.

MANTA’s Solution?

MANTA’s ability to show all the dependencies within the environment prior to migration helped the customer approach the problem proactively as a complex matter. This saved the customer a lot of time and allowed them to avoid future complications. Otherwise, the customer may have found these relations much later, such as after the migration when one of the reports stopped working properly. Then they would have needed to go through thousands of tables manually to see where the problem was. MANTA enabled the customer to be prepared and gave them the opportunity to make the necessary decisions ahead of time, before any problems occurred.

Difficult Decisions

Multiple reports in the environment were sharing tables with each other. But the reports were of different levels of importance. In one case, the customer wanted to migrate a report to the newly established cloud because it was a frequently used report that was worth keeping. But the report shared some tables with another report that had much lower priority for the customer’s company. However, regardless of its lower priority, it wasn’t exactly a report that could be deleted. It was still used occasionally.

The customer could migrate all the reports and tables involved, but that would inconveniently inflate the amount of data being migrated. Alternatively, the customer could skip these reports and tables and not include them in the migration, but then they would not be able to use the reports in the newly established cloud environment. The customer could duplicate all the data, but what if something changed? Having the same data in multiple places always requires effective synchronization to make sure the data is identical and up-to-date everywhere, which is usually a really complex problem.

So what should the customer do? Have you experienced a similar situation, or are you afraid that this is your case too and you need MANTA’s help to find out for yourself? Contact us at manta@getmanta.com or join the discussion under our post on social media!

GDPR: “What’s Next?” The Aftermath

The General Data Protection Regulation (GDPR) took effect last Friday. At this moment it is important to ask: “What’s next?”

The General Data Protection Regulation (GDPR) took effect last Friday. At this moment it is important to ask: “What’s next?”

Were you able to ensure your company’s compliance with GDPR, send all the mandatory privacy policy update emails, and happily complete what you thought might be one of the most challenging tasks for your company in 2018? Well, we are sorry to break it to you, but it doesn’t stop there.

The GDPR was formed as a regulation to protect users and customers, and they have indeed been given the reins. It might not have happened to you yet over the last few days, but there will come a time when customers will request that your company delete all existing data about them. Or your company might have recognized that there are still large amounts of data in your BI environment that need to be anonymized. Either way, there is a heap data staring back at you, waiting to be handled.

The Big Mess

Some of the consultancy companies that MANTA works with have recognized that the biggest struggle connected with GDPR compliance is most likely tracking customer data across multiple databases. Companies must identify the various locations where sensitive or uncompliant data is being stored as well as discover the relationships between this data. Not all records are equally sensitive, not all need to be anonymized. Sometimes, only parts of the data need to be re-written (e.g., a name and a country code in the same table will most likely not lead to the identification of a customer, but adding a city name can end up leading to quite precise identification).

The complexity of today’s BI environments makes it almost impossible to search for these relations manually. Luckily, MANTA can automatically analyze all database objects and data processing logic within your database, and if an additional description about the level of sensitivity of each record is provided, it can identify the locations of sensitive records. Then you can eliminate all potential threats to compliance as well as make sure you didn’t fail to find a complete record of customer data.

Lucky for you, MANTA is still available, no matter whether the GDPR countdown has ended or not. If you think that you could use some serious help with tracking your data and automating your data lineage, schedule an informative call with our pre-sales guys. They are really friendly!

Or contact us at manta@getmanta.com. We are always up for a chat.

Key Features of MANTA: Current and Past Revisions

Due to popular demand, we bring you another episode from our series “Key Features of MANTA”. This time, we will take a look at MANTA’s ability to automatically report the differences between current and last revisions.

Due to popular demand, we bring you another episode from our series “Key Features of MANTA”. This time, we will take a look at MANTA’s ability to automatically report the differences between current and last revisions.

Exactly which differences are reported?

Usually, you are not the only user in your bu¨siness intelligence environment. Many employees and groups of employees within the company access specific resources, edit database schemas, or just upload new information, e.g.:  add new customer records, edit customer information such as addresses or phone numbers, and so on.

When various changes are made in your database, it is possible that some relations within tables are changed or deleted, and that can lead to spare tables suddenly getting “lost” in traffic or being modified, resulting in a change in the course of the data lineage itself.

Suddenly, the data lineage you have been exporting from MANTA doesn’t fit in the database anymore. This could mean that you no longer have an accurate overview of the data being used in your database, or even, in the worst case scenario, that you are no longer compliant with regulations.

Practical explanation of how revisions work can be found in our technical demo of MANTA + Microsoft SQL Server. Take a look (video will start at 5:09, where Lukas starts to explain how revision system works):

We’ve got you covered!

Because MANTA is specifically tailored to meet customers’ needs, we have noticed this point of frustration and have developed a specific feature that allows you to avoid any unpleasant situations in connection with changes to your database. Every time you log in to MANTA, it will let you know what has changed since it last exported data to your metadata management solution. We’ve simply got you covered!

Is there anything else you’d like to know about MANTA? We are always available at manta@getmanta.com.

 

We cherish your privacy.

And we need to tell you that this site uses cookies. Learn more in our Privacy Policy.