Overcoming Data Overload with MANTA Filters

Overcoming Data Overload with MANTA Filters

Engineer's Notes MANTA Business
February 11, 2020
Zosia Szczech

No matter what your job position is and how tech-savvy you are, you most likely find the amount of data surrounding you overwhelming. And there’s nothing wrong with that—I get it, I feel you, and I have something to ease the pain.

Back in My Day…

Do you remember the good old days, when you were not being bombarded with information everywhere you look? No smartphone in your pocket, hence no notifications popping up every few minutes, no checking your email wherever you are whenever you want to, no updating company databases from your phone or tablet. Things are completely different today. Our computers and phones have better technical specifications than the Apollo 11 computer had. They are now able to process tons of data that flow in every direction at lightspeed, providing us with more information than ever. On one hand, you can never have too much good data. On the other hand, making good use of it is a completely different story.

Visualize It!

Let’s imagine you are working on a project for which it’s necessary to identify a specific set of elements in your environment. You get all your ducks in a row, you open MANTA, you run the visualization to get your hands on the relevant part of the lineage so you can work with it, and you see this:

With a sigh of resignation, you zoom in, you analyze each object, table, and transformation one by one, and one eternity later you finally get what you need. Sure, you could do that, but such an approach has two flaws.

1. It would break our MANTA hearts if we knew you were spending your precious time doing it.

2. Mistakes are waiting to happen.

Don’t Panic

If you analyze all the data flows manually, it is a lot of pressure on one person to decide which element represents what and to distinguish which information is relevant and which is not. That’s when MANTA and its filters step in to save the day.

Filter It!

Filtering information is extremely important, yet challenging. When you are sandboxing or working on a particular project, perhaps concerning just a small part of the environment, identifying what to work on saves time and allows you to focus on project delivery rather than the manual analysis of a huge database. Before hitting the Visualization button in MANTA, you can choose from our four filters.  The Everything filter is set by default. Choosing this one shows you all your objects in all analyzed technologies. Depending on your needs, you can set the following filters.

  • Databases, Files & Reports — view only database objects and files, not transformations
  • Database objects — view only database objects, not files or transformations
  • Important objects — view only table lineage and data obtained from reporting tools.

Selecting the right filter will allow you to narrow the scope of visualization and take you from the stage you saw above to this:

If you are wondering if it is possible to dig deeper and only apply a filter to particular steps, the answer is yes, you can! It only takes two clicks to get your results from here:

to here:

Tune It However You Like!

Our basic filters are just a start. You can play around to make sure that the visualization results show you exactly what you need and hide information irrelevant to the project. Have you run the visualization and realized that some elements are not helpful at all? Hide them on the go. Do you want to check the transformations one by one and hide those that you are not interested in at the moment? Start with the big picture and let MANTA help you by taking you straight to what you need in no time. Whenever you are working on a task that involves using your organization’s data, MANTA is the chief cook and bottle washer. Our Unified Lineage Platform will show you lineage adjusted to the level of detail that meets the needs of everybody on your team.

Are you ready to explore what else MANTA filters can do and how you can use them in your environment, no matter how complex it is? Request a Demo or shoot us a message at manta@getmanta.com.