All Topics

Well, that’s a wrap! 2018 is behind us, so as you sit in your office chairs in the first few work days of 2019, read our last blog post about 2018 written by our CEO, Tomas Kratky.

For most people around the world, especially those in areas with Christian roots, the time around Christmas and New Year is somehow special. It starts with Christmas parties. There are so many of them everywhere that you can literally spend the whole first half of December going from one to the next. Later in December, no matter how busy our year is, we usually tend to slow down a bit and think about spending more time with family and friends. Even workaholics consider taking some time off. This magic is what I love so much about Christmastime.

The end of the year is also a good time to look back and thank the people who have supported us on our journeys. At Manta, we would like to thank, from the bottom of our hearts, all our customers, partners, and supporters. 2018 was an amazing year – we experienced triple-digit growth in customers, employees, and revenue. We also completed our first investment round last summer and welcomed two VCs on board. With their support, we are continuously improving our operations and internal processes to prepare ourselves for the coming years of growth.

I could fill pages with all the great things we achieved in 2018, but what would be the point in doing that? The most important thing is to say how proud we are to serve such great companies and how it is even better to see and enjoy the excitement that individual users and data professionals experience as a result of the lineage automation capabilities Manta brings them. In those moments, we feel like all the hard work and sleepless nights are worth it.

The end of the year is also a time flooded with different Top X predictions. Sometimes it is so boring to read again and again how technology X will disrupt the whole industry next year. This year, we are all talking about AI and machine learning. It is funny to see every technology out there suddenly “powered by AI & ML”, “bringing AI & ML to everyday life”, etc. But you know what – things are really changing, and AI & ML buzzwords are getting more and more real. Which is great. But like with big data a couple years back, we have to govern new initiatives properly and make them part of our data governance programs from day 1, otherwise we will end up with a big mess (as we did with data swamps). With no trust in the data you use for AI and ML, how can you trust the outcomes and results of your intelligent algorithms. Remember – mess in, mess out!

The end of the year is also the time for New Year’s resolutions and wishes. In Manta, we have no special resolution, just to keep on doing the same as we have been doing every year since we started (aligned with our mission to provide our customers with fully automated and complete navigation through their entire data and application landscape) – to push the boundaries of lineage automation and again understand a bit more, always maintain a can-do attitude, and continue to become slightly better versions of ourselves.

And my wish for 2019? As my grandma always says: “I wish you good health, my boy, and you will take care of the rest!” And she is right. Health is critical, not only for people but for data too. With bad data, we very much limit our ability to use it to its full potential. But the same way we sometimes harm ourselves (drinking, smoking, no sleep, no fitness, etc.), we can also do a lot of harm to our data. So my wish for 2019 is the best possible health to all of you and your data as well!

Happy New Year! Šťastný Nový Rok! Frohes Neues Jahr! Feliz Año Nuevo!

Depending on where you are, either snow is slowly covering the ground and rooftops or, well, not. Here in Prague where MANTA’s development center is, it’s freezing outside. Let’s grab a hot chocolate and dive into our last software release of 2018.

Each new version of MANTA supports a new technology. In the last release of this year, we are bringing you not one, but two! They are connectors for Talend Data Integration and Sqoop, and from Microsoft’s SQL Server toolbox, we now fully support SSAS. If your BI environment includes Talend projects, jobs, and subjobs with SQL code, you can now include them in your data lineage in MANTA’s native visualization, IBM IGC, Collibra DGC, and TopQuadrant.

For now, SSAS is only supported in MANTA’s native visualization. In the previous release, we added support for tabular models. Now, we support all SSAS models, including multidimensional models with cubes. MANTA now has the ability to automatically read SSAS dimensions and measure groups with all their hierarchies and sets.

The second big hit in this release is a Sqoop connector. If you have Sqoop scripts written in shell, MANTA can now add them to the data lineage in its native visualization, IBM IGC, Collibra DGC, and TopQuadrant. Before the end of the year, we will add a detailed Sqoop page to the Scanners & Integrations section on our website, so keep an eye open for that!

Our development teams have also improved existing connectors. The Apache Pig connector can now read macros, and the connectors for PostgreSQL, Amazon Redshift and Greenplum can now read stored procedures. The biggest improvements, however, have been made to our Informatica EDC connector. MANTA can now upload data into EDC fully automatically. This is possible mostly thanks to Informatica’s new API. Besides automated upload, we are now also able to load metadata from Teradata Parallel Transporter (TPT) scripts.

Last but not least, our installer, configurator, and updater now work for all integrations in our tech hub. You can now integrate each and every one of your connectors from the comfort of your own user interface – preferably from your own couch with cookies and hot chocolate. Happy Holidays!

Don’t forget that MANTA does a more detailed technological breakdown for every release together with screenshots and more, made especially for our partners & customers. If you haven’t gotten yours, contact us at manta@getmanta.com and we will make sure it arrives in your inbox.

MANTA has attended a fair number of conferences this year. What has that been like and what have we gotten out of it? Read this exclusive article by Jan Ulrych to learn more.

Hi, everyone. My name is Jan Ulrych and I am taking over MANTA’s blog for the day. My message is simple: I would like to share with you my insights as a pre-sales expert who has been to literally every MANTA conference. Talking to partners, potential customers, and other industry experts gave me a different view of things.

I decided to look at the matter from others’ point of view. You see, it’s hard for us. MANTA is a tech startup with weird software that solves a very specific problem – how to get end-to-end data lineage of your data warehouse. And let’s say that not everyone wants or needs that.

So, here I bring you a finite categorization of the people you will meet at tech conferences and what to tell them, from a data lineage point of view:

Type 1: The clueless guy

This first type of people doesn’t see data lineage as an issue their company should solve. Why? Simply because their solution should already be able to solve this need. But does it?

We at MANTA know that the third-party metadata management solutions from all the companies out there are very capable in a general sense. But because they solve “general” issues, it turns out that they are not able to focus on detailed matters such as accurate, detailed, complete data lineage down to the column level and automation of the mapping process, deep down, all the way to the column level. And if you add in multiple programming languages and custom programming code in the database – well not all solutions can handle that.

These people are usually senior managers and are not very familiar with the solutions their company uses on an operational level. But that is okay. It takes us some time, but we try to explain to them the differences between MANTA and other third-party metadata management solutions that MANTA integrates with, and we often get a lead right there at the stand.

Type 2: The type who needs convincing

Okay, technically this has nothing to do with data lineage. It’s more like psychology. There are people like this everywhere – the type who needs convincing. They do get the overall idea of tracking their data and seeing where the data they use in the data warehouse comes from in the database, but up until now, they have not realized it is an issue. Or, they have not realized it is something worth spending the money in their budget on.

The fun part is once you explain how many man hours and, just in general, resources and money they can save by deploying an automated solution like MANTA to handle the manual processes within their company, they become quite eager customers. But they are indeed the ones who want to see the proof, the case study, the success story, the solution.

Type 3: The type who knows

I promise, there really are people like this out there! But they are sadly the minority, even at the most field-specific niche conferences. But we look out for them. They are always a pleasure to talk to, and they become our customers. People who know that data lineage is necessary and who are willing to put forth effort to achieve it are our kind of people.

Type 4: I’m here for the catering

Thanksgiving is just around the corner. Speaking of Thanksgiving, have you ever realized that large-scale data warehouses are a lot like your auntie’s famous Thanksgiving casserole? Never? Well, let us tell you a tale then!

It’s a familiar scenario. The family is gathered around a table covered with an impressive number of unorganized trays and plates full of food, with the turkey on a golden platter. As soon as you are done saying your prayers of thanks, you look for your favorite dish, still served in that worn baking tray as it has been for what seems like the last 300 years – your auntie’s Thanksgiving casserole. When you finally get your hands on that juicy looking dish, you shove in the serving spoon in pure ecstasy to scoop out that little piece of heaven – only to realize that you have served yourself mostly potatoes and green beans, and almost no bacon.

Shoot, the good stuff’s barely there. As you can see, the image of that perfect wholesome dish that you keep in your mind isn’t exactly accurate. There is a ton of ingredients that make up that little piece of heaven, and you don’t always get the same share. And well, your Data Warehouse is just like that.

It might seem like one perfect wholesome storage system that runs in the bowels of your company, but it’s not. Just like your auntie’s casserole, it is overflowing with ingredients. You might have a steady SQL-code foundation, which may include some tricky SQL overrides, but there could also be some JavaScript, deceitful PostgreSQL, and don’t even get us started on integrated Hive databases and pieces of custom code used to stitch the environment altogether.

That’s what it is, a mess. And the mess gets worse the bigger the environment is, the older the databases are, the more languages and technologies that get piled up, and the more developers take charge of integrating the environment and writing necessary parts of code to keep it up-and-running, no matter what the caloric intake – I mean no matter how much time and effort it costs the company. And when the need to automate, upgrade, or migrate such an environment arises – well, easy is not a word that comes to mind.

Luckily, MANTA is a powerful tool. Stronger than your will to fish more bacon out of the casserole dish; better than your favorite post-Thanksgiving fat-burning exercise. From the beginning, the main advantage of our software has been that it can automatically parse even the most difficult custom programming code. Since we have taken the pleasure of adding more and more supported technologies and programming languages to our technology stack over the last few years, we’ve got you covered.

MANTA can deal with the most heterogeneous of environments and automatically map information flows in your data warehouse. We can tell you how exactly your databases are related and what sensitive data they share amongst each other and create an overall map of end-to-end data lineage in our own visualization or push it into your existing metadata management solution.

For specific case studies or a full demo, contact us at manta@getmanta.com or talk to Mark in our friendly bot on the right. He is far less annoying than he looks, we promise.

We are upgrading our technology stack again and need you to try out our connectors! Since the last release, MANTA has supported Pig, and before Christmas, you can look forward to a gift from us: software release 3.23 with support for Talend.

Every time we begin supporting another technology, it is a big step for us. We always look for pioneers who have this technology in their metadata management environment and are not afraid to take the challenge of testing it in the field, giving us feedback, and allowing us to tailor the technological connector to fit the actual solution.

The Pig Connector

When we came out with MANTA 3.22, we provided our customers with the Apache Pig connector.

Apache Pig is a platform for analyzing large sets of data in Apache Hadoop. Pig Latin programming language is a high-level, procedural language for expressing data analysis programs. Pig’s structure enables substantial parallelization.

MANTA understands Pig Latin and is able to analyze and visualize it. MANTA is able to scan:

• Pig Latin statements
• Relations
• Bags
• Tuples
• Fields

The Talend Connector

In our last release of the year, we are going to introduce our Talend connector. Talend Data Integration is an open-source-based ETL tool. The software is driven by the commercial open-source vendor Talend and is used for the visual design of data transformations with a number of ready-to-use components and connectors.

MANTA understands both Talend Open Studio for Data Integration as well as the commercial edition. MANTA will be able to export and analyze:

• Projects, jobs, and subjobs
• Connectors
• Components
• Expressions
• SQL overrides

Did we get you interested in becoming a MANTA Pioneer for Pig or Talend? Or is there another technology on our scanners & integrations list that you are interested in testing? Drop us a line at manta@getmanta.com. We are looking forward to hearing from you!