MANTA Cases

MANTA Cases #5: The Only Reason Developers Exist

Today’s edition of “MANTA Cases” was inspired by a recent MANTA Sales training. Our sales folks were discussing one of our largest customers, a Fortune 500 bank, and their struggle to track changes in an ever-moving environment.

Today’s edition of “MANTA Cases” was inspired by a recent MANTA Sales training. Our sales folks were discussing one of our largest customers, a Fortune 500 bank, and their struggle to track changes in an ever-moving environment.

Data lineage is not always a platform for proving the existence of security processes and achieving regulatory compliance. Sometimes data lineage just provides an elementary function: it tracks the changes and transformations in the data environment and visualizes them in a detailed and accurate map. And many of our customers use MANTA just for that!

Our customer, a Fortune 500 bank, had really comprehensive, high-level internal systems. In order to keep them up and running, it had its own team of developers following an agile development process. There were several reasons why the customer chose MANTA in the effort to automate their processes.

Reason #1: The Inevitability of Change

The bank’s management was very much aware that the development processes and changes in the environment needed to be recorded, not only to continuously improve future development but mainly to understand their data. But from the beginning, they were not quite sure how to achieve that. Their solution was to make developers write down, record, and notify the rest of the team about every change that they made in the environment. That didn’t work, mainly because it is almost impossible. After all, the only reason developers exist is to make changes in the environment.

Reason #2: The Pain of Documenting Stuff

Often—and this case was no different—teams of developers have their know-how, their undocumented knowledge. They know how they have built the environment, and they are the ones continuously working on it. Sometimes it is just impossible to get that knowledge out and documented. Also, people come and go. Often, there was no prior documentation for a task that was about to be completed simply because no one had ever done that before in this company. That is why the idea of MANTA is so promising for developers. It can read and document data automatically without prior assumptions of how the data flows should look. Which leads us to…

Reason #3: Make-believe Lineage

This is usually more of a post-data-lineage realization, but it is one of our favorites. Often, developers do have everything documented and know exactly what transformations are performed in their data warehouse—or so they think. After seeing MANTA’s data lineage, customers are often surprised by the visualization. They say they “could have sworn” it was “like this”. Well, you can’t fool MANTA.

The Result?

The customer deployed MANTA in several company teams to help with process automation. All processes and reports were delivered by MANTA in a matter of hours, replacing the months of manual work done in the past. They really took advantage of MANTA’s broad spectrum of connectors, because they had multiple data governance solutions in their company for different tasks. The developers used MANTA’s native visualization to track transformations in their data warehouse; the financial department used MANTA in combination with Collibra’s solution; another team used MANTA together with IBM Information Governance Catalog to calculate impact analyses; etc. With MANTA, the options are really unlimited.

Need MANTA for a new connector? Visit our Tech Hub! Don’t see what you are looking for? Write to us at manta@getmanta.com and find out where it is on our roadmap. 

MANTA Cases #4: Patient Zero (How it all began)

Have you ever wondered how it all started back in the day? Well, as another part of our MANTA Cases series, we’ll introduce to you the very first way MANTA was ever used, “Patient Zero”.

Have you ever wondered how it all started back in the day? Well, as another part of our MANTA Cases series, we’ll introduce to you the very first way MANTA was ever used, “Patient Zero”.

Patient Zero of MANTA was a member of the Société Générale Group, the Czech bank Komercni Banka. Back then, MANTA was no more than an internal tool of a Czech consultancy company, Profinit, which was hired to solve a data quality issue within Komercni Banka.

The problem was that some parts of Komercni Banka’s data warehouse had different coding because of special Czech diacritics that were more space-consuming than regular letters and symbols. This was creating a lot of mess throughout the piles of custom code. The customer was using a Teradata database, whose price varies according to the amount of data stored. As Komercni Banka had to store a phenomenal amount of customer data, they were trying to store it as compactly as possible to save money on database space.

In a Teradata database, you can store data in Unicode or in ASCII. In Unicode, you can store Czech letters like “š” or “ů” or any other language-specific symbols, as it has a larger format. ASCII is able to store only classic letters and numbers, but therefore takes up less space on the database. As not all Czech words and names have diacritics, this was an opportunity to store some data in a more compact format. (This use case could work with any language that has language-specific symbols, not only Czech.)

Komercni Banka first looked into the source systems to identify all source columns in the database that might have contained Unicode and those that contained only ASCII, and with the help of MANTA, they identified where the Unicode data was propagated and where it was clear that there was only ASCII data. After that, the company used MANTA Checker, a tool that MANTA originally developed to quickly fix errors in code, to perform an automatic add conversion function to the data that could be converted from Unicode to ASCII.

The bank has estimated that thanks to the use of both MANTA and MANTA Checker, they have saved quite a lot of money on storage fees, freed up several FTEs in the organization, and have been able to both monitor their environment and develop new code more efficiently than ever before.

Want more information about MANTA or MANTA Checker? Don’t hesitate to sign up for a free trial and schedule a call using the form on the right, or directly at manta@getmanta.com

MANTA Cases #3: (Not Always) Agile Development

Many companies seek to achieve an agile development strategy. Sometimes it is not exactly needed and might bring some difficulties (as our CEO Tomas Kratky already mentioned in one of his older articles here).  But sometimes an agile-type strategy naturally evolves in the development department. In the third part of our MANTA Cases series, we will take a look at how one of our customers uses MANTA in a (not agile) development team. 

Many companies seek to achieve an agile development strategy. Sometimes it is not exactly needed and might bring some difficulties (as our CEO Tomas Kratky already mentioned in one of his older articles here).  But sometimes an agile-type strategy naturally evolves in the development department. In the third part of our MANTA Cases series, we will take a look at how one of our customers uses MANTA in a (not agile) development team. 

One Too Many Patches

One of our customers, an international bank, was using a workshop-type software to release internal production software. Because this internal database software is so complex, they have 10 to 15 patches of code each day between releases. To make sure that the environment doesn’t crash, they have to deploy these patches of code all at once. Each patch of code is created by a different development team, and the individual teams are not aware of the dependencies between their parts of the code. Because of this, the patches were often deployed in the wrong order causing the entire environment to crash.

These crashes often made the deployment process last more than one day, with numerous other patches needing to be developed to fix the problems created while deploying the previous patches.

Understanding the Environment

The patches of code the development team wrote were all improvements to their internal database system (e.g., changing database logic, repairing certain objects, even adding new columns or creating more tables and views). Such patches resulted in changes in paths or even in values within the environment. And given the size of our customer’s database – which contained multiple banking systems, each having an average of 180 thousand scripts – and the amount of data being added to the database each day, it was necessary to perform the 10-15 patches daily.

The only way the customer could keep up with the (not) agile development pace and effectively find the dependencies between the patches was to apply automated software like MANTA.

Post-MANTA

Now that the customer has MANTA on their development team, they take all the patches that need to be deployed that day and let MANTA run them. MANTA draws out all the dependencies between the individual objects that the patches are related to. Then the customer exports them using MANTA’s API and has an automatically sorted order that it needs to deploy the patches in so that the environment will survive. This has cut the entire deployment down to a couple of minutes and has also made it a one-person job.

This one person now only needs to:

  • Gather the patches
  • Upload them into MANTA
  • Use MANTA to determine the order of deployment
  • Order the scripts accordingly
  • Release them in that order
  • Done!

Conclusion

Now, the customer can be sure that the patches will be deployed correctly and that it will be done in just a couple of minutes, making the daily-release strategy actually work out every day.

MANTA has been an enormous help to their situation. And well, what can we say: this case is proof that not every development team chooses to be agile. Some just have to adjust their development pace to the complexity of their environment.

Be sure to read more of our MANTA Cases! Any requests? Any questions? Send your comments to manta@getmanta.com.

MANTA Cases #2: Pure Development Efficiency

In the second part of the “MANTA Cases” series, we will take a look at a US investment management company that deployed MANTA, primarily for their development team to perform impact analyses and to boost the overall efficiency and agile development processes of their internal software. 

In the second part of the “MANTA Cases” series, we will take a look at a US investment management company that deployed MANTA, primarily for their development team to perform impact analyses and to boost the overall efficiency and agile development processes of their internal software. 

The Customer had a fairly small yet, for its size, very complex data environment. The environment consisted of three databases:

  1. One was used as the main data warehouse that collected data for the core business
  2. The second database was a special investment data mart database (this database had specific requirements, making it even more challenging for it to co-op with new releases)
  3. The third database contained resources for the sales team

The entire environment contained about 15 hundred scripts, of which 20 to 30 percent were legacy code. The other 70 to 80 percent were newer code that was evolving at a fast pace due to the customer’s agile development standards. The customer used Microsoft SQL, most of the ETL code was in T-SQL, and some scripts were in SSAS.

Given the user requirements for each of the separate databases, the environment was growing and its expansion was necessary – otherwise, it would have become a burden. And, as is always the case in development teams, that familiar question was floating around in everyone’s heads: What’s gonna break if we make changes? The customer had developed an automated testing process for new code involving Unitask software, but for the legacy code, there was nothing like that. This made it necessary to deploy a tool that would define the dependencies between various parts of the code.

Now that MANTA has become part of the customer’s development process, MANTA can show the development team the impact a new release will have on existing code, making it easier to catch most of the larger issues during the testing phase. The issues the customer faces after a release have been, in their words, reduced by half and are in general much smaller.

Another benefit of MANTA for the development team is that the overall testing time before a new software release has been cut by 10 to 20 percent, and the number of developers needed to perform a release is now 2 people!

In this case, MANTA helped the customer maintain an agile development strategy for their internal software and improve the overall performance of impact analyses. However, due to the benefits that came with it, such as the reduction in overall testing time and the number of people needed to perform a release, MANTA could easily be beneficial for companies having trouble with staffing as well as for other small development and business analytics teams.

Do you have any questions about this specific case, or would you like to learn more about how to deploy MANTA in your company? Write us at manta@getmanta.com.

MANTA Cases #1 (Pilot): Market Data Tracking

We have created a new series for you on our blog: MANTA Cases. Real-life examples of how to use MANTA are what interests our readers and prospective customers the most, therefore we have decided to periodically publish some of the most interesting and creative cases on our blog. You will be able to find the entire series in the category called “Use Cases & Case Studies” on the right side of the page. Enjoy!

We have created a new series for you on our blog: MANTA Cases. Real-life examples of how to use MANTA are what interests our readers and prospective customers the most, therefore we have decided to periodically publish some of the most interesting and creative cases on our blog. You will be able to find the entire series in the category called “Use Cases & Case Studies” on the right side of the page. Enjoy!

The first part of our series is dedicated to using MANTA for market data tracking. This interesting way of using MANTA was thought up by a company that provides analysis and reporting services to their customers. Such companies handle large amounts of data. They themselves also need to buy enough data for their analyses and reports from other companies, e.g. market data from Bloomberg and others.

The problem, in this case, was not a matter of personal data, therefore they did not need to comply with GDPR. (Compliance projects are, by the way, one of the most common uses of MANTA.) This was data such as past and current prices of shares, market research data, and other data that has a different kind of “sensitivity”.

Here, there were two main problems with the sensitivity of the data:

  1. When the company buys market data from third parties, they often need to assure the seller that the data is being handled in a safe way, with minimum risk of leaking, and it usually must be used according to the terms of a license agreement for data use.
  2. The second thing to look out for in relation to this data is that such data is usually priced according to the number of users who have access to it.

Based on these two points, MANTA had to help the customer with the need to comply with the licensing agreements, to be able to prove how and where the data is being stored; and to be able to prove the number of users, to monitor the profiles of the users working with the data.

How does MANTA solve these problems?

MANTA documents what market data sets are used in which individual reports. When combined with access privileges to the individual reports, the company has a clear and documented understanding of which data set is used by how many end-users, and they pay for the particular data set accordingly. Moreover, with people coming and leaving and any changes happening in the actual data use, the always up-to-date lineage gives them exact numbers at any point in time.

Do you have any questions about how MANTA can solve a problem at your company? Contact us at manta@getmanta.com or use the cute little bot on the right. 

We cherish your privacy.

And we need to tell you that this site uses cookies. Learn more in our Privacy Policy.