Control Damage Caused by Incorrect Data in Teradata DWH

June 19, 2015 by
Control Damage Caused by Incorrect Data in Teradata DWH

Imagine your data warehouse has been hit by invalid input which has messed up your reports. The business users are angry, and the business intelligence department is getting ready for a couple weeks of emergency operations.

Control Damage Caused by Incorrect Data in Teradata DWH

Imagine your data warehouse has been hit by invalid input which has messed up your reports. The business users are angry, and the business intelligence department is getting ready for a couple weeks of emergency operations.

One day a financial manager came to work and found out that according to reports he had gotten from the BI department that morning, the bank had sent more than 2 trillion euros somewhere. (That’s this much money: 2,000,000,000,000, by the way.) Instead of having a heart attack, he called the BI department to ask what had happened.

They easily figured out that one of the source systems had sent a wrong amount for one of its transactions. So the financial manager could stay calm, but he insisted that he needed a corrected report as soon as possible and that there were also other reports that could be corrupted.

Finding the rotten apple in the basket

How could they find out which reports could potentially be corrupted and therefore need to be recalculated? And not only reports, they should also find all workflows and scripts transforming the source data and launch them again in the right order. Doing this manually would take a lot of time and effort – they would either rely on the potentially obsolete or inaccurate documentation or browse code in a lot of scripts – while running a whole day‘s load would stop the whole BI department.

manta_teradata_readmore_630x100

The original Manta to the rescue 

Fortunately, they had one of the first versions of Manta. (The product name Flow didn’t exist back then.) So, how to fix this problem, then? They had the name of the faulty extract from the source system, and they also knew the precise column with the erroneous value. But they needed to find all scripts and workflows that use or transform that value and determine what order they should be executed in, so every value in the data warehouse that depended on the erroneous value would be recalculated. Using the right settings they were able to investigate how the data from the erroneous column was propagated throughout the data warehouse and visualize it.

muck

With Manta Flow they reduced the time spent controlling data warehouse damage caused by incorrect data by more than 90%.

This was one of the topics from our webinar. Keep your eye on our Twitter, so you won’t miss another one. Also, use the form on the right if you have any questions.

We cherish your privacy.

By using this site, you agree with using our cookies.