Monitoring Data Source Delay in Business Dashboard

In building a living or production business dashboard, we always expect data to be up to date. This is a concern that can not be relieved by soly knowing the refresh time stamp (tableau provides this information). It doesn't mean that the data is up to date. So we decide to create our own data monitor.

The main business data sources are time series. We have transaction data usually up to the second, minus the delay in data preparation pipeline. In an automated environment, we expect the data to be available within one day. Beyond one day, something might go wrong somewhere in the data pipeline. Traffic jam may happen as well as server mishap. The monitor alerts us when something goes wrong and help us to diagnose and fix the data pipeline so that it will stabilize.

Assume we have 3 data sources and all of them are time series and are refreshed every day. We need to know how each of them are up to, relative to today's date. Here is how we designed our data monitor.

1.Create a table of 3 rows with a single column: Source ID

2.Associate each ID with a Data Source name

3.Get the Source Max Date of each data source

Note that we don't need to blend with data in secondary data sources. Just drag date dimension over. (Ignore any Tableau alert that reminds you to blend.)

4.Create a bar chart showing delay of each data source relative to today. We assume "Today" is 12/31/2020 in the chart. (Today label is an annotation in Tableau.) Replace it with Today() in your production business dashboard.

5.Calculate the Max Delay of all the data sources.

6.Create a button sheet and place it high in dashboard. Add the delay bar chart to viz-in-tooltips. The button also shows the max delay of all in days.

7.Create a viz alert on condition where the delay is -2 days or more. You may set your own criteria. For this purpose you need to create a collapsible container button. Place the delay bar chart inside the container. Use the horizontal axis to create the alert.

The workbook can be downloaded here, where you can find more details. Let me know if this is useful to you.