Saturday, October 24, 2015

Evolution of the Order of Operations Diagram

[Update: There are a few new updates on this topic in the comment section.]

Initially Tableau published this diagram on the Order of Operations in a white paper.
Last month (September 2015), I updated the diagram to include Set Filters and called it the Precedence of Filtering. Note that I make the pipeline get narrower when going down, because all but the last filters are reducing the data set at the record level. The data set is getting smaller and smaller.

The context filters are actually regular filters that we have manually assigned higher priority in execution. On the other hand, all filters provide context for the filters that follow.
At Tableau Conference 2015 this month in Las Vegas, Jen Vasser, a sales consultant at Tableau presented her version of the order of operations or the Query Pipeline. She included:
  • Data Blending 
  • Forecast
  • Annotations, Reference Lines, Trend Lines, Tooltips, Pages, Titles
I was very happy to have learnt a few things from her excellent session filled with examples. Moreover, I found someone who cares as much about the order of operations! It is very important for viz design. It took me a while to understand and master it.

Now I try to integrate some of Jen's elements into a new diagram as follows. Also added primary and secondary data paths. Context filters can only be applied to the primary.

I couldn't find my original PowerPoint file in which I drew my first Precedence of Filtering diagram. And I found that using PPT to draw is a bit tedious. It's not easy to align text and may not be easy to update.

So I decided to draw the diagram in Tableau. And voila the result. Tableau can easily handle the drawing and labeling as well as tooltips. I no longer needed to struggle to align the text here and there. The diagram is based on Gantt chart.

Hope this new diagram helps you get a better understanding of Tableau's inner working.

at

9 comments:

  1. Air_Cooled_NutOctober 25, 2015 at 8:29 AM

    I took a picture of this and was going to post it, too. Thanks for getting it out to the community!

    ReplyDelete
  2. UnknownOctober 26, 2015 at 11:20 AM

    Alex, thanks for posting this! The order of operations is extremely useful in understanding how Tableau gets to an end result (especially if the end result doesn't match expectations). Something doesn't quite seem right, though. For one thing, filtering based on dimensions doesn't happen post-blending. It happens for each data source (primary and secondary/ies) individually before blending. Based on yours, I put this diagram together. It may not be entirely correct (so I'm open to corrections) and definitely simplifies the pipeline (I know there are likely exceptions), but it corresponds to my current understanding of the order of operations. See you what you think:

    http://vizpainter.com/wp-content/uploads/Order-of-Operations.png

    ReplyDelete
    Replies
    1. Alexander MouOctober 26, 2015 at 10:50 PM

      Joshua, thanks for the reply. Filtering on the secondary data source is always a challenge for me. The diagram above is mostly about filtering the primary. Could you come up with some examples? This reply in blog is not good for discussions because we can't attach pictures or screenshots etc. Let's move to the forum where more people may jump in. http://community.tableau.com/thread/192365

      Delete
  3. Alexander MouNovember 10, 2015 at 2:23 AM

    The diagram is updated http://vizdiff.blogspot.com/2015/11/update-of-order-of-operations-diagram.html

    ReplyDelete
  4. Kelly HottaNovember 24, 2015 at 5:20 PM

    Hi Alex! Here's the version we used at TC15: https://www.dropbox.com/s/l3m8s2lu7a6dzji/Pipeline.png

    Glad you enjoyed the session :)

    ReplyDelete
    Replies
    1. Alexander MouDecember 3, 2015 at 1:45 AM

      Thanks Kelly!
      A couple more updates since:
      http://vizdiff.blogspot.com/2015/12/the-order-of-operations-within.html
      http://vizdiff.blogspot.com/2015/11/update-of-order-of-operations-diagram.html

      Delete
  5. Alexander MouDecember 13, 2015 at 12:20 AM

    The order of operations diagram is updated again on 12/10/2015. The change is to make dimension filters take precedence over data blending.
    http://vizdiff.blogspot.com/2015/12/the-order-of-operations-diagram-update.html

    ReplyDelete
  6. DaveRWNovember 4, 2016 at 3:29 PM

    This comment has been removed by the author.

    ReplyDelete
    Replies
    1. Alexander MouNovember 4, 2016 at 4:05 PM

      Those extract filters, data source filters, or even dimension filters are before row level calculations. Those filters are excluding the data not relevant to the view. Then row level calculations start.

      Delete