Coloring Logarithmically for Optimal Human Perception

A recent post by James Eichinger inquired about the best practices of manipulating color values because his viz is swamped by high intensity colors. A solution is mentioned by David Walp and Joshua Miligan as logarithmic transformation:

• Use natural logarithm LN([Measure]) for coloring, instead of [Measure].

Joshua even had a great article about its use in mapping.

Here is an example I produced using the Superstore data to visualize sales by city. You can notice the stark difference between the two coloring schemes. The workbook can be found here.

This reminds me of the human perception theory that we see things in a logarithmic way. The earliest scientific discovery regarding this topic is by Greek astronomer Hipparchus, some 2000 years ago when studying the relationship between star brightness and human perception. In the 19th century, a formal theory is created as Weber-Fechner Law.

From Wikipedia:

"The law states that the just-noticeable difference between two stimuli, is proportional to the magnitude of the stimuli, (and the subject's sensitivity), i.e. if you sense a change in weight of .5 lbs on a 5 pound dumbbell, you ought to feel the extra pound added to a 10 pound dumbbell."

That is, the human sensation/perception difference ∆P to weight W or to any external stimulus is proportional to ∆W/W instead of to ∆W, namely,

• ∆P=K(∆W/W)

Through integration we get

• P=K*LnW + C

K and C being constants.

There are more recent articles on this ever-fascinating topic in Quora, Huffington Post and IEEE Spectrum.

In star gazing, the stellar brightness is attenuated logarithmically by the sheer distance. It's the same for sound during its propagation. For data visualization, it happens on computer screen which is very close to our eyes. The force of nature does very little to attenuate the screen light. My postulation in this case is that we see light intensity as it is, much in a linear way, rather than logarithmic. To mimic the force of nature for the best human perception, we can mathematically transform the color intensity by ln().

It applies best when there is a wide range of color intensities, in which case low intensity colors tend to be obscured by high intensity counterparts if we linearly scale the color. By applying the logarithmic transformation, we curbed the high intensity and see more low intensity colors. It gives us more insights into the shade, at the expense of non-differentiation of high intensity colors.

It is a great tool with limitations. My suggestion is, use it with care and without misleading viewers by labeling clearly.

BTW, I proposed an idea to Tableau to include logarithmic transformation as an option in color editor. Please vote it up if you like it.

PS. Perception is everything, except being linear. It is logarithmic.