The parable of the histogram

I must be some kind of heretic. I"m a statistician, and here I am pointing out the problems in yet another common statistical tool.

We"ll see how the histogram, which is a very popular way of displaying the distributional shape of a set of data, must be viewed with a good deal of caution.

Even though histograms are often found in the media, the problems with histograms are almost unknown among the general public. Indeed, most places that teach statistics at university completely fail to mention them.

I"d like to say that the problems are well known among professional statisticians, but that might be too strong. Certainly problems have been pointed out in the literature, and many statisticians are aware of the problems, but it seems many still are not, and the appropriate cautions are not always explained.

I"m going to show you a simple example.

Here"s some data (40 observations in this sample), which I"m going to draw a histogram of. I have rounded the numbers off to two decimal places:

3.15 2.28 2.06 3.43 4.85 3.22 4.01 4.43 
5.46 3.12 5.53 5.50 5.56 5.52 5.31 4.96 
3.28 4.10 5.19 2.54 1.89 1.84 2.56 1.90 
4.20 3.42 2.39 3.64 4.84 4.31 5.11 5.60 
1.98 3.91 1.88 4.33 5.75 2.00 2.58 1.92

I give the numbers so you can (if you are so inclined) confirm for yourself what I will tell you in my little parable.

The parable

This data set was given to a student, Annie. She constructs her histogram of the data by counting the number of values between 0 and 1 (but not including exactly 1), between 1 and 2, and so on, and then drawing a series of boxes each of whose base covers the subset of values that the count came from and whose height is the count for that range of values. Annie"s histogram is shown in the top-left of the picture below.

She obtains a histogram whose shape corresponds to a distribution that is skewed to the left. See, for example, this description of using histograms to assess distributional shape here; while it"s basically a MATLAB tutorial, they give a pretty good basic idea of the way people generally read ("eyeball" is how the author puts it) histograms - it"s pretty much precisely the way many elementary books on statistics describe the way to assess the shape of a distribution (and usually it"s going to give you the right sort of impression).

Note that I could remove the scale and I could still describe the shape - I don"t need to know the numbers on the scale in order to arrive at my description.

Three of Annie"s friends, Brian, Chris and Zoe (Hah! Psych!) also get data sets with 40 observations, and they all do exactly as Annie did. Their histograms are given below (Annie"s data is V1, Brian"s is V2 and so on).

(click pic for a larger image)

Correspondingly, Brian describes his distribution as symmetric (and he might add "uniform"). Chris describes his as skewed to the right. Zoe describes hers as symmetric and bimodal (it has two main peaks).

So far so good - this is exactly how the books tell you it all works.

So while they"re comparing their histograms, Annie idly starts looking at Brian"s actual numbers. She realizes something odd is going on. She quickly places all their data sets side-by-side.

"Look, Chris!" Annie says, "all Brian"s values are smaller than mine by 0.25. All yours are a quarter smaller than Brian"s, and Zoe"s are a quarter smaller than yours!"

They all confirm that she is correct - each set of values is the same, but with its origin merely shifted a little. Their data sets are identical in shape, but the resulting histograms are not.

That is to say, assessment of distributional shape in histograms can be dramatically affected by choice of scale (specifically, by the choice of the origin and width of the histgram bins). Here ends the parable.

It usually isn"t this dramatic, of course, but the fact is, if I can generate (in a matter of a few minutes of thinking, with a little help from Excel"s RAND() function) a seemingly innocuous set of numbers whose histogram will look completely different (and for which many people will assert the distributional shape is completely different) every time I merely add or subtract a quarter, it can happen with real data too. And it does happen. Mostly the difference in impression is more modest... but not always.

So if you see a histogram, just keep in the back of your mind that it"s perfectly possible that a different choice of bin boundaries would yield a somewhat different impression of the data.

Imagine I want to show some students that I write "easy" tests (I don"t know why this should be such an object of fascination for students since they all do the same test, and marks are generally scaled, but it is). In preparation, I draw a histogram and it turns out to like Chris" - it looks like most students score below the middle of the range of marks. But lo, I discover with a bit of fiddling around that if I make my bin centres where the edges were (and so on), the completely opposite impression is given - just like Annie"s histogram. Yay, "easy test" ... and many fewer worried queries from students in the run-up to the test, because they tend to feel there"s a good chance of scoring "above the middle".

Did I lie? No. Did I fudge the data? Well, no. I did something, though. Or rather, I didn"t do something.

This is a sin of omission. I fail to explain what the data would have looked like given a different choice of bin location.

Clearly, when circumstances are right, the ability to choose the location and width of the bins can give us the opportunity to somewhat alter the impression given by a histogram. Without fudging the numbers themselves, we can sometimes fudge the impression they give.

What do statisticians do? Well, there are other ways to look at distributional shape. Kernel density estimates are popular, and they completely get rid of the "bin-location" issue, though there"s still the equivalent of a "bin-width" issue (choice of bandwidth, also called the "window"), which is often dealt with by looking at more than one choice of width (usually a width that gives a nice smooth result and then one that is smaller, giving a "rougher" result, in order that we can see there"s nothing unsual hiding away - like the blue and green curves in the graph at top left right** at the wikipedia link a few lines up). But there are a variety of other tools that might be used (which I don"t plan on going into here).

**(did I ever mention that I have trouble with correctly attributing the words "left" and "right"? - well as you see, sometimes I do. But not when describing the shape of a distribution, isn"t that odd?)


What can you do? Well, assuming you don"t have anything more sophisticated that a basic histogram tool, at the least (with continuous data, anyway), try shifting your bin starts forward or back a fraction of a bin-width (if you"re lazy, maybe try something near a half, otherwise maybe try a couple of values). Also try a narrower bin width. If you do a few different histograms that all give the same general impression, it doesn"t matter much which one you use. And if they don"t give the same impression, you better either say so, show more than one, or find some other way to convey the information.

[Or you can do a kernel density estimate readily enough - many packages (including some free ones) will do them; there are pages online that can draw them if you just paste in some data. Implementing a kernel density estimate of your own is fairly straightforward - you can compute one in a spreadhseet easily enough - if anything, it"s probably slightly simpler to compute one than it is to compute counts for a histogram, which is in itself pretty straightforward. ]


Caveat Emptor