You’re on a road trip with your buddies and your iPod is the one churning out the tunes, when a Captain and Tenille song comes on.  The guy riding shot gun shoots you a look and skips to the next song.  After that one finishes, there they are again, “Do that to Me One More Time” is killing the good vibes in the car. Skip.  Then, of all things, “Muskrat Love” comes on? “Dude, how much Captain and Tennille do you have on your iPod?”  Turns out you only have a greatest hits album and you have something like 8 days worth of other music on that Jobsian box.  Clearly the iPod can’t generate a random playlist worth a damn.

Noise, chance, whatever you want to call it.  Often times people misunderstand what randomness actually is.   Randomness is a sequence of things such that there is no intelligible pattern or combination.  The problem is we expect that random means that things will be equally spaced out or that there will be “no coincidences.”    In fact, one of the ways you know your are looking at randomness is that there will be repetition or related items in the list.

One of the most common experiments is examining runs of heads or tails in coin flips.  If you ask people to make a list of 100 coin flips to simulate flipping a coin 100 times, they will often rarely put in runs of more than about 3 heads in a row or 3 tails in a row, because they perceive more than that to be too rare of an event to actually occur if they were really flipping a coin.  In reality, a run of 5 or more heads or  tails in a row in 100 coin flips isn’t  unusual.  Therefore, it is easy for people who understand this to identify a list of real flips versus a person writing Hs and Ts down on a sheet of paper.  Here is an example.

As with many statistical properties, we can see examples of this phenomenon in sportz.  Many people assume that, say, scoring baskets in basketball are not independent events.  That is, we think that once a player scores a couple in a row that he is more likely to make the next one (he’s on fire!).  Statistics don’t indicate that this is the case.  If a player makes 60% of his shots and shoots thousands of time, you expect several runs of makes and misses, some of them quite long.  If anything players have shorter runs of baskets and misses than we’d expect, probably because they take shots of different difficulties (some close and some far away) throughout a game.

Another sportz analogy is that of hitting streaks in baseball, like those of Joe DiMaggio and Pete Rose.  Of these two, Pete Rose’s streak (hits in 44 consecutive games) is actually more unusual than Joe DiMaggio’s (56 games) given the player’s respective batting averages. But given all the players in MLB (somewhere around 1000) and the number of games played each season (162), we expect that someone will hit many games in a row every so often (presumably someone with a high batting average).

Now, back to your embarrassing road trip situation.  Apple claims that its shuffle feature is “refreshingly random,”  contrary to the Captain and Tenille  incident of 2010.  This has been questioned numerous times (including by people in my own lab).  Supposedly, the iTunes shuffle feature works by shuffling your songs the way cards get shuffled in a deck of cards.  So, if you were going to listen to all your songs in one sitting, no song would repeat until you had been through all of them.  However, pretty much no one does this.  So when you stop and start again the deck is reshuffled and some of the songs you heard before might play again.  But really, this is expected in a random shuffling routine.

Here is one study that examined the shuffle feature and has come up with a conspiracy theory involving the music labels.  They expect that every song should be in their playlists an even number of times.  Notice that they are effectively reshuffling the deck every time they make a new playlist and that the playlists are short, allowing the effects of randomness to creep in.  This is a good simulation of how the average user probably experiences the results of the shuffle feature (unfortunately, like you and your road trip buddies)  A better test was done here.  These guys let the shuffle feature run for hours.  They find extremely even results over their library, with every song being played between 24 and 26 times. When large numbers are involved randomness does approximate equal spacing.

So your iPod probably really is random (or as close to random as you can get), you’re just unlucky.  Don’t worry, next time it will be Celine Dion.

The two sportz examples I used in this post as well as other real world stats illustrations can be found in a book by John Allen Paulos, Innumeracy: mathematical illiteracy and its consequences.