How To Win A Medal (Or Come Kinda Close)

Another interesting data-centric post appeared over at NCCSEF, and when it comes to data, I just can’t help myself but comment.

This time we’ve got some slides that seem to be trying to draw a relationship between WJC results and winning a medal at (I believe) the Vancouver Olympic Games. We’re only shown the (partial) results for six skiers, so I’m not sure what exactly the lesson is supposed to be.

We seem to be mixing sprint and distance results together as an indicator for future success. That seems strange to me, but I’m certainly not an expert in that sort of thing. We’ve also selected a curiously successful subset of Olympic medalists to examine. Absent is Pietro Piller Cottrer, who’s best (and only) result at WJC was 32nd (admittedly, a long time ago). Also missing is Aino-Kaisa Saarinen who’s WJC results were 15th and 23rd. How about Tobias Angerer (WJC: 18th, 26th, 28th)? On the other hand, we are shown Marcus Hellner, who’s WJC results were good but not spectacular: 15th and 21st.

The further information provided at the bottom regarding time to an athlete’s first podium also contains mostly skiers who achieved this feat fairly young, but then also two who did not (Gaillard and Rickardsson).

What am I to learn from this? That the right path is to podium at WJC (Northug), except when it isn’t (Bjørgen, Haag)? That the right path is to be successful early in your 20’s on the WC (Northug, Harvey), except when it isn’t (Gaillard, Rickardsson)?

When I read stuff like this, I’m left feeling mostly confused, like I’ve been presented a bunch of data, but that no one has gone to the trouble to transform this data into information. The reader is left alone, drifting in a sea of numbers, wondering what exactly was the author’s point.

I’m absolutely not going to argue with the idea that skiers who show considerable promise early on are more likely to develop into successful WC skiers. Indeed, I’m less interested in the nuts and bolts of what results mean at a given age than I am in effective and clear presentation of data.

I’ve written about connections between WJC results and medal on another occasion and I tried to emphasize the fact that when you look at all the data, there’s certainly a connection, but the different paths that skiers take toward success can vary so much that it’s difficult to create many useful generalizations just from the data.

But let’s revisit this idea with a few simple approaches and see if we can organize the data in a way that’s informative (and maybe interesting too!). First, I’m going to broaden the scope from medals to top ten results at either Olympics or World Championships. The problem with looking only at medalists is that there are just too few of them. Much can be learned by imitating a single good skier, but there’s always the danger that what worked for them only worked because of something unique about them, rather than having stumbled across some universal truth of skiing.

Let’s tackle the connection between WJC results and whether or not someone achieves a top ten result at the Olympics or World Championships. I fit a simple model (actually, not so simple; no OLS regressions here!) and plotted the model’s predictions for the probability of a top ten result at a major championship based on that athlete’s best result at WJC (sprint or distance): Read more

Do More Consistent Skiers Ski Faster?

In a word, no.  But the relationship between consistency and speed is a little subtle.  To look at this question let’s take the distance results from major international competitions (OWG, WSC or World Cups) and restrict ourselves to those times when an athlete did at least ten races in a particular season.  Then for each season we’ll calculate a how variable their results were and also the average of their best five races. Read more

Andrew Newell Sprint Qualification Redux

So there was this post a while back where I looked at the oft repeated CW that Andrew Newell tends to qualify rather fast and then slip back a bit in the heats.  My goal was just to see whether and to what degree this actually happens.  It turns out that my attempt to tackle this topic is slightly cursed because I keep messing it up.

The first time around I mistakenly mixed sprint races with 16 and 30 people moving on into the heats.  More recently, Nat Herz reminded me of something that should have been obvious from the start: people who qualify well are more likely to move backwards simply because there are only so many spots available to them to move up.

For example, if you qualify 3rd, in some respects you more likely to do worse in the heat simply because there’s more room behind you than in front of you.  To give you a sense for how right Nat was about this effect, check out this plot:

As you can see, it’s much more common for people who qualify well (i.e. top 5-10) to end up losing places, and vice versa for people who qualify slowly.  However, the blue trend lines probably aren’t as steep as you might expect.  This is because there’s another general trend working in the opposite direction.  Skiers who qualify well do actually tend, on average, to be faster skiers.  So it’s quite common for them to do as well or better in the finals.  You can see this tendency in the plot by looking at the size of the points, which indicate how often a particular combination occurred.

How do we fix this?  Simple: rather than looking at the difference between each skier’s qualification and final places, we instead look at how that difference compares to the median difference over all skiers who qualified at that rank.  For instance, say I qualified 5th and then ended up in 9th after the heats.  That’s a change of 5-9 = -4.  We then look at all the people who qualified 5th and calculate the median change across all these skiers, suppose it’s -2.  That means that I moved back two more places than is “typical” for people who qualified 5th.

Here are plots focusing on the men and women who’ve done at least 15 WC sprint races over the past 4 years or so, complete with error bars: Read more

Technique Preferences: Norway

The third in the series, looking at differences in performance in skating vs. classic races for particular nations.  We’ve already done Japan and Italy.  Now, by request, we have Norway.

To avoid repeating myself, I’m going to skip much of the introduction and explanation for the methodology and graphs.  Go read the Italy post if you need a refresher.

Distance

First we’ll look at the estimates for each individual skier over their entire career:

There are a ton more skiers here, even after filtering out those with fewer than 2 skating and classic races, so I’m sorry that the names are a little hard to read.  As always, negative values (on the x axis) indicated better performance in skating races relative to the median WC skier.

As we might have expected, most Norwegians shown here are nominally better at classic skiing (positive value) and up near the top there are quite a few that even appear statistically significant (error bar doesn’t overlap zero).  Good old Odd-Bjoern, hanging out up there!

Unlike with Italy, we don’t see much of a difference between the men and women.

Here’s how things have changed over time for the entire team:

As before, there are some years that look unusual: 1996-1997 Women, 2007-2008 Men+Women, maybe 2004-2005 Men.  In a more “serious” analysis we’d go back and check those out to make sure our model isn’t doing something funky.  Perhaps I’ll get to that sometime…for now, though, I wouldn’t necessarily leap to the conclusion that the Norwegian women were suddenly way worse at skating (or better at classic) in 1996-1997 and then returned to normal the next year.  I’d be similarly cautious about interpreting 2007-2008.

And of course, the 2010-2011 estimate is preliminary.

Overall, though, we’re seeing a mild preference for classic (positive values) with the men that’s been mostly stable since the mid-90’s.  The women have seen more of an up and down, but only if we ignore the “problem seasons” of 1996-1997 and 2007-2008.  Ignoring those aberrations it looks like the Norwegian women went from preferring classic to being roughly even and then really preferring classic and then finally coming back down toward being roughly even in recent years. Read more

Technique Preferences: Italy

My post on the differing performance of Japanese skiers by technique (classic vs. freestyle) got a lot of positive responses and a few requests that I use the same methods on some other countries.  First up is Italy.

I’ve tweaked and refined my model a fair bit, hopefully for the better.  The basic idea is the same: using a hierarchical linear model to estimate differences in performance in skating and classic races (I’m omitting pursuits of all varieties).  There are some technical things I’ve changed to be able to accomodate changes over time.  Mostly this means making some adjustments for the occasional small sample sizes you find from season to season.  This allows me to provide an estimate even in seasons where a skier did races of only one technique, although naturally those estimates come with a bit of a grain of salt.

In the results by athlete for their entire career, I’m only going to display information on only those athletes that did a minimum number of races of each technique (2) for space and clarity reasons.

The final big change is in the distance category.  There are some technical reasons why FIS points are somewhat of a nuisance to use as a response variable in models like these, so I’m using something else: percent back from the median skier.  I’ll save a more detailed description for why I’m doing this and how this measure is useful for another post.  Here all we need to know is that 0% back represents the median (or middle) WC skier.  Negative values mean you’re faster and positive values mean you’re slower.

Going into this, our intuitive notion is that the Italians have been generally better at skating.  And that does turn out to be the case.  But some other fascinating stuff pops up as well. Read more

Do The Japanese Prefer Classic Skiing?

In my race recap for the Davos distance race I noted the strong performance of Masako Ishida, and pointed out what an extreme classic specialist she is.  A certain world famous XC skiing journalist wanted to know if the conventional wisdom he’d heard was correct, that Japan’s skiers typically do better in classic skiing overall.

This was particularly fun to tackle, since it turned out to be a situation where simply graphing the data in a clever way wasn’t enough.  We actually have to do statistics!  Woo hoo!  Don’t worry, though, while the techniques I ended up using for this analysis are fairly sophisticated, the results are pretty easy to understand and explain.

I’ll start with my first pass:

My first thought when approaching this kind of question is always to simply throw the data up on a graph and see what I can see.  So this is all of Japan’s WC, WSC and OWG results back to 1992.  Note that I’ve plotted rank, not FIS points, to keep the distance and sprint panels on the same scale.

The classic preference is clear in recent years on the women’s side, but since we already know that Masako Ishida has a monster proclivity for classic skiing, this could just be due to her results.  And other than the recent results for the women, it’s tough to make out any obvious patterns.  That’s because this graph treats all of Japan’s results from every athlete as a single group.  But clearly different skiers will have different abilities in skating vs. classic.  So we need a way to look at each individual skier’s races.  The problem is that there are more than 20 Japanese skiers (in my database at least) with a fair number of WC starts.  Can you imagine looking at a similar graph but with more than 40 panels (distance and sprint for each athlete)?  That wouldn’t be very illuminating, I think.

One option would be to artificially limit myself to a small number of skiers, but then our answer would apply only to those skiers we picked, not all of Japan’s skiers.  If we really want a good answer to this question we really need to include as much data as possible.

The solution is to use a model (gasp!).  In particular, a hierarchical linear model.  I’m not going to bore people with a detailed description of how this worked; if you’re really curious ask questions in the comments.  The bottom line is that this tool allows me to estimate the difference in results performance both overall and for each skier individually at the same time (by doing both at the same time, it often does a better job at each).

I probably could have squeezed this all into a single model, but I decided it would be easier to explain to folks if I modelled sprint and distance races separately.  That also allows me to use FIS points as a measure for distance races and rank for sprinting, which makes somewhat more sense anyway.

In distance races Japanese skiers (men and women) tend to ski about 3.49 FIS points slower in freestyle races (95% CI -1.75,8.73).  That little parenthetical just now meant that the 95% confidence interval for this effect ranges from -1.75 FIS points to 8.73 FIS points.  Since this interval includes zero, we would typically say that this does not meet the threshold for “statistical significance”, meaning that we can’t say with much confidence that the real difference isn’t actually zero.  Also, 3.49 FIS points is not a very large difference in practical terms.

But remember that this fancy-shmancy model I’m using doesn’t just estimate the overall effect, it also estimates this difference for each individual skier.  The following graph displays the results, along with their associated 95% confidence intervals: Read more

World Cup Survival Analysis

When most people say that World Cup skiers are animals, they probably mean they are fierce, strong competitors.  I got my PhD in statistics in a department that found itself working quite often with very strong wildlife biology and ecology departments, so for me that reference leads me to think, “Well, what if they really were animals?  What sorts of statistics might I end up doing on these data?”

A common statistical analysis when your subjects actually are wild animals is called survival analysis.  Very generally, the aim is to determine what factors influence the survival of, say, bears1.  The poor biologist would spend countless hours over multiple summers capturing, tagging and then tracking and recapturing the shrews or slugs or whatever2.

The end result would be a bunch of lifetime data (along with other variables) on individual organisms.  Then the question is, which variables seem to influence survival rates?  There are all sorts of technical details with this kind of data (censoring, mainly) on how to model it that I’m not going to get into here.  If some nerdy biologist is reading this and wants more details, let me know, and I’ll put them in the comments.

Read more

  1. Usually, the organism isn’t nearly this exciting.  Typically I’d see data on something like the western spotted shrew, or the golden mantled ground squirrel.  One of those animals I made up, the other I did not.
  2. Stats grad students would frequently talk about how grateful we were that we didn’t have to do field work.

Next Page »