Do The Japanese Prefer Classic Skiing?

In my race recap for the Davos distance race I noted the strong performance of Masako Ishida, and pointed out what an extreme classic specialist she is.  A certain world famous XC skiing journalist wanted to know if the conventional wisdom he’d heard was correct, that Japan’s skiers typically do better in classic skiing overall.

This was particularly fun to tackle, since it turned out to be a situation where simply graphing the data in a clever way wasn’t enough.  We actually have to do statistics!  Woo hoo!  Don’t worry, though, while the techniques I ended up using for this analysis are fairly sophisticated, the results are pretty easy to understand and explain.

I’ll start with my first pass:

My first thought when approaching this kind of question is always to simply throw the data up on a graph and see what I can see.  So this is all of Japan’s WC, WSC and OWG results back to 1992.  Note that I’ve plotted rank, not FIS points, to keep the distance and sprint panels on the same scale.

The classic preference is clear in recent years on the women’s side, but since we already know that Masako Ishida has a monster proclivity for classic skiing, this could just be due to her results.  And other than the recent results for the women, it’s tough to make out any obvious patterns.  That’s because this graph treats all of Japan’s results from every athlete as a single group.  But clearly different skiers will have different abilities in skating vs. classic.  So we need a way to look at each individual skier’s races.  The problem is that there are more than 20 Japanese skiers (in my database at least) with a fair number of WC starts.  Can you imagine looking at a similar graph but with more than 40 panels (distance and sprint for each athlete)?  That wouldn’t be very illuminating, I think.

One option would be to artificially limit myself to a small number of skiers, but then our answer would apply only to those skiers we picked, not all of Japan’s skiers.  If we really want a good answer to this question we really need to include as much data as possible.

The solution is to use a model (gasp!).  In particular, a hierarchical linear model.  I’m not going to bore people with a detailed description of how this worked; if you’re really curious ask questions in the comments.  The bottom line is that this tool allows me to estimate the difference in results performance both overall and for each skier individually at the same time (by doing both at the same time, it often does a better job at each).

I probably could have squeezed this all into a single model, but I decided it would be easier to explain to folks if I modelled sprint and distance races separately.  That also allows me to use FIS points as a measure for distance races and rank for sprinting, which makes somewhat more sense anyway.

In distance races Japanese skiers (men and women) tend to ski about 3.49 FIS points slower in freestyle races (95% CI -1.75,8.73).  That little parenthetical just now meant that the 95% confidence interval for this effect ranges from -1.75 FIS points to 8.73 FIS points.  Since this interval includes zero, we would typically say that this does not meet the threshold for “statistical significance”, meaning that we can’t say with much confidence that the real difference isn’t actually zero.  Also, 3.49 FIS points is not a very large difference in practical terms.

But remember that this fancy-shmancy model I’m using doesn’t just estimate the overall effect, it also estimates this difference for each individual skier.  The following graph displays the results, along with their associated 95% confidence intervals:

The dots are the estimated effects, so positive values means that skier does worse in skating races and negative values means that skier does better.  The bars are the 95% CI’s.

This rather wonderfully captures several important things about statistics and data analysis!  First, you’ll note that here too, basically all of the confidence intervals overlap with zero, meaning that the sticklers among us will simply throw up their hands and say the results are inconclusive.

There are only two skiers that seem to conclusively prefer one technique or the other: the aforementioned Masako Ishida and Mitsuo Horigome.  But this misses an important fact, that the overwhelming majority of Japanese skiers have estimated effects larger than zero (although many of these effects are also quite small), indicating at least some preference for classic skiing.  The important lesson here is that just because your results aren’t “statistically significant” doesn’t mean we can’t learn interesting things from the data.

In this case, the fact that so many Japanese skiers have a nominal preference for classic events (despite each individual effect not being significant, or perhaps small) plausibly suggests that as a group, Japan does perform somewhat better in classic races.

To satisfy all those statistical significance sticklers out there, we have our second important lesson.  That is that XC skiing is a volatile business and so large amounts of variability are just par for the course.  That variability is what’s muddying the waters here and preventing us from getting nice clean “significant” results.  So while it may be fair to say that Japanese skiers are generally better at classic events, we should keep this variability in mind and use caution when we’re watching a specific skier or a particular race.

The trend is there, but it’s imprecise.

Ok, what about sprinting?  Same deal, but instead of FIS points, we’re just going to use rank:

Remember we’re talking about places now, not points.  So an effect estimate of 5 means that skier finishes around 5 places worse in freestyle sprints, on average.  So positive values would indicate a nominal preference for classic and vice versa.

There are more clear cut cases here, to be sure, with ~13 different skiers having a preference that is significant.  The overall effect of skating for Japanese skiers as a group is 1.11 finishing places (95% CI -2.06,4.27).  Once again, this effect appears small and is not statistically significant.

However, we still have this phenomenon of considerably more Japanese skiers preferring (albeit by small, often statistically insignificant margins) classic races.  Additionally, among the more stark cases, ~10 prefer classic while only ~3 prefer skating.

Of course, now that I’ve done this for Japan, what other countries might be interesting…?

Ok, if you don’t care about annoying technical details, feel free to stop reading now.

I checked for an interaction between technique and gender and failed to see anything worth reporting (let alone anything that was statistically significant).  My discussion/interpretation of confidence intervals has been meant for a lay audience, so try to give me a break if I glossed over something you think is important.  Feel free to gripe in the comments, though.  However, if you complain about multiple comparisons problems, I will not respond and instead will roll my eyes and make fun of you privately with all my stats friends.  ;)

Related posts:

  1. Head-to-Head: Marit vs Justyna
  2. Most Improved: Women’s Distance
  3. Skiing Performance and Age
  4. Trends In US Skiing Performance: Distance
  5. Trends In US Skiing Performance: Sprint

About Joran

Comments

8 Responses to “Do The Japanese Prefer Classic Skiing?”
  1. And what about the Norwegians? I’m quite sure they are better in classic than in skating – at least for the men. But I think it is getting evened out a bit now? If it takes you only half an hour to toss the data into your Japanese models, why not try… :)

  2. I bow in your direction. Superlative stuff!

  3. Nagano skiing says:

    Great work. Interesting, but there are simple explanations that might throw some extra light on this. Taking Nagano for a base (one of the big three ski prefectures along with Nagano and Hokkaido, with Akita a little behind) at elementary level the kids all learn to ski classic, and the prefecture champs up to age 11 are only in classic style. They may skate around for fun, but the classic is the base. Junior high school (12- 15 year olds) they now race both classic and skating, but the prefectural champs in skating in Nagano have only been raced more recently (10 – 15 years, sorry for vagueness…. can’t track the first competition dates for skate) – partly influenced by the cost of two sets of equipment.

    So if you track back you might find a better or more equal skating performance among younger skiers, rather than the those who have been on the circuit for quite some time like Ishida and Natsumi who definitely favor classic. Just a extra bend to the stats. My personal take is that generally speaking the younger skiers are now better at free, and find classic harder to master – though I’m not so clear cut on those who rise through to WC level who are the elite.

    • Joran says:

      Thanks for the inside info on Japan’s development scene! That’s interesting that kids start entirely doing classic…

      Your comment also somewhat tangentially points to an issue that I didn’t raise in the post: time. As you noticed, the “overall” results in this post are averaged across all 18 seasons or so, and the estimates within each athlete are averaging over their entire career which in some cases is >10 years.

      Including time trends in this kind of model is a bit tricky, mainly due to sample size issues. You’ll notice how few of the actual effects ended up being “statistically significant” as it is; if you estimate a term for each athlete within each season you dramatically reduce your sample size, the result being that the CI’s will be even wider. And that’s not even dealing with the issue of the many skiers who may have only done races of one technique in a given season. I have some ideas for how to deal with this, but it requires a little more thought and, well, time.

  4. seattleskier says:

    The results of comparison are relative to non-Japanese WC skiers. Seems to me, it’s not a “preference” of classic over skate, but a relative “weakness” of skate results relative to classic versus non-Japanese skiers. This makes me wonder about efficiency of technique, or physical differences between Japanese and non-Japanese WC nordic skiers, such as power output, leg length, body mass, height, years of training, etc.

  5. Having reread this post now, I’m even more impressed by the analysis. If I’d have to recommend another subject of study about national aptitudes, I’d nominate Italy, which seems (in my understanding and/or memory) to be a far better skating nation than classic nation, especially in distance races.

    This finely-grained analysis also gets at something that I’d mentioned in an email – a way of determining which skiers are best at particular techniques and distances. For instance, we know (to mention only male racers) that Northug and Cologna and Legkov are “all-rounders,” as were Teichmann and Angerer at their peaks, or Alsgaard a while back. But which racers are one-technique ponies? Filbrich, Hofstad (though retired), Hjelmeset come to mind as consummate one-technique racers. What about Johnsrud Sundby? Is he really the classic specialist that he’s made out to be? These questions can be partly answered by scanning season-end WC rankings and such, but those aggregated records don’t easily let someone see where a skier is decidedly better at one technique and just mediocre (or worse) at the other.

Trackbacks

Check out what others are saying about this post...
  1. […] This post was mentioned on Twitter by WoXC_D, statskier. statskier said: Do The Japanese Prefer Classic Skiing? http://goo.gl/fb/5H88B […]

    [WORDPRESS HASHCASH] The comment’s server IP (208.74.66.43) doesn’t match the comment’s URL host IP (74.112.128.10) and so is spam.

  2. […] like many Japanese skiers, Ishida performs dramatically better in classic races. If we look at only classic races, measured […]



Speak Your Mind

Tell us what you're thinking...
and oh, if you want a pic to show with your comment, go get a gravatar!