Skip to content

Top US Results Have A Freestyle Bias

I don’t think it’s any secret that US skiers tend to have somewhat better results on the World Cup (and OWG/WSC/TdS) in freestyle events. Let’s quantify that a bit, starting with distance races:

I’ve plotted major international results of both techniques (so no skiathlons) for the US men & women for the last decade or so. I’m just using finishing place as the measure, to keep things simple and easily interpretable. The lines represent the 50th & 10th percentiles for each technique. I had to use two sets of lines with the same colors (blue = freestyle, red = classic) so I labelled the sets as 50th and 10th percentile.

There’s a distinct separation between techniques for the US women that has persisted for quite a while. For the US men, I’d say that the 50th percentile result is about the same for each technique, with a very small edge for freestyle in the 10th percentile, not terribly significant.

Here is the same plot for sprints:

Some interesting differences here. The women still have a consistent bias towards freestyle, but it’s considerably smaller, and the difference for the 10th percentile has narrowed in recent years. The men seemed to have very little difference between techniques until recently when a fairly substantial gap has developed, again favoring freestyle. Part of me wonders how much of that is the result of just one skier, Andrew Newell, declining slightly later in his career and then mostly retiring.

Jessie Diggins’ Best Classic Races

I noticed an interesting Tweet from Chad Salmela regarding Jessie Diggins’ race in today’s Falun 10k classic mass start:

Ultimately, of course, Jessie Diggins is the only person who truly knows what her best classic performance is. But for data people like me it’s always a fun game to try to tease out some objective measure from the peanut gallery. When I read Chad’s tweet my gut reaction was disagreement.

Comparing race performances, even limiting ourselves to just classic distance races, is hard. There are lots of different confounding variables but the big obvious one is race format. Mass start events are just very different in modern skiing than individual start races, and of course pursuit starts are probably worth excluding all together.

Let’s look at Diggins’ major international classic results using three measures:

  1. finishing place (rank),
  2. FIS points, and
  3. PBM points (a point system analogous to FIS points based on the median skier and the spread of skiers in an attempt to capture the strength of the whole field rather than just the top finishers)
Generated by wpDataTables

First, let’s sort the table by finishing place (rank). Even if we exclude the pursuit start race in Canmore, the 7th in today’s (2021-01-30) Falun 10k mass start is bettered by several events including an interval start race also in Falun in 2016. There are also numerous other 7th & 8th place results that should be in the conversation by this measure.

But that’s just one measure, and Chad specifically referenced percent-back, which is what FIS points are based on, so if we instead sort the table on FIS points we see basically what Chad describes as the top three (again we’re ignoring that Canmore pursuit start).

But do you notice something interesting here? When we focus only on the percent-back from the leaders some strong interval start races suddenly look much, much worse. Sort of disturbingly worse.

Here we start to butt up against the reality that mass start and interval start events may be so different, tactically and physiologically, that we should start to at least consider the possibility that they aren’t even really comparable at all. Not as incomparable as distance and sprint events, maybe, but different enough that what happens at the front of the field may not adequately capture a skier’s efforts between those formats.

As all cross-country ski racers know, today’s mass start races often have relatively slow paces for much of the race, ending with a frantic sprint over the final few kilometers, or perhaps even only the final few hundred meters. Interval start races tend to involve a more consistently high pace (and effort) for the entire race. Old farts like me tend to be biased towards interval start events (it is “real” racing, after all!) but if I’m being honest, it’s just different and requires somewhat different skills.

Circling back to Diggins’ classic results, it’s these sorts of things that make me wary of percent-back based measures when we’re mixing interval and mass start events. It should give us pause that Diggins can finish 5th & 7th in interval start races and yet have as much as ~3x the FIS points (roughly 3x the percent-back) as a 7th place mass start finish. Should we really be discounting those interval start results that severely? Diggins was much closer, in time, to the handful of skiers up the track in the mass start race, but was that because she skied faster, or was it because those skiers in front of her were only really “racing” at a different pace for a short time and so didn’t have the road to put more time on her?

In general, I think percent-back based measures like FIS points tend to pretty severely underplay interval start results. Many of the choices I made in designing a percent-back from the median skier revolved around these sorts of concerns. I’ve found that the middle of WC fields tend to be a bit more stable a benchmark for measuring performance. You avoid situations where a single skier like a Bjoergen or Johaug just outpaces the field by a minute. Additionally, I included an adjustment for how “spread out” overall the results for that day are, to penalize slightly situations where the pace is slower, resulting in a very bunched field.

So if we finally sort the table by PBM points we get very different results indeed! This measure appears to give much more credence to Diggins’ various interval start results, rather than just putting all the mass start races at the top. But there are some very different, and possibly surprising, races at the top using this measure.

First, it really likes the 2018 TdS 10k classic mass start in Val di Fiemme, for instance. This does make some sense. It was a mass start race that fractured pretty badly (albeit with a small, late-Tour field), so Diggins ended up way ahead of the mid-pack skiers, but she finished 4th, within a respectable distance from the leaders. This measure also really likes her 5th place finish in the 2016 Falun 5k interval start. This also makes sense: Johaug outpaced the field by ~19sec that day in only 5k and that’s the sort of thing that can really distort percent-back from the winner measures.

But I’m burying the lede here because where on this list is today’s 7th in Falun? Holy cow it’s all the way down in 18th! Why so low? Well, in today’s race Diggins wasn’t unusually distant from the middle of the pack (~1min in a 10k) but the field as a whole was very “bunched” suggesting that the overall pace wasn’t doing much to separate the field and so we should put somewhat less stock in being close to the leaders at the finish.

Ultimately, my gut tells me that my PBM points are being perhaps a bit too harsh on today’s race, and the truth is probably somewhere in the middle. Today’s classic performance almost certainly wasn’t Diggins’ best ever, but I’d say top-10 is fairer than ranking it all the way back in 18th.

My thought process when reading results actually does synthesize all three of the measures we just discussed, so in practice I don’t think any one is preferable to the exclusion of the others. My internal monologue today went something like this:

Wow, Jessie finished 7th in a classic race! And very close to the winner! That’s very good for her! But, hmm, it was a mass start, and gee, the top of the field looks like it was very bunched together suggesting either a relatively slow pace, or very fast conditions, or both. Let’s see, how far from the middle of the field was she? Hmm…pretty average actually. Ok, so that was a very solid classic race for her. Not Earth shattering, but very good.

Swedish Men’s Struggles

It’s always fun to take one of Devon Kershaw’s rants and make it concrete. In this case, his apparent despair over the struggles of the Swedish men on the international racing scene. He’s ranted about it more than once, but in this particular case he referred to 2019-2020 as being the worst season for them ever, more or less.

So let’s look at the Swedish men’s distance results in World Cup, Olympic and World Championships, tally them up in categories and give each season a score. My entirely made up points are to award 10 points for a win, 5 for a podium, 2 for a top 10 and 1 for a top 30. (Results are counted in each category they fall in, so a win contributes 10 + 5 + 2 + 1 = 18 points.)

Swedish men’s distance results. One point for top 30, two points for top 10, 5 for podiums and 10 for wins.
Season Wins Podiums Top 10 Top 30 Score
2019-2020 0 0 7 53 67
2018-2019 0 1 7 53 72
2017-2018 0 3 12 57 96
2016-2017 2 5 23 81 172
2015-2016 0 2 4 37 55
2014-2015 1 5 22 72 151
2013-2014 1 8 24 59 157
2012-2013 3 6 29 81 199
2011-2012 3 5 25 62 167
2010-2011 3 9 24 67 190
2009-2010 2 9 32 67 196
2008-2009 2 5 15 42 117
2007-2008 1 3 16 43 100
2006-2007 0 2 15 46 86
2005-2006 3 6 22 61 165
2004-2005 0 2 13 34 70
2003-2004 3 7 17 52 151
2002-2003 6 14 37 69 273
2001-2002 4 10 20 47 177
2000-2001 8 10 22 47 221
1999-2000 0 5 18 58 119
1998-1999 2 4 30 67 167
1997-1998 0 2 19 59 107
1996-1997 0 1 22 60 109
1995-1996 0 2 16 57 99
1994-1995 1 3 20 63 128
1993-1994 0 4 17 50 104
1992-1993 2 4 17 46 120
1991-1992 1 3 18 42 103

2019-2020 isn’t technically the worst on this list, but close enough. It was pretty bad. I don’t quite recall what happened in 2015-2016. I know Marcus Hellner was still racing then but he had a pretty terrible season. My guess would be some combination of injuries or illnesses, but I have a terrible memory for that stuff.

For comparison, here’s the same thing for sprinting:

Swedish men’s sprint results. One point for top 30, two points for top 10, 5 for podiums and 10 for wins.
Season Wins Podiums Top 10 Top 30 Score
2019-2020 0 1 13 51 82
2018-2019 0 0 14 57 85
2017-2018 0 3 14 42 85
2016-2017 1 3 11 41 88
2015-2016 0 0 10 34 54
2014-2015 0 0 7 29 43
2013-2014 2 7 19 43 136
2012-2013 4 6 17 43 147
2011-2012 3 8 20 52 162
2010-2011 7 8 31 51 223
2009-2010 4 8 26 54 186
2008-2009 1 2 11 37 79
2007-2008 1 6 19 41 119
2006-2007 0 5 24 56 129
2005-2006 7 12 30 49 239
2004-2005 1 5 20 50 125
2003-2004 3 10 24 49 177
2002-2003 4 11 30 52 207

In this case 2014-2015 and 2015-2015 both stand out quite a bit more than any of the more recent seasons, although they all have a distinct lack of wins.

Hopefully they can turn things around!

US Development Graphs

I posted these graphs on Twitter a few days ago with basically zero commentary, so I thought I should at least describe what I did in a tad more detail.

I make these graphs because they get at an aspect of something the US Ski Team leadership talks about fairly often: being “on track” for top international results. They are not meant to represent any actual artifact or analysis that the US Ski Team actually uses for making decisions. They are merely my translation of the general concept. There are lots of different ways you could actually approach this idea and end up with somewhat different conclusions.

First we need a definition of “top international results”. For distance events I took that to be finishing in the top ten in WC/WSC/OWG/TdS events at least three times. For sprints, reaching the semifinals in WC/WSC/OWG/TdS events at least three times.

Then I took those “successful” skiers and plotted a band that represents the 10th-90th percentile of their race FIS points by age. So this band represents what the bulk of these skier’s race FIS points looked like at different ages.

Then I plotted on top the actual race FIS points from some of the top US skiers according to the most recent FIS points ranking lists for distance and sprint. I skipped a few people who I believe retired, and generally tried to focus on younger skiers in places. But mostly I was just running down that ranking list.

Finally, the red trend line is tracking the median result for each skier. I was a tad lazy and didn’t tweak the smoother too rigorously for each skier, which results in some poor (overfit) results in a handful of cases. Those are the red lines that are really, really squiggly. If I were taking this more seriously, I’d replace those with a more rigid linear fit, or maybe just omit the trend line for those skiers entirely.

National Team Selection & Age

Springtime means national team nominations, which is always one of people’s favorite things to argue about. This has become particularly true in recent years with some national teams having a fairly explicit preference for younger skiers built into their criteria. It seems like every year there are 1-2 skiers pushing 30 (or beyond) that have skied well that season but are passed over for younger competitors who have equal or even worse results to boast of.

National team selections, along with Olympic and World Championship team selections, are delicate and fraught decisions, and so I try to write very carefully on the topic. So be prepared for lots of qualifications and caveats. If you wanted a straight answer, you shouldn’t have asked a statistician!

One of the first problems with looking at patterns between age and international skiing success is defining “success”. Is it podium results? Top 10? Scoring World Cup points? Is it doing any of those repeatedly, and if so, how many times? This is the first subjective decision point in this ostensibly objective exercise.

For the purposes of this post I’m going to go with success being a top ten result in a distance race (WC/OWG/WSC/TdS) or a making a sprint final (top 6). Why? Because this is my blog and so I get to do whatever I want. Let’s start with some simple descriptive stuff. At what age to skiers tend to achieve these things?

Starting with distance:

The way to read this is to start on the y-axis and move horizontally across. So start at 50% and move across to see that around 50% have gotten their first top 10 distance result by the age of 25. 75% of men have gotten their first top 10 by around age 26 and women closer to 28. Another way to put this is that “only” around 25% of people with a top ten distance result manage to do that for the first time past age 26-28.

Now, 25% is a minority, but you might also say it’s a pretty big minority. Big enough that it isn’t really surprising when it does happen. (Does this kind of language feel familiar to certain probabilistic forecasts of a certain US presidential election?)

Before we get into more technicalities and caveats, lets look at the same thing for sprinting:

These curves are pretty similar, maybe shifted back by a year or two. So here around 50% of skiers have their first sprint final by age 23-24 and around 75% have achieved that by around age 26.

An interesting thing to think about with this data is what direction the causation (if any) is going in. Or more precisely, what specifically is it about being 25-30 that makes it harder to land a strong result? Do older skiers have an inherently lower ceiling on their future improvement? Or are there structural barriers that give them fewer opportunities to improve?

It is certainly true (or at least plausible) that on average your potential to improve decreases as you age. I mean, I’m 41; trust me, getting older is a thing. But what if because everyone “knows” that older skiers have a lower ceiling they are subtly but systematically given fewer opportunities that artificially contributes to their lack of success?

I’m not trying to be inflammatory here. Consider the case of Martin Johnsrud Sundby, who was left off the Norwegian national team this year. Norway probably could field a pretty darn competitive national team comprised entirely of skiers over the age of 27-28, at least from the perspective of us in North America. But in an environment filled with talented skiers under 23 (again, compared to North America) why on earth would you?

So what if this pattern toward success at an early age is being driven by the hyper-competitive environments of Norway, Russia, Sweden, etc. where if you haven’t had World Cup success by age 24 there are 5-10 more super talented 22 year olds biting at your heels for a chance?

The problem with this is that if you’re looking for a data set of skiers who have plugged away at competing on the World Cup at a middling level for a long time to see how many of them suddenly get much better past age 28, that is a very small and peculiar set of skiers. Hard to do “unbiased” analysis on those folks. (I mean, I have looked at it, and these folks really do rarely improve, but it’s such an oddball group of skiers I’d hesitate to generalize much from them.)

Now that I’ve pummeled you with that barrage of caveats, let’s actually look at what really does happen after a skier’s first top 10 distance result or their first sprint final. First up, distance:

For people who got their first top 10 WC/OWG/WSC/TdS result between 2005-2015, this shows the number of subsequent top 10s they had over the next five years. The points at the bottom get pretty crowded (and I didn’t want to make you think about a log scale) so I’ve highlighted all the 0’s by coloring them blue and dropping them down slightly. So the blue dots are people who were one and done.

So this is why things look pretty grim for older skiers, particularly men. Look at that drop off for the men at age 26-27! The women have a handful of folks who developed into regular top 10 skiers past age 25, but they are a distinct minority. Next up, sprinting:

More or less the same story, but shifted 1-2 years younger.

Again, one of the things I worry about when I look at plots like these is what I was rambling on about before. How much of this pattern isn’t because skiers can’t improve dramatically after the age of 25, but instead that so many hyper-competitive skiing nations can afford to be really brutal about tossing them aside because they have a seemingly endless supply of talented 22 year olds right behind them?

Like everything else, it’s probably not one or the other, but a mix of both. This is where we start to incorporate other sources of information. For example, I’m not an exercise physiologist but I can imagine evidence from that field either in support of or against skiers having a lower ceiling past age 25 that is independent of race result patterns. Also, coaches will typically have knowledge about a specific athlete that isn’t captured by knowledge about the “average athlete”. This can be a dangerous source of information, but it is real and valuable. But now we’ve strayed past what could reasonably be encoded into a concrete national team selection criteria.

Don’t say I didn’t warn you! Ask a statistician and you’ll likely get a lot of graphs and data that mostly lead to more questions.

Do US Women Have Better Results Earlier In The Season?

I received an email recently asking the above question, and since responding to these sorts of things is often kind of challenging in an email I figured I’d just write it up here. I’m sure I’ve written about this sort of thing before, but it was probably a long time ago. Regardless, I’ve been looking at skiing results data for long enough now that I’ll admit to having some pretty strong priors when people ask me questions like this. Specifically, my gut reaction is to respond that whatever pattern you’re responding to is probably not going to be particularly evident if we start looking at it systematically. Then again, I’m sort of a statistical Debbie Downer so take that with a grain of salt.

Setting my depressing priors aside, let’s see what we see. So let’s start by taking all US women’s major international results over a sizable chunk of recent history, say since the 2013-2014 season. We’ll exclude pursuit starts, since they rather famously don’t compare well with anything else, and let’s also exclude events in USA/CAN to avoid having to worry too much about events with enormously expanded American starters. (This last one isn’t a huge number anyway.)

The FIS race calendar is mostly pretty predictable, but it does shift around from year to year. But let’s be simple about this and divide the races up by month. However, I’m going make a special allowance for the Tour de Ski and basically consider it a “month” by itself. Probably I’d guess that people who’ve done it might be ok with labelling it as a month’s worth of racing, anyway. So our “months” are Nov, Dec, TdS, Jan, Feb and Mar. That’s a little weird, but hopefully it makes sense.

Again, simple is the best place to start, so let’s just count up American women top 3 and top 10 results in those time periods since 2014:

Three things immediately jump out: a noticeable lack of top results in November (i.e. Ruka), an apparent spike in good distance results during the Tour de Ski and a very consistent rate of top results in sprinting after November.

Yes, yes, I hear you all shouting at me through the internet about needing to look at rates not counts, so let’s divide all these by the total number of results (i.e. how many US women finished a race in that time period):

See, now that’s kind of interesting! It’s almost like things completely reversed between the distance and sprint results. We still see pretty low rates of top results in November, but now the rate of top distance results is the more stable of the two and the rate of top sprint results has an obvious peak during the Tour de Ski.

Circling back to the original question, my Debbie Downer-ism was mostly correct in that there isn’t some very obvious peak in Nov/Dec that steadily declines throughout the season. But that doesn’t mean this was all a waste, of course.

What’s going on here, I think, is two things: I suspect the US may have made the Tour de Ski a focus for their season in ways that perhaps other nations have not, and if you do focus on the Tour and finish it you’re likely to be rewarded with some significantly reduced field strengths and sizes in the later stages.

Frankly, what’s most interesting to me about all this is the relative lack of success at the Ruka races in November. I don’t know if that’s the result of intentional strategic choices or if perhaps the races and formats haven’t really suited as many of the US women as other events have. Perhaps I’ll have to write a follow up that digs into that a little further.

Chinese World Cup Performance

I got another friendly homework assignment from The Devon Kershaw Show today. This one was pretty simple, though, as Jason was just wondering about the performance of Chinese skiers on the World Cup (or similar level events). This was prompted by Chunxue Chi finishing 28th in today’s 10k freestyle race in Oestersund.

Jason was curious whether this was one of the best performances by a Chinese skier on the circuit, and the answer is: kind of.

The Chinese women have had a handful of roughly equivalent distance performances, but not recently. In general they’ve had more success with their male sprinters. In this particular case, though, Chunxue Chi finished 24th just a few weeks ago in Nove Mesto. Still, it hasn’t been very common to see the Chinese women scoring points in distance events.

Tagged ,