Applying FIS Point Development Curves To The USST

Last Friday I used an interesting post by Pete Vordenberg as a starting point for looking at how we could expand on the idea of (roughly) measuring athlete development using FIS points. Since the USST announced their nominations last week as well, I thought it might be interesting to see how applying a tool like this might work in the real world.

This isn’t a critique of the USST selections. This is simply meant as a demonstration of how informative (or not informative!) some simple data analysis could be in the context of team selection. I will be pointing out plenty of ways in which the data is ambiguous or unhelpful, trust me.

FIS points as a measure of skier ability have plenty of well known weaknesses, but they can be informative, particularly if we’re aware of their weaknesses, so that we’re less likely to be fooled. Keep that in mind as we do this.

Recall from last Friday that I presented some curves that represented a rough estimate of the ‘typical’ (as opposed to the best) FIS points at a given age earned by skiers who went on to place in the top ten (WC, OWG or WSC). I’ve had a few people comment privately that maybe that’s a little too loose a criteria; there may be lots of skiers with a single top-ten result who we aren’t really interested in comparing ourselves to. That’s a fair point, so let’s restrict things down to people who’ve finished in the top three in a WC, OWG or WSC race. That feels reasonable to me, since medals are such a constant stated goal of the USST.

What we’ll do here is simply overlay the corresponding data for USST members on top of these reference curves, starting with the A team:


  • I’m using medians here, not the average of someone’s best five races, which is how we typically think about FIS points.
  • Since this mixes together FIS points earned from WC levels races down to regular FIS races, the amount of ‘adjustments’ I can apply to the FIS points themselves is limited.
  • The black line represents the median FIS point race for a skier at a given age who went on to reach the podium at least once. The black vertical bars represent the 1st-3rd quartiles, so the  middle 50% of the data.
  • The colored lines represent the median FIS points for each athlete at that age. Keep in mind that the ages are approximate, and that I’ve adjusted them, since otherwise an athlete’s races at age 25 would include Jan-Apr of one season and Nov-Dec of the next, which isn’t quite what we want.
  • The colored lines give no indication of the number of races an athlete did at a particular age; at the younger ages this is often quite a low number.

Kris Freeman and Andrew Newell fit the development curves fairly well in their respective specialties. Kikkan Randall wasn’t as strong in qualifying (which is all that sprint FIS points refer to) at some point in her early twenties. It’s tempting to look at this and conclude that Randall’s sprinting ability is being undervalued here, and I think there’s something to that. But then again, the points only measure qualification speed, and the benefit to qualifying 1st rather than 20th is debatable.

Some interesting stuff is going on in the women’s distance panel, though. Randall’s general improvement is of course evident. Notice, though, that using this development curve as a guide, Randall appears well out of the mix until recently. Also, notice that both Randall and Stephen’s medians in their most recent season both sloped up somewhat. That doesn’t seem right at all. This is mostly the Marit Bjørgen Effect. Randall and Stephen both had fairly strong seasons on the WC circuit for them, but on occasion their FIS points were quite low thanks to Bjørgen (or Johaug or Kowalczyk) ditching the field by a minute or more. If you spotted each of them a generous adjustment and supposed that their median results dropped by the amount it actually rose you’d see Randall inching a bit closer to the median development curve, but Stephen would still be well above it (though improving).

Moving on to the B Team folks:

Things look good for Noah Hoffman here, and Jessie Diggins as well, although she’s still quite young. I’m generally less sure what meaning to draw from trends in sprint FIS points, but by this measure Simi Hamilton isn’t far off where you’d like him to be either. One thing this view of the data doesn’t account for is technique specialization. Tad Elliott, for example, is generally better in freestyle races, so looking at all of his races may hide a promising trend in a single technique.

And finally the D Team folks:

These guys are still young, but they aren’t far off in sprinting.


How’d We Do? USA/CAN Season Review 4

Last, but not least,  the women’s sprinters, which was easily the highlight of the season for the Americans:

The Canadian women recovered somewhat after two rough seasons. While they didn’t have the eye-catching results of Kikkan Randall, the Canadian women have three different skiers (Crawford, Gaiazova, Jones) who qualified for the heats more than once, and each of the three advanced to the semifinals on one occasion. It’s nice to have more than one skier capable of making a run into the heats, but the news isn’t all rosy:

Gaiazova has improved her sprinting compared to 4-5 years ago, but didn’t see much improvement over last year, although her best result was slightly better. Crawford has been fairly inconsistent from year to year, including a season lost to injury, and her best seasons are now 3 and 5 years in the past. It was neat to see a third Canadian able to advance past the quarterfinals, but it’s not clear from Jones’s trend whether that will prove to be an anomaly or not.

For the American women, it was obviously a banner year for Kikkan Randall. I’m not going to spend much time singing her praises, since I think we’re all pretty familiar with her season. Instead, let’s look a little more closely at the fact that there’s Kikkan Randall, and then there’s everyone else:

‘Other’ refers to American women other than Kikkan Randall. Randall’s median performance slipped a bit back in 2009 thanks to a few more sub-par races than usual, but she’s turned things around dramatically since then. Some people have noted that both of the folks who finished above her in the sprint World Cup points (Follis and Majdic) are retiring next year. However, we should also note that Marit Bjørgen was only 24 WC points behind Randall and did two fewer sprint races this season. With Bjørgen likely to do the Tour de Ski next year, and with the sprint races evenly split between classic and freestyle, I think Bjørgen’s a safe bet for an overall sprint podium. The next closest women to Randall were Kowalczyk (113 points back) and Falla (122 points back). Kowalczyk did 10 races, the same as Randall, but Falla only did 8.

I’m betting that in order to repeat her overall sprint podium next year, Randall will have to outpace one of those three: Bjørgen, Kowalczyk or Falla. (And of course there’s always the unpredictable newcomer.) But to do that, Randall will surely have to raise the level of her classic sprinting, as all three of those ladies had much more balanced results between techniques this year and the schedule will be evenly split 6C/6F next year, not 5C/7F.

Getting back to the graph above, it’s clear that Randall’s sprinting success hasn’t really filtered down enough to show up in WC results, at least not yet. Along with quite a few others, I was excited by Ida Sargent’s early season racing this year, but an unfortunate car accident seemed to gum up the works. On the one hand, I’m optimistic that Sargent might have some success if she earns some WC starts next year, particularly in classic sprints. On the other hand, I’m following the classic sprint stage of the Sun Valley SuperTour final as I’m writing this, and while Sargent was the closest American to Randall in qualifying by a hefty margin, she was still slightly more than 4 seconds back. Sargent beat Randall in qualifying in her one classic WC sprint (in 33rd), but when Randall has qualified, 4 seconds generally would push her Sargent of the top 30.

This is all pretty negative for what was easily the best season by a US skier in recent history, and probably the best since the days of Bill Koch and Co. So let’s end on that reminder of what a fun season it was!

How’d We Do? USA/CAN Season Review 3

This week it’s the sprinter’s turn in the limelight, starting with the men. As you might imagine, there are quite a few more cheery topics when talking about North American sprinters. As before we’ll begin with a simple plot showing the number of results per race over time:

The American men have seen a steady decline in top thirty sprint performances, mostly due to the recent struggles of Torin Koos. For about three seasons, the US men had two skiers regularly qualifying for the heats, but Koos’s results have dropped off, leaving only Andrew Newell as the top performer. Simi Hamilton, while surely talented, hasn’t yet made up the difference created by Koos’s absence. Hamilton only did five major sprint races this season and qualified twice, but was well back of 30th the other three times. It has sounded from the news reports like he has struggled a bit with injury and illness this season, so perhaps he can show that he can be a regular in the heats next year if he stays healthy.

Outside of Newell, the US men have seemed inconsistent. Of the top thirty performances from someone other than Newell since 2002, 27 belong to Koos, and another 11 to Chris Cook and Garrot Kuzzy. All three either didn’t ski particularly well, or in Kuzzy’s case didn’t ski well enough to get a real chance at any big starts. As for Newell himself, his results were actually fairly similar to what he’s done over the past 3-4 years: Read more

Week In Review: Friday Apr 22nd

I took a break today from the retiring skier posts, but they will resume next Friday…

  • The first two parts of a brief season review for the North American skiers this week, focusing on the men’s and women’s distance skiers.
  • I did a little exploration of how we might put together some data on (very rough) FIS point benchmarks for developing skiers, based on a post by Pete Vordenberg here.

FIS Points Development Guide

Pete Vordenberg and Bryan Fish have an intriguing post up over at NCCSEF about the FIS point profiles of the top ten skiers in the world. The intention is to give developing skiers a rough tool to use to gauge their progress against the development paths of the best skiers in the world.

The FIS points awarded at any given race, of course, are intended to provide this sort of information, but that’s a comparison to the best skiers in the world right now, not the best skiers in the world when they were your age.

Also note that the term ‘FIS point profile’ means (I assume) the average of an athlete’s best five races over an entire calendar year that is used for generating the FIS point lists used in seeding. (Since this average is recalculated several times a year, I’m not sure which list they used at a given age; I presume they used the year-end list.)

My thoughts, as a stat-head:

  • Ten skiers isn’t a huge ‘sample’ to learn from
  • The sample is even smaller for many of the younger ages; for instance for the male sprinters it isn’t until age 21 that all ten have FIS point profiles
  • The choice of display (four big sets of staggered columns of numbers) makes it hard to get sense of what a ‘typical’ FIS point profile is at a given age, and how much variability there is between athletes.

Let’s see if we can provide some similar information, geared towards the same purpose, but maybe in a format that’s a bit easier to digest. My version will be different in several respects, but my aim is still to provide a rough guide to where you stand compared to the best skiers in the world at a given age.

Rather than just the current top ten skiers in the world (however you choose to measure that), let’s instead look a bit wider at anyone who has achieved a top ten result in a major international competition (WC, OWG, WSC). Also, rather than looking at an athlete’s best results, let’s look at all the results achieved by top-ten caliber skiers at a given age. Here’s the result in chart form: Read more

How’d We Do? USA/CAN Season Review 2

Continuing on from last time, now we move on to the women’s distance events. We’ll start out the same way, with a basic graph showing the number of results per race for various levels of performance:

The Canadian women built up steadily under the Beckie Scott/Sara Renner days, but haven’t quite found any replacement for these ladies (yet). The American women have generally bounced around flirting with mostly top-30 results, seeing a steady increase in these over the past few seasons. As recently as 2006-2007, the US women didn’t have a single top-30 result in 12 attempts. Since then they’ve progressed all the way to ~56% of their results being ‘in the points’ in 48 attempts. Obviously, Kikkan Randall’s been leading the charge here, along with Liz Stephen and Morgan Arritola.

Kikkan Randall’s 11th place finish at the Lahti pursuit this year was remarkably good. I can find a grand total of 9 top-15 results by US women in WC, OWG, or WSC races since 1992, Randall owns four of them and all of them are either 14th/15th except for this one 11th place. Her FIS points from the Lahti pursuit (50.96) were deceptively high, thanks to Johaug and Kowalczyk skiing away from everyone. If you use standardized percent back from the median skier, that race is, by a healthy margin, the best US female distance result since 1992. In fact, it is one of only two that are more than one standard deviation better than average. Randall scored a -1.33 and the next best result by this measure is Leslie Thompson’s 17th place in a 10km freestyle all the way back in Davos, 1993, which netted her a -1.17.

Here’s a look at some of the individual US women who did international racing this year: Read more

How’d We Do? USA/CAN Season Review 1

I think most people generally have a sense for how the past World Cup season went for the North Americans. What I’m going to do over the next few posts is to simply show some data that hopefully provides some context for what you already know. I’m going to split them into four posts for men/women and distance/sprint. Today we’ll start with the men’s distance performances.

Let’s start with the simplest of metrics, finishing place, and a style of graph that I’ve used before that shows the number of results per race at a given level:

Read more

Next Page »