I noticed an interesting Tweet from Chad Salmela regarding Jessie Diggins’ race in today’s Falun 10k classic mass start:
Ultimately, of course, Jessie Diggins is the only person who truly knows what her best classic performance is. But for data people like me it’s always a fun game to try to tease out some objective measure from the peanut gallery. When I read Chad’s tweet my gut reaction was disagreement.
Comparing race performances, even limiting ourselves to just classic distance races, is hard. There are lots of different confounding variables but the big obvious one is race format. Mass start events are just very different in modern skiing than individual start races, and of course pursuit starts are probably worth excluding all together.
Let’s look at Diggins’ major international classic results using three measures:
- finishing place (rank),
- FIS points, and
- PBM points (a point system analogous to FIS points based on the median skier and the spread of skiers in an attempt to capture the strength of the whole field rather than just the top finishers)
First, let’s sort the table by finishing place (rank). Even if we exclude the pursuit start race in Canmore, the 7th in today’s (2021-01-30) Falun 10k mass start is bettered by several events including an interval start race also in Falun in 2016. There are also numerous other 7th & 8th place results that should be in the conversation by this measure.
But that’s just one measure, and Chad specifically referenced percent-back, which is what FIS points are based on, so if we instead sort the table on FIS points we see basically what Chad describes as the top three (again we’re ignoring that Canmore pursuit start).
But do you notice something interesting here? When we focus only on the percent-back from the leaders some strong interval start races suddenly look much, much worse. Sort of disturbingly worse.
Here we start to butt up against the reality that mass start and interval start events may be so different, tactically and physiologically, that we should start to at least consider the possibility that they aren’t even really comparable at all. Not as incomparable as distance and sprint events, maybe, but different enough that what happens at the front of the field may not adequately capture a skier’s efforts between those formats.
As all cross-country ski racers know, today’s mass start races often have relatively slow paces for much of the race, ending with a frantic sprint over the final few kilometers, or perhaps even only the final few hundred meters. Interval start races tend to involve a more consistently high pace (and effort) for the entire race. Old farts like me tend to be biased towards interval start events (it is “real” racing, after all!) but if I’m being honest, it’s just different and requires somewhat different skills.
Circling back to Diggins’ classic results, it’s these sorts of things that make me wary of percent-back based measures when we’re mixing interval and mass start events. It should give us pause that Diggins can finish 5th & 7th in interval start races and yet have as much as ~3x the FIS points (roughly 3x the percent-back) as a 7th place mass start finish. Should we really be discounting those interval start results that severely? Diggins was much closer, in time, to the handful of skiers up the track in the mass start race, but was that because she skied faster, or was it because those skiers in front of her were only really “racing” at a different pace for a short time and so didn’t have the road to put more time on her?
In general, I think percent-back based measures like FIS points tend to pretty severely underplay interval start results. Many of the choices I made in designing a percent-back from the median skier revolved around these sorts of concerns. I’ve found that the middle of WC fields tend to be a bit more stable a benchmark for measuring performance. You avoid situations where a single skier like a Bjoergen or Johaug just outpaces the field by a minute. Additionally, I included an adjustment for how “spread out” overall the results for that day are, to penalize slightly situations where the pace is slower, resulting in a very bunched field.
So if we finally sort the table by PBM points we get very different results indeed! This measure appears to give much more credence to Diggins’ various interval start results, rather than just putting all the mass start races at the top. But there are some very different, and possibly surprising, races at the top using this measure.
First, it really likes the 2018 TdS 10k classic mass start in Val di Fiemme, for instance. This does make some sense. It was a mass start race that fractured pretty badly (albeit with a small, late-Tour field), so Diggins ended up way ahead of the mid-pack skiers, but she finished 4th, within a respectable distance from the leaders. This measure also really likes her 5th place finish in the 2016 Falun 5k interval start. This also makes sense: Johaug outpaced the field by ~19sec that day in only 5k and that’s the sort of thing that can really distort percent-back from the winner measures.
But I’m burying the lede here because where on this list is today’s 7th in Falun? Holy cow it’s all the way down in 18th! Why so low? Well, in today’s race Diggins wasn’t unusually distant from the middle of the pack (~1min in a 10k) but the field as a whole was very “bunched” suggesting that the overall pace wasn’t doing much to separate the field and so we should put somewhat less stock in being close to the leaders at the finish.
Ultimately, my gut tells me that my PBM points are being perhaps a bit too harsh on today’s race, and the truth is probably somewhere in the middle. Today’s classic performance almost certainly wasn’t Diggins’ best ever, but I’d say top-10 is fairer than ranking it all the way back in 18th.
My thought process when reading results actually does synthesize all three of the measures we just discussed, so in practice I don’t think any one is preferable to the exclusion of the others. My internal monologue today went something like this:
Wow, Jessie finished 7th in a classic race! And very close to the winner! That’s very good for her! But, hmm, it was a mass start, and gee, the top of the field looks like it was very bunched together suggesting either a relatively slow pace, or very fast conditions, or both. Let’s see, how far from the middle of the field was she? Hmm…pretty average actually. Ok, so that was a very solid classic race for her. Not Earth shattering, but very good.