Is xFIP useful to predict future performance?

One of the sabermetricians favorite metrics is xFIP. By now, most of you should have at least a rough idea of what xFIP is, but if you don't here is the definition:

Expected Fielding Independent Pitching (xFIP) is a regressed version of FIP, developed by Dave Studeman from The Hardball Times. It’s calculated exactly the same as FIP, except it replaces a pitcher’s homerun rate with the league-average rate (10.6% HR/FB) since pitcher homerun rates have been shown to be very unstable over time. A pitcher may allow a homeruns on 12% of their fly balls one year, yet then turn around and only allow 7% the next year. Homerun rates can be very difficult to predict, so xFIP attempts to correct for that. Along with FIP, xFIP is one of the best metrics at predicting a pitcher’s future performance.

Sounds great! I'm always looking for an edge in my Rotisserie League. Sign me up!

Ten Most Favorable ERA/xFIP Differentials 2010 (min 100 IP)

Pitcher	2010 ERA	2010 xFIP	2011 ERA	Predictor?
Josh Beckett	5.78	4.54	2.89	xFIP
James Shields	5.18	4.24	2.82	xFIP
Kevin Correia	5.40	4.71	4.79	xFIP
Jeff Francis	5.00	3.88	4.82	ERA
Jason Hammel	4.81	3.7	4.76	ERA
Ricky Nolasco	4.51	3.86	4.67	ERA
Brandon Morrow	4.49	3.16	4.72	ERA
Bud Norris	4.92	4.17	3.77	xFIP
Nick Blackburn	5.42	5.07	4.49	xFIP
Aaron Harang	5.32	4.6	3.64	xFIP

The chart above lists the 10 pitchers who pitched at least 100 innings in both 2010 and 2011 and had the most favorable xFIP differentials compared to their ERA. The expectation would be that – generally speaking – these pitchers’ ERAs would have almost all improved in 2011. At a glance, then, xFIP looks like a rousing success. Eight out of 10 of the pitchers on this list saw their ERAs improve; only Nolasco and Morrow took a step back.

However, a closer look indicates that it isn't quite this simple. Despite the fact that Francis and Hammel’s ERAs improved, the improvement was marginal. As a predictive tool, you would have been better off using ERA and simply ignoring xFIP in four out of the 10 cases in the chart above. Looking at it through this lens, six pitchers out of 10 had a 2011 ERA closer to their 2010 xFIP than their 2010 ERA.

But 10 pitchers is an admittedly paltry sample size. What happens if you expand this to include all 100+ inning pitchers in 2010 and 2011?

It doesn't help. 

One hundred and three pitchers pitched at least 100 innings in 2010 and 2011. Of these pitchers, 2010 xFIP was a better predictor of future ERA for 53 pitchers, 2010 ERA was a better predictor for 49 pitchers, and Tommy Hanson "tied" using this model. Expressed as a percentage, xFIP was better than ERA at predicting future ERA 52% of the time.

However, there are some seasons in here that are statistically close. Doug Fister posted a 4.11 ERA and a 4.10 xFIP in 2010. While saying xFIP is a better predictor in this case is technically correct, you could have used ERA or xFIP to predict future success and come up with more or less the same result.

In order to compensate for this, I took out all of the pitchers whose 2010 xFIP and ERA came within 0.3 of each other. Eliminating these seasons, xFIP was a better predictor of future ERA in 38 cases and prior ERA was better in 31 cases. This helps xFIP's cause somewhat, but it is still only a better predictor of future ERA 55% of the time.

Does this mean that xFIP is worthless? Certainly not. However, there wasn’t a strong corollary last year between prior xFIP and future performance. At the very least, analysts should use caution when simply trotting out xFIP and saying that it's certain that a pitcher is going to get better because his xFIP says so.


I don't exclude myself from this conclusion. Last year, I used xFIP a lot without performing a simple test like the one above. I sometimes take others to task for drawing broad conclusions on the strength of metrics without backing them up, so I have to offer my own mea culpa here and say I'll be more vigilant in the future.