Obviously, this still begs the question on exactly how to use FIP but it does provide a blue print for how to begin to decipher players who particularly over or under perform their FIP, xFIP, K/BB, etc.I agree with Ross, although this was one of the things I was trying to get at yesterday. We can look at the numbers and draw conclusions, but we would need a comprehensive resource (something like Baseball Reference, only for these advanced metrics) going back through at least 20-30 years of data to do the kind of analysis to prove how useful or worthless FIP and xFIP are.
Since I do use FIP/xFIP and I do look at these numbers, perhaps as a starting point I can share with you how I use these numbers.
1) If the pitcher is a veteran, does he historically pitch above or below his FIP?
This one almost seems too obvious. Yet many analysts seldom if ever bother to ask this question. If a pitcher is throwing up an FIP that's half a run higher than his ERA in 2010 but this is close to his career norms, it's not likely he's going to implode. If, on the other hand, he is a pitcher that normally has an ERA/FIP that are identical, I'd worry about regression.
2) I still like LIMA.
FIP is useful, but I still look at the LIMA indicators (K/9, BB/9, HR/9) when trying to decide whether or not a pitcher is going to get better, maintain, or get worse. The K/9 matter most to me, followed by BB/9, and then HR/9. I'd rather have a pitcher who strikes out a batter per inning than one who strikes out six per nine; the pitcher with more strikeouts has more control over the game. The pitcher who walks fewer batters is less susceptible to bad luck on balls in play than a pitcher who walks the park, and obviously a pitcher who allows fewer home runs isn't going to have a high ERA. This leads right to my next criterion.
3) I prefer ground ball pitchers.
While a ground ball pitcher can still be prone to the big inning, a ball hit on the ground isn't going over the fence. The data on HR/FB% and how much control a pitcher has (or doesn't have) over fly balls is murky at best. But I know that a pitcher who has a high G/F rate is less likely to destroy my ERA/WHIP than a pitcher with a low one. Yes, there are exceptions to this rule (Justin Masterson, I'm looking right at you).
4) Is a pitcher "clutch"?
This criterion is at the bottom of my list because it's easy to get sucked into small sample sizes. However, for a veteran who has been around for 4-5 years, we can look at his numbers with men on base vs. with the bases empty and draw certain conclusions. It might just be perception, but certain pitchers seem to perform better with no one on base while others seem to dig deep and pitch well with men on and the game on the line.
As I said above (and yesterday), there is a lot of work that needs to be done in this area before we come to too many meaningful conclusions. However, I recommend all of these points above as good jumping off points when trying to decide whether or not a pitcher is worth buying at auction or trading for in the off-season or mid-year.