What Makes a Hall of Fame Reliever?

Several years ago, on the now defunct Rule V Baseball Blog, I started writing a series of posts wondering what makes a relief pitcher a Hall of Famer. I’ve always been curious about the true value of a relief pitcher and what their representation in the Hall of Fame would be.

My original hypothesis was that too few relievers were in the Hall. Rich Gossage and Bruce Sutter were not yet in, so it was just three (Dennis Eckersley, Hoyt Wilhelm, and Rollie Fingers). I’ve considered a series of stats to evaluate relievers—starting with the likes of ERA+ and eventually settling with Wins Above Replacement (WAR)—and if you dig deeper into the numbers, my original hypothesis doesn’t really hold up too well.

Obviously, relievers have a smaller body of work to analyze than your typical starting pitcher or hitter. This is the main factor that limits the career value of relief pitchers. However, it must be noted that while relievers appear in fewer situations, often (not always) they are performing in more important situations. So, how does that affect their overall career value? In the past that has been tough to quantify. But now that we have WAR, we finally have a statistic that is helpful in figuring out overall career value.

What goes into WAR? Let’s take a look at the WAR numbers on Baseball Projection for Mariano Rivera. You’ll see not only the WAR column, but also a series of components that go into it. Here’s how the site describes them:

IP, Runs – Actual Innings pitched and runs allowed totals

Rep – Estimated runs that a replacement level pitcher would allow, given this pitcher’s opponents, ballpark, league, and defensive support

Def – Estimated runs saved by this pitcher’s defense, using TotalZone range, DPs, OF arms, and catchers, prorated by the number of balls in play allowed by the pitcher

LI – Leverage Index, licensed from Inside the Book, which measures the win impact of the game situation faced by each pitcher

WAR- Wins above replacement – Runs above replacement, adjusted for leverage, divided by a runs to wins converter. This is normally close to 10 runs per win but is less in lower run scoring environments. Pitchers are credited with extra leverage halfway between their actual leverage and 1.0. This is due to the effect of chaining, a relief ace who normally has a leverage of 2.0 and is injured. He will be replaced by a setup man, who is normally an above average pitcher himself, and all pitchers will move up into higher impact situations. The net effect is less than if we assume that the relief ace is directly replaced by a replacement level pitcher.

R+ – Runs allowed above or below league average. Unadjusted for any context.

BB+ – Run value of walks allowed above or below league average given batters facing pitcher. Unadjusted for any context.

HBP+ – Run value of hit batsmen allowed above or below league average given batters facing pitcher. Unadjusted for any context.

SO+ – Run value of strikeouts above or below league average given atbats against pitcher. Unadjusted for any context.

HR+ – Run value of homeruns allowed above or below league average given number of fair contacted balls allowed. Unadjusted for any context.

Hit+ – Run value of hits allowed above or below league on balls in play. Unadjusted for any context.

X – Runs saved or allowed beyond measurable impact of hits, homers, walks, strikeouts, and HBP. This is a catch-all category that includes holding and picking off runners, defensive support including the DP, errors in the field, timing of events (pitching better or worse with runners on base), or any other explanation you can think of.

As you can see, a lot goes into it. Personally, I love the inclusion of leverage index. That means relievers with fewer innings can get a boost by the fact that those innings have a higher impact on the outcome of the game than normal. The extra dimension leverage index brings is one of the reasons I’ve settled on WAR as my metric of choice for comparing a reliever’s total value to other types of players.

Of course, there’s one issue with using WAR to rate relivers. It’s pretty damning for them. And maybe this is a good thing. In order for a relief pitcher to have a career WAR total on par with other Hall of Famers, he’s gotta be damn good for a long time. There are a lot of “ace” relievers that flame out after a few years. These guys may finish with sparkling stats over a 1000 inning sample. But will their total career worth be on par with an average Hall of Famer? Nope.

I’m looking at you, Bruce Sutter.

It’s rare that a reliever pitches at a very high level for a very long time. But when a reliever does this, he should be recognized as a Hall of Famer.

I’m looking at you, Rich Gossage.

What’s the magic number for career WAR? Well, looking at Baseball Projection it seems that a WAR of 40 gets you in the Hall of Fame conversation. There are plenty of players above 40 WAR that aren’t in, but there are also quite a few with less than 40 WAR who are in. Here’s a list of some of the sub-40 WAR Hall of Famers:

Player WAR
Dizzy Dean 39.6
Chuck Klein 39.2
Lou Brock 39.1
Ernie Lombardi 39.0
Hack Wilson 38.8
Chief Bender 38.5
Rabbit Maranville 38.0
Burleigh Grimes 37.2
Pie Traynor 37.0
Herb Pennock 36.9
Roy Campanella 36.3
Ross Youngs 36.2
Jesse Haines 33.8
George Kell 33.5
Jack Chesbro 33.2
Catfish Hunter 32.5
Jim Bottomley 32.3
Chick Hafey 29.4
Freddie Lindstrom 29.1
Rube Marquard 28.5
George Wright 28.5

While some of these players are among the weaker selections in history, there are others who were great major leaguers. If there’s a relief pitcher that provided as much total career value as Lou Brock, for example, I’d be perfectly happy seeing him in the Hall of Fame.

So, let’s go back to my previous reliever analysis and look at the Reliever WAR table. I’ve cut it off at around 23 WAR (way below 40 WAR). I made a couple of other modifications (in addition to updating the career stats for Mariano Rivera, Trevor Hoffman, and Billy Wagner).

  1. I removed Dennis Eckersley. Eck was a great reliever. However, he wouldn’t have made the Hall of Fame as a reliever without his career as a starter and he wouldn’t have made it as a starter without his relief career. His totaled just under 800 relief innings, so the workload comparison would be Tom Henke. Eck produced 16.6 WAR in his relief years while Henke produced 23.1. Henke didn’t even sniff the Hall. Still, if you add Eck’s contributions as both a starter and a reliever, he’s clearly a Hall of Famer. Eckersley was truly a unique case and doesn’t really apply to this debate.
  2. I also removed Ron Reed as most of his value came from being starting pitcher as well.

So, here’s the list of top relievers by WAR, with the WAR components added as well:

Player WAR IP ERA+ Runs Def Lev R+ BB+ HBP+ SO+ HR+ Hit+ X
Mariano Rivera (B-R) 49.9 1090.0 202 295 -17 1.8 313 36 0 83 78 48 64
Hoyt Wilhelm (B-R) 41.3 2254.1 146 773 45 1.3 295 7 -6 96 70 78 48
Rich Gossage (B-R) 40.0 1809.1 126 670 -14 1.6 187 -26 -6 132 23 27 36
Trevor Hoffman (B-R) 31.5 1042.0 147 347 -25 1.9 197 21 9 104 7 37 17
Lee Smith (B-R) 30.3 1289.1 131 475 -13 1.8 137 -10 4 116 12 -22 34
John Hiller (B-R) 28.2 1242.0 134 438 -1 1.5 133 -27 6 89 -20 13 73
Billy Wagner (B-R) 27.0 833.2 182 248 8 1.8 186 2 1 143 -8 26 22
John Franco (B-R) 25.8 1245.2 137 466 -6 1.9 141 -11 3 34 48 -14 81
Bruce Sutter (B-R) 25.0 1042.1 136 370 -8 2.0 105 13 1 67 -10 13 18
Kent Tekulve (B-R) 24.8 1436.1 132 526 -5 1.4 132 5 4 -17 68 26 47
Rollie Fingers (B-R) 24.4 1701.1 119 615 21 1.6 168 30 -2 87 14 10 28
Dan Quisenberry (B-R) 24.3 1043.1 146 356 16 1.5 159 59 4 -58 69 8 78
Lindy McDaniel (B-R) 23.4 2139.1 109 924 18 1.2 58 34 12 19 40 -37 -11
Tom Henke (B-R) 23.1 789.2 156 252 8 1.7 150 5 2 105 0 12 20

Are you seeing what I’m seeing? There are three guys at 40+ WAR. Not only that, there’s a pretty severe dropoff before the next group. These three, of course, are the truly remarkable Mariano Rivera, the ageless stalwart Hoyt Wilhelm, and perhaps the best combination longevity and dominance (this side of Mariano Rivera) Rich Gossage. These three pitchers make up our first tier. I’ll call that tier:

Tier 1: Relief Pitchers who deserve to be in the Hall of Fame, no questions asked

These players provided enough value out of the bullpen that they compare favorably even with Hall of Fame starting pitchers and hitters. They deserve induction under any circumstances.

  1. Mariano Rivera
  2. Hoyt Wilhelm
  3. Rich Gossage

While there is a large gap after these three pitchers, there are still a couple pitchers who exceeded 30 WAR and also carried some significant career accolades. Both Lee Smith and Trevor Hoffman either was (in the case of Smith) or is (in the case of Hoffman) the all time career saves leader (Note: this is the first time saves were mentioned so far).

I’ll be honest. I’ve been such an anti-saves guy over the years that I think I was too quick to dismiss Lee Smith. He was actually a fantastic reliever and I’d have no problem seeing him in the Hall of Fame. He certainly wouldn’t be the worst player in the Hall.

Trevor Hoffman, like Mariano Rivera, will inevitably be elected to the Hall of Fame on the strength of what will be 600+ saves. Stat geeks can take solace in the fact that his induction won’t be a travesty based on an inflated, meaningless stat. Hoffman passed Smith in career value in 2009 and still isn’t done. Hoffman posted 2.5 WAR last year, his highest total since 1998. He currently sits at 31.5 and could conceivably push himself towards 35.0 by the time he’s done.

So, I’d put Smith and Hoffman in the next tier:

Tier 2: Relief Pitchers who could be elected, but would be near the bottom of the Hall in terms of career value

These players were among the best at their craft, but when compared to other Hall of Famers are on the extreme borderline. If elected, they’d be near the bottom of the Hall roster in terms of total value, but still wouldn’t be the worst players elected.

  1. Trevor Hoffman
  2. Lee Smith

Then there’s the elephant in the room. We’ve already got two guys in the Hall of Fame—Rollie Fingers and Bruce Sutter—who fall well below even Tier 2. So while my original hypothesis called for the induction of more relievers, my findings suggest we may actually need to trim the list. Yikes!

The fact of the matter is all relief pitchers beyond Tier 2 significantly fall behind the typical Hall of Fame standards of other players. If you want more relievers in the Hall of Fame than those five (and two of them are still active and not yet eligible), then really that means you believe in at least one of these statements:

  1. Relief pitchers should be held to a lower career value standard than starting pitchers and hitters.
  2. Fingers and Sutter are in the Hall of Fame already, so you can’t leave guys out who were better than them.

So, if you’re on board with one of those statements, there are five more pitchers you should consider.

John Hiller is just behind Smith in WAR (28.2). In 1973, he posted a WAR of 6.9. That is ridiculous for a reliever. Remember that from 1987 through 1998, Dennis Eckersley totaled 16.6. That’s the type of magical relief year that reminds me of Mark Eichhorn’s 1986 rookie year (6.4 WAR). Not only did Hiller set a new saves record with 38 in 1973, but he also recorded a record 17 relief wins in 1974. He did all this after suffering a heart attack in 1971 and working his way back to the big leagues (losing a year and a half of his prime). Hiller appeared on the Hall of Fame ballot just once, getting 2.6% of the vote. While I’m not sure he deserves induction, he certainly deserved a longer look.

Billy Wagner recently returned from Tommy John surgery and used a strong 15-appearance stint with the Red Sox to land a new contract with the Braves. Hopefully Wagner can return to his Hall of Fame pace. Wagner is one of the greatest pitchers of all time in terms of ERA+, but he’s also a one-inning guy so WAR is helpful for evaluating him. He has reached a career high WAR of 3.9 twice. The remarkable thing is he’s able to do this in 86 and 75 innings. That’s a lot of value packed into a little workload. This will be Wagner’s age 38 season and there’s no reason to think he can’t add to his WAR value. He’ll be an interesting Hall of Fame case. I have confidence that Hoffman will easily be inducted, but I bet Wagner is not.

John Franco piled up WAR the same way he piled up saves and an impressive ERA+. With consistent solid years and a lack of dominance. He peaked at 3.2 WAR in 1988, while he was still with the Reds. Franco has 400 saves and was second all time when he retired. He also has over 1000 career appearances (third all time). I have the feeling he’ll be on one Hall of Fame ballot and then be finished. He’s intriguing though. Kind of reminds me of a relief version of Tommy John—slowly piling up value to finish with an impressive career total.

Kent Tekulve, like John Hiller, is one of those guys that makes you say… really? Tekulve is kind of a hybrid closer. He pitched a ton—over 1000 appearances—but also averaged about an inning and a half per outing. So, he wasn’t strictly a one-inning guy, but also not quite an old school fireman. He was a bespectacled submariner, though… and that makes him awesome in my book. He peaked at 3.5 WAR in 1983 and exceeded 3 WAR a couple other times. Again, nothing flashy. Just a solid reliever for a long time—which is more rare than you think.

Dan Quisenberry has always been my pet case. In fact, he’s the reason I started researching relief pitchers. I couldn’t understand how a guy with such an incredible six-year peak who was also tied for third all time in ERA+ (at the time—since then, Rivera and Hoffman have reached the 1000 inning minimum to qualify). In the six seasons from 1980 to 1985, Quisenberry placed in the Top 11 in the MVP voting five times, Top 5 in Cy Young five times, and was Rolaids Relief Man of the Year five times (no other pitcher has won the award five times). You might think that sixth season (1981) was a down year. Nope, he happened to have a 1.73 ERA—his best in the entire six year span.

What holds Quisenberry back? It is just a lack of work. He is incredibly similar to Bruce Sutter, though the two were much different pitchers. Both barely cleared 1000 innings. Both dominated while they pitched, but didn’t have long careers. In Quiz’s case, he got a late start at age 26 and pitched until he was 37. He tore his rotator cuff after five games in 1990 (his first serious injury) and called it a career. Sutter, on the other hand, started at age 23 but threw only 64 innings after age 32. There are many things I don’t understand about Hall of Fame voting history. But the biggest is how the BBWAA can elect Bruce Sutter while Dan Quisenberry fell off the ballot after one year at 3.8%. I believe it’s the Cy Young Award and the pioneering of the split-finger that got Sutter in. That six year peak was packed with so many accolades that I think it pushes Quiz over the line, as well.

Sadly, Dan Quisenberry passed away in 1998 after battling brain cancer.

I also want to briefly mention Tom Henke here. Go back and look at Henke’s stats. He was remarkable. But with under 800 innings, there’s no way to seriously consider him for the Hall of Fame. He only pitched 100 innings before age 28 and retired at age 37 after saving 36 games with a 1.82 ERA. He was remarkable for his entire career. it just wasn’t long enough.

To recap, here is Tier 3. I’m ordering them by how much value they provided compared to their innings pitched.

Tier 3: Relief Pitchers who you can make the “If Fingers and Sutter are in, why not him?” argument with

These players are below the normal career value for a Hall of Famer. But they also put up similar or better value than two relievers who are actually already in the Hall of Fame. So, if you believe in fairness, perhaps they should be in, too. That, of course, is a slippery slope.

  1. Billy Wagner
  2. Dan Quisenberry
  3. John Hiller
  4. John Franco
  5. Kent Tekulve

So who would I support?

I don’t have a Hall of Fame vote, but I sure wish I did. If I did, here are the relief pitchers I would support:

  1. Mariano Rivera: As highly regarded as he is, he may actually be underrated. There has never been a relief pitching remotely close to him. He’s knocking down the career value of Hall of Fame starting pitchers as well, which is simply remarkable.
  2. Hoyt Wilhelm: According to WAR, Wilhelm gets in on longevity. According to ERA+ (146), he gets in based on dominance. He’s a good combination of the two, always steady but never flashy. He was a pioneer.
  3. Rich Gossage: While Goose also had longevity, it is his peak of stardom that cements his case.
  4. Trevor Hoffman: I’m willing to venture into Tier 2. Forget about the saves—Hoffman packed a ton of value into just over 1000 innings. He’s not done yet by any means and could still add a few WAR to his total.

If we were starting the Hall of Fame from scratch, that is where I’d stop. I’ll even add the caveat that Hoffman may not be a Hall of Famer if he hung them up right now. But I believe he has a couple solid years left. I really don’t think I’d include Rollie Fingers and Bruce Sutter in the Hall of Fame. Fingers I can understand because he had the save record for quite a while. But his performance certainly lags behind those top four.

But, since Fingers and Sutter are already in, I can’t bring myself to leave these guys out:

  1. Lee Smith: I have to say… he’s better than I gave him credit for. He’s pretty much the dictionary definition of borderline, but I think being the all time saves leader for a good period of time (as much as I dislike the save statistic) gives him points towards the “Fame” part of “Hall of Fame”. I’d put him in, but I wouldn’t be upset to see him left out.
  2. Dan Quisenberry: I’m sorry. This is my pet case. He was the best—by a lot—for a six year period. He was unorthodox, but he always got the job done. It’s too bad he got a late start to his career (he was undrafted) and also had an early exit (due to a torn rotator cuff). Of course, neither of these compares with the fact that he lost his battle with brian cancer. I’d put Quisenberry in. Heck, I have him ahead of Fingers and Sutter (and possibly Smith as well—he did almost as much as Smith in far fewer appearances). I may be clouded by the haze of fandom, but I believe Dan Quisenberry is a Hall of Famer.

Add Fingers and Sutter and that’s eight relief pitchers (keep in mind that two have not retired yet, but also keep in mind that I’m not counting Eckersley). A third active reliever, Billy Wagner, has a chance at induction. We’ll have to see how he finishes his career. He has certainly dominated so far, but I just don’t think he has the number of innings to be seriously considered.

Francisco Rodriguez was on a Hall of Fame path up until his 2009 hiccup with the Mets. After a five year span of 3.5, 2.9, 4.1, 2.6 and 3.2 WAR, K-Rod slumped to just 0.2 last year. Still, in 519.2 innings, he has accumulated 18.2 WAR. That’s a lot. Over 1000 innings, that translates to 35.0 WAR. That’s enough to get you in the conversation, for sure. Had K-Rod continued on his pace, he’d be looking at something more liked 40 WAR.

Joe Nathan didn’t become a closer until 2004, but over the past six seasons he has made up for lost time. He has collected 22.1 WAR in 685 innings. Entering his age 35 year, he’ll need to produce at this level for a few more seasons though. He has shown incredible consistency (rare for a closer), so it is possible. The last active pitcher worth keeping a close eye on is Jonathan Papelbon. While he hasn’t been around too long, he already has earned 14.7 WAR in just under 300 innings. Prorate that to a 200-inning season and you’re looking at just under 10.0 WAR. That’s pretty nuts. It’ll be tough to keep up that pace, but if he does he’ll get in the conversation.

And there you have it. There are three relievers who deserve induction no matter what your standards are and a couple more who could pretty easily be in as well. The third tier is interesting, though. Two guys are already in… who else (if any) should be in from that group?

What do you think?