Answering criticisms of Win Score

Last night I was pumped up because I had finished my regression analysis and found that the metric I use, Marginal Win Score, is highly consistent and therefore highly predictive — better than any other metric I could find.  The wins a player is credited with last season have a .88 correlation with the wins he produced this season.  That’s a monster victory in the war to make MWS48 legitimate.

So anyway I’m searching on-line because I wanted to find out if anyone had done a similar regression for Win Shares, the metric used by Basketball-Reference.  Instead what I found was unbelievable vitriol against Win Score and its creator, Professor Berri.  It gave me pause whether I wanted to continue with this thing.

On message board after message board they attacked Berri’s metric, his method, and most astonishingly, his character.  In all of that, though, I only read one solid criticism of Win Score.  But let me address some of the ones I read.

1. He “weights rebounds heavily”

This is not true, completely untrue.  Well, if it is true, one must also say “He weights points heavily” and “He weights steals heavily” and “He penalizes turnovers heavily”, because Berri’s model weights all of those things equal to rebounds.  Points and possessions get full value.  There is no preferential treatment of rebounding.

2. “His weighting is arbitrary”

Somehow the belief has gotten out there that the weights Win Score places on each box score stat is just something Dave Berri pulled out of his ass.  Some clown even commented “Berri could just claim Mourning produced the most wins by weighting blocks”. 

Okay, first of all, Win Score is based upon regression analysis done to determine how the traditional box score stats correlate with wins.  Those stats in turn were the product of lengthy research.  Berri didn’t simply conjure up a metric.

3. It undervalues the difficulty of scoring

Here is the crux of the matter.  Its not that basketball fans love Allen Iverson, or whomever.  Its that basketball fans fundamentally believe scoring is difficult and rebounding is pretty easy.

The best most intelligent criticism of Win Score I’ve read was on FreeDarko.  In essence the author agreed with Berri that scorers are overpaid for the amount of wins they produce.  But, he argued the “scorer premium” was necessary because under a Win Score regime, scoring involves risk (if you miss you’re deducted one WS point).  Focusing one’s efforts on possession creation involves no risk.  Therefore under a Win Score regime those that take the risky path should be rewarded.

Here’s my rebuttal.  The author is creating a straw man argument.  He is being too simplistic. 

At one point the author argues that players will opt for assists rather than shots.  He ignores the fact that assists involve the risk of turnover.  In fact, he ignores every risk of negative WS points other than those created by a field goal attempt.  So I think his argument is a bit disingenuous. 

But further, under Marginal Win Score you don’t have the problem of risky shooting.  No one is necessarily required to take a shot unless and until the opponent takes and makes a shot.  At which point the incentive to avoid shooting that the author speaks of disappears.

I have to go.  I’ll continue this discussion later.

Advertisements

11 Responses to “Answering criticisms of Win Score”

  1. brgulker Says:

    I really look forward to hearing more. I’ve yet to find a criticism of Win Score that is as scientifically rigorous as Win Score itself.

    I would add to your rebuttal of #1, though. It isn’t Dr. Berri that weights rebounding heavily; it is the game of basketball that weights rebounding heavily.

    Once one understands that a rebound = a possession, and once one understands that the game of basketball is fundamentally about maximizing the number of and taking advantage of one’s possession while minimizing and making the possessions of the opponent more difficult, one begins to understand why rebounds are so valuable (and Win Score (and its derivatives) begin to make a lot more sense).

    The best critiques I’ve read are in their essence critiques of the Box Score. Because Win Score (oversimplified) regresses box score statistics on wins, it makes sense to criticize the box score (because it’s pretty hard to levy a scientific critique against the approach). If the Box Score is incomplete, then Win Score is incomplete. Dr. Berri wants to track charges and credit the player who takes the charge with creating a TO. I agree.

    I’d also like to see deflected passes and altered shots on defense and on the ball and/or ball screens that lead to a score on offense — but I don’t have the foggiest about how one would accurately and objectively track those things.

  2. Alvy Says:

    The criticism are sort of brought up by people who haven’t read Wages of Win. I don’t see how legit one can actually take those remarks.

  3. BadgerBucco Says:

    Ty,

    I like Win Score, but the biggest criticism is the value of creating your shot. If you look at Eli Witus’ work, it shows that as you take more shots, your efficiency goes down. You can’t have a team full of possession-creators who only used 15% of the teams possessions on offense. If you have such a team, some of those players will be forced “out of their comfort zone” and their efficiency will decrease. The best metric normalizes scoring efficiency based on possession-usage.

  4. dberri Says:

    BadgerBucco,
    Are you referring to the Eli Witus model where he regressed a dependent variable on exactly one independent variable? This is another problem with the on-line stats community. The understanding of how to construct a regression model seems very limited. Not only did Eli Witus fail to control for anything in his model, when this obvious problem was noted, his colleagues in the on-line community just reacted with more anger and personal attacks.

    I will add… the link between field goal attempts and shooting efficiency is quite weak. This is noted in Stumbling on Wins.

    And of course, thanks Ty for the post.

  5. BadgerBucco Says:

    Professor Berri,

    I must admit that, as a lawyer, I’m not an expert on regression. However, I think that what Witus was looking at (which I didn’t state correctly), was really offensive rating and possession usage rather than field goal attempts and shooting efficiency. Have you looked at the link between those two statistics?

  6. dberri Says:

    BadgerBucco,
    No. I have not done any studies with offensive rating. There are problems with that model (although not as many problems as we see with PERs and plus-minus models).

    The story people have told is that WP underestimates players like Allen Iverson (who takes many shots) and overestimates the value of players who take few shots. But shot attempts and shooting efficiency do not have the relationship people argue exist.

    On the Witus study (assuming we are talking about the same Witus “study”)… what he did was regress one variable on a single independent variable. He then proceeded to discuss the value of the estimated coefficient. This coefficient, though, doesn’t mean much because nothing was controlled for in the model.

    Unfortuanately, many people in the APBRmetrics community are like you BadgerBucco. They simply do not have much training in regression analysis. From what I understand, Eli Witus is also a lawyer and I think I read once that he has no formal training in how to do regression analysis.

  7. BadgerBucco Says:

    Professor,

    I’ll admit that Witus’ analysis may not be the best based on your more educated opinion. If what you say is true, the study should be redone properly by someone such as you.

    That said, my other point still stands: I think we should be looking at possession-usage and some form of overall offensive contribution rather than some form of field goal percentage and shot attempts as you’ve analyzed. If that shows no relationship, then it shows no relationship. I’m not one to stick with an opinion just to stick with it.

    My intuition is that usage is overvalued in some systems and am fully on board with you about Allen Iverson’s possession usage skills not being very helpful given his poor offensive contribution when he uses a possession. However, just because one player uses too many possessions, doesn’t mean that the relationship doesn’t exist. If I am correct, it just means that he should lower his usage by foregoing the worst of his offensive opportunities, thus increasing his overall offensive contribution.

    I’m not sure how you feel about thought experiments, but your argument seems to support the idea that if I had a team of Ben Wallaces (or the like at the relevant position), I would win the championship. Given his limitations on offense, I can’t imagine him maintaining his offensive contribution by forcing him to step out and take the 15 foot jumpers or contested hook shots that he normally foregoes.

    By the way, I sense some form of defensiveness in your responses. That may be because of the reactions you’ve received. Please don’t paint me with that same brush, though. I seek the truth (or as close as we can get) and a good back and forth helps the dross to fall away.

    Finally, what is the problem with offensive rating?

  8. R Says:

    If you set your model up with total rebounds as a variable you will get different results than if you set it up with offensive rebounds and defensive rebounds as separate variables.

    • tywill33 Says:

      But why would you distinguish between the two? Its the same act whether it occurs on your end of the court or on your opponents end of the court.

      In both instances the basketball is “loose” in other words possessed by neither team. The player that grabs it gains possession for his team.

      I don’t understand the desire to reward offensive rebounds or consider them more valuable than defensive rebounds. If offensive rebounds are so valuable, then a defensive rebound must be equally valuable because it denies an offensive rebound to the other team.

      The reason people want to downgrade defensive rebounds is because offensive players are rarely aggressive on the boards. Generally an offensive player will not attack the glass unless he’s “in the vicinity”. Watch tonight’s game. I’ll bet you won’t be able to fill up the fingers on one hand counting the number of times a player attacks the glass from the perimeter. Players just don’t do it. They either consider it too difficult, a waste of energy… just plain not worth it. But there’s no reason on Earth that ANY rebound should go uncontested.

      I’m willing to bet the farm on this point. If teams started paying premiums to offensive rebounders, you would quickly see the number of uncontested defensive rebounds go from 30% to nearly nothing overnight.

      In fact, if players were given an incentive to relentlessly bang the offensive glass, the true value of ALL rebounds would very quickly become apparent to non-believers.

  9. R Says:

    I should have also said some models up with offensive rebounds and defensive rebounds as separate variables get significantly different weights for them but at least one doesn’t. Too much time & expertise involved to fully review this issue for me right now.

    Most teams don’t hit the offensive glass all out because transition offense by the opponent if they still succeed in getting the rebound- which they usually do to the rate of at least 65% of the time even against the greatest offensive rebounding lineups- is more efficient than half-court offense by a pretty large amount.

  10. R Says:

    Most of the weights for the stats in Wins Produced are quite firmly in order with each other. There are a few very tiny differences from a simple subjective value of possession but no other regression of boxscore stats I’ve seen shows this degree of consistency, this degree of similarity to a set of logical value of possession assumptions.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: