On the surface this seems like an easy thing to do: who is winning the most? Take a look at Cups, at career winning percentage, and bingo-bango, you have your answer–these are the best coaches. There’s something very tidy in that simplicity, but obviously there’s more to it. Oddly, there isn’t a lot written on the subject, but I’ve scrounged around and found some material to dig into.
Jonathan Willis focuses on how fans and the media tend to criticise the wrong things (or focus on only a few factors) when it comes to coaching, but within that piece is this gem about Scotty Bowman:
The one thing my dad’s always been so good at, I think, is he’s been able to adjust…. [F]or a guy who’s ‘old school’ and has been around so long, he’s incredibly progressive and willing to try new things, willing to do things which are not the norm, and that’s what made him successful as a coach … he was very unpredictable … I think all coaches today are kind of – I don’t want to say programmed – but they’re led to do a certain thing. So if you can force yourself to try things maybe a little different or take a different approach, it’s going to give you that advantage. Ironically I think what makes him so exceptional is that he didn’t think he had all the answers.
Stan Bowman is absolutely right about how robotic some NHL coaches are–it’s their way or the highway.
Dave Berri spoke quite generally about coaching changes and in it he cites a study from The Sports Economist which concluded:
We find that for particularly poorly performing teams, coach replacements have little effect on team performance as measured against comparable teams that did not replace their coach. However, for teams with middling records—that is, teams where entry conditions for a new coach appear to be more favorable—replacing the head coach appears to result in worse performance over subsequent years than comparable teams who retained their coach.
This echoed other studies, including one on the NHL:
The effect of a change of coach on team performance in the NHL has been estimated in a parametric model, also based on match-level data. Ordered probit regression has been used to represent the discrete and hierarchical structure of the ‘win-tie-lose’ match-results-dependent variable. The use of lagged match results data provides a control for the phenomenon of mean-reversion in team performance. The empirical results suggest teams that changed their coach within-season tended to perform worse subsequently in the short term than those that did not. However, the detrimental effect appears to be short-lived, and over a longer time horizon the effect is almost neutral. In the broader context of the debate concerning the managerial influence on organizational performance, the results suggest that a change of management in the midst of a crisis is unlikely to improve performance by more than might have been expected through the natural tendency for mean-reversion after a spell of poor performance.
Berri concludes:
That suggests that coaches in sports are not very different from each other. It may be true (and more than likely very true) that you are better off with a professional coach than with a random person grabbed from the stands (or no one at all). But it doesn’t appear that the choice of professional coach matters much.
If your mind revolts at the idea that coaches are a rotation of robots whose performance is simply an indication of how good their roster is, I understand the feeling. Let’s keep in mind that there are always exceptions to the rule and, indeed, there are signs that it’s not so simple.
A newer study by Jean-Rene Gauthier finds that hiring younger, more inexperienced coaches has a more positive impact (presumably a more adaptive group); coaches with historical losing records also provide as boost, as do (paradoxically) those who have won a Stanley Cup (all the post-lockout winners, other than Randy Carlyle, are coaching at the moment). He concludes that firing has an overall positive effect (which runs contrary to earlier studies).
Nick Emptage offers a much narrower and more analytics-focused look, which you can peruse at your leisure (Carlyle was a bad coach kids–who knew?). In a similar vein check out THN or Benjamin Wendorf. Taking a look at these numbers we can see that, while it may not always be born out in the win column, certain coaches employ much better systems and better player usage and that shows in their possession and other metrics. It’s this latter reason that you can look at Mike Babcock’s lousy 21-32-10 record with Toronto and understand the team is performing better with him than without him.
What, ultimately, can we conclude from this mountain of data? Like everything else, comparative analysis will ultimately tell you which coaches are having a positive impact, regardless of their win-loss record.
This article is written by Peter Levi (@eyeonthesens)
6 Comments
Comments RSS TrackBack Identifier URI
eyeonthesens 
[…] 1-3-1 style, believing that after his first season teams adapted to it. When I researched coaching success back in March I came up with […]
[…] that line of logic too far and a coach is never responsible for anything). When I looked at coaching success earlier this year it was clear that a good coach can’t make a bad roster better–what […]
[…] the oddly-built Columbus Blue Jackets to romp through the league? I’ve looked at coaching before and the collective analysis over the years broadly concludes that 1) results are mostly due to the […]
[…] sets of analysis point back to what I found looking at the impact of coaching about a year ago, where it’s extremely difficult to separate out the effect of coaching […]
[…] roster needs to be truly superlative to overcome the inadequacies in charge of them. I looked at the impact of coaching a year or so ago (something still poorly understood; my link in that piece to Nick Emptage’s […]
[…] I’ve been working on a detailed breakdown of the BSens PK (similar to what I did on the powerplay a month ago), but while that’s in progress I thought I’d go over the team’s player usage vs success on both special teams up to this point (if coaches are going to have any impact on a team, this is one of the most noticeable places for it–for the difficulty in understanding the impact of coaching see here). […]