Dr Bill Gerrard of Leeds University’s Business School has provided getstats with an insightful account of life as a statistical analyst with the Oakland A’s and the Saracens. It is the story of the role analysts play in evidence-based coaching in elite team sports.

   _______________________________________________________

Moneyball: The Art of Winning an Unfair Game, the best-selling book by Michael Lewis and now a Hollywood movie, tells a real-life David-and-Goliath story of how the Oakland A’s, one of the smallest franchises in Major League Baseball, have tried to compete with the mega-rich teams such as the New York Yankees and Boston Red Sox. The story focuses on Billy Beane, the General Manager of the A’s (played by Brad Pitt in the film), and how he has employed statistical analysis to gain a competitive advantage. Ultimately professional team sport is about transforming money into wins by signing talented players. Billy Beane realised that he could buy wins more cheaply by using the available player performance data better. Statistical analysis showed that the conventional metrics used by baseball teams to evaluate hitters and pitchers were not the best statistical predictors of success. Billy Beane exploited this market inefficiency to identify players who were undervalued and achieved efficiency gains of around 60% better than the league average between 1998 and 2006. In recent years the A’s have struggled to maintain their competitive advantage as competitors with more financial resources have emulated their methods. But the lesson is clear: statistical performance analysis (SPA) can be a powerful tool to support coaching decisions.

So is the Moneyball approach transferable to other team sports such as football and rugby? It is no surprise that the innovation emerged first in a striking-and-fielding sport where the contributions of individual players are highly separable and performance data are relatively easy to collect. In the more complex invasion team sports such as football, rugby, hockey and basketball, there are considerable technical problems in gathering and analysing performance data. Technological developments have largely resolved the data gathering problems through video analysis, image recognition and GPS systems. And top teams are beginning to employ statisticians to analyse and interpret this data. However there remains considerable coach resistance to the use of SPA especially in football. There is a real Catch-22 situation – outsiders like myself trying to develop SPA need access to the data to show the practical value of SPA but it is difficult to gain access without having already established the usefulness of SPA. Indeed such as been the scepticism amongst football teams in England that most of my work in elite sport in recent years has been outside the UK and/or outside football. Much of my recent development of SPA has been in collaboration with Billy Beane and the Oakland A’s ownership group. The A’s also own a football team in Major League Soccer (MLS) and I worked with them over a three-year period to investigate what was possible from applying SPA to football. One of our key innovations was a Value-For-Money rating for every MLS player that could be used to inform player trading decisions when operating under a tight salary cap.

After finishing my assignment with the A’s, I was introduced to the coaching staff at Saracens, one of the leading teams in English rugby union. Under their then Director of Rugby, Brendan Venter, a qualified GP, Saracens had already adopted an evidence-based approach strongly influenced by medicine. Their philosophy puts an emphasis on long-term progress in improving their own controllable performance levels. This philosophy provides a great context for applying SPA, identifying key performance indicators (KPIs) and working in training to improve these to increase the probability of success. Games are analysed closely to identify the controllable factors that significantly affected the outcome. Everyone is disappointed to lose but at Saracens there is a real commitment to treating losses as feedback from which to learn and improve.

Evidence-based coaching at Saracens involves a five-stage process. It starts with the coaching vision of the “perfect game”, a tactical game plan covering the different aspects of play – attack, defence and set-plays. After every game each coach analyses his area of responsibility and evaluates how well the players have performed relative to the game plan. Particular emphasis is put on evaluating the decisions made by players and their skills implementation. The next stage is for the data generated by the coaches to be analysed. This leads on to monitoring team and player KPIs using a dashboard with a traffic-lights system to classify performance levels. These KPIs are not only tracked and interpreted by the coaches but are also made available to the players who can access these KPIs after every game on their iPads and laptops. There is a “self-coaching” approach with players encouraged to take responsibility for their own professional and personal development, guided by the performance metrics and supported by the coaches. This leads to the final stage of the coaches determining the appropriate interventions to improve future team performances. If any decision KPI is classified as red then this signals a need to improve the understanding of that aspect of the game plan. If it is a skill KPI that is red then this necessitates additional practice time in this area of the game.

In many ways Saracens have gone “back to the future”. Instead of using the activity data produced by the various sophisticated tracking systems that many other teams employ, the coaches at Saracens rely on their own game video analysis much as previous generations of coaches have done. The crucial step is the systematic recording of the coaches’ expert evaluations of player performance. Systematically recording these evaluations creates the data that can then be analysed. Importantly it is only the Saracens coaches who can determine how effectively the players are implementing the Saracens game plan. Third-party commercial data providers can only provide activity data of how much players have done. Only the team coaches are in a position to produce effectiveness data.

My contribution as the statistical analyst is to search for patterns in the data, transforming the data into evidence to help inform coaching decisions on player recruitment, tactics, team selection, match preparation and training priorities. At Saracens statistical analysis provides additional oil in an already well-oiled evidence-based engine. Statistical analysis did not win the Aviva Premiership for Saracens in May 2011. But it certainly contributed towards helping an exceptional set of players and an exceptional set of coaches achieve well-deserved success.

Dr Bill Gerrard
September 2011

Print PDF