Key takeaways
– Based on OptaPro’s attacking ratings, the margin between the All Blacks at the top and second-placed South Africa is the same as the gap between South Africa and Fiji, who rank eleventh.
– However since October 2017 New Zealand’s defensive strength has declined slightly, while the other contenders’ performance levels have all increased, meaning the defensive rankings are more tightly packed.
– Despite home advantage, Japan only have a 46% chance of reaching the knock-out stages according to OptaPro’s projection with New Zealand, England and South Africa all having projections in excess of 90%.
After close to a decade at the top of the official international rankings, New Zealand recently found themselves supplanted by reigning Six Nations champions Wales. A whole fortnight passed before the All Blacks returned to the top, only to again find themselves dropping back to second spot, this time behind Ireland. Clichés regarding buses have abounded since.
With the 2019 Rugby World Cup (RWC) beginning today, questions have been asked about whether the rankings are an indicator of New Zealand’s relative decline, while also potentially signalling Wales and Ireland’s improved prospects.
Given the All Blacks dominance of world rugby over the past decade, during which time they’ve won both editions of the RWC as well as seven of the ten editions of the Tri-Nations/Rugby Championship, seeing them fall back into the pack would represent a significant shift in the hierarchy.
In order to investigate whether the All Blacks performance levels have declined, as well as assess the contenders for the RWC, we’ve developed a model to quantify team strength based on the final score lines of international matches from 1983-2019. This period covers the four years prior to the inaugural RWC in 1987, up to the present day.
The model is built using a Bayesian modelling framework with PyMC3 to define a hierarchical Markov Chain Monte Carlo (MCMC) model that factors in home advantage, the era of the match to adjust for changes in the points awarded for a try, as well as the attacking and defending strength of the opponent. This allows us to determine team ratings for both attack and defence separately. These ratings are calculated over two-year periods running from October-to-September e.g. ratings for 2019 are based on results from October 2017 – September 2019. For the ratings below, we exclude RWC matches so that we can measure performance in the lead-up to tournaments.
Further details of the model’s structure are included at the end of the article.
Team performance levels
The figure below displays the attacking strength of the participants of the 2019 RWC over the past two years. The values are converted to the mean expected points they would score against an average Top-8 opponent (based on the model’s ratings over time), with the modern reference point being the current Argentina side.
Consistent with their skilful reputation, New Zealand are comfortably the strongest attacking team going into the RWC. The margin between them and South Africa in second is the same as the margin between South Africa and Fiji in eleventh place.
RWC 2019 hosts Japan sit in eighth place, ahead of France, reflecting the diminishing flair and attacking prowess associated with the latter over the past decade. Japan’s attacking performance levels have improved considerably over the past two years and are now slightly below that of an average team from 1983-2019.
The figure below focusses on the defensive strength of the 2019 RWC teams. Again, the ratings are converted to the mean expected points total they would concede against an average Top-8 opponent. The modern reference point in this case would be the current Australia team shown above – the attacking strength of top-tier teams is somewhat lower than their historical counterparts, potentially reflecting a greater emphasis on defence rather than attack in the modern game.
Performance levels for the top-teams is much more tightly packed in the case of defence, with little to separate the main 2019 RWC contenders. New Zealand aside, the bedrock of these teams is defence, with the top-9 teams all ranking as either average or better-than-average compared to the best top-level teams over the period from 1983-2019.
As far as the hosts are concerned, their Achilles heel is their defence, ranking thirteenth of the RWC teams. Of the comparative minnows, Georgia exhibit the greatest defensive strength and are on a par with Argentina and better than Italy.
Overall, the clear stand-out team when combining attacking and defensive strength is New Zealand, who would be expected to beat an average Top-8 opponent by around 19 points at a neutral venue (i.e. excluding home advantage). England and South Africa sit in second and third respectively, with a further gap between them to a grouping of Ireland, Australia and Wales. Beyond that, performance levels are at an average level or worse.
Performance levels over time
The results above indicate that these top-6 teams are the main contenders for the forthcoming RWC tournament, with the All Blacks as the leading light in contrast to the official rankings. The above does not examine the potential decline or ascent of these contenders though. The ‘form’ of these teams is judged by comparing their attacking and defensive strength from 2015-2017 to 2017-2019 to further assess their credentials and place current performances in a slightly longer-term context.
The attacking strength of the main contenders for the 2019 RWC is shown below via a ‘slope graph’, which connects their ratings in the 2015-2017 period to those from 2017-2019, as well as indicating their performances relative to their rivals.
The stand-out feature in the attack ratings is the decline in performance level of New Zealand by four points against an average Top-8 opponent. While four points sounds like a relatively small amount, in terms of win probability, it amounts to approximately a 4% shift against such an opponent. In closer contests against stronger opponents, this shift is magnified e.g. their win probability would drop by approximately 8% against the current England team.
Of the other teams, England and Australia’s attacking performance level has dropped by two points, while South Africa’s has increased slightly. Wales and Ireland have maintained their ratings since the last RWC.
The defensive strength of these teams is illustrated below; note the vertical axis is flipped, with lower point values indicating a better defence.
New Zealand’s defensive strength has declined slightly, while the other contenders’ performance levels have all increased. This tightening of the pack on the defensive side coupled with the All Blacks attacking decline does indicate that the teams are closer going into the RWC than they were in the period after the last edition. While clearly still a strong team, this is the weakest All Blacks team going into a RWC tournament since the very first iteration in 1987. However, their strength relative to their peers is greater than the 1987 team, as well as being greater than in 2015. The strength in depth of their immediate peers is also lower than in 2011 and 2015.
Wales’ shift towards defensive fortitude, aligned with the kicking ability of Biggar and Halfpenny is a potential recipe for success, particularly in closely fought knock-out tournaments. Against an average Top-8 opponent, their improved overall rating increases their win probability by 7%, making them solid favourites in many matches. The current Welsh team boast their best defence heading into a RWC, although the attack ranks towards the lower end of their historical performance level and is 8 points below the attacking verve of their 2005 Grand Slam winning team. Overall though, this is Wales’ best side heading into a RWC.
While the above analysis contrasts with the official rankings in terms of the gap between New Zealand and the other RWC teams, it does illustrate the closing of that gap over the past two years as the former’s form has declined and the chasing pack has improved.
Tournament forecast
Taking the attack and defence ratings from the team strength model, we can generate expected score-lines and thus match probabilities. For example, the tie of the opening weekend sees New Zealand face South Africa, where the All Blacks have a 69% chance of victory. The closest match of the group stage sees Wales with a 50% chance of defeating Australia, with the Wallabies sitting on 47% and a draw at 3%.
These match probabilities and expected score-lines can then be used to simulate the potential timelines of the tournament and estimate the likelihood of team’s qualifying for each stage, as well as the ultimate winner.
The immediate concern for the participants is to navigate the group-stage and qualify for the quarter-finals. Given the strength of the top-teams and the seeding of the group stage, the top-6 teams examined above are heavily favoured to progress. France and Scotland are solid favourites to join them.
Despite home advantage, Japan are less favoured to reach the knock-out stages, although they do benefit from a relatively weak group and are most likely to cause an upset. Argentina’s attacking performance has declined significantly over the past two years, making the chances of them repeating their semi-final place in 2015 less likely. Of the genuine outsiders, Fiji are given the greatest chance of progressing by upsetting either Wales or Australia in Pool D.
As far as the tournament winner is concerned, New Zealand’s attacking and defensive strength results in them winning in 47% of the simulations, making them the favourite for the tournament by a huge margin. The nature of probabilistic forecasts is that they attempt to estimate the most likely (or unlikely) outcomes – in fact, the most likely scenario is that New Zealand won’t be victorious.
Should the All Blacks fail, the main contenders are their Pool B rivals, South Africa and England. Wales, Ireland and Australia round out the most likely candidates to win the tournament; their chances are lowered by a combination of their overall ratings being slightly lower plus somewhat tougher draws in the group stage.
Overall, the model illustrates the relative strength of New Zealand compared to their rivals, but does point toward concerns regarding their weakened attacking performance in particular. Such a decline coupled with improved defensive performances by the other contenders has seen the gap narrow and increase the chances of the latter in the RWC.
Whether it is enough to prevent the All Blacks from landing a third consecutive RWC win and fourth overall is the major question on the eve of the tournament.
Appendix: model hierarchical structure
The hierarchical structure of the model firstly accounts for the relative level of the team; the international governing body defines these as Tier One, Tier Two and those outside it, which are the groups used here. The key assumption is that the performance levels of these tiers are comparable e.g. Hong Kong and Chile are closer in strength than they are to England and Wales. This structure aids the determination of team strength for nations that play relatively few matches.
The next level of the model sees the team’s performances over a single two-year period sat beneath the team’s overall performance levels e.g. England’s performance from 2017-2019 is related to their performances from 1983-2019. The prior used is relatively weak, so performances can and do vary significantly over time; England’s best attacking and defensive strength ratings occur from 2001-2003 in the build-up to their RWC win for example.
Two-year periods were chosen to reflect shorter-term form as they performed better in out-of-sample predictions than periods of one or four years. Two-year periods also provided more accurate forecasts of performances in the following RWC tournament.
