1st5R metric

[donmcdazzle]

You guys been having a look at this new metric? Pretty interesting, I always appreciate guys trying to quantify what we are seeing when we watch the games. Rugby has been really slow to adopt metrics (at least with the viewing public). US sports are awash with this type of thing.

Anyway it's an interesting thing to keep tabs on to see if your analysis matches the maths.

http://www.stuff.co.nz/sport/rugby/...5r-rating-super-rugbys-playmakers--week-three

 

[dasNdanger]

Very interesting - thanks for sharing. I'm going to re-post the link in a thread I started for the 10 position in general, over in the general RU forum.

Looks like Cruden is the guy to beat right now. Hopefully he'll stay healthy.


das

 

[Draggs]

I touched upon it briefly last year: TRF

Personally I find the stats thing overdone at times. I like this article, although football, not rugby: Link

 

[donmcdazzle]

^ I don't think they are overdone at all in rugby. Even in football/basketball I think if you use them to supplement what you are watching rather than just relying on them 100% it can be useful.

 

[1st5R]

Hi everyone, Stephen Ryan here the formula's creator,

Glad to see you guys (mostly) like it, if you have any questions about 1st5R I'll be happy to answer them or you can find me on twitter @1st5R.

My aim with this was to generate debate about who should wear the All Blacks number 10 Jersey at the rugby world cup considering Carter, Barrett, Slade and Cruden are all really good options, but it works well for keeping tabs on 10s from other teams that you might not get to see the games of.

 

[Draggs]

I find it interesting no doubt, it's a different way of looking at things.
I wasn't griping about this specifically, just how people tend to try and reduce everything to stats instead of the game itself.

Will you update every few rounds, Stephen?

 

[goodNumber10]

Hi everyone, Stephen Ryan here the formula's creator,

Glad to see you guys (mostly) like it, if you have any questions about 1st5R I'll be happy to answer them or you can find me on twitter @1st5R.

My aim with this was to generate debate about who should wear the All Blacks number 10 Jersey at the rugby world cup considering Carter, Barrett, Slade and Cruden are all really good options, but it works well for keeping tabs on 10s from other teams that you might not get to see the games of.

it's a nice piece of analysis, but it is difficult to quantify performance, do you mind sharing a bit about how the marks are assigned?

 

[1st5R]

I find it interesting no doubt, it's a different way of looking at things.
I wasn't griping about this specifically, just how people tend to try and reduce everything to stats instead of the game itself.

Will you update every few rounds, Stephen?

Yup, there's a new article giving the super rugby ratings on stuff.co.nz every Tuesday at noon New Zealand time.

 

[1st5R]

it's a nice piece of analysis, but it is difficult to quantify performance, do you mind sharing a bit about how the marks are assigned?

Its a scale from 1-100 where 50 is (approximately) the average score based on the average stats of the 252 40+ minute super rugby games played by 1st 5's in 2014. So if a player gets the average stats, he gets a score of 50, gets more than average its above 50, if he gets less its below. (Side note, in the early running of 2015, average stats across the board are down from 2014)

There are 11 categories a player can contribute to his overall rating, tries, running, line breaks, turnovers, tackling, goal kicking (not included obviously for players like Matt Toomua), territory kicking, possession kicking, kick errors, attacking work rate and defensive work rate. Each category is broken down into different stats has a different weighting to match its overall impact on the game, e.g. tries/try assists are worth more than line breaks/assists. Each category has a minimum and maximum amount of points, so a player who only does one thing, but does that one thing brilliantly he won't get a brilliant score. If someone scores 6 tries in 7 games, but misses 40% of his tackles and 50% of his kicks it's going to balance back down to average.

The stats it takes into account are minutes played, tries, try assists, total touches, number of runs, metres run, line breaks, line break assists, defenders beaten, turnovers, tackles made, tackles missed, kicks made, kicks missed, (territory kick metres, number of territory kicks, possession kick metres, possession kicks, number of penalty kicks to touch, and kick errors).

The bracketed stats require difficult and unique stat collection and so aren't included for super rugby, so you'll notice that an excellent out of hand kicking game, like the one Sias Ebersohn had in the forces win over the Waratahs in round one won't rate as highly as they should. That's purely financial decision and if it were up to me those stats would be included. If the public get out and support and it becomes financially viable, then it will get added back in later on.

 

[goodNumber10]

Its a scale from 1-100 where 50 is (approximately) the average score based on the average stats of the 252 40+ minute super rugby games played by 1st 5's in 2014. So if a player gets the average stats, he gets a score of 50, gets more than average its above 50, if he gets less its below. (Side note, in the early running of 2015, average stats across the board are down from 2014)

There are 11 categories a player can contribute to his overall rating, tries, running, line breaks, turnovers, tackling, goal kicking (not included obviously for players like Matt Toomua), territory kicking, possession kicking, kick errors, attacking work rate and defensive work rate. Each category is broken down into different stats has a different weighting to match its overall impact on the game, e.g. tries/try assists are worth more than line breaks/assists. Each category has a minimum and maximum amount of points, so a player who only does one thing, but does that one thing brilliantly he won't get a brilliant score. If someone scores 6 tries in 7 games, but misses 40% of his tackles and 50% of his kicks it's going to balance back down to average.

The stats it takes into account are minutes played, tries, try assists, total touches, number of runs, metres run, line breaks, line break assists, defenders beaten, turnovers, tackles made, tackles missed, kicks made, kicks missed, (territory kick metres, number of territory kicks, possession kick metres, possession kicks, number of penalty kicks to touch, and kick errors).

The bracketed stats require difficult and unique stat collection and so aren't included for super rugby, so you'll notice that an excellent out of hand kicking game, like the one Sias Ebersohn had in the forces win over the Waratahs in round one won't rate as highly as they should. That's purely financial decision and if it were up to me those stats would be included. If the public get out and support and it becomes financially viable, then it will get added back in later on.

that's brilliant thanks, makes perfect sense.

How do you code the games, i presume that's manually done on a visual watch through?

And how many watches does a game get?

Sorry, don't' mean to ask for all your secrets, just find this fascinating as find coding accurately the hardest part of analysis.

 

[1st5R]

that's brilliant thanks, makes perfect sense.

How do you code the games, i presume that's manually done on a visual watch through?

And how many watches does a game get?

Sorry, don't' mean to ask for all your secrets, just find this fascinating as find coding accurately the hardest part of analysis.

It is done by a manual watch, but not by me. Fairfax media (the ones that have the rights to it for Super Rugby) have the connections to the statisticians so its all run through them. So sorry but I can't be of much help to you on that one.

 

[donmcdazzle]

Thanks for the replies Stephen. Do you think you'll roll this out for other positions too eventually? I like that you have it rooted back to averages, would make someones score relatively comparable against another position e.g. this centre is 60 (i.e. 10 points better than average) compared to a lock at 45 (ie 5 points worse than average). Really interesting stuff.

I guess there is some subjectivity around how each stat is weighted, for example I don't think tries are that important for certain positions as you could just be catching the ball and falling over and that probably gives you a fair few points. Then again tries are what actually affect the game the most.

 

[1st5R]

Thanks for the replies Stephen. Do you think you'll roll this out for other positions too eventually? I like that you have it rooted back to averages, would make someones score relatively comparable against another position e.g. this centre is 60 (i.e. 10 points better than average) compared to a lock at 45 (ie 5 points worse than average). Really interesting stuff.

I guess there is some subjectivity around how each stat is weighted, for example I don't think tries are that important for certain positions as you could just be catching the ball and falling over and that probably gives you a fair few points. Then again tries are what actually affect the game the most.

I'd love to roll it out for other positions but 1st5R has to prove there's an audience for it first, if it can do that (share it with your rugby loving mates) then the next logical position would be full backs and then we'll go from there.

There is, my own and a couple of fairfax reporter's, subjectivity for the weighting and at the moment I would say its biased towards 1st 5s who get the ball in the red zone often, as opposed to 1st 5s who get their team to the red zone then let their forwards score tries or draw penalties. Though arguing which way is better to score a try would take a month of Sundays as it has so many different factors from the strength of the forwards, to the defence to the preference of the coach.

You're right different positions demand different weightings, e.g. Victor Matfield's lineout prowess destroyed the All Blacks in the late 2000's without scoring (if I recall correctly) any tries. It's reasonably easy to give a high score for a good try over a fall over the line, e.g. Julian Savea's effort against the Force this weekend just gone had him break two tackles and run 50 metres. He'd get points for the touch, try, beating defenders and a long run, compared to Cory Jane's try where he caught the ball and ran all of 2 metres, he'd only get points for the try, 2 metre run and a touch.

 

[dasNdanger]

I'd love to roll it out for other positions but 1st5R has to prove there's an audience for it first, if it can do that (share it with your rugby loving mates)

Oh, I think there's an audience for it - fans (esp. men...I think it's a math thing) love stats. More importantly, they love arguing the stats. I'm just wondering how it would be done on a larger scale - how many people would be needed to give full stats on every position on a specific team, for instance. It just seems so huge to me. Even limiting it to 1st 5s seems huge...lol. (I actually think it would be rather handy for rating forwards, esp. when it comes to those who do - and who don't - keep a scrum stable.)



das

 

[TRF_stormer2010]

Like Das mentioned sports fans and men in general love sports stats and this is something I feel rugby has been lacking; the relaying of effective stats to the viewing public. I can see such a catch-all average for a player being used for gaming purposes as well. Keep it up, Stephen! I sincerely hope you find a way of making your project financially viable. It can only add to rugby's appeal (and will be vital for rugby as a sports product to make headway in such stat-heavy sports markets as Australia and the US in particular).

 

[Icemn]

good thing morne steyn headed for the hills... i would love to see Naas Botha's stats lol

 

[Kiwiwomble]

http://www.stuff.co.nz/sport/rugby/...t5r-rating-super-rugbys-playmakers--week-nine

very interesting

 

[RedruthRFC]

Fantastic thread and idea. It's particularly at home on this forum where lots of people are fond of using stats to try to prove their point or disprove other people's along the lines of "A is a better defender than B because A has made x tackles compared to B's y". These comparisons are often made over tiny sample sizes or fail to account for other significant factors (e.g. number of missed tackles, damage done by missing the tackle or the effectiveness of the tackle), so the question is, do we understand what the stats we're referring to mean, how important they are and what they actually say about any given player's value to the team? In the UK at least, I believe that the stats we fans get to see and hear given importance to by commentators and journalists are hopelessly naive. I suspect New Zealand is already ahead of the game in this regard as they have been thinking more deeply about the subject since 2012 (http://www.stuff.co.nz/sport/opinion/6456754/Moneyballs-mantra-helped-ABs-win-World-Cup).

As it happens, I am reading Moneyball at the moment, I'd recommend it to anyone with an interest in sporting statistics. Of what I've read so far, chapter four (I'm near the end of it at the moment) is particularly worthwhile if anyone's interested. Applying the thoughts and ideas that I've read so far, the fundamental question to me about 1st5R is will it ever be possible to assess how effective the metric is? In baseball, Billy Beane was able to conclude that his approach was effective because of the results it delivered in the draft. Bill James (one of the early poineers of meaningful baseball statistics) was able to apply his formula to previous season's statistics and see if they predicted what would happen correctly. @1st5R has already said that some of the statistics that his formula factors in are subjective, it appears to me that the only way to assess the accuracy of the metric is to compare it to an entirely subjective measure (e.g. asking a panel of "experts" to rate the 10s in question in preference order). There are also the noted dangers of the metric favouring 10s who play a certain style or for successful teams. I understand that 1st5R is working within constraints, but rather than comparing different 10s in different teams, I would be interested to hear what the metric says about how different 10s perform in the same team, for example Farrell vs Ford for England or Ford vs Devoto for Bath.

I don't mean to be too negative, just to stimulate debate, but this seems like an incredibly ambitious idea given the complexity of rugby.

 

[1st5R]

Fantastic thread and idea. It's particularly at home on this forum where lots of people are fond of using stats to try to prove their point or disprove other people's along the lines of "A is a better defender than B because A has made x tackles compared to B's y". These comparisons are often made over tiny sample sizes or fail to account for other significant factors (e.g. number of missed tackles, damage done by missing the tackle or the effectiveness of the tackle), so the question is, do we understand what the stats we're referring to mean, how important they are and what they actually say about any given player's value to the team? In the UK at least, I believe that the stats we fans get to see and hear given importance to by commentators and journalists are hopelessly naive. I suspect New Zealand is already ahead of the game in this regard as they have been thinking more deeply about the subject since 2012 (http://www.stuff.co.nz/sport/opinion/6456754/Moneyballs-mantra-helped-ABs-win-World-Cup).

As it happens, I am reading Moneyball at the moment, I'd recommend it to anyone with an interest in sporting statistics. Of what I've read so far, chapter four (I'm near the end of it at the moment) is particularly worthwhile if anyone's interested. Applying the thoughts and ideas that I've read so far, the fundamental question to me about 1st5R is will it ever be possible to assess how effective the metric is? In baseball, Billy Beane was able to conclude that his approach was effective because of the results it delivered in the draft. Bill James (one of the early poineers of meaningful baseball statistics) was able to apply his formula to previous season's statistics and see if they predicted what would happen correctly. @1st5R has already said that some of the statistics that his formula factors in are subjective, it appears to me that the only way to assess the accuracy of the metric is to compare it to an entirely subjective measure (e.g. asking a panel of "experts" to rate the 10s in question in preference order). There are also the noted dangers of the metric favouring 10s who play a certain style or for successful teams. I understand that 1st5R is working within constraints, but rather than comparing different 10s in different teams, I would be interested to hear what the metric says about how different 10s perform in the same team, for example Farrell vs Ford for England or Ford vs Devoto for Bath.

I don't mean to be too negative, just to stimulate debate, but this seems like an incredibly ambitious idea given the complexity of rugby.

Thank you for taking the time to write out your thoughts. What you've written is extremely accurate, and the current constraints on 1st5R are the ones you have stated and I'm looking at ways to work them out. Sadly that is a full time job in itself and until the product proves commercially viable, we're half way through super rugby so we're starting to get an idea since the cricket world cup took a lot of shine off the super rugby start here in NZ, I can't afford to put in the hours necessary on top of a full time job + family.

The metric originally took subjective ratings to a sample of games to ensure that it stood up to the 'eye test' so I find most people are against the idea in principle (which is weird since the All Blacks have been doing this sort of thing since 2001 and the Chiefs won back to back ***les because of it) they very rarely disagree with the numbers the formula delivers.