Friday, 5 June 2015

Est Systems Review for Season 2014/15

Similar to previous seasons, I’m going to review all the various sets of systems, one at a time. It can become quite boring writing and reading this stuff if I’m honest but as always, I’ll try to highlight anything that I find interesting and potential things we can look out for going forward.  I’m going to work from the spreadsheet I sent out to all members at season end which is called “TFA live results to end of 2014-15 season”. This way, anyone reading this with access to the sheet can recreate any of these tables themselves. Usually, I work from my own results sheets I have which contains results for each set of systems and has some formatting applied to it but I quite like this new sheet I’ve created and I like the idea that anyone using the sheet can see these tables for themselves.

So, this review will look at the performance of the first two rating algorithms (sys 6 & 21) for the season as well as the 11 Est systems. I’ll also look at the latest live results to date and make some commentary on these hopefully.  As always, I’ll try to summarise at the end and hopefully provide some sort of conclusion.

Here’s the results for this season for the first rating algorithm (system 6):

Not a fantastic season by any means but a profit of 10.8pts from 808 bets, an ROI of 1.3%. At a high level, a solid enough season for this rating algorithm although as we will see, trying to filter the bets on this algorithm proved very difficult. 

We can see from the table here that the value ratings didn’t really work at all. There were 650 bets with a value % of less than 10% and these did very well, generating a profit of 29pts at an ROI of 4.46%. I think this is about as good as you can expect from these bets and I would say they hit the target this season.

The issue with this algorithm (it will be a common theme across reviews) is that the bets with a value of >10% lost 18.2pts from 158 bets. This is ultimately the issue with this algorithm this season and unfortunately, for whatever reason, these bets have underperformed massively this season.

If we look at the results since inception of the algorithm (value ratings were not shared prior to the 2013/14 season), then we see that this trend last season is not common across the value bands:

Since inception of the algorithm, even with last season’s good returns in the lower value band, backing all bets with less than 5% value has not been profitable. Backing all bets with a value of at least 10% has generated a profit of 5.5% which although below expectations, is acceptable. Therefore, seeing a loss of over 11.5% last season is disappointing.

One of the aims last Summer with this algorithm when I was refreshing the ratings was to try to increase the bet number a little and as the below table shows, this was a success.

However, what I haven’t managed to do (which was my aim!) was to reverse the downward trend on the ROI. The ROI has been declining each season since the 2011/12 season and although I refreshed the algorithm last Summer, I didn’t actually manage to improve on the returns. Now I’ve seen the underlying results and the fact that last season wasn’t too tough overall, I’m disappointed by the returns from this algorithm (about to become a common theme I think during the reviews!).

Homes were slightly more profitable than Aways but interestingly, both Homes and Aways were profitable.

Looking at results since inception, Aways are twice as profitable as Homes which is down to the risk/reward I suspect of the higher odds.

Overall then, not the greatest season by any means for this algorithm but considering the performance of the other 3 algorithms, this was my best performing UK algorithm (ignoring the Draw algorithms). The worry is the clear downward trend on the ROI and if this continues, we’d probably be looking at a losing season next season. Saying that, making a profit 5 seasons in a row for any football algorithm is not easy and therefore, it needs to be applauded for this at least!

Here’s the results for the second rating algorithm (system 21):

A loss of 12.2pts from 812 bets. An ROI of -1.5%. The first losing season the algorithm has had since inception.

A similar story to the first algorithm in the sense the value ratings are all over the place. Coming into this season, the lowest value bets were loss making over the previous 4 seasons but this season, the lowest value bets somehow created a profit of 17.3pts at an ROI of 3.5%. Very good and probably better than I would expect given the value % for these bets is between 0%-5%. However, all other bets then with a value % of 5%+ made a loss of 29.5pts from 322 bets. An ROI of -9.2%. Very, very poor.

Here’s the results since inception for this algorithm:

Same sort of story as algorithm 1. Bets with less than 10% value have generated a profit of 18.7pts from 1,590 bets over 5 seasons. Bets with more than 10% value have generated 122.2pts from 1,322 bets. Nothing in these results would lead me to believe that we would have suffered losses from the over 10% value last season. Variance is probably at play due to the low bet numbers.

Here’s the results over time for this algorithm:

One of the big changes this season as a result of the tweaks last Summer was the increased bet number. Great that this algorithm now has more bets as it wasn’t generating enough bets but unfortunately, the P&L has probably been impacted by the changes too. I’ve let too many bets sneak onto the algorithm that haven’t been profitable last season and therefore, the challenge I guess is to try to refine the algorithm again this Summer to reduce the bet number a little but increase the ROI.

Unfortunately, Aways were poor and a loss on Aways caused the issues for this algorithm.  Here’s the results below since inception:

Again, the results since inception don’t back up what we saw last season and Aways have generally been the strongest bets on this algorithm since inception.

Overall then, both algorithm’s performance last season was below par, more so on algorithm 2. Unfortunately, the algorithms didn’t perform like they did in previous years although how much of this was down to the increased turnover on each algorithm is unclear. Even so, as I said last Summer, none of the changes to the algorithms were expected to reduce profitability and therefore, I wouldn’t put any blame of the underperformance on these tweaks last Summer.

So, if it was impossible to filter the rating algorithms using the value ratings, how did the system filtering do last season? Unfortunately, it was a similar story.

Here’s the filtering on the systems on algorithm 1 for last season:

A nightmare really for these systems and as you tried to filter the bets, it became worse and worse. System 7 made a 9.1% loss which is very poor considering the fact system 6 made a profit and system 8 did even worse and managed to make a loss of 13.3%. A very poor performance and the last couple of seasons have been similar.

Here’s the results for the filtering of these systems based on the first 3 seasons of results:

A perfect example of how I hoped the filtering would work on the systems and as you reduced the bet number, the ROI increased and for those that wanted to reduce the bet number by using higher system numbers, the strategy appeared to be working well.

Here’s the same results for the last 2 seasons only:

Quite amazing how badly the filtering has worked for the last two seasons and therefore, I have massive question marks over whether or not system 7 & 8 should continue next season. At the end of the day, if the systems aren’t adding anything to system 6, then what is the point in having them.

Based on the last two seasons, you could use system 7 as a filter on system to weed out the worst bets on system 6!

Here’s the full picture for the last 5 seasons:

Based on this, you have to say that dropping system 7 and 8 wouldn’t be the worst idea. If you use system 7 as a filter on system 6, you end up with system 6 improving its ROI and reducing the bet number. You end up with 2,287 bets and a profit of 124.5pts. An ROI of 5.4%. The unfiltered ROI is 3.7%. Hence, you can increase the ROI on system 6 by 45% by excluding all system 7 bets!

The crazy thing is that after the first 3 seasons, it’s hard to think this strategy could have possibly have worked the last two seasons. Whatever worked for the first 3 systems on this algorithm has stopped working for the last two seasons.

Here’s the filtering on the systems for algorithm 2 for last season:

A similar story to algorithm 1 where the filtering has actually worsened the performance of algorithm 2. A 5.5% loss on system 22 compared to a 1.5% loss on system 21.

If we look at the first 4 seasons this algorithm was live (rather than the first 3 as the filtering worked fine last season), we have the following performance:

As you can see, the filtering has worked perfectly well up until this season and the ROI increased from 7.3% to 12.3% as you moved from system 21 to system 22.

More importantly, system 22 in isolation had looked a very good system until last season and here’s the returns by season for this system:

Again, I’m not sure I could have predicted system 22 would struggle this season. It looked one of the best systems although as always, sample sizes were small and therefore, there was no guarantee there was any edge here at all in the previous 4 seasons.

I think unlike system 7 & 8, there appears to be good rationale for continuing with system 22. It actually adds a fair bit to the performance of algorithm 2 and I think I would happily write off last season based on the fact that the previous 4 seasons were very good. More difficult to do this for systems 7 & 8 given it has happened 2 years in a row on these systems.

So, we have seen that filtering the algorithms based on value ratings didn’t work this season (worked previously) and we have now seen that filtering systems didn’t work either last season (worked previously). The last place to filter bets is obviously using the combined systems and hoping to improve the performance of the basic algorithms by cross referring the systems.

Here’s the results for last season for the 11 systems in total:

Overall, a loss of 118.4pts from 3,604 bets and an ROI of -3.3%.

Clearly, systems 7 and 8 are responsible for a large part of the losses as it has a knock-on impact on the combined systems.

Systems 6-21 and 6-22 had disappointing seasons but being honest, the results weren’t that bad at all. Same comment probably goes to system 7-22 and 8-22 although these did have poor seasons. Systems 7-21 and 8-21 had an absolute nightmare of a season and anyone following these systems would have struggled badly.

These systems have obviously been around since the inception of the service. Here’s the updated live results after the season past:

I think now we have 5 seasons of worth results, we can draw our own conclusions.

Going back to what I said earlier in this review, I’m not sure what system 7 or system 8 adds to the party. Clearly, the filtering from system 6 using these two systems doesn’t work well enough at all. I either need to look into this over the Summer or I simply need to drop system 7 & 8. Of course, dropping 7 & 8 would mean dropping 7-21 thru to 8-22.  Looking at the results, I’m not sure I would like to drop systems 7-21 thru to 8-22. I know one person who has followed system 7-22 since inception with a little tweak applied and I’d probably lose this member if system 7-22 was removed!

I have always been a fan of following system 6-21 rather than just following system 6 or 21 and I think this table backs this up.  System 6-22 is also an improvement on system 22 and therefore, this justifies having system 6-22 in the portfolio.

Given the fact that systems 7 & 8 are poor, you really need the cross reference with 21 & 22 to achieve any sort of meaningful return. Therefore, if you are going to keep system 7 & 8 going, you really need to keep 7-21 thru to 8-22.

Overall then, a poor season for these systems but long-term, the results still look OK. I guess the question is around whether or not I can improve on the algorithms and base systems in the summer to hopefully try to stop the slide on the returns over time. Another season like last season and I suspect these systems are dead (and the service likewise!).

Summer Actions – Look to do some form of re calibration on both algorithms over the Summer to hopefully increase ROI whilst at the same time, looking at the 10 base systems that sit behind systems 6,7,8 and possibly look to improve these if possible. Not sure if I will look at the base systems on algorithm 2 or not, will see how much time I have this Summer. 

No comments:

Post a Comment