Friday, 19 June 2015

New Systems Season Review 2014/15

If we thought it was a poor season for the Established Systems, this review is going to make even worse reading. Going into the season, I would say this set of systems probably carried more weight than any other set of systems in subscriber’s portfolios and betting strategies but these systems crashed and burned badly this season.

These systems have always been a little bit of boom and bust since their inception and this was the 3rd season that all systems had been live. Systems 31-33 have been live for 4 seasons. The first season all systems were live wasn’t great, the second season was amazing and the 3rd season has been poor. Overall, the results don’t look too bad although they now lag a little behind the Established Systems.

Going into this season, system 31 looked like the best system I had and probably one of the best systems I’d ever seen with 3 season’s of live results behind it. Roll on another season and the results don’t look anywhere near as impressive after a very poor season. Half way through the season, System 31 hit the magic <1% P-Value and I thought, finally, a system that is statistically significant. It barely hit a winner after this!

Here’s the results for the 3rd algorithm (system 31):

A poor season for this algorithm as discussed above. A loss of 24.5pts from 943 bets for a -2.6% ROI. Looking at the breakdown by value, all the profits were generated in two bands. Bets less than 10% value lost 33.5pts ( low value bets were profitable historically unlike the other algorithms) and the bets with more than 20% value lost 21.4pts. The sample sizes for the higher value bets were very low but it is these bets that historically, have really made the difference on this algorithm as we’ll see later. Only hitting 1 winner from 20 bets at 25+ value is a killer although the ave odds are pretty high as you can see.

Unlike the first two algorithms, the low value bets didn’t do well and I find it slightly annoying that on the first two algorithms, low value bets were great and were terrible historically and yet on this algorithm, low value bets did poorly and yet, were profitable historically. Very irritating!

If we look at the results on this algorithm since inception, we can see how annoying these results this season are:

I guess if I was data mining, I could say that the value band 25%-30% has struggled since inception and last season’s losses aren’t a great surprise but the highest value band usually does very well.

Since inception, the overall algorithm has generated a profit of 4.3% even after last season and therefore, it is obviously very disappointing to see a losing season.

Here’s the results of this algorithm for the past 4 seasons, side by side:

I think this picture reiterates the fact that seeing a losing season for this algorithm is very disappointing. The first season wasn’t great by any means but still a helluva lot better than a loss. A 10% ROI two years ago was maybe the best performance I’ve ever seen by a rating algorithm over a season. To go from this to a loss in two seasons is very disappointing.

You could potentially point to the increased bet number as a mitigating factor for the deterioration in performance but as I said with the previous review, I don’t think the increased turnover is to blame for the poor results. Always impossible to prove this but nothing was done to the ratings to worsen the expected performance going forward.

Here’s the Home & Away split for last season:

Homes were slightly better performers than Aways last season but overall, both lost money.

Here’s the same split since inception:

Again, similar to everything else last season, the results didn’t match the historical results. Aways have been the most profitable bets historically and these were the worst bets last season.

Going into the season, the best performing league on this algorithm was the Championship by far. Of course, 437 bets wasn’t a big sample but even so, an ROI of 16.3% was impressive and if I was having to bet on a league performing well on an algorithm, the Championship on algorithm was nailed on. Here’s the results before this season:

Amazingly, the P-value for the results in the Championship was 2% going into the season. Here’s the results from last season:

Well, you can obviously guess but the worst performing league was of course, the Championship. A loss of 25.7pts when on average, the league was making 25pts a season before last season! Grrrr….

I think as always, these sample sizes are small but it shows the difficulties people have with trying to draw conclusions by league or by any other factor they may use to filter bets. It isn’t easy and in any season, I honestly believe the results are going to be random and will bear little resemblance to historical trends.

Overall then, a first losing season for algorithm 3 although at one stage last season, it looked like the algorithm would make a profit. The wheels fell off mid season and never really recovered if I’m honest.

Going forward, there will obviously be doubts about this algorithm now and I think they are justified. However, one poor season can’t undo the work of the previous seasons and it’s important this lost is kept in context. I’ll tweak a few things in the Summer to try to improve the algorithm and check the changes made last Summer didn’t impact the performance negatively.

Here’s the results for the 4th algorithm:

Not much to say here apart from this algorithm isn’t very good. Coming into the season, it was definitely my poorest performing algorithm and nothing last season made me change my mind. If anything, these results reinforce the fact the algorithm isn’t very good.

Annoyingly, the bets that made a profit were the bets with less than 5% value. Here’s the results coming into the season:

If there was one value band on any algorithm I wouldn’t have touched with a bargepole, it was low value bets on algorithm 4. What happens? This is the value band that makes 12.1pts profit, the other value bands made a loss combined of 65.5pts from 372 bets! Terrible.

Here’s the updated results for the algorithm since inception:

Even after last season’s disaster, the algorithm is still pretty much break-even after 3 seasons and 2,639 bets. Although it’s disappointing it hasn’t performed better, it’s not a total disaster and if you were filtering the bets from system 41, you are at least starting from a break-even position. There is clearly no edge here though as things stand at the moment.

Given I basically rebuilt this algorithm last season on the same basis as the previous algorithm, it’s clear that this method no longer works. If this algorithm continues, it needs a full rebuild again and probably ignores any previous method or rules for this algorithm. Not sure I have the appetite to do this!

Over the last 3 seasons then, here’s the results:

Not much to say, even when the algorithm performed well, it was still rubbish!

Not going to show any more tables for algorithm 4, not much point really given the poor results. Any members can do this for themselves using the spreadsheet.

So, looking at the results for both algorithms, it’s hard to imagine the other system results being much better. Here’s the filtering on algorithm 3:

Basically, the filtering made very little difference on algorithm 3. The results were similar for all 3 systems and basically, the filtering didn’t make things better but didn’t really make things worse either.

Here’s the results since inception:

Interestingly, we can see clearly that system 32 and system 33 don’t improve things at all. If anything, they make things a little bit worse in terms of the results. Therefore, there is little point in having these systems.

Did system 42 rescue the 4th algorithm? Here’s the results this season:

In a word, no!

Here’s the results since inception:

Same story here, system 42 adds nothing to system 41.

Based on these system results then, we already know the combined systems aren’t going to look great. Here’s the results for all 11 New Systems this season:

A terrible set of results all round. I guess the only plus point is that the cross referring of the systems ensured the overall impact of systems 41 and 42 were reduced somewhat but even then, if you cross refer two losing algorithms, it’s impossible to do well.

System 33-42 somehow made a profit even though systems 33 and system 42 were terrible, so it shows you can actually make a profit cross referring losing systems but it’s easier to do that with lower turnover systems. Trying to do it with higher turnover systems is much more difficult.

Here’s the same results since the inception of these systems (have to split it in two since systems 31-33 been alive for an additional season):

Overall, the returns from most of these systems have taken a significant hit after last season and therefore, all these systems are now under the microscope I think. I think it’s clear that systems 32,33 and 42 don’t improve the returns on either 31 or 42 and therefore, not sure what the point of having these systems is. The filtering doesn’t work and therefore, there is no need for these systems.

My only concern with scrapping them is that they are used by some people for various methods of filtering the bets but I suspect that next season, these won’t be official systems in the proofed results. They will be retired!

Overall, a really poor set of results and it’s a shame that these systems have had such a poor season after a solid start for a few seasons. I need to think long and hard about what I do with these systems and whether or not they all continue (unlikely) and if I drop some, they need to probably still highlight the bets they would have had for people using the number of times a team appears as a filter.

Summer Actions – Decide what I do with this whole bunch of systems. 31 is the only system that is guaranteed to continue next season although it’s likely I’ll do a deep dive on the rating algorithm to see what I can do. Not sure about system 41 or any of the other systems 32,33 and 42. Something needs to be done though! 

No comments:

Post a Comment