Before I get on with directly answering this, I’d like to ensure that everyone is fully aware of why the backtested results of my systems (or any system?) may not look anything like the results once they go live.
Aside from the fact that variance can impact the returns any system makes in a live environment, and aside from the fact that a system may lose its edge over time, why is it then that the live results may not look like backtested results?
Well, if you think about it, one set of data is using odds at kick-off (the backtested/backfitting) and the other set of data is using odds at a time when I release the bets (typically a Thursday evening). Quite simply, by comparing backtested and live results on my systems, we are comparing apples and pears.
Let’s take a real life example from last season. My minimum away odds on all systems for a selection is 2.75 apart from extreme cases – regardless of value calcs. There was a bet thrown up on all 20 systems last season but it was only available at 2.75 in two or three places and therefore, when it came to releasing the bets on a Thursday evening, the bet was dropped (rightly so as I doubt very few would have got 2.75 on it).
At kick-off, the team was available to back at odds of 4.00. Why was this? Well, when the best non-league football tipster (Skeeve) tips a bet on the Home team, this was probably enough to ensure the Away team would drift massively and I think the Away team also had some injury issues which just added to the drift.
So, if I look at this game in the data now and backtest last season, it will be thrown up on all 20 systems last season as a bet. The Away team won (won’t be often I’ll disagree with Skeeve and hit a winner I’m sure!) and therefore, it’s a very simple example of a team appearing in backtesting and not live.
More likely, the opposite will occur. The vast majority of system 7/22 bets are much shorter at kick-off compared to the odds I’ve backed at on a Thursday evening. Therefore, if I backtest system 7/22 for last season, most of the teams who we backed live on system 7/22 wouldn’t be thrown up as bet. Therefore, the results for backtesting system 7/22 last season look nothing like the live results we experienced. The same is true for many of the marginal big priced Away teams I backed last season. They were shorter at kick-off and therefore, they wouldn’t be bets during backtesting. Instead, I’d have another set of these big priced away teams who would appear during backtesting and they may or may not have outperformed the teams we backed!
I hope the above is understood and explains why it is dangerous to go from backtesting to live results. Of course, many systems people follow aren’t sensitive to odds movements and therefore, you can be pretty safe you can use backtested results as a guide to the future performance. The way my systems are built (and the complexity of the layers involved in the system building process), quite often, all it takes is a team to go from 2.25 to 2.20 and it may get dropped off a single system. So, it may move from system 8 to only being a system 7 bet. That may then stop it from being a system 8, 8-21,8-22 bet. Likewise, a bet that gets dropped from system 22 to only 21 will get dropped from 6-22,7-22 and 8-22 also if it appeared on them.
My live system bets are based on odds at a point in time and of course, the backtested results are based on a point in time too. However, they are not the same point in time!
Although I haven’t discussed it on the blog, I ran a trial of a few Under/Over 2.5 goal systems in the SBC forum last season. The backtested results looked good but I was wary of the impact of moving from odds at kick-off to odds on a Thursday night as quite simply, the markets on a Thursday evening for the leagues I play in aren’t developed which means you have a higher over-round but more importantly, there hasn’t been much money placed in these markets, so the odds aren’t very accurate.
When I trialled these systems live, the results looked nothing like the backtested results and were almost embarrassing. I have never shared the backtested results for the very reason I was worried about how good the systems would do live and my instinct was actually correct.
The 20% return during backtesting became about 1% when live. As soon as the season ended, I did a quick check to backtest the systems on the data at kick-off and only around 10% of the bets for backtesting matched the bets live! Hence, I was simply trialling bets that had nothing to with the way the systems were built basically!
The backtested systems only achieved about an 8% return overall last season, so it was a poor season and going from the backtesting to live results brought this 8% down to 1%.
I’ll cover this off when I’m starting doing the Euro system reviews next week but I am going to use the drop-off in returns between backtesting and live results on the UK returns to hopefully guide us when it comes to the Euro systems.
For example, we know that 17% on system 6 during backtesting has become about 5% when live. Therefore, we can probably expect the 17% on E1 to become 5% when live, assuming the impact is the same on the Euro systems. Of course, this is a major assumption but what else have we got to go on at the moment with no live results?
Right, apologies for the delay to replying to Rowan’s question. I just think it’s important people fully appreciate the risks involved as we move from a set of systems with good backtested results to a live environment.
For those that didn’t see it, here is Rowan’s question he posed:
OK, I'm going to be getting a little ahead of myself and yourself here, but there has been a recurring theme throughout these posts on the Euro systems. I've a feeling you may well be intending to address this point a little further down the line. If so, tell me to pipe down and be patient. :)
Being straight with you, my intention has always been to follow the Euro systems next season, but not with hard cash. As you say yourself, next season is to be judged on the performance of the UK systems and that's good enough for me.
But, the recurring theme that you keep touching on are the similarities to the results that you experienced when you were developing the UK systems. I see there are differences between weightings of home bets vs away bets, but in terms of the results..."Quite spooky how close these returns are to the UK systems!"
Should these similarities not give us a lot more confidence in backing these systems? Am I being a little too cautious in thinking I should just watch these systems perform over 2012/13 to see how it goes? I'm not suggesting we wade right in (you know me! :) ) but at the same time...
Right, tell me to bugger off if you like, and to stop trying to rush things.
The title of the post is probably a giveaway I think in terms of where my answer is going to lead us to but let’s see how we get on.
As I said to Rowan in my initial reply to his question, I think the best place to start has to be to do a quick benefit analysis, outlining the advantages and disadvantages of having the Euro systems as part of your betting portfolio next season. This is only my take on the advantages/disadvantages by the way, it isn’t meant to be an exhaustive list!
Advantages of following the new ‘unproven’ European systems
No price pressure and bets can probably be followed easier than the UK systems – I think this is the main advantage of following the European systems next season. The UK systems are now part of the Secret Betting Club’s Hall of Fame services and therefore, the service will have more members next season. Take into account there will also be a higher average stake being played on each selection (new members will have more confidence and existing members will be increasing stakes), and it doesn’t take a genius to figure out it won’t be as easy as it has been to date to match the system results. I don’t expect too many people to be following any European systems next season, so the prices won’t be under pressure and bets can probably be placed at any point up to kick off.
No liquidity issues – Even though there weren’t many members following the UK systems last season, I still had to split stakes to get my stakes on some of the bets on in League Two and the Bsq Prem. If following bets in the top European leagues, you’ll be able to place as much money on the teams as you want. In addition, it probably opens up the door of using Betfair if you prefer Betfair to bookmakers, so it opens up more options from a liquidity sense.
Greater potential to achieve higher returns – This is a tricky one as I seem to spend my time trying to talk down the European systems to ensure no one gets carried away but what if (and it’s a big if), the European systems do better or as good as the UK systems did when they first went live? In the first 4 months of going live, the 6 UK combined systems made a profit of 175pts with an ROI of 43%. What if the Euro systems do the same? Not only that, what if we find out next season that the Euro systems are actually better systems than the UK systems. How would we feel about not following these systems when they went live?
Paper trading doesn’t give the full picture of how the systems perform on a daily, weekly or monthly basis – I think this is a very important aspect and shouldn’t be understated. I’d love to ask all my current subscribers from last season if they were aware of the daily ups and downs that the systems would provide them with last season? I’d be amazed if no one was caught out. I had 4 seasons of backtesting results, one season of live results behind me and I was still taken aback by some of the ups and downs last season. The only way to really understand a system’s performance is to follow the system live.
Are these the systems of the future? – Impossible to know this for sure I think and it’s far too early to even consider this but I suspect that some people following this blog would love to see these systems perform as well as the UK systems or better. If it is the case that we’re all going to be following the European systems in future, by not following closely in the first season, do you run the risk of falling behind the curve? A few people who have joined for next season have mentioned that they’d wished they’d joined last Summer but didn’t want to take the risk of joining until they saw that the first season wasn’t a fluke. I suspect they now regret that decision and feel like they’re playing catch up with others.
I think that’s the 5 main advantages I can think of for following the Euro systems next season. Here’s the main disadvantages I can think of!
Disadvantages of following the new ‘unproven’ European systems
There are no live results – Without a doubt, the biggest concern for me is the fact there are no live results. I’ve used this quote elsewhere and on the blog but I strongly believe that one seasons live results is worth 100 seasons of backtested results. I can’t stress enough how much more weight I place on live results than backtested results. Hence, in my opinion, the backtested results don’t mean much as soon as a new system goes live!
Time can be better spent following the UK systems – This is an interesting one and it’s one that I find it hard to argue with. If people want more bets, why don’t they just follow another system or two from the UK systems? If the new Euro systems don’t turn out to be what we hoped they’d be, then spending time following them next season has been time wastet and could have been spent following the proven UK systems.
They can’t be staked properly anyway – I think this is a valid point and one that is linked to the previous point. Even those that decide these Euro systems are worth following next season, they can’t really be staked aggressively with no live results behind them and therefore, why bother? Is it not better just increasing stakes on the UK systems and making profits that way, rather than messing around with smaller stakes on unproven systems?
I’m sure there are more disadvantages of following these newer systems but they are all going to relate to the fact the systems have no live results I suspect!
OK, now I’ve laid out the advantages and disadvantages that I can think of, what do people think? Well, here’s my take on it. I should add that this is my personal view only (as in the view of a punter, not the view of a tipster running a very profitable service!). I’ve already given my view as a tipster (watch the Euro systems next season only!!!)
I’m a great believer in the phrase intelligent risk taking and it is something that underpins everything I do with betting. During the first season of the UK systems, I had nothing to go on but 4 past seasons results and it didn’t stop me from following the systems in the first season and making a profit. It was based on the fact the historical results were very good and my belief that I could make a profit from following the systems. Aside from making a profit, it also taught me an enormous amount about the systems and how I could use them going forward to make a profit.
Last season, I followed some of the new UK systems in a separate portfolio for much of the season even though they too had no live results. Although the ROI wasn’t great, I still managed to make a profit and of course, I learnt a lot about the systems and some of these systems will make their way into my main portfolio of systems next season.
So, what do you think I’m going to do next season? Well, not surprisingly, given what I’ve done for the first two seasons when new systems have appeared from the TFA laboratory, I will follow some of the new Euro systems in a separate portfolio from my main football betting systems and see how I get on.
Like I have always done, the staking will reflect the fact there are no live results and ultimately, my profit on the season will hinge on the UK systems but the Euro systems will carry enough weight to ensure that I want the systems to do well. No point playing £2 stakes and not caring about what the results will be every week. The stakes need to be set in such a way that if the systems perform well, I’ll see a difference to my portfolio return but if they perform poorly, I don’t wipe out all the profits made by the UK systems!
What's my advice from one successful investor to another? Well, it's up to you really. I'm not going to tell you a definitive Yes or No for following these systems. We all know the risks at this game, regardless of whether a system has live results or not. If people are comfortable following a new system, knowing the risks, then they should think about doing so. If people aren't comfortable, then don't follow these systems next season. It really is as simple as that I think!