Before I get on with directly
answering this, I’d like to ensure that everyone is fully aware of why the
backtested results of my systems (or any system?) may not look anything like
the results once they go live.
Aside from the fact that variance can impact the returns
any system makes in a live environment, and aside from the fact that a system
may lose its edge over time, why is it then that the live results may not look
like backtested results?
Well, if you think about it, one set of data is using
odds at kick-off (the backtested/backfitting) and the other set of data is
using odds at a time when I release the bets (typically a Thursday
evening). Quite simply, by comparing backtested and live results on
my systems, we are comparing apples and pears.
Let’s take a real life example from last season. My
minimum away odds on all systems for a selection is 2.75 apart from extreme
cases – regardless of value calcs. There was a bet thrown up on all 20 systems
last season but it was only available at 2.75 in two or three places and
therefore, when it came to releasing the bets on a Thursday evening, the bet
was dropped (rightly so as I doubt very few would have got 2.75 on it).
At kick-off, the team was available to back at odds of
4.00. Why was this? Well, when the best non-league football tipster
(Skeeve) tips a bet on the Home team, this was probably enough to ensure the
Away team would drift massively and I think the Away team also had some injury
issues which just added to the drift.
So, if I look at this game in the data now and backtest
last season, it will be thrown up on all 20 systems last season as a bet. The
Away team won (won’t be often I’ll disagree with Skeeve and hit a winner I’m
sure!) and therefore, it’s a very simple example of a team appearing in
backtesting and not live.
More likely, the opposite will occur. The vast majority
of system 7/22 bets are much shorter at kick-off compared to the odds I’ve
backed at on a Thursday evening. Therefore, if I backtest system 7/22 for last
season, most of the teams who we backed live on system 7/22 wouldn’t be thrown
up as bet. Therefore, the results for backtesting system 7/22 last season look
nothing like the live results we experienced. The same is true for
many of the marginal big priced Away teams I backed last season. They were
shorter at kick-off and therefore, they wouldn’t be bets during backtesting.
Instead, I’d have another set of these big priced away teams who would appear
during backtesting and they may or may not have outperformed the teams we
backed!
I hope the above is understood and explains why it is
dangerous to go from backtesting to live results. Of course, many
systems people follow aren’t sensitive to odds movements and therefore, you can
be pretty safe you can use backtested results as a guide to the future
performance. The way my systems are built (and the complexity of the layers
involved in the system building process), quite often, all it takes is a team
to go from 2.25 to 2.20 and it may get dropped off a single system. So, it may
move from system 8 to only being a system 7 bet. That may then stop it from
being a system 8, 8-21,8-22 bet. Likewise, a bet that gets dropped from system
22 to only 21 will get dropped from 6-22,7-22 and 8-22 also if it appeared on
them.
My live system bets are based on odds at a point in time
and of course, the backtested results are based on a point in time too.
However, they are not the same point in time!
Although I haven’t discussed it on the blog, I ran a
trial of a few Under/Over 2.5 goal systems in the SBC forum last season. The
backtested results looked good but I was wary of the impact of moving from odds
at kick-off to odds on a Thursday night as quite simply, the markets on a
Thursday evening for the leagues I play in aren’t developed which means you
have a higher over-round but more importantly, there hasn’t been much money
placed in these markets, so the odds aren’t very accurate.
When I trialled these systems live, the results looked
nothing like the backtested results and were almost embarrassing. I have never
shared the backtested results for the very reason I was worried about how good
the systems would do live and my instinct was actually correct.
The
20% return during backtesting became about 1% when live. As soon as the season
ended, I did a quick check to backtest the systems on the data at kick-off and
only around 10% of the bets for backtesting matched the bets live! Hence, I was
simply trialling bets that had nothing to with the way the systems were built basically!
The backtested systems only achieved about an 8% return
overall last season, so it was a poor season and going from the backtesting to
live results brought this 8% down to 1%.
I’ll cover this off when I’m starting doing the Euro
system reviews next week but I am going to use the drop-off in returns between
backtesting and live results on the UK returns to hopefully guide us when it
comes to the Euro systems.
For example, we know that 17% on system 6 during
backtesting has become about 5% when live. Therefore, we can
probably expect the 17% on E1 to become 5% when live, assuming the impact is
the same on the Euro systems. Of course, this is a major assumption but what
else have we got to go on at the moment with no live results?
Right, apologies for the delay to replying to Rowan’s
question. I just think it’s important people fully appreciate the risks
involved as we move from a set of systems with good backtested results to a
live environment.
For those that didn’t see it, here is Rowan’s question he
posed:
OK, I'm going to be getting a little ahead of myself
and yourself here, but there has been a recurring theme throughout these posts
on the Euro systems. I've a feeling you may well be intending to address this
point a little further down the line. If so, tell me to pipe down and be
patient. :)
Being straight with you, my intention has always been
to follow the Euro systems next season, but not with hard cash. As you say
yourself, next season is to be judged on the performance of the UK systems and
that's good enough for me.
But, the recurring theme that you keep touching on are
the similarities to the results that you experienced when you were developing
the UK systems. I see there are differences between weightings of home bets vs
away bets, but in terms of the results..."Quite spooky how close these
returns are to the UK systems!"
Should these similarities not give us a lot more
confidence in backing these systems? Am I being a little too cautious in
thinking I should just watch these systems perform over 2012/13 to see how it
goes? I'm not suggesting we wade right in (you know me! :) ) but at the same
time...
Right, tell me to bugger off if you like, and to stop
trying to rush things.
Rowan
The title of the post is probably a giveaway I think in terms
of where my answer is going to lead us to but let’s see how we get on.
As I said to Rowan in my initial reply to his question, I
think the best place to start has to be to do a quick benefit analysis,
outlining the advantages and disadvantages of having the Euro systems as part
of your betting portfolio next season. This is only my take on the
advantages/disadvantages by the way, it isn’t meant to be an exhaustive list!
Advantages of following the new ‘unproven’ European
systems
No price pressure and bets can probably be followed
easier than the UK systems – I think this is the main advantage of
following the European systems next season. The UK systems are now part of the
Secret Betting Club’s Hall of Fame services and therefore, the service will
have more members next season. Take into account there will also be a higher
average stake being played on each selection (new members will have more
confidence and existing members will be increasing stakes), and it doesn’t take
a genius to figure out it won’t be as easy as it has been to date to match the
system results. I don’t expect too many people to be following any European
systems next season, so the prices won’t be under pressure and bets can
probably be placed at any point up to kick off.
No
liquidity issues – Even though there weren’t many members following
the UK systems last season, I still had to split stakes to get my stakes on
some of the bets on in League Two and the Bsq Prem. If following bets in the
top European leagues, you’ll be able to place as much money on the teams as you
want. In addition, it probably opens up the door of using Betfair if you prefer
Betfair to bookmakers, so it opens up more options from a liquidity sense.
Greater
potential to achieve higher returns – This is a tricky one as I seem
to spend my time trying to talk down the European systems to ensure no one gets
carried away but what if (and it’s a big if), the European systems do better or
as good as the UK systems did when they first went live? In the
first 4 months of going live, the 6 UK combined systems made a profit of 175pts
with an ROI of 43%. What if the Euro systems do the
same? Not only that, what if we find out next season that the Euro
systems are actually better systems than the UK systems. How would we feel
about not following these systems when they went live?
Paper
trading doesn’t give the full picture of how the systems perform on a daily,
weekly or monthly basis – I think this is a very important aspect and
shouldn’t be understated. I’d love to ask all my current subscribers from last
season if they were aware of the daily ups and downs that the systems would
provide them with last season? I’d be amazed if no one was caught out. I had 4
seasons of backtesting results, one season of live results behind me and I was
still taken aback by some of the ups and downs last season. The only
way to really understand a system’s performance is to follow the system live.
Are
these the systems of the future? – Impossible to know this for sure I
think and it’s far too early to even consider this but I suspect that some
people following this blog would love to see these systems perform as well as
the UK systems or better. If it is the case that we’re all going to be
following the European systems in future, by not following closely in the first
season, do you run the risk of falling behind the curve? A few
people who have joined for next season have mentioned that they’d wished they’d
joined last Summer but didn’t want to take the risk of joining until they saw
that the first season wasn’t a fluke. I suspect they now regret that decision
and feel like they’re playing catch up with others.
I think that’s the 5 main advantages I can think of for
following the Euro systems next season. Here’s the main
disadvantages I can think of!
Disadvantages of following the new ‘unproven’ European
systems
There
are no live results – Without a doubt, the biggest concern for me is
the fact there are no live results. I’ve used this quote elsewhere and on the
blog but I strongly believe that one seasons live results is worth 100 seasons
of backtested results. I can’t stress enough how much more weight I place on
live results than backtested results. Hence, in my opinion, the backtested
results don’t mean much as soon as a new system goes live!
Time
can be better spent following the UK systems – This is an interesting
one and it’s one that I find it hard to argue with. If people want more bets,
why don’t they just follow another system or two from the UK systems? If the
new Euro systems don’t turn out to be what we hoped they’d be, then spending
time following them next season has been time wastet and could have been spent
following the proven UK systems.
They
can’t be staked properly anyway – I think this is a valid point and
one that is linked to the previous point. Even those that decide these Euro
systems are worth following next season, they can’t really be staked
aggressively with no live results behind them and therefore, why bother? Is it
not better just increasing stakes on the UK systems and making profits that
way, rather than messing around with smaller stakes on unproven systems?
I’m sure there are more disadvantages of following these
newer systems but they are all going to relate to the fact the systems have no
live results I suspect!
OK, now I’ve laid out the advantages and disadvantages
that I can think of, what do people think? Well, here’s my take on it. I should
add that this is my personal view only (as in the view of a punter, not the
view of a tipster running a very profitable service!). I’ve already given my
view as a tipster (watch the Euro systems next season only!!!)
I’m a great believer in the phrase intelligent risk
taking and it is something that underpins everything I do with betting. During
the first season of the UK systems, I had nothing to go on but 4 past seasons
results and it didn’t stop me from following the systems in the first season
and making a profit. It was based on the fact the historical results were very
good and my belief that I could make a profit from following the
systems. Aside from making a profit, it also taught me an enormous
amount about the systems and how I could use them going forward to make a
profit.
Last season, I followed some of the new UK systems in a
separate portfolio for much of the season even though they too had no live
results. Although the ROI wasn’t great, I still managed to make a profit and of
course, I learnt a lot about the systems and some of these systems will make
their way into my main portfolio of systems next season.
So, what do you think I’m going to do next season? Well,
not surprisingly, given what I’ve done for the first two seasons when new
systems have appeared from the TFA laboratory, I will follow some of the new
Euro systems in a separate portfolio from my main football betting systems and
see how I get on.
Like I have always done, the staking will reflect the
fact there are no live results and ultimately, my profit on the season will
hinge on the UK systems but the Euro systems will carry enough weight to ensure
that I want the systems to do well. No point playing £2 stakes and not caring
about what the results will be every week. The stakes need to be set
in such a way that if the systems perform well, I’ll see a difference to my
portfolio return but if they perform poorly, I don’t wipe out all the profits
made by the UK systems!
What's my advice from one successful investor to another?
Well, it's up to you really. I'm not going to tell you a definitive Yes
or No for following these systems. We all know the risks at this game,
regardless of whether a system has live results or not. If people are
comfortable following a new system, knowing the risks, then they should think
about doing so. If people aren't comfortable, then don't follow these systems
next season. It really is as simple as that I think!
Bingo! I do believe we're on the same wavelength! :)
ReplyDeleteOK...this is what I'm thinking.
When I look at a service or system that potentially I might want to follow financially, I think there are so many more factors to consider other than (or at least, in addition to) what the simple bottom line profit figures say. Let's take for granted that the figures state that a profitable edge has been found.
Two of the major factors to consider have got to be: can I get the money down on the selections, and can I get it down at a decent price, ie. before the price is smashed in (or before an intelligent market brings the price in by virtue of being an intelligent market).
So when you talk of the ability to get money into the major European markets with little difficulty, which I know from personal experience to be true, and the fact that there are very likely a fewer number of people actually following the Euro systems than the UK systems next season, to my mind, they are both extremely relevant points.
I must add though, I'm 100% convinced that by following professionally and diligently, and by having bookmaker accounts set up properly (which is a whole different topic) getting on the UK system selection bets at the correct prices won't be a problem this season coming.
Anyway, before I digress further...I am also very aware of the weaknesses that paper trading hold. You're right, without actually putting money on selections, it is near impossible to gauge the ups and downs involved in following. As you know, wherever possible with the footie, I now take an age literally going through each day's past results from a system/service, just to get an idea of the psychology needed to follow it live. Doing this, you see some bad days (remember the day back in the 2010/11 season when West Brom were 3-0 up at half time against West Ham, and drew 3-3 - the day where every other bet seemed to lose aswell?) and you can start to menntally prepare. This is massively important I think, when it comes to gambling. Once all the analysis and poring over figures has been done, it's rendered useless if you can't handle the mental side of following something live.
To be continued...
....
ReplyDeleteThirdly, re. not following and falling behind the curve. I'm not too concerned personally about potentially missing out on profits with a view to following once there is a season of "live" performance to give that bit more confidence. But I also think it would be a shame to miss out entirely, IF there is sufficient evidence to suggest that by taking a calculated risk by following, there is a good chance of profits being made. The emphasis being on the word 'calculated'. What suggests to me that the risk may well be worth taking are the similarities in performance to the UK systems that became apparent during the development of the Euro systems. Perhaps it is coincidence, but reading your blog posts, I simply don't believe it is. So we have one, proven analyst, developing new systems using similar methodologies to those that have already been proven to be extremely successful, coming up with something that produces extremely similar results. You see where I'm coming from.
One other point. As you know, I followed your UK systems in their first live season, and did rather well. So, a) if I was prepared to do that with the UK systems, what logical reason is there for not doing so with the Euro systems now that I've seen the UK systems perform so superbly, bearing in mind the paragraph above? And, b) as a (hopefully) wiser gambler now, I'd not bugger up in the same way as I did when following the UK systems in their first live season by tinkering with staking, changing which system I followed, etc. Still made great profits in that first season, but not as great as they should have been. Lesson learnt, and by learning it, I'm sure that even if the Euro systems didn't live up to our hopes next season, any losses would be very controlled.
Finally (phew!), there's a long way to fall from the roi's quoted - even allowing for the fall between backtested and live results - to produce a loss. My estimation at this stage would be that if the first live season of Euro systems turn out to be disappointing, it would be because the profits made were smaller than hoped as opposed looking at losses.
So to my mind the risk:reward ratio, to me anyway, looks like the risk is worth taking. With an older head on my shoulders though, I need to look at staking. Perhaps wading in at full stakes is simply unnecessary at this stage. It would upset that risk:reward calculation.
I have a feeling I've just written paragraphs confirming that we're thinking along similar lines! :) Maybe I'm just coming at it from a different angle...that of the punter/follower as opposed to the Systems Analyst/Developer/Owner.
Now I'm off for a rest...
Rowan
Hi Rowan.
ReplyDeleteApologies for the delay in responding mate, was determined to finish the Euro systems download document at the weekend, so didn’t really read your comments until today. I need time to digest your comments! ;)
Having read your comments, I think it’s safe to say we’re both on the same wavelength completely. I think the one point you make which I missed in my post is something I’ve said elsewhere a few times. There is such a big drop-off needed from making a profit to a achieving a loss on the systems, I take great comfort from this fact and I think others should too. As you say, a disappointing season next season is probably a small profit, rather than a gigantic loss on the systems, touch wood.
I’ve given the new UK systems from last season a lot of stick since they went live and yes, they have disappointed me but 10 of the 11 systems achieved an ROI of 3%+ last season. As picked up during the system reviews, some of the ROC’s were actually very decent due to the low level of volatility on many of these new systems meaning the banks were fairly low. Hence, with hindsight, it wasn’t anywhere near as bad as it seemed on these systems. I got too sucked in by the low ROI and of course, by comparing the returns to my other systems.
It’s hard to say what I want these Euro systems to achieve next season. As long as the UK systems do well again, I’d take any sort of profit on the Euro systems as a bonus. Of course, if the UK systems don’t do well, it will be a poor season, regardless of what the Euro systems do. Hence, my preference of course is for the UK systems to do well first and foremost but I suspect that view is shared by everyone following, regardless of how many of us dabble with the Euro systems next season.
The risk:reward ratio will vary for every individual as it depends on your risk aversion ultimately but I’m sure I won’t be alone in dabbling with a few of the Euro systems next season. At the very least, it means I have an interest in even more games taking place next season!
I reckon there may be 500 system bets some weekends. Should be fun trying to keep track of all of them over a weekend. lol
Can’t wait. :)
Graeme
PS. I reckon that blog post was my longest ever (that's saying something for me!) but your reply was the longest I'd ever seen on a blog post too. You should really get a blog with your writing skills. Oh, wait a minute....