Two Steps Forward...One Step Back

It's also frustrating to take a step backwards in any project, but sometimes it's necessary. I noticed that the correlations in my earnings per start data were lower for 1999 to 2000 than the earlier year, and suspected that there was something funny going on in the data. It appears I was right. Because the data was collected sometime around late 2002 or 2003, the year 2000 crop had only a very limited number of starts in the data, and this likely resulted in more random variation and less correlation to previous years. Because the average correlations I've been reporting include this crop, I'm going to go back and recalculate them without it. What I know is going to happen is that the averages will all go up. What I don't know is whether there was any systematic bias in the results. By having one crop where the horses' four year old and up seasons weren't included, were sires whose offspring mature late being 'punished'? Would this have had any impact on the analysis I've been doing? Probably not, but there's certainly enough question that I'll eliminate the year 2000 crop so that the data represents something closer to a complete record of the performance of the included offspring.

Experimental Design

In response to the series of posts on my research into predictive factors for sires' offsprings' earnings per start, 'Winston...not really' suggested the basic design for an experiment involving breeding multiple stallions to the same band of mares to avoid the possibility that what we're measuring when look at stallion performance is simply a result of a systematic difference in the quality of mares that are being sent to them. His suggestion is definitely a good idea...if you have a few billion dollars to spare. For anyone who doesn't, statistical analysis should be sufficient. I've mentioned before that we're going to need to make adjustments to the sire statistics to take the quality of mares into account, and its something I'd like to tackle in the near future. However, if we're going to adjust for 'quality' of mare, it may make sense to at how to measure that quality first. Racing success? Previous breeding success? Quality of bloodlines? I know that some studies on the topic have indicated that racing success is more important than bloodlines, but I'm not sure anyone has tried to measure the relative predictive value of racing success versus success of previous progency...and how the balance changes as a mare ages and we're able to gather data on a slightly larger number of her progeny.

EPS Research Update

I've continued working on the earnings per start research that I discussed recently here. Some of the new findings:

1. Looking at the overall mean earnings per start for a sire's offspring, rather than the mean of the offsprings' earnings per start, did result in a slightly better predictive result (.45, instead of .40). Put more clearly (I hope), it was better weighting the EPS by race rather than by horse. However, this still wasn't as good a forecasting tool as the median (which had a crop-to-crop correlation of .50). I definitely need to try out something called a "weighted median" so that I can try weighting the median by race instead of horse. I expect this to provide an even more accurate prediction. And in fact, I'm guessing that weighting each race evenly isn't optimal either. We learn more information about a horse's ability from its first or second race than we do from its twentieth race, and the statistics will probably reflect that.

2. Using only the median data, comparing crops that were more than one year apart (ie looking at the correlation between a sire's 1997 and 1999 crops) SEEMS to show a slight trend towards less correlation than crops in consecutive years. The effect isn't clear enough to be definitive, but it would make some sense, since over time the quality of mares that a sire is sent to may change.

3. Also using only the median data I tried using two year averages (means) of the individual year medians as a predictor of the following year (for example, the average of the 1998 median and the 1999 median to predict 2000 results). It didn't seem to work any better or worse than one year. Even a three year average didn't work any better. This surprised me a little, but what may be happening is that the benefit of greater sample size that the multi-year averages are giving us could be cancelled out by the effect I described in #2 above. There's also a good possibility that this is simply due to random chance, and over large studies, the two year averages would have a slight benefit.

4. The most interesting result of the new work I did was that the average of the 1996 and 1997 medians was a GREAT predictor of the average of the 1998 and 1999 medians. Even without refining the 'predictor' to use a weighted median and to adjust for quality of mares, the correlation of the sires' earnings per start medians between these two year sets of data was .77.

Obviously these results have suggested all sorts of productive follow-up research. One of the things I'll need to look at is how the multi-year studies I did for median would perform for mean...since that's really what people are looking at when they look at a sire's career AEI.

Changes In Behavior

The first study I did for Thoroughmetrics compared the success of horses that had been sold at yearling auctions against the success of those that were sold at two year old in training auctions. What I found was that there was a drastic difference. Not only did those in one of the types of auctions perform better overall (and at all price levels), but the performance was more predictable too...there was a much stronger relationship between price level and performance. Really valuable information for anyone spending hundreds of thousands of dollars (or more) per year at auctions of unraced horses.

That said, it recently occurred to me that there's a real problem with studies which look at any human behavior that changes drastically over time. In this case, the conclusions of the study will only be valid as long as buyers use the same criteria to make their decisions about what a given horse is worth, and as along as sellers enter the sames types or quality of horses in each type of auction that they did at the time of the study.

A similar problem exists with any data on the return on investment (ROI) of any handicapping system. Data gathered on the ROI will only be predictive as long as handicappers as a group don't change how they make decisions about how to allocate their money to horses with various characteristics. So although ROI is what the handicapper ultimately cares about, it is also one of the least stable measures to look at.

For handicappers, the solution is to actually create a 'line' based on the key characteristics of the horse. If you know what the odds on a given horse should be, you only need to compare those to the actual odds to know if a bet has a positive expectation. So instead of thinking something like "lone speed in a 6F race on average returns $2.30 for every $2.00 bet, so I'll bet on #4", handicappers should be thinking along the lines of "lone speed in a 6F race increases a horse's chance of winning by 15%...let's see if that makes #4 a good value at his current odds".

This type of change in thinking applies to ownership as well. Instead of doing studies that focus on general measures that change (like which type of auctions provide the best value), we're better served by looking at more stable measures. For example, if we find that offspring of AP Indy on average receive Beyer ratings 2 points higher in their races than offspring of Silver Charm, that's not affected by changes in human behavior (unless you think that there's a systematic change in how Beyers are being subjectively adjusted over time). So instead of simply knowing which type of auction tends to provide better value, we could go into any auction and evaluate what each horse is worth based on their Beyer ratings, and whether they're a good value based on the bidding.

I realize that in essence I'm saying that most people have the right approach, and that while the findings of my original study were valuable, they will become less valuable over time as behavior changes. However, where most people go wrong is that they judge each horse's value subjectively, or using flawed statistics that don't have much predictive value. It doesn't help to judge individual value, if you don't have a valid model for valuation.

Earnings Per Start

I finally got my hands on some data that will allow me to do research on the predictive value of various measures of sires' success. I have about ten years worth of basic data on the performance of offspring of the top one hundred sires at a point in time several years ago. The data has a number of minor flaws for my purposes, but it's still a treasure trove of valuable information and will allow me to start answering some of the questions I'm interested in.

I started playing around with it today, and took a look at some data for the 82 sires that had offspring in every crop from 1996 to 2000. So far I've found that the average correlation for consecutive years of the mean of earnings per start of the sires' offspring is .40. The average correlation for consecutive years of the median of earnings per start of the sires' offspring is .50.

What does that mean? Actually, we can't really draw any conclusions about the absolute levels of the correlations until we make some adjustments. It's very possible that a large part of the correlation is due to the relative quality of the mares being sent to each stallion. I have the data I need to make some adjustments for that, but it's not in 'machine readable' form. If anyone with access to a copy of American Produce Records wants to volunteer to help me enter the CI values for about 20,000 mares, I'm taking volunteers! In the meantime, my findings do show one valuable piece of information - that the median earnings per start correlates better from year to year than the mean of the offsprings' earnings per start does.

My next step (which I'll get to in the next few days) is to look at the actual mean earnings per start (in other words, counting each race once, instead of counting each horse once). I believe that will be a closer approximation of the popular AEI, so it will be interesting to see if that performs any better than the measures I've already tried. After that, I'm very interested to see how the different measures do at predicting eachother. In other words, is next year's crop's mean earnings per start better predicted by this year's crop's mean earnings per start or this year's median earnings per start. It would not be a huge surprise if the median turns out to be the best predictor, since it is less impacted by a single huge success (like Curlin). It would certainly be an indictment of using AEI to evaluate sires' quality if it can be outperformed by something as crude as the median earnings per start.

True Nicks

One of the more popular (and controversial) factors that many people use in breeding decisions is the past success of 'nicks' between various bloodlines. Some people wouldn't consider a mating where the nick has been unsuccessful in the past, while others think that the entire theory is pseudo-science with no predictive power. One of the most popular ways of using nicks is to use the ratings provided by Truenicks. Their ratings are provided free for stallions whose farms have 'sponsored' them, while they are sold for around $20 per hypothetical mating for other stallions. The Truenicks people use the following statistics to back up their claim that their ratings are a powerful predictive factor in the success of matings:

"1. While only 13% of the entire Thoroughbred population earn “A” rankings (A to A++), 37% of the stakes winners rate as “A’s.”

2. Horses rated “B” or better (B to A++) represent just 30% of the entire population, yet 3 out of 4 (77%) stakes winners rank “B” or better.

3. Almost half of Thoroughbreds in general–44%–are on the low end of the scale (rated “C” through “F”), yet only two in 25 stakes winners (8%) have these lower rankings."

There's just one problem. As I've discussed in the past, this type of analysis is "cheating". Those who develop automated trading systems to predict movements in financial markets are all too familiar with the pitfalls of 'overoptimizing' your theory to fit past data. The systems developed do a great job of predicting the past, but have little predictive power going forward. The problem here is that the ratings themselves are based on the same data that is being used to demonstrate their success. To do a valid study of the predictive power, we'd need to look at rating in some point in time and then track the success of horses with various rating going forward.

The great news is that we have the data available to do that! Because of the free ratings available for many stallions, we can create a list of ratings for several hundred (or more) horses now, and then track their performance on the track over the coming months and years. It will be several years before we have the final results of the study, but the amount of work involved isn't overwhelming, and it will be a fun study to follow as the results begin to come in. I'll be talk more in future posts about the specific study design I'll be using for this, and how I'll deal with a few of the issues or pitfalls that it will face.

How Much For The Holy Grail?

Ok, it may not be the 'holy grail' of pedigree analysis. In fact, it's really just a starting point, not a final destination. But after my realization a few days ago that sire data by crop simply isn't available for free, no matter how much manual work I'm willing to do, it's beginning to feel like it.

To recap, what I'm hoping to do is evaluate the predictive power of existing measures of sire success such as AEI and AEI/CI, and ultimately to come up with something that works better. In order to do that, I need to get data on multiple crops of offspring for a reasonable large group of sires. Realistically, I'd say I'll need about 50 sires included to have full confidence in my results.

Looking at the BRIS and Equineline web sites, it looks like I can get the data I need for $18 per sire. So it turns out that the holy grail costs about $900. Based on the sample reports on the BRIS site it looks like the data is human readable, but not in a good format for computer processing, so I'll probably have a long, boring data entry project to put it into Excel or Access, but that's something I can live with. If anyone with $900 to spare wants to invest in this project, I'd be happy to share the full results with you, reimburse your investment as I generate sales of the research, and throw in some free advertising on my site (if you run racing partnerships).

Or Maybe Not...

Argh! The calculations I discussed in my last post may have been based on some faulty assumptions about how sire statistics are reported. Can anybody answer the following question...When BRIS presents statistics on 'leading second crop sires', are they including data only on the horses that are actually part of the sire's second crop, or data on both the first and second crop children of sires whose second crop are currently racing? I had assumed that it's the former, but after looking over the data some more I'm thinking it must be the latter since it doesn't seem like ANY horses have more starters and winners in their 'second crop' year than their 'first crop' year. If so, that's incredibly frustrating and is almost certainly going to mean that research in this area will involve paying for a substantial amount of data and doing a lot more work than I had anticipated. Why does it always seem like BRIS and Equineline go out of their way to present their data in a way that makes it useless for any serious research?

Finally Getting Somewhere

As my regular readers (all four of you) know, one of the research projects I'd like to tackle is an evaluation of the existing measures of breeding success, and ultimately coming up with some statistics that do a better job of predicting future success. As I mentioned yesterday, the main challenge has been finding the data I need to study it. This morning I had a very minor breakthrough. Although BRIS is the data provider for The Bloodhorse, they don't always present the data in the same format. It turns out that BRIS has a version of the leading freshman, leading second crop, and leading third crop sires where they include SPI. As far as I can tell, SPI appears to be the same thing as AEI, although I haven't yet checked the numbers to see that the calculations are being done exactly the same.

I took list of leading first crop sires of 2007 and the list of leading second crop sires of 2008, both ranked by SPI. Of the 75 names on each list, there were 59 in common. The correlation between the SPIs was .65. Removing the horses with very few starts (less than 20 starts in the 2007 list and less than 30 starts in the 2008 list) didn't make much difference. The remaining 34 horses had a correlation for their SPIs of .63. On the surface, these correlations look pretty high, and indicate consistent performance by sires across crops. Of course, the problem is that a lot of that consistency may be coming from the fact that some sires are covering better mares than others, so we'll need to make an adjustment for that before we can have any confidence in the results we're seeing.

Data...Looking For Help

The single greatest impediment to doing statistical research for the thoroughbred industry is the lack of quality data. That may seem surprising, considering that racing is a very data intensive sport. But what I've found is that virtually every study I'm interested in doing requires data that either needs to be painstakingly entered into spreadsheets or databases manually, or simply isn't available. In many cases, the data is out there, but not in a form that's usable for statistical research. For example, I've got a copy of the American Produce Record...but the data in there can only be searched one horse at a time...not exactly ideal for statistical research where I'm looking for patterns over samples of hundreds or even thousands of horses. I've talked about some research I'd like to do evaluating the predictive value of AEI and AEI/CI...someone with direct access to the BRIS or Equineline databases could do the study in about ten minutes. Unfortunately, neither organization thinks that's how we should be accessing their data, so I would apparently need to pay for the data per horse. Again, not exactly ideal for statistical research.

If any of you have data that you can share that you think might be useful for me, I'd love to hear about it. I'd be happy to use it for research that you might find useful or interesting too.

Vacation

I'll be away from Sunday until Friday, so there won't be any new posts until after I return.

Dehere

In the True Nicks Blog at The Bloodhorse today they discuss how the entire future of Dehere as a sire was changed when his daughter Arrested Dreams lost the 1998 Matron by a nose. The loss cost Dehere the chance to be the leading freshman sire, and ultimately probably resulted in his being exported to Japan (and not getting to cover the same quality of mares that he would have) before his value as a sire was recognized several years later. I can't really comment on whether this analysis of events is accurate, but if so it would show how unbelievably superficial a lot of the pedigree analysis in the thoroughbred industry, and how bad the tools that many use for major financial decisions are. Does anyone REALLY believe that an evaluation that comes up with a substantially different answer depending on which horse finishes a nose ahead in one race has any possible predictive value for the future?

2009 Kentucky Derby

Yes, I know it's early to be thinking about the Derby, but I am anyway. Like breeding and owning horses, handicapping can offer some nice opportunities based on people's inability to correctly take sample size into account in their analysis. The top two two year old colts in training in the US (Vineyard Haven and Midshipman) are being sent to Dubai to prepare for the 2009 Derby. The common thinking is that because this "has never been successful", it can't work. On the surface, this makes sense. The two year olds that have followed this path before have always either missed the Derby entirely or not performed well. My instinct upon hearing that two more top colts were being shipped out was something along the lines of "shoot, there go two more horses we'll never heard from again". On the other hand, when everybody has the same strong emotional reaction, it's worth thinking about logically to see if their may be an opportunity. Here are a few reasons to think that the trip to Dubai may not be the kiss of death that most people think:

1. Limited sample size. How many horses have actually followed this path? Ten? Fifteen? In any case, the number is small enough that on average, we'd expect maybe one Derby winner out of the group and one or two other good performances. Godolphin may just be suffering a run of bad luck.

2. Vineyard Haven and Midshipman are far more accomplished than most of the other two year olds that were sent to Dubai. Most (such as Etched and Numaany last year) have simply been impressive maiden winners. The two colts this year are both multiple Grade 1 winners.

3. Conditions have changed. The first colt I remember following this path was Worldly Manners. When he won the UAE Derby, I'm fairly certain I remember hearing that the entire field was owned by the ruling family of Dubai. That's got to make you wonder how tough a prep race it was back then. Now it's a world class race against Southern Hemisphere three year olds (four year olds by US standards). So it's likely that horses following the Dubai route to the Derby now are getting much better preparation than ten years ago, and the sample of valid comparisons is even smaller.

Based on all this, I'd suggest keeping an eye on Vineyard Haven and Midshipman for an opportunity to get good value betting on them. If they're part of the 'pool' for Derby futures, that could be your chance. Of if they make it to the starting gate after performing well in the Dubai prep races, they may go off at higher odds than they deserve.

More on Trainer Winning Percentage

In my previous post I mentioned that trainer winning percentages appear to be very stable from year to year. For the 73 trainers who were among the top 100 in the US in wins in both 2005 and 2006, there was a correlation of .83, which means that the list is exceptionally stable. The highest percentage trainers are pretty much the same from one year to the next to an extreme degree. In fact, it was so extreme that I could see the consistency just eyeballing the numbers. Interestingly, the correlation for 'in the money percentage' was slightly lower at .79 for reasons which aren't totally clear to me. I had assumed that it would be a little higher since the sample size of horses in the money is greater.

One thing to keep in mind is that the exact correlation should probably be taken with a grain of salt, since there is a selection bias at work here. The value is probably being artificially increased by the fact that we're not looking at the trainers who had a good enough winning percentage to get them into the top 100 in wins one year, but not the other. This is likely being offset to some degree by the fact that we're also excluding a much greater number of trainers whose winning percentages weren't high enough to make the list in either year. By doing so, we're creating a more homogenous group, which tends to lead to lower correlation values. Either way, if there was no correlation, we'd expect to see a value of 0 in our sample, and a value of .83 clearly indicates a great deal of consistency. It's likely that this is due partly to a trainer's "skill level", but even more to how aggressively they place their horses.

Trainer Data

I'm in the middle of doing some research on the future success of horses after they're claimed, based on the 'type' of trainer they're claimed from. I'm defining those trainer types based on their winning percentages and their average earnings per start. While I'm going to be charging for the end results of my research, I've compiled aggregate data for all trainers who were among the top 100 in the U.S. (ranked by number of wins) during any year from 2004 to 2006. If anyone is interested in a spreadsheet with this data, I'd be happy to share it for free...just send me an email or put a request with your contact information in the comments here. One interesting thing that's showing up already is that trainer winning percentage appears to be very, very consistent from year to year. I'll be discussing that in more detail once I get to do a little more analysis.

Auction Strategy

Sorry about the lack of recent posts. I'll see if I can make up for it over the next few weeks.

In his latest blog post at The Bloodhorse ( ) Scot Gillies asked readers to weigh in with their thoughts on strategy for buying broodmares at auction. I thought I'd post my response here.

Basically, I think we need a little more information than he's provided...what is his total budget? Does that included the money allocated for ongoing expenses? What price range are the sixteen mares he's looking at likely to fall within? What his goals for his purchases? Is he planning to sell the offspring of his broodmare, or to race them?

Assuming that his budget is typical of a small-time owner, and not someone like IEAH or Zayat, and that he's breeding to race, I think he's better off going for a single broodmare. The expenses on two horses will be so much higher than one, that's each one is likely to have a negative contribution to profitability.

Also, I think he should wait until later in the sale, when the bargains are likely to be steeper. Yes, he may miss out entirely on getting one of his targets. But really, what's the harm in that? There's always another sale, with more mares available. In the meantime, it looks really unlikely that the overall economy or and/or the 'thoroughbred economy' will recover so fast that bargains won't still be available in a few months.

Oops...Blood-Ex Going Ahead After All

I just received confirmation from the folks at Blood-Ex that they will be going ahead with their launch over the Winter after all. The two members of their team that have been publicly identified (Valentine Feerick and Nick Carthy) are both still involved, and they intend to update the website soon. That's definitely good news, as I'm very interested in to see things play out.

Very Quick Update

When someone asks a good question in the comments on an old post, I'm never sure whether they'll see my response if I address it there. Winston asked whether I'm going to have trouble with my study of how lousy horse names impact auction prices, because most two year olds sold at auction are unnamed. The answer is a resounding 'yes'! I hadn't realized how few are named prior to the auctions, so it will take quite a while to find enough that have lousy names for any kind of study. I probably won't bother.

I've mentioned baseball and fantasy baseball a few times, mostly as an excuse when I don't post for a while. My inspiration for starting Thoroughmetrics was the statistical research that's been done in baseball over the past 25 years or so. As an example of how strong the predictive value of that analysis can be, I just won a car by coming in first in a fantasy baseball competition with over 14,000 contestants. While the details are different, I'm using the basic statistical techniques in my analysis of horse racing, and I expect to find similarly valuable results.

I'm also going to have agree with Winston's assessment that Blood-Ex failed before it ever launched. Their website hasn't been updated since the end of June.

AEI and CI Revisited

A few weeks ago I wrote about an idea I had for calculating AEI and CI by crop. It turns out that one part of what I wrote was accurate - things are rarely as straightforward as they seem in statistical analysis!

The reason I want to calculate AEI and CI by crop is to see what kind of crop to crop correlation there is for AEI and AEI/CI for sires and evaluate the value of these measures as predictive statistics.

It does look like there's a way to do this, but it's more complicted than I had anticipated. First, I'll need to calculate my own version of AEI by crop based on following a group of stallions data through The Bloodhorse's 'Leading First Crop', 'Leading Second Crop', 'Leading Third Crop' lists. Basically, I'll figure out average earnings per starter in each of those crops.

Then I'll need to figure out the CI for each crop for those sires. I should be able to do this by using the following formula:

Crop CI = ((Total CI * Total Starters) – (Previous Year’s Total CI * Previous Year’s Total Career Starters))/Starters In Crop

Basically I'll be looking at changes in career CI from year to year, and based on the number of starters in this crop compared to the number of career starters the sire has had, figuring out what the CI for the current crop would have had to be to impact the career CI by the amount it changed by.

Unraced Mares and Unraced Sires

Starting today, Scot Gillies' blog at The Bloodhorse will be featuring a series of mares for sale at the upcoming Keeneland November Sales. He's looking for reader input, and one of the questions he asked in regards to today's feature mare is 'what do you think of unraced mares'.

I'll give my opinion on that here. Most horses, regardless of breeding are relatively unsuccessful on the racetrack. Certainly most are not successful enough to warrant much interest as broodmares. So statistically, you have to go on the assumption that most unraced mares would not have been good enough to be worth much for breeding. This is even more true once you consider that they may have remained unraced due to soundness issues that they could potentially pass along to their offspring. No matter how well bred, I would assume an unraced broodmare should be a cheap broodmare.

An obviously extension of this line of thinking is that an unraced stallion is NEVER worth trying out. I don't care that he's a son of Storm Cat out of a top mare. The odds that he would have been successful enough to warrant a career at stud are incredibly low...certainly never high enough to justify an expensive breeding experiment.

High Percentage Trainers

I realize I've left a few loose ends by not finishing my discussion of lousy horse names, and my attempts to calculate AEI/CI per crop. I'll get back to those shortly, but wanted to mention part of another study I'm working on. This is one that I'm doing for a client - we're swapping research for a small share in a 2YO filly that his partnership owns. The study is going to focus on which factors can be used to improve the percentage of claimed horses that go on to become stakes quality horses. I'll be looking at a number of factors, but the one I'm the most excited about at this point is the idea of restricting claims to trainers with high winning percentages. On the one hand, a low percentage trainer is generally going to be a worse trainer, and may make more mistakes...including not identifying a potential star. However, I think this would be more than outweighed by the fact that a high percentage trainer likely gets that high winning percentage by entering horses in spots where they can win...which means generally entering them at or below their 'true' class. They realize that its tough to be profitable at the claiming game, and accept the fact that in order to be profitable, they're going to have to lose a certain percentage of their horses.

The Worst of the Worst

My post over at Thoroughbred Champtions generated just the kind of discussion I was looking for, with over 75 awful names being suggested. Some of the ones that I think are particularly bad (possibly bad enough to have impacted auction prices prior to their racing careers):

Dirty Dish Mitch
The Love Master
Viagra River
Stinky Twinkie
What a Gaylord
Isitingood
Bodacious Tatas
Stinke Pant's (yes, they put the apostraphe in the wrong place)
Kirby's Glue Foot
Uno Tess Tee
Annie Dirt Bag
A Colt Named Sue
Erotick Mountain
Smelly
Judge Smells
Chickenonastickbabe
King of Scat
Dead Rabbits
Pissed Apache
Schoonerwharfbardog

In case anyone is wondering whether I could just study how these horses performed relative to their auction fees, the answer is 'definitely not!'. Without even looking at the results, I can tell you what we'd find - they substantially outperformed expectations, and would appear to indicate that purchasing horses with lousy names is a great way to get a bargain. The problem is, that these were the horses that people were able to think of. In general, that means they probably had some success on the track...or at least made it to the track enough time that some people have heard of them. Using this kind of after the fact selection of horses for a study will ALWAYS show strong performance. You can see why by trying a more extreme example. Let's test whether giving a horse a name that starts with 'Z' makes them more likely to excel on the track. Offhand I can think of...Zenyatta, Zaftig, Zarkova, Z Fortune, Z Humor, Zanjero...WOW! Definitely name all your horses with 'Z' names! You can see what the problem is here.

It's possible that you could get reasonable results using the second study design (compare auction prices to sire's stud fees) I mentioned in my previous posts with the horses people thought of that had bad names, but even that might be impacted by the fact that people will tend to remember superior horses. Perhaps the horses on the list, in addition to generally being relatively fast, also tend to have good conformation (which presumably tends to go together with speed). If that's the case, then they likely would have sold for higher prices than their sire's stud fees would indicate...so the study would tend to show that horses with awful names are a bad deal.

In any case, I'll definitely get more reliable results if I select names in a more random fashion. I'll go through the full list of horses for sale at a two year old auction and select those with awful names. I'll then compare their auction prices to their sire's stud fees, and see if they tend to sell any worse than the average horse at the same sale. My selections will focus on those names that include explicit sexual references, punctuation and spelling mistakes, and anything else that would make a name really embarrassing...not just a little unappealling.

Useful Angle For Auctions?

A not too serious post on the dos and donts of claiming horses by Frank over at That's Amore Stable generated some discussion about awful horse names. That got me thinking...are there any names that are so awful that it would impact potential buyers' interest in purchasing the horses? Never one to avoid an interesting research topic, I'd like to try to answer this question. The study design shouldn't be that hard:

1. Go through a list of horses about to be sold, and select the worst of the worst names.
2. Collect the racing of an unbiased sample of horses sold at the same auction.
3. Compare the racing results of the oddly named horses with the rest.

This could be done retrospectively as long as I'm not familiar with ANY of the horses sold in the auction, since I wouldn't want that to bias which names I select.

Actually, an even simpler study might begin to answer the question:

1. Go through a list of horses about to be sold, and select the worst of the worst names.
2. Compare the auction prices (relative to sire's stud fee) of the poorly named horses with those of all horses in the auction.

In order to do either of these studies, I need a good understanding of what names people really hate. Which horses have such bad names that somebody would be embarrassed to tell their friends if they owned the horse? I polled the folks over at the Thoroughbred Champions Forum to get some ideas of the worst of the worst names. I'll share the list in my next post.

Calculating AEI and CI By Crop

Things are rarely as easy as they should be when doing research on thoroughbred racing and breeding, and a lot of the blame for that has to fall on those who have the data. Rather than making it available (at a price) in its 'raw' form, so that we can do the research WE find interesting, they insist on spoon feeding it in the format that THEY find interesting, which often makes it nearly impossible get the information we want.

One example of this is that I'm interested in seeing how successful the most popular statistics for ranking sires (AEI and AEI/CI) are in predicting future success. What I'd like to do is get a list of sires along with their AEI and AEI/CI and see what kind of correlations the statistics show over time. The problem is that I need to look at the data BY CROP. If I go to Bloodhorse.com the data is presented by year (never mind that its in a PDF file and I need to spend time on 'data hygiene' to even get it into a usable format). The problem is, looking at the data by year, of course there will be high correlation...because we're comparing some of the same horses to themselves. Smart Strike's results in 2007 and 2008 will both include Curlin...that's obviously going to make the correlation between his AEI and AEI/CI higher, but what is it really telling us? Not much.

What we really need is the AEI and AEI/CI data separated out by crop, and I think I may have figured out a way to calculate this, and will share it in a few days, once I've had the chance to test it out.

Odds and Ends

Just another six more days of baseball season, and then my focus will shift entirely to horse racing!

In the meantime, I wanted to post on a few assorted items. At this stage there are only a handful of people reading this blog because I haven't gotten to promote it much yet (and wouldn't necessarily want to until I'm posting more regularly). In an odd coincidence though, one of those few readers is apparently Chuck Simon...the man who trained Comic Relief to her only career win. I made two posts about Comic Relief and why I think she'd be an underrated broodmare prospect (despite the lack of success of her first few children). Chuck provided some really detailed information on her. To me, the key points are:
1. The injury that forced her retirement is not something that would change my assessment of her promise for breeding.
2. She should probably be bred to sires whose offspring do well on turf.
3. Her connections believed she would have been a potential stakes winner.

I'm currently working on a study of claiming horses that go on to be stakes winners or stakes placed, and how to identify claimers who are more likely to do so. While I have a great capacity for completing length, boring tasks, if anyone can point me in the direction of a complete list of U.S. stakes winners for 2007 and 2008, I'd appreciate it. Otherwise, I'll be typing the complete list in by hand.

Breeders Cup Handicapping

In general I'm trying to focus on the inefficiencies in the thoroughbred industry that owners can take advantage of. Beggars can't be choosers though, and sometimes the inefficiencies aren't where we want them to be. On big days for international racing, such as the Dubai World Cup or the Breeders Cup, there can be some great opportunities. For example, in his race in Dubai earlier this year, Notional was the favorite for an extended period of time, and ended up going off as the 2nd choice. Keep in mind that this was a Grade 2 quality horse, who was completely out of form at the time, running on grass, against many of the best turf milers in the World. Obviously, his presence and odds should have made it pretty easy to find good value on other horses. With so many Europeans horses aiming for the Breeders Cup this year, its likely that similar values will exist...particularly if you have access to betting pools in both the UK and the US. There may even be races where you can bet on every horse in the race and be guaranteed a profit. Bet on the US horses in the UK, and the UK horses in the US, and proportion the bets so that they all return a similar amount.

Is The Bubble Bursting

According to The Bloodhorse, Sheik Mohammed's Bloodstock Manager, John Ferguson said, "If somebody is thinking of buying a racehorse, this is a good time to do it because I think there will be value to be had throughout the course of the year." It's unclear whether that was before or after yesterday's relatively weak Keeneland September Sales results.

He may be right. You can certainly buy horses for less money now than the same horse would have cost a year ago. However, I think the current environment for unraced horses has some of the signs of the early stages of a post-bubble bear market in financial markets.

First, some of the typical characteristics of a financial bubble:
1. Prices go higher than anyone thought was possible.
2. There is no fundamental basis for the level that prices reach.
3. Many people are buying purely as a speculation, because believe they can resell to someone for even more.
4. Some people believe that prices will never go down because 'this time things are different'.

How does the situation in the very recent past meet these criteria:
1. Prices are at all time highs (although in fairness, not so much different than in the 1980s at the top end of the market).
2. The 'fundamentals' (earnings from actually racing the horses) make it impossible for something like 90% of the horses sold at auction to ever earn a profit for whoever owns them during their racing careers.
3. The entire concept of 'commercial pedigrees' as opposed to 'breed to race' pedigrees is an open admission that many breeders subscribe to a version of the greater fool theory...they believe that there are buyers who don't care about winning, and will buy their horses at prices unjustified by the horse's potential to earn money on the racetrack.
4. I've heard plenty of people say "prices won't go down much as long as the Sheiks and Coolmore are around". This kind of misses the point of what happens if they choose to stay home one year.

If the current drop in prices is indeed the early stage of a bursting bubble, then the following may happen:
1. Prices will go down further than many thought possible.
2. Eventually fundamentals will drive the market...there will be no interest in 'commercial pedigrees', and much more of the market will be driven by the idea that the horse being purchased has a reasonable chance of eventually earning the money back.

I don't want to overstate the latter point though...people do buy racehorses for reasons other than making a profit (obviously), so the link to fundamentals won't be as strong as in other markets.

My overall point is that now may not be a good time to buy at all, despite the apparent bargains. It might be a good time to sit on the sidelines and wait for the bottom in prices. Just remember...when the true post-bubble bottom is reached, most people are usually screaming that its the end of the world and the market will never come back.

Exclusive Quality

In my last two posts I talked about a mare named Comic Relief. It appears that she lives in Florida, based on the stallions she been bred to so far. So who would be a good stallion to send her to? How about the fastest son of Elusive Quality? No, not Smarty Jones! Exclusive Quality. He won three out of four starts. Tied a track record at seven furlongs. In his three wins he defeated Diabolical, In Summation, Songster, and Bernardini!!! He's currently standing for $4,000 in Florida. Predicting a sire's success at stud is always a gamble, but I'd be happy to bet on that stud fee going up once his children have been racing for a few years (his first crop will be two year olds in 2010). Does Exclusive Quality match up well with Comic Relief? I have no idea. But until someone shows me convincing statistical evidence that concepts like 'nicks' have predictive value, I'll go with the approach of 'breeding the best to the best'. And given his brilliance on the track, I believe that Exclusive Quality is a lot closer to 'the best' than his stud fee would indicate.

More Comic Relief

A little more detail on the offspring of Comic Relief:

2004 - Gelding by Notebook named Satire. I haven't found details on his racing career yet.
2005 - Colt by Montbrook named My Friend Tingui. Racing in Puerto Rico.
2006 - Unnamed colt by Montbrook.
2007 - Colt by Bernstein. Sold for $7,000 in February 2008 to W.P. Stiritz.

Comic Relief

I've read in a variety of places that those thinking of becoming thoroughbred owners should only do so if its something that you'll enjoy enough to make up for the money it will cost you. The assumption is that, barring extreme good fortune, it's not a "beatable" game. A close look at the economics of owning racehorses is certainly a sobering reminder that the odds are stacked HEAVILY against you. That said, there are clearly massive inefficiencies in the industry, and if I didn't believe that discovering and measuring those inefficiencies had at least the potential to provide enough of an edge to owners to have a profitable expectation, then I wouldn't have started Thoroughmetrics.

That said, it's often useful to write out the exact nature of our 'edge' in any game. In general, if you can't detail specifically where your edge comes from, then you likely don't have an edge. My edges in evaluating breeding, racing, buying, and selling opportunities right now:

1. The results of a single study I did on buying horses at Two Year Old Auctions vs. Yearling Auctions, and in what situations each of them provide better results.

2. My knowledge of correct statistical analysis techniques that allow us to test predictive theories for their ability to forecast the future, rather than just explaining the past.

3. I have shiny, white teeth*

In other words, I have lots of work left to do.

In the absence of a proven edge, sometimes you have to rely on old fashioned common sense. Just don't assume that it will lead to the kind of results you're looking for. My common sense says that most potential owners are overvaluing horses where they can put a check the box next to 'Black Type', and possibly undervaluing horses who might have been equally good, but never earned any black type. I've begun compiling a list (note: there's only one horse on the list so far) of mares who meet the following criteria:

1. Did NOT earn black type.
2. Very lightly raced.
3. Showed potential stakes ability.
4. Career cut short due to problems unlikely to be passed on to offspring.

The first mare on the list is named Comic Relief. Actually, I haven't even verified that point #4 above is true for her, but she meets the other criteria - she won her only start (at a mile and a sixteenth) with a Beyer of 83. She's well bred for whatever that's worth (by Kingmambo). She's being bred to stallions in Florida who stand for $10K-$20K (Bernstein for one), and her offspring haven't yet accomplished anything worth mentioning (the most accomplished was last seen racing in Puerto Rico as far as I can tell). I think she and her offspring are a great value, particularly if I can find out why she was retired, and confirm that it was something relatively unlikely to be passed on to her offspring.

* I'm taking some poetic license here. While shiny, my teeth are not particularly white.

Some Updates

Thought I'd update people on a few things. As the baseball (and more importantly, fantasy baseball) season winds down, I'll have more time to work on racing and pedigree research. My new projects have gotten held up for a few reasons - lack of time, some difficulty getting ahold of the right data to do the studies properly, and most importantly, I went back to make some changes to the original study comparing the results of horses at different types of auctions. I showed the study to a few people, and the feedback from everyone was similar to my own opinion. "Great study, valuable results, but you need to factor in some sort of analysis of cost of ownership and ideally would include a wider variety of auctions". Since my goal is to provide the best research available anywhere, I'm working on making those changes. When I'm done the study will include a profitability analysis for each horse in the study, based on how many years they raced. It will also include some less 'elite' sales than those that were originally included.

I'm also looking for an update on something. Does anyone know what happened to Blood-Ex? They did their 'pilot launch' in June, and there hasn't been a peep out of them on their website or in the news since then. I emailed them several days ago, and didn't get a reply.

The Iron Horse

Scot Gillies from The Bloodhorse posted a comment here asking what I thought of their new study on 'The Iron Horse'. I'll have some more comments on it soon, but thought I'd start by sharing the email I sent to him:

Hi Scot. You had asked me (via my blog) what I thought of the Bloodhorse study 'The Iron Horse'. I've read it several times now, and believe that (as the study itself said) it provides a useful starting point, but doesn't really allow us to draw a lot of conclusions. Basically it shows that across many segments of the business, horses race less than they use to. It does not indicate anything one way or the other about whether the breed is actually less durable, or whether training methods have trained. Like Rob Whiteley, I would tend to doubt whether there has been enough time for the breed to change as drastically as many people seem to think. The most interesting thing to me is that even at the lower levels of competition, where there would seem to be financial incentive to race horses more often, rather than less often, racing schedules are much lighter than they used to be. Is it possible that the entire industry has missed the boat on this one, and are making a financial mistake in an effort to copy the techniques of the people at the high end of the industry? It's something I'll be giving a lot of though to, and hope to come up with some answers to.

Sorry

Sorry to anyone who has been checking in here that I haven't been posting at all. I'm finding (again) that there's only so much time to devote to my hobbies, and fantasy baseball (and to a lesser extent, fantasy horse racing) take priority over the blog, because at this point I've got a lot more potential money riding on them. I can't guarantee what my posting schedule will look like going forward, but if anyone has any topics they'd like to discuss with me, feel free to email me and I'll definitely respond.

Sample Size

Part of my purpose here at the ThoroughMetrics Blog is to help you evaluate the available research on the thoroughbred industry. As I've said before, the majority of it is flawed in one way or another, and insufficient sample size is one of the most frequent offenders.

One blog that I read fairly regularly is Scot Gillies' Five Cross Files at The Bloodhorse. Scot has a lot of interesting thoughts on the breeding industry, and he's also been very generous with his time, providing me with really detailed feedback on some of the research I'm working on. That said, in his latest post, Scot falls prey to the temptation of basing conclusions on ridiculous small sample sizes. The article is about undiscovered 'breed to race' sires. He does warn the reader that some his selections are lightly bred, and that he's tried to keep the small number of offspring in mind when ranking them. However, he goes on to include Raj Waki and Intidab in his top six. Raj Waki has only 29 starters, but "over 10% stakes winners". Intidab has 27 starters, but "tops 11% black type winners". The problem here is that "over 10%" of 29 is 3, and "topping 11%" of 27 is also 3. A sample size of 3 is not good news, and there is almost certainly no predictive value to the statistics these sires have compiled. We simply have no idea if they're any good or not. When you're thinking about whether a small sample size is meaningful, there's a good 'mental test' you can use. If you removed 1 or 2 from the total, would the result still be impressive? If I remove 2 from 3...well we wouldn't be talking about Raj Waki with 1 out of 29 stakes winnering offspring. When you have small sample size, the results have to be mind blowing to have any predictive value.

In fairness, drawing conclusions based on insufficient sample sizes can be very, very tempting. When I read Scot's article, my first thought was "he should have included Repent instead of one of those others". I made the exact same error as Scot! Repent has less than 100 offspring so far and just a handful of stakes winners. It's too soon to really tell if he's going to be a star at stud.

More About Blood-Ex

Along with horse racing, and financial markets, I’m also an obsessive fantasy baseball player. Last year, I participated (and finished 8th out of 4,000+ people) in a new game that used a stock market type trading mechanism. Because it was the first year the game had been run, nobody knew exactly how the ‘market’ would function. Two lessons that I learned there, that I think apply to Blood-Ex as well:

1. There’s no way to know exactly what will happen once trading starts, or what strategies will be successful.
2. Whatever happens, you’ll be in a better position to succeed if you’ve already given some thought to the possibilities prior to the start of trading.

I think the second point is especially true, because ‘immature’ or new markets are often the least efficient, and provide the greatest opportunities for profit. Once Blood-Ex has thousands of participants, including some with large sums of money available to them, the inefficiencies in the market will begin to disappear, and opportunities will be harder to find.

In the meantime, here are four things to think about as you prepare for the start of trading:

1. The 5% commission on sales is huge. It’s very possible that it will turn all short term trading strategies into losers. It should be less of an issue for those who plan to buy and hold shares. Also, it appears that you will not have to pay the commission when horses you own shares of are sold at auction. So if you’re holding shares of a valuable horse, it will generally be better to simply hang on to them until they’re sold than to sell them on Blood-Ex.

2. Most male horses end up their careers worthless as bloodstock. While trading in fillies and mares may have some resemblance to stock trading, with relatively modest price gains and losses, trading in colts and stallions is going to look like options trading…a few big winners, and a lot of 100% losses. Because of that, any long-term ‘portfolio’ of shares should hold very small positions in colts, and diversify across many of them. Because of the lower volatility, it should be ok to hold fewer, more concentrated positions of shares in fillies and mares.

3. Pay attention to the spread, or the difference between the best bid (offer to buy) and the best ask (offer to sell). Generally speaking, the spread will act as an additional transaction cost. If you want to make sure you get shares in a horse, you’ll have to make your offer to buy at a price equal to the best ask. But if you try to sell those same shares, even if the price hasn’t moved at all, you’ll need to sell them at the level of the best bid to guarantee yourself a buyer. So you’ll lose money on the round trip. That said, if you’re patient there are situations where the spread can work to your advantage. Instead of placing your buy orders at the best ask, place them just above the best bid, and wait to see if someone is willing to lower the asking price of their shares. And instead of placing your sell orders at the best bid, place them just below the best ask, and wait to see if an eager buyer is willing to pay what you’re asking. Until trading starts we really have no idea how wide the spreads on Blood-Ex are going to be. Most likely they’re going to be relatively narrow on well known horses which are actively traded. But for more obscure horses, spreads could be 15-20% of more. If that’s the case, there’s an opportunity to overcome the impact of the commission simply by placing buy orders on horses at a price just above the best big and placing sell orders on those you acquire just below the best ask. However, if you try this, you should probably not leave your orders open during the horse’s races, when the price (and perceived value) of the horse is likely to move sharply in one direction or the other.

4. Pay attention to how responsive the market is to news. Initially there may be relatively few participants, and prices may react slowly to news. This is likely to be most true for horses that aren’t stars. If you follow the news closely, you may be one of the first to learn of an injury or a particularly fast workout. Once the market has more participants, it’s likely that price moves in reaction to news will be so fast that there isn’t an opportunity for most to profit from it. One aspect of this that should be particularly interesting to see is how prices will move during the running of a race. As far as I can tell, trading in horses will not be frozen during their races, although it wouldn’t surprise me if this is done to avoid wild price swings.

Blood-Ex

I'll be writing a LOT more about this over the next few months, but wanted to post something to let everyone know about it right away. The world's first online bloodstock trading exchange is close to it's official launch. Blood-Ex will start with UK horses being listed, but plans to expand to the US and other countries later this year. Basically owners will put up shares in a horse's breeding rights for sale on the exchange, and traders will be able to buy and sell those shares. Each share is equal to 1/150th of 1% of the horses breeding rights. It comes with no obligations to pay for expenses, and no rights to any share of the horse's earnings on the track. Owner's who post shares to the exchange sign an agreement that they will eventually put the horse up for public auction in order to set a final value for the shares of the horse. For someone like myself who is fascinated both by horse racing and financial markets, this is exciting stuff, and I'll certainly be an active participant, and will post my thoughts on Blood-Ex here on an ongoing basis

ABE Examples

I thought it would be interesting to look at some examples of ABE’s for auction buyers. These are from the 1999 Fasig-Tipton Saratoga Selected Yearlings Sale. I only included the four buyers listed in the data who bought at least five horses at the sale. Several of these were acting as agents on behalf of other buyers. Keep in mind that the sample size’s here are microscopic. To draw meaningful conclusions from the data, you’d need to compile data across multiple sales. And even then, ABE suffers from the biases and problems I mentioned in the previous post.

Baden P. Chase- Horses:6, Total Cost:$1,190,000, Earnings:$279,759, ABE=0.24
Gatsas Thoroughbreds LLC- Horses:5, Total Cost:$282,000, Earnings:$364,099, ABE=1.29
John C. Oxley- Horses:5, Total Cost:$1,780,000, Earnings:$533,010, ABE=0.30
Todd A. Pletcher- Horses:7, Total Cost:$850,000, Earnings:$874,726, ABE=1.03

At some point I'd like to do a study across multiple sales looking at this data, and determining whether ABE is a 'repeatable skill' with predictive value...in other words, whether successful buyers are just getting lucky or not.

Auction Buyer Effectiveness

To the best of my knowledge, nobody has ever tried to measure the effectiveness of buyers at thoroughbred auctions and to compare the success of different buyers. Until now, that is. I'll be writing about some of my thoughts on how best to measure this over the next few weeks. I also plan to study whether buyer success is consistent over extended time periods. I'll start by introducing a very crude measure of buyer success. I'll call it 'Auction Buyer Effectiveness' (ABE). To calculate a buyer's ABE, simply add up the total earnings of the horses purchased, and divide by the total amount spent. As I said, this is a VERY crude measure, and it remains to be seen whether it provides any really valuable information. I believe we can come up with some better measures that address some of ABE's shortcomings, but for now will simply list a few of the problems with ABE:

1. Ignores variability in total expenses caused by different lengths of career.
2. Ignores impact of expenses of buying younger horses vs. older horses.
3. Like any measure based on earnings, may be skewed by state bred races.
4. Like any measure that uses a total or mean (rather than median) may be skewed by a single outstanding success.
5. Likely to be systematically higher for lower priced horses than higher priced horses.

If I have time, I'd like to take a look at whether even a crude measure like ABE shows correlation from one year to another within various price tiers.

Pinhooking

In the course of doing research, I'm finding that I put together data from a number of sources, since the 'keepers of the data' for the industry (BRIS and Equineline) choose not to make the raw data of their database available at any price. In the course of putting data together for studies, I'm finding a lot of data that's interesting, without being valuable enough to justify asking people to pay for it.

I did a study on horses sold at yearling auctions compared to those sold at two year old auctions. Because I selected data from sales in consecutive years, there are actually some horses that were sold as yearlings in 1999, and then sold again as two year olds in 2000. I thought it might be interesting to look at those:

Capeless, 1999: $140,000, 2000: $50,000, Earned: $35,750
Cat Tracks, 1999: $150,000, 2000: $250,000, Earned: $85,262
De Rose Colony, 1999:$100,000, 2000: $250,000, Earned: $58,200
Fax and Go, 1999: $40,000, 2000: $675,000, Earned: $0
Fistfite, 1999: $150,000, 2000: $250,000, Earned: $158,183
Illusionary, 1999: $300,000, 2000: $360,000, Earned: $176,274
Lady Katie, 1999: $65,000, 2000: $30,000, Earned $152,728
Lady Victoriate, 1999: $130,000, 2000: $90,000, Earned: $48,405
Let's Behave, 1999: $110,000, 2000: $285,000, Earned: $294,029
Perfect Stranger, 1999: $90,000, 2000: $150,000, Earned: $159,690
Red Carpet, 1999: $375,000, 2000: $825,000, Earned: $37,760
Songandaprayer, 1999: $470,000, 2000:$1,000,000, Earned: $380,480

What can we learn from this? Probably not that much. The sample size is just too small, and even with a larger group it's not clear what we could conclude, since many horses bought with the intention of being resold may not have been successfully sold. The two things that stand out here are that the originally buyers of Fax and Go did an unbelievable job reselling a horse who ultimately never earned a penny on the track, and that while Songandaprayer was resold for a huge profit, his later buyers obviously got their money's worth if they held him until his stud career took off.

Survivorship Bias

One of the things that I'd like to do with this blog is help create more 'educated consumers' of the vast amount of information that's available on the thoroughbred industry. An awful lot of that information is presented to bolster arguments that may or may not be supported by fact. Sometimes the information is intended to mislead, but more often the authors are simply unaware of some of the easier statistical errors to make when forming theories.

One of the easiest mistakes to make is the error of 'survivorship bias'. This is a well known phenomenon in the financial industry, where it can distort the appearance of past performance of indexes or funds. An example would be if a new ‘index’ is created which tracks the prices of a number of the largest companies in a specific industry. To show how the index would have performed historically, the historical price of the index is reported, going back many years. Invariably, these calculations show unrealistically strong performance. The problem is that by using the current largest firms, any firms that went out of business, or simply did poorly enough to shrink substantially were left out of the index. So the index by definition includes the firms that have done the best in the past. This doesn’t give any indication of how it might do in the future, or how it would have done if the largest firms from some time in the past had been used to create the index.

So what does this have to with the thoroughbred industry, and how would it impact research on racing or pedigree? Here’s an example, where I almost designed a study with the same flaw. I’ve mentioned before that I want to study what factors might predict potential future ‘breakout performance’ for claiming horses. I have access to a database of several thousand races of past performance data, and was thinking I could use that data for the study. I’d look at former claimers who had allowance or stakes wins, and identify some patterns in their history, and then use the past performance data to test the performance of all horses that exhibited the same patterns. The problem is that by using past performance data to test the patterns, I’d be automatically excluding all the horses that had the same pattern, but then didn’t make it back to the races, and I’d be reducing the impact on the overall data sample of horses that weren’t good enough to run often after exhibiting the pattern. The bias this would introduce to the data would have made my findings almost useless. It’s subtle problems like this in most existing research that lead me to believe that there’s a need for better research in the industry.

First Report Complete

If you don't want to read any shameless self promotion, feel free to skip this blog entry. That said, my first full research report is complete and available for purchase. It compares the racing results of horses purchased at two year old auctions (specifically Fasig-Tipton Selected Two Year Olds in Training at Calder) with the results of those purchased at yearling auctions (Fasig-Tipton Selected Yearlings at Saratoga). The results are a real eye-opener...if you're spending money at the wrong type of auction for the price range of horse you're buying, you're potentially wasting a lot of money, and could achieve much better results in the long run by limiting yourself to the right type of auction. I'll be putting some information about the study up on my business site ThoroughMetrics in the next few days, but for now, if you'd like to learn more about the study, send me an email at zelvin30@hotmail.com.

Sometime in the next few days, I'll start on my next study. I'm tentatively planning to look at which is a better predictor of success for the offspring of a mare...the mare's racing ability, or the success of her previous offspring. I haven't thought through all the details yet, but I picture the following high level steps:
1. Get a fairly random list of horses foaled in the same year.
2. For each horse, record some measure of their racing success (probably SSI), sire, sire's stud fee in year conceived, dam, dam's SSI, # of previous foals for dam, median SSI of dam's previous children, and mean SSI of dam's previous children.
3. Calculate correlations between horse's SSI and dam's SSI, horse's SSI and dam's median offspring SSI, and dam's mean offspring SSI.
4. Divide horses into tiers based on stallion's stud fee and dam's SSI/dam's median offspring SSI/dam's mean offspring SSI and compare horses' SSI in each category. This should control for the fact that dam's of one 'type' (good racer, good breeder) may tend to be sent to better stallions.

The thing that makes this interesting to me (other than the possibly utility of the results) is that there's a tradeoff here. The success of a dam's previous offspring is obviously a more direct measure of what we're looking for - the ability to pass on racing ability to children. However, it's also something that has VERY high variance. So many great racehorses have had half (and full) siblings who were duds on the track. The success of a mare during her racing career is a much more indirect measure of what we're looking for...but it's also much lower variance. Despite the equally small sample size, the results of a horse's races are much more consistent than the performance of their children.

If I had to guess, I'd expect the racing ability to have slightly more influence the the success of previous offspring. But the beauty of statistical research is that I don't have to guess...I can actually measure it.

I'd love to hear people's thoughts on the idea...is it a useful study? Are there any obvious flaws in the high level study design that I've described?

Unique Opportunity

I think there are two approaches that can be profitable in any type of speculation, whether it's as a thoroughbred owner, stock market investor, or gambler. My usual preference (and the approach my Thoroughmetrics business uses) is look for confirmed 'edges' that can be applied many times, leading to a virtual certainty of long term profit. This requires situations that come up often enough that statistically valid back-testing can be done to test theories, and that the edge can be applied frequently enough in the future to be worth something.

The other approach is to be opportunistic, and wait until a unique situation presents itself that has so many factors in its favor, that your experience and intuition tell you that you have an immense edge. These situations can't be statistically validated, because they occur so infrequently, and each one is relatively unique. The key to these is to be patient and impartial when evaluating them. You should also be able to articulate in clear and specific terms your reasons for thinking that they represent a profitable opportunity...otherwise it's likely that you're fooling yourself, and just WANT to find such an opportunity.

I believe that there's a chance that one of these opportunities may present itself in the Belmont Stakes this year.

Most people seem to expect Big Brown to go off as about a 1-5 favorite. I think most would agree that he actually deserves to be somewhere around a 1-2 favorite. He's beaten most of his opponents, won at a longer distance than any of them, he's unbeaten, run faster Beyers than his opponents, and run a much faster Ragozin than any of them. While there are several good horses in the race, none of them are proven stars yet. His strongest opponent (Casino Drive) has only raced twice, and won a Grade 2 race by 5 lengths with a Beyer of 101. He has never run longer than a mile and an eighths. Yes, he looked good in the Peter Pan, yes his pedigree suggests he should love the distance of the Belmont, and yes Bernardini had similar credentials before the 2006 Preakness...but I believe that fair odds on him would be around 5-1. His next strongest opponent (Denis of Cork) finished around 10 lengths back in the Kentucky Derby. Yes, he made a big comeback from last place in that race, but that running style tends to be overrated for the Belmont Stakes. He also probably deserves to be about 5-1 or 6-1. Assuming everyone elses odds should be substantially higher, I believe 1-2 is about right for Big Brown.

So is the opportunity I'm looking at to bet against Big Brown, since people expect him to be at far lower odds?

I don't think so. I believe there's a chance that he may go off at much higher odds than expected. Possibly even money or worse. Here's why:

1. The Japanese have showed up in large numbers for overseas races in the past, and bet their horses down to ridiculously low odds. I'm not sure that will happen for Casino Drive since he wasn't an established star in Japan, but if it does happen it could have a huge impact on the betting.

2. Many pedigree geeks seem to be overrating the influence of Casino Drive's pedigree. Yes, he almost certainly can handle the mile and a half distance of the Belmont, but there's not some mystical ability that Better Than Honour has passed on to her children to automatically win the race.

3. The ten previous failures since the last Triple Crown winner seem to have convinced a lot of people that it's become almost impossible. Instead of understanding that the near misses by horses like Real Quiet and Silver Charm showed that it CAN still be done, people are interpreting the bad luck and near misses as showing that it CAN'T, and some people will bet against Big Brown simply because of this.

4. Big Brown's quarter cracks are going to scare off some more potential support at the windows, despite the fact that many horses have run with worse, with little or no impact on their performance.

In summary, I think there's a real possibility that Big Brown could go off at around even money or slightly higher. And if so, I think it's a fabulous opportunity to back a likely winner at generous odds.

Searching For Hidden Gold

I’d like to get some suggestions from readers on a study I’ll be starting on in a few weeks. Basically I’ll be looking for factors that influence the likelihood of a claiming horse eventually competing successfully in allowance or stakes races. If you have any suggestions for factors to look at in the study, please let me know!

Some of the factors I’m already planning to study include:
1. Age – I assume a younger horse has a better chance of substantial improvement than an older horse. I’d like to measure just how much better.
2. Experience – I assume the fewer previous races a horse has had, the better chance that it has some hidden abilities. I might also try to factor in its experience on a variety of surfaces and tracks.
3. Sex – The sex of a horse could affect the likelihood of it being put into claimers despite having the potential for improvement.
4. Trainer – I’d actually prefer to look at owner/trainer combinations, except that I think most wouldn’t have large enough sample sizes to learn much of value. I’d guess that some trainers aren’t as good at recognizing a horse’s future potential than others, and that some may enter their horses more aggressively at the lower claiming levels in order to try to win races.
5. Breeding – I’d guess that the higher the sire’s stud fee, the more of a dud the owner and trainer have to consider the horse before they’ll enter them in mid and low level claiming races.

Anybody have any other suggestions?

Predicting the Past

I’ve mentioned before that I think most research done on the thoroughbred industry doesn’t use correct research methods and statistical analysis. One of the really common errors I’ve noticed is studies that don’t actually evaluate the predictive value of their findings. Theories are tailored to produce the best possible fit with what’s happened in the past, but it is just assumed that they will predict the future equally well.

There are at least three things we can do to avoid drawing false conclusions of this sort:

1. Apply common sense. While the basic premise of this blog and my Thoroughmetrics research business is that relying on common sense isn’t enough, that doesn’t mean we should do without it entirely. For an extreme example, no matter what the data tell us, we wouldn’t put any faith in a theory that say “dams whose names begin with the letter S outperform all other letters”. While that may accurately describe what has happened in the past, there’s no logical reason to expect that to help us accurately predict the future.
2. Look for ‘smooth’ patterns in the results. For example, before the Florida Derby, there was a lot of talk about how Big Brown’s outside post position would make it tougher for him to win the race. Horses in the outside positions in routes run at Gulfstream have very low winning percentages. Some people discounted this with the argument that ‘horses in the 4 slot also have a low winning percentage’. That’s nonsense. If 1, 2, 3, 5, 6, and 7 have solid winning percentages, while 4, 8, 9, 10, 11, and 12 have low winning percentages (with 11 and 12 having no winners prior to Big Brown), it’s a safe bet that post position 4’s low winning percentage is a fluke, since 2, 3, 5, and 6 do just fine. On the other hand, the outer post positions impose a real handicap on a horse’s chance of winning…none of them have yielded good results.
3. Test multiple, independent samples to confirm the results. For example, if you’re looking at a variable that you think will best identify the top sires, don’t just lump all the data together. Try looking at it for individual years, and seeing how consistent the results are from one year to the next. You may find that rankings that you thought had real predicative value, simple describe what has already happened, rather than predict what will happen. This is particularly true with variables that can be thrown off by a small number of aberrations. Because of Big Brown, Boundary is likely to be one of this year’s leading sires based on total earnings of offspring. I hope nobody will now consider Boundary a top sire (forgetting for the moment that he’s been pensioned already).

Inefficient Markets

Stock market experts have argued for the past four decades about whether the stock market is efficient or not. Those who argued for efficiency were basically saying that future performance of stocks was essentially random, because as soon as some mispricing or inefficiency offered a profit opportunity, someone, somewhere would take advantage of it, and in the process eliminate the opportunity for others. Those who argued against it pointed to the performance records of some of the world's greatest investors, compiled over many years, and how unlikely that level of performance would be if it was truly impossible to have an edge. More recently, both camps have generally adopted the more moderate (and correct) opinion that the stock is very (but not completely) efficient. Opportunities for outperforming the market as a whole do exist, but they're VERY hard to find, because you essentially need to be the only one to identify them to profit from them, and the patterns that work now will be discovered and eliminated, requiring constant adjustments to strategy.

Whenever I'm evaluating an opportunity for speculation (and I'd include both investing and handicapping in this category), I think about whether I'm operating in an efficient 'market', and what that means for me.

In my opinion, the reason the stock market is so efficient (and tough to 'beat') is that there are an unlimited number of participants (so there's a greater chance that somebody is going to find each profit opportunity) and participants can 'bet' as much as they want (so just one or two people finding an inefficiency can completely eliminate it).

The same situation exists with parimutuel betting. Anyone can participate (even more true with the advent of online betting), and they can bet as much as they want. This means that (like the stock market) you really need to be one of the very best handicappers to come out ahead. That's particularly true when you add in the fact that handicapping (unlike the stock market generally) is a negative sum game...overall the payouts are less than the total amount invested.

Ownership of horses does not necessarily operating as efficiently. While in theory, anybody can participate, not everyone does. At an auction, or when evaluating a horse to claim, you may really only be competing against a few hundred other people. If you're operating at the high end of the market, the number may be even lower. How many people really are involved in the bidding for Big Brown's stud future right now? 2? 3? Maybe 4? Only one of those has to overvalue him for IEAH to hit the jackpot. The less efficient a market is, the more important it is to know who you're competing against, and the more profit opportunities you'll find if you're good, but not the best at something. As an owner, you should be looking for opportunities like those found at a poker table or in a fantasy baseball league, where you're competing against the same 8 or 9 people over and over, and if you're the best, you're generally going to come out ahead in the long run.

This might mean playing the claiming game at a smaller track with less claiming stables, it might mean hording a top stallion's offspring because you know there's a buyer willing to consistenly overpay for them, or it might mean privately purchasing horses so that you're in situations where you're competing one on one with an unsophisticated seller.

Claiming Races

While the premise behind Thoroughmetrics is that the thoroughbred industry is short on really good statistical analysis and statistically valid research, that's not entirely true. The problem is that the vast majority of the quality research has focused on what handicappers need to know, not what owners and breeders need to know. A perfect example that occurred to me is what kind of research into trainer or trainer/owner performance in claiming races would be most useful. Handicappers are most interested in what happens while a horse is under the care of a trainer. What patterns does the trainer have success with? Do the horses the trainer claims improve while under his care? Owners on the other hand, should be more interested in what happens AFTER horses get claimed from a trainer or owner. Were they trained especially hard, causing ongoing health problems? Is a drop in class from that trainer (or owner) a warning sign that something is wrong and the stable is trying to unload the horse? If I can get my hands the right data (which, it appears, is going to be a recurring theme for me) I'd like to study whether there are stables whose horses should never be claimed, or should be avoided in certain situations (for example, a sharp drop in class coming off an ok performance).

Turning An Idea Into A Study - Part 2

Actually the other three issues I identified that need to be taken into account in designing my study looking at which sires produce sound offspring are relatively simple to address. As a reminder, here they are:

2. Was sample size sufficient to draw meaningful conclusions (the poster actually mentioned this one).

3. Were results being skewed by the fact that some horses in the data might not have finished their racing careers yet? If so, that would have more impact on the data for younger sires with fewer crops who had copleted their careers.

4. Were unraced horses included in the data? In most cases, the reason for not racing would presumably be lack of soundness.

Sample size can be addressed easily...just include a relatively full set of data. I'm mostly interested in sires that have been successful to some degree, so all of them will have had many children in each crop. I'll look at all of those children, unless data is hard to find for some (like those who raced overseas).

The issue of data being skewed by horses who haven't finished their careers yet will not affect the study as I'm approaching it, since we're looking at year by year results, rather than 'full career' result. Yet another advantage to doing it this way!

I definitely want the data to reflect horses that never raced. Other people have shown this data (% starters for the sire), but I think it's an important part of the data on soundness, and I'll include it too.

Turning An Idea Into A Study - Part 1

In my last post I talked about one approach to determining which sires tend to produce sound offspring, and some potential pitfalls to studying the the topic using that approach. Here I'll discuss some steps that can be taken to ensure that a study that uses this approach eliminates the first (and most complex) of the biases introduced by those issues:

Offspring of better sires will tend to retire earlier due to breeding value.

One thing to keep in mind is that this will affect males and females differently. The effect will be strongest among males, but only those with enough ability (or good enough breeding) to be sent to the breeding shed. In females, the effect will be weaker, but will affect virtually all of them. So we can't fully solve the problem by stuying just one sex or the other, but perhaps by limiting the study to males who are not ultimately used for stud duty. Another option would be to look at geldings only...but then we run into sample size issues...particularly for the top sires, where owners will be very reluctant to geld their sons.

One possible approach to eliminating this as a factor would be to look at average starts per year of racing, instead of average starts per career. If we did that, we'd need to make sure to control for age in some way though, since horses will tend to race more in the prime of their career than as two year olds. One bias in the data we should be aware of here is that cheaper horses will tend to race more, regardless of soundness. So the pampered children of star sires like AP Indy may come out looking less sound than they really are.

Next post, I'll start looking at the other three issues I brought up regarding this research topic.

Producing Sound Offspring

While I truly believe that statistical analysis can ultimtely answer many questions about what factors impact breeding and racing results, I also believe that most of the answeres are not as straightforward as they seem.

As an example, someone on one of the discussion boards I visit recently asked, "Which sires consistently produce sound offspring?" In response, someone else posted a list of some well known sires, along with the average numbers of starts for their offspring. As a general approach, this seems reasonable. And it's far better than going based purely on reputation. That said, none of the following potential sources of bias were addressed:

1. Were the offspring of better sires retired earlier because of breeding value, rather than unsoundness?

2. Was sample size sufficient to draw meaningful conclusions (the poster actually mentioned this one).

3. Were results being skewed by the fact that some horses in the data might not have finished their racing careers yet? If so, that would have more impact on the data for younger sires with fewer crops who had copleted their careers.

4. Were unraced horses included in the data? In most cases, the reason for not racing would presumably be lack of soundness.

In light of the common opinion that horses today are being bred for speed rather than soundness, the question of which sires pass on soundness is a good one. And the approach of looking at starts per offspring is valid. But a lot more than that goes into doing statistically valid research. Next post, I'll talk a little about how a study could be designed that would eliminate or reduce the impact of each of the four problems I discussed above.

ThoroughMetrics

One of the inspirations for starting ThoroughMetrics was my belief that well designed statistical research can help performance in almost any area of study. One of the most well known (and well studied) subjects is baseball. The statistical study of baseball (popularly known as sabremetrics) was popularized by Bill James, and became known more widely due to the popularity of Moneyball, which told the story of how a team with limited resources was able to use better statistical analysis as a competitive advantage. While many people have read Moneyball, and agree with it's conclusions, I think most people take too narrow a view of it's lessons. It isn't about specific strategies for building a competitive baseball team...it's about how objective analysis can provide an edge over those who stick to common sense, intuition, and statistically invalid, biased research. It's a lesson that can be applied profitably to anything where data can be collected and results depend on the accuracy of forecasts or predictions.

The ThoroughMetrics Blog

I'm starting a new business doing statistical research for the thoroughbred breeding and racing industry. I've got a lot of ideas for topics to study, and how things can be done better (and more profitably) than they're being done now. While I'll be charging a substantial amount for subscriptions to the in-depth research studies, I'm going to be using this blog to talk about some of my ideas for future research, and thoughts and observations about the industry. I think a lot of people will find the blog entries valuable in their own right, and those considering subscribing to my research service will get a feel for the type of work I'll be doing. Welcome to ThoroughMetrics!