Recent Articles
Categories
Archives
Technorati
|
April 20, 2008
A Case for Prediction Markets
The case on Google’s internal corporate prediction market that I wrote with Peter Coles and Karim Lakhani is now available for wide distribution (a teaching note for this case is also available to faculty). The case’s introduction explains what prediction markets are, and why they might be interesting to business leaders:
"Prediction markets were very much like stock markets. They contained securities, each of which had a price. People used the market to trade with one another by buying and selling these securities. Because traders had differing beliefs about what the securities were worth, and because events occurred over time that altered these beliefs, the prices of securities varied over time.
In a stock market like the New York Stock Exchange the securities being traded were shares in companies, the prices of which reflected beliefs about the value of the companies. In a prediction market, in contrast, the securities being traded were related to future events such as an American presidential election. In this case, the market could be designed so that each security was linked to a candidate, and its price was the same as the estimated probability that the candidate would win, according to the market’s traders.
Prediction markets on the Internet had proved to be remarkably accurate at predicting the results of political elections and other events, and the Googlers had wanted to see if they could also be productively used within companies to forecast events of interest such as the launch date of a product or whether a competitor would take a specific action. The experiences of the previous seven quarters had shown that Google Prediction Markets (GPM) were in fact quite good at predicting such events. Googlers put none of their own money at risk when they traded within GPM; instead, they bought and sold securities within GPM using “Goobles,” an artificial currency."
I’m going to teach this case on Tuesday in my MBA course, and am really looking forward to it. It’s one of my favorite classes of the semester, and will be made even better by the fact that Bo Cowgill, the Googler who initiated prediction markets within the company, will come to Boston to share his insights with my class (and also with Tom Malone‘s at MIT).
Cowgill has written a paper with Justin Wolfers and Eric Zitzewitz analyzing data from Google’s markets, and Wolfers and Zitzewitz also wrote a more general overview of prediction markets. The Wikipedia article on the topic is another good resource. Prediction markets on the Web include the Iowa Electronic Markets, InTrade, NewsFutures, and the Hollywood Stock Exchange.
Our case concentrates on two issues: how to encourage more trades and more liquidity within a corporate prediction market like Google’s, and how business leaders can and should use the information provided by the market.
After writing the case, teaching it a few times, and spending some time understanding the mechanics and utility of prediction markets, I share the puzzlement articulated by James Surowiecki in his book The Wisdom of Crowds:
". . . the most mystifying thing about [prediction] markets is how little interest corporate America has shown in them. Corporate strategy is all about collecting information from many different sources, evaluating the probabilities of potential outcomes, and making decisions in the face of an uncertain future. These are tasks for which [prediction] markets are tailor-made. Yet companies have remained, for the most part, indifferent to this source of potentially excellent information, and have been surprisingly unwilling to improve their decision making by tapping into the collective wisdom of their employees."
Why is this? It’s not because the technology is hard to acquire: Inkling Markets, Xpree, and Consensus Point, among others, will happily provide a company with Web-based prediction market software. So what is the real stumbling block? Is it that companies don’t really want the most accurate information about future events to come out and be widely known?
Leave a comment and let us know what you think, or what your experience has been. I’ll post more on this topic after our class on Tuesday.
I have come to conclusion that availability and quality of management tools greatly exceed ability of corporate managers to use them productively. Many corporate executives, I have worked with, are much more comfortable with conceptual reasoning and analysis of historic data, than use quantitative methods for predicting future outcome of their decisions. Perhaps it has something to do with a fear of accountability, but they seem to treat forecasting as a matter of “art” rather than “science”.
Hi Professor - I have hit many roadblocks at my company, which actually you and I spoke about a few months ago. Here’s an overview of what I have encountered, in no particular order…
1) The myriad usage scenarios within an enterprise actually makes it harder to get the investment necessary to try prediction markets. Why? There is reluctance to take the risk yourself when there are benefits for the enterprise as a whole. In other words, if it has that much potential, someone higher up the food chain should fund it. This isn’t merely a funding issue, however. It’s also a measurement issue. Very few people are given a target metric for anything that doesn’t fit neatly into the balance sheet, with managing quarterly expenses at (or near) the top of the list. Naturally this leads to the opinion that if it’s not being measured, it’s not a priority. We acknowledge this needs to change, but gaining agreement on what those new metrics should be is a big hurdle unto itself.
My point here is that, at least initially, it likely would be easier to find a sponsor if the business case was more targeted, which is what I’m trying to do now.
2) Despite all the press regarding how accurate prediction markets are, there is a great deal of skepticism regarding their accuracy for our company, or where specifically they would be most accurate. If we pilot the technology in the wrong area, or otherwise don’t set up the markets properly, the whole thing could get the hook quickly.
3) The ROI is not immediate, nor is it necessarily quantifiable as part of the initial business case. We can cite how terrible most companies are at forecasting or decision-making in various areas, but we really need our own ‘forecasted vs. actuals’ historical data as a starting point. Gathering that data from various business units is quite a chore to say the least. We also need to run traditional methodologies and prediction markets concurrently to see how the results match up.
4) Despite the ROI extending beyond forecast accuracy to areas such as social networking, knowledge management and information flow, these benefits are less tangible. I believe this will change over time, but most of the companies that have even deployed social software still are not doing real social network analysis...they’re just trying to increase collaboration and team productivity. Their ability to gain insight from the underlying data is far down the road, when they’ll wake up one morning and realize that Enterprise 2.0 and Business Intelligence have converged (with ROI more readily apparent).
5) Social software adoption is already an issue, so there is concern regarding the participation level necessary to make the markets as accurate as possible. Aside from the usual inhibitors to the adoption of new tools, the concepts of online reputation, digital identity, virtual currency and overall openness are novel to most employees. There is reluctance to contribute in any form, via any medium, without maintaining anonymity.
6) Assuming the CIO and/or IT Dept makes all funding decisions for internal technology deployments, prediction markets would have to show greater business value than the approved/funded projects currently in plan. So this gets back to the ROI issue (and points to the potential scenario of using prediction markets as part of that same IT funding/planning process).
7) Traditional methodologies for forecasting and decision-making are well within the comfort zone of most executives. Even when they are willing to try other methods, those experiments are very controlled and merely supplement the formal methodology. It is believed that instinct plays factor as well, but not just anyone’s instinct (that’s crazy talk).
Also, there is limited quantitative data on prediction market accuracy for longer term events. So any businesses which try to produce mid-term to long-term forecasts would take a higher risk, have to compensate for market liquidity issues, etc. Still a worthwhile exercise, however, in my opinion.
8) Prediction market results which conflict with or undermine project status reporting, for example, would potentially give managers and executives less decision-making power and less credibility. If managers administered the market themselves, and made the necessary adjustments early enough, I think this concern would be mitigated. Today, however, self-preservation is a big factor, often at the expense of the larger organization...and credibility could go out the window anyway.
9) If the enterprise happens to be an E2.0 software vendor ("I have this ‘friend’...") then there is the build vs. buy vs. partner issue, strategic fit, line-of-business ownership, go-to-market strategy, etc.
Despite these and other obstacles (like that pesky day job), I’m continuing to promote prediction markets all over the corporation, and have built up quite a bit of interest. I’m not stopping until I find an executive sponsor or receive a rational explanation as to why we will not pursue the topic further.
Frankly, I don’t think a rational explanation exists. Then again, corporate decisions aren’t always rational, which brings us back full circle.
Happy to discuss further if you’d like…
BP
(Go Sox!)
I think the wisdom of crowds concept is great, and I think prediction markets make sense for external events, but I wonder about conflicts of interest on internal event betting. If you’re betting against a project being on time (or betting for it to come in at a later date), don’t you have an incentive to “slow roll” any requests that come your way from that project? Or worse if you’re actually a member of that project?
And think of the political games that could get played if someone wanted to make another team look like it was failing by betting heavily against their metrics? Panicked execs saying “Well, the market says your project is going to be a train wreck...”
Professional sports has dealt with this problem for a very long time. It’s obviously a big no-no to have players betting on - or especially against - their own team.
Not that I have any deep rooted concerns on this topic, but I thought I might play devil’s advocate.
“Is it that companies don’t really want the most accurate information about future events to come out and be widely known?”
Why I think companies may be right to be cool towards predictive markets:
1st: An information market at a company is probably not efficient, (I know that the article claims that the Google market is “reasonable efficient”, but so was Thailand’s currency market before George Soros) and at best, the predictive market at a company is the weak form of market efficiency where the current price is reflective of the historical series of events. Way too much room for manipulation in a market like this.
2nd: Time value of money: The instances when having an accurate prediction is worthwhile is when the stock value is at “$1” not at “$80”. Sure, I can very accurately predict when a movie ends when the credits roll, but I am a poor predictor when I am watching the trailers. The problem is that businesses need accurate predictions during the trailers, not during the credits.
3: Relationship of the outcome significance to the prediction market performance:
It is one thing to predict the if a new product will meet sales goals in yr 1, it is quite another to predict the success of breakthrough products like the ipod. Why my concern? I look to physics. The biggest shakeup in physics over the last 300 years was the theory of relativity. Problem, when Einstein developed the theory, scientists were almost universal in predicting the theory to be wrong/incorrect as compared to Newtonian physics. Well, they were wrong. Granted, this point doesn’t dismiss predictive markets, but it does make me wonder how effective they really will be?
Having mention the above, if predictive markets do work, I certainly can foresee a time when companies, as a work performance measure, provide their employees with equal amounts of Goobles and have their employee’s bet away. The most successful will be evaluated and cultured for positions in strategic thinking roles within the firm. Interesting
Oh, by the way, both Sox sux. Go Cubs!
As noted in a recent article in Inside Knowledge Magazine, the number of companies that have implemented an internal prediction market is modest (lower bound was in the 30s as of 2006, although growing rapidly). Feedback from companies utilizing prediction markets has identified critical deployment considerations, including trader knowledge, question/security design, and trader participation however the limited trials and low adoption rate of prediction markets indicates that resistance to their use pre-dates these deployment considerations.
Tom Davenport, a Professor as Babson College, has identified traditional hierarchical organizations as a major obstacle to the use of prediction markets (as well as trader participation once deployed). At BitInsight (http://www.bitinsight.com), a consulting firm utilizing prediction markets as part of a decision support service, our experience would concur with this organizational resistance. The resistance ranges from disbelief in the process, concerns about data leakage, concerns about the impact on current business processes, and concerns that the process will “undermine management” (to use the words of Microsoft in their presentation on their prediction market, PredictionPoint at the recent Conference on Corporate Applications of Prediction/Information Markets).
By definition prediction markets are highly visible and if not carefully managed can be controversial, note the furor over DARPA’s terrorism futures market. This visibility adds to the personal risk from failure and precludes “skunk work” executions in most cases. The (ideally for many questions) cross organizational nature of the trading pool also adds organizational obstacles to market introduction.
Prediction markets are the data mining tool to access the unstructured data stored in the enterprise’s distributed human capital (broadly defined). It provides another source of data to be considered and leveraged; it does not, per se, make or even recommend any decisions to the corporation. Defining the tool as such may make it more palatable to senior management whose buy-in and support are absolutely critical to successful trial and on-going incorporation of this technique within the enterprise’s business processes.
Several corporate executives, I have worked with, are much more comfortable with conceptual reasoning and analysis of historic data, than use quantitative methods for predicting future result of their decisions
Page 1 of 1 pages
|
|
