Well, as my first conference on Prediction Markets (Yahoo Confab Dec 2006) this one left me a little puzzled, even with James Suroweicki and Robin Hanson there in person. So here are my thoughts, questions, and reactions to what was presented, talked about, questions asked and examples thrown out to us. (These comments were first posted at Arik Johnson's ReconG2)
What seemed a way to learn about the concept and applications, successes and failures, ended up being a way for Google, Yahoo and other vendors to tout a process they just happened to have new tools for. The 4 hours seemed to be for the "true believers", and really glossed over PM 101, even though presentations by both Surowiecki and Hanson were meant to address the basics of the concept.
Reps from Microsoft and HP really had the most valuable lesson for me - they have been testing prediction markets in various forms and for various business functions, but neither are using them to make real business decisions at the group, department, business unit or corporate level. Yes, the markets can be very accurate, more so in many cases than internal and external "experts" (PM's are cheaper, too!), but no, they aren't taking over from the more entrenched ways of forecasting and decision making.
I heard a lot about how to get people to trade in the market, pros and cons of real money vs. other forms of "payoffs" (social standing and reputations), how to address the issues of risk attitudes and "quality" of trading over time (people getting it right, or close to it).
One really useful set of thoughts came from Adam Siegal of Inklingmarkets.com ( See them here) - lessons learned as to why PM's will fail within any group:
- No concept of what a PM is
- Using software that is too complex and not easy to use, to set up, trade in and run a market
- Market structure was wrong
- Asking for an opinion - questions must be quantifiable, even if just yes or no
- Description or rules of the market not clear/understood by the trading populous
- Questions posed are biased
- Time frame of the market is too long - apathy over time and the market becomes static
- No information is available for the traders to make a reasonable judgement - trades are based on information and knowledge (duh!)
Now this last one really struck me - I thought the whole idea in a PM was to bring out hidden tacit knowledge in an organization in a way to focus it on an issue or challenge to meet a specific business need. It would seem that there is a issue here of simply bad selections of the trading group members. I would also say there is a real issue on selected traders vs. self selected traders?
In this same vein, someone asked about the issues of "insider trading"! Wait, I thought, don't you want "insiders" who have the information and knowledge necessary to make informed trades? Of what use is the market without them?
It would seem that the ideas of an informed market, an efficient market, a rational market, etc., as requirements within a PM keep a lot of these folks up at night. Well, as Steve Barth and I have written about, people don't act rationally and unemotionally within any market.
I noticed that in the research presented on the accuracy of PM's, the was a lot of assumed cause and effect, with no seeming basis in fact other than events happened to have occurred at the same time or one before the other.
Leslie Fine of HP noted that perhaps the best use of PM's was as a vehicle to bring up the elephant in the room no one to talk about. For one thing, HP is using PM's to try and get knowledge out of their sales force for better decision making - always a challenge anywhere!
Todd Proebsting of MSoft noted that people who do trade love to move the price, and to move it fast. More importantly, he said, "...inside an organization, if the market activity does not result in actual business value, well, it's a great theoretical exercise, but..."?
While there are tons of public PM's out there, no one at this conference could tell me that they knew of any company actually using PM's as a major sensing, sense making, or decision making tool - I have to look a little deeper and see if that's the case...surely as hyped as this topic is, and as much buzz going around, someone is using it successfully within a business (and I mean as part and parcel of the sensing, sense making and decision making process)...If you know of one or two, let me know, OK?
In retrospect, the same old issues of how to get people to volunteer their tacit knowledge, and how to contextualize it around the things that are important to an organization rears it's head. As a tool to do this, PM's have some very good attractors (but so do other tools). BUT, you have to understand the failures and learnings to really make them work as they are capable of..it seems PM's have more reasons they can fail as do other tools and processes.
I am sure that these things are a really valuable way of soliciting tacit knowledge, and generating more accurate predictions, however, at the end of the day, no matter how cool these things are, how much more accurate they are - if the leaders within an organization won't use them...?
Well, I am off to the research desk and books to learn more...
Comments