August 27, 2007
The Great Decoupling
Like many people who are intrigued by the digital world, I was floored and deeply influenced by William Gibson‘s science fiction, which I first came across in Omni magazine in the late 1970s, when I was a deeply nerdy pre-teen. His 1984 novel Neuromancer popularized the term ’cyberspace,’ was prescient about how people would be using computers and networks 20 years later, and remains a fantastic read.
Early in the book, which is set in a vaguely specified near future, he tells the reader how to visualize the action in the novel’s Boston-Atlanta Metropolitan Axis:
"Program a map to display frequency of data exchange, every thousand megabytes a single pixel on a very large screen. Manhattan and Atlanta burn solid white. Then they start to pulse, the rate of traffic threatening to overload your simulation. Your map is about to go nova. Cool it down. Up your scale. Each pixel a million megabytes. At a hundred million megabytes per second, you begin to make out certain blocks in midtown Manhattan, outlines of hundred-year-old industrial parks ringing the old core of Atlanta..."
Gibson’s point, of course, is that in a market information flows follow the money. But what about within a hierarchy like a single company? What’s the principle there about where information flows?
Until recently there was also a pretty simple rule of thumb within hierarchies: information flows followed decision rights. A decision right is exactly what it sounds like: the power to make a call, settle on a course of action, arbitrate a dispute, etc. A Gap store manager has decision rights over whom to hire, but not where to open a new location; that decision right resides much higher up in the company.
Information flows have historically followed decision rights for two simple reasons: effective decision-making requires information (often a lot of it), and information has historically been expensive to gather and transmit. As a result, it made sense to be stingy when amassing and sending information, and to only send it where it would be put to best use -- as an aid to decision-making.
As MIT’s Tom Malone has been pointing out for a while now, most recently in his provocative book The Future of Work, one of these two reasons no longer holds. Information is now essentially free to transmit, thanks to the Internet’s pricing structure and to the vast increases in bandwidth brought by the laser and fiber optics. Processing, memory, and storage also continue to decline exponentially in price, making it cheap to store and process information as well. The net result of these increases in computing muscle is that most of us no longer need to be stingy about transmitting, holding, and analyzing information. Instead, we can be profligate.
In addition, it’s also becoming much cheaper to amass information. Enterprise IT collects masses of detailed data about the ongoing activities of a company: its transactions, events, activities, status changes, etc. It can be difficult, of course, to make sense or extract value out of this data, but it’s not difficult to acquire it over time.
Malone draws pictures like the ones below to show what happens over time as information costs go down (I’ll use the broader term ‘information costs’ to emphasize that it’s not just communication costs that decline sharply over time.). Centralization first increases as information collection and transmission become feasible for the first time, then decreases as costs do. Centralization is eventually replaced by lateralization:
But lateralization of what, precisely? The pictures above are probably highly accurate illustrations of how information flows within a hierarchy change over time as information costs plummet. If we could graph current information flows within IBM, Renault, Li & Fung, or The US Army I’d be astonished if they didn’t look like the rightmost picture above. And if we could do the same for the information flows of 10, 20, and 40 years ago I’d be equally astonished if they weren’t much more centralized.
Malone’s theory, however, goes much farther than just outlining how information flows change. It also predicts how decision right allocations will change as a result. His thesis is a simple and powerful one: decision rights will also become more lateralized as information costs plummet, leading to greater power and autonomy at lower levels within a hierarchy -- in short, greater decentralization. As he says in The Future of Work:
"There is ... [a] crucial factor that affects where decisions are made in business, and this factor is changing dramatically almost everywhere...
What is this factor?
It’s the cost of communication...
With new communication technologies… it is now becoming economically feasible -- for the first time in history -- to give huge numbers of workers the information they need to make more choices for themselves."
The strong form of this hypothesis is that companies will become less hierarchical and authoritarian, and more democratic and autonomous. Command-and-control approaches will become archaic. Pyramids will become pancakes. The fleetest, most innovative, and most competitive companies will be those that push decisions downward and empower the people closest to the action. The information gathering and filtering bureaucracies that most large companies have built up will become superfluous, and will be pruned. They’ll be replaced by networks of interdependent yet autonomous units that are given the decision rights necessary to pursue the company’s goals.
This is a powerful argument and there’s clearly a great deal of truth to it. Any of us could point to our favorite examples of technology-enabled decentralization and local empowerment, both across and within companies. Mine include eBay, Innocentive, open source communities, Cambrian House, and Rite Solutions.
But the strong form of the Future of Work hypothesis -- that decentralization of decision rights is a main result of vanishing information costs -- rests on the assumption that decision rights and information flows are inherently coupled. It’s easy to see where this assumption comes from. Because the two have been so tightly coupled up to now it’s reasonable to think that they will they will continue to be, and that big changes in information costs will ‘pull’ decision right along with them.
But the fundamental rule about where decision rights should go has nothing to do with information costs themselves. Instead, it has to do with knowledge. The ground rule is: align decision rights with relevant knowledge. In an economist’s formulation, relevant knowledge is the sum of general knowledge and specific knowledge. General knowledge is just what it sounds like -- knowledge that is widely known and easily transmitted. Specific knowledge is the opposite; it’s knowledge that is confined to one entity (a person, or perhaps a team or a lab) and hard to extract from that entity and send somewhere else.
General knowledge and specific knowledge are close in many ways to a sociologist’s conception of explicit and tacit knowledge, respectively. The reason tacit knowledge stays tacit isn’t just because people don’t want to share, or give away their comparative advantage; it’s because, as the philosopher Michael Polanyi elegantly summarized "We know more than we can tell."
Let’s say that a mortgage company realized that a few of its loan officers were just better at assessing credit risk than all the others. For whatever reasons (intelligence, experience, intuition, etc. ), they just had superior specific knowledge. In that situation, it would make good sense not to decentralize, but instead to centralize that decision right within the company, taking it away from the other loan officers. All the general knowledge (income statements, credit histories, etc.) would be sent to these few people, who would apply their specific knowledge to it and made decisions. In this example low information costs are still important; they allow all the general knowledge to be zipped to the few good officers. But the effect of low information costs isn’t decentralization and greater empowerment. Instead, it’s centralization of an important decision right and reduced autonomy for most loan officers.
Thought experiments like this one indicate to me that the net result of disappearing information costs won’t necessarily be decentralization. It will instead be the decoupling of information flows and decision rights. Organization designers will be able to allocate decision rights without worrying about how costly it will be to get required information to deciders. Leaders will be able to ask "Who should make this decision?" without adding "Keeping in mind that it’s going to be slow, difficult, and expensive to get them the general knowledge they’ll need."
Will this work always, or even usually, lead to more decentralized organizations? I find myself less confident than Malone that this will be the case. I agree with him that we’re at a very interesting point in the history of technology and the economics of information, but I’d label it a great decoupling (of information flow and decision rights) rather than a broad decentralization (as decision rights lateralize along with information flows).
I’m also less confident of a single broad trend toward decentralization because of another fundamental aspect of today’s information technologies: they often serve to change the mix of general and specific knowledge. For example, the mortgage industry has in fact moved to greatly centralize decision rights about whether or not to extend a loan, but in a way that doesn’t quite follow the example given above. Instead of giving this decision to a small group of people, it’s been largely given over to computers.
After a lot of analysis, the formerly somewhat intuitive and qualitative decision about whether to extend a loan has become a quantitative one. For individuals, the single-number FICO score does a pretty good job of capturing the likelihood of defaulting. In this case, the knowledge required to answer the question "Will this person repay their loan?" is no longer specific; it’s become general. Computers and the people who program them have become quite good in several domains at converting specific knowledge into general knowledge. With enough history, data, and intellectual energy many formerly intuitive tasks have been codified to the point that they can be turned over to computers for superior performance. Humans, for example, are probably no longer the world’s best chess players.
They’re still the world’s best radiologists, however. It’s not the case that specific knowledge is headed for extinction. Our brains are powerful and highly specialized computers, and they’re just better at some things than the silicon kind are. In their wonderful book The New Division of Labor: How Computers are Creating the Next Job Market Frank Levy and Richard Murnane explain that we humans excel at pattern recognition, case-based reasoning, and other similar skills that they group together under the label "Expert thinking." Expert thinking results from specific knowledge, and can’t be mimicked with general knowledge alone.
As the examples of chess and mortgage scoring show, today’s expert thinking can become tomorrow’s computer program. Some kinds of expert thinking, in other words, can be codified to a very high degree or substituted for by brute force computational power (As radiology, driving in traffic, and playing the Asian board game Go demonstrate, however, many other human skills appear safe, at least for the time being. ). When this happens, it makes sense to take related decisions out of people’s hands, and give them to computers. This is a type of centralization.
However, just as computer programmers work diligently to convert specific knowledge into general knowledge, the rest of humanity works to do the opposite. We build up our specific knowledge in large part by assimilating lots of information, some of which comes from information technologies. The food service company SYSCO used business intelligence software to help identify which of its current customers were most likely to defect, but it was still the sales rep’s call how to best approach these customers and get them to stay. I find this a very interesting case study: even though the BI software converted some formerly specific knowledge ("What makes a customer likely to defect?" ) to general knowledge, related decision rights did not move away from the sales rep because SYSCO believed that the rep’s specific knowledge about the customer was still critical.
One last point: with all of this information and information technology piling up, it’s sometimes hard to tell where the relevant knowledge lies, and how much of it is specific vs. general. Most large apparel retailers, for example, have highly centralized their sales forecasting activities. Small teams at headquarters determine what’s going to sell 12-24 months in advance. They rely on their intuition, and often on forecasting algorithms that predict demand based on previous sales. The Spanish clothing company Zara, meanwhile, has radically decentralized the work of predicting sales. Zara asks its store managers what garments will sell in their locations over the next couple weeks, then gets those clothes to them in a couple days. Headquarters also regularly asks store managers what trends they’re noticing and designs new clothes throughout the year to capitalize on them. Zara’s leaders believe that for the stylish clothes the company sells the relevant knowledge is almost all specific knowledge, and that it resides in the heads of its store managers around the world. Zara’s recent performance suggests that they’re right about this.
Most of what I’ve seen recently strongly indicates that the sudden near-disappearance of information costs is bringing up a fascinating and consequential set of questions for organization designers and corporate leaders. They now have the freedom to place decision rights where they wish without being hampered by information costs. What are the long-term consequences of this great decoupling? Rather than a steady rise in decentralization, I think we’re going to see an extended period of innovation and experimentation. I think Malone might well be right that the "market share [of centralized management] is likely to decrease," but I also think there will be strong movement in the opposite direction -- toward more centralization of some decision rights—and a lot of very interesting hybrid models, some so interesting that they’ll look like science fiction.
What are you seeing? How are decision rights migrating within your organization, and what role is information technology playing? Is IT central or peripheral to the trends you’re observing?
Great artickle. Thanks
The rag trade is notorious for experiencing forecasting difficulties but I am surprised that Zara is collecting weekly data in this manner. I would have thought it could do much better by collecting SKU data via the EPOS system, augmented by local data that it could then crunch through an analytics engine.
Excellent post, Andrew!
It really got me to thinking about how the concepts you’ve discussed impact the implementation of a Trust Fabric for the Social Graph in Web 2.0 for Business vs Social Web 2.0.
See my blog post for more:
http://smoothspan.wordpress.com/2007/08/28/business-web-20-demands-a-different-trust-fabric-than-social-web-20/
I agree with the analysis that organizational models may change due to declining information/communication costs (and increasing information/communication capabilities) but that it is hard to say if there will be decentralization per se. Organizational structures are complex and involve people. If one takes a look at computer networks, the decline of information/communication costs has stimulated the development of decentralized (P2P) systems but there is still a lot of debate about these systems. See for example the discussion on the BBC iPlayer.
Hi Andrew.
Lets divide data, information, knowledge and intelligence.
A lot of data is passing through the network, some of that becomes information. With reasoning this info can be converted to knowledge. The use of the knowledge is intelligence.
Business Intelligence is the present of decision making. With the proper tools much of the info can be analyzed to even convert in business rules.
I remember the time when the weather forecast was just for the next hours. With computer power the time is now 10 days, we can reasonable predict.
What no many people realize is that the computer tools and the computer muscle is not enough to solve the problem. We need a solid theory behind. For most markets we do not have these theories. So we are “try-and-error.”
The problem with this is sometimes we find a pattern that perfectly works for a period of time, but we do not why. So when the market begin to change the “try-and-error” also begins.
Is BI the ultimate decision making process? No. It is the present. For the time-being BI is the best we have to take advantage.
Mario Ruiz
@ http://www.oursheet.com
Fantastic post andrew, you really have helped put a different perspective on this complex topic and have provided a valuable resource for a blog post I will be doing soon but I would certainly say that IT is central and core to everything in life now let alone businesses.
Interesting article.I’ll have to relook into these aspects as of now for my organization and then post about it later.
Great article Andrew!
My company, JackBe (http://www.jackbe.com), provides an enterprise mashup (http://blogs.jackbe.com/2007/07/defining-mashups.html) platform and as such is on the cutting edge of providing enterprises with necessary technology for information exchange and collaboration. We have personally experienced the ‘hybrid’ model that you describe and we too believe that it is required for enterprises to exchange information while ensuring an environment of trust.
In essence we are seeing that the exchange of information and migration of rights (that Malone describes very well), as well as the technology needed to facilitate this exchange, can occur for consumers and enterprises alike but enterprises have very different needs in their exchanges (http://blogs.jackbe.com/2007/08/difference-between-consumer-and.html). Our experience is that enterprises have to ensure a level of security, governance and reliability that is both tough enough to meet their risk criteria and still balanced enough can it be implemented and maintained.
In short, no enterprise will formally allow nor motivate information exchange without governance. And hence the migration cannot occur.
very impresive, thank you
Rule engine technolgy is helping organizations to redefine decision rights. Actually, my daily job is to develop decision services using software such as Blaze Advisor and JRules etc. A centralized business rule repository typically takes away decision rights from individuals. This centralization helps organization to better manage core decisions and also makes improvment easier.
See more in Tom Davenport’s blog: http://discussionleader.hbsp.com/davenport/2007/07/automated_decisions_sneak_up_1.html
and James Taylor’s blog: http://www.edmblog.com/
Great post. Thanks!
I blogged a response here.
JT
The EDM blog
Author of Smart (Enough) Systems
Great post! Very insightful, it’s like free college classes!
Hi Andrew,
Very nice read ... The way I see it, this is set to change the very nature of the organization. I have written about this at http://atulrai1.blogspot.com/2007/09/how-knowledge-is-changing-organizations.html though I must say that I was not really able to put my finger on decoupling, which is the concept you have elaborated quite well on.
Thanks, Atul.
I was interested to read about Zara getting their forecasts from store managers. While I’ll grant that forecasts for the next couple of weeks are better than forecasts for next year, forecasts are always wrong. That’s just the way they are. If Zara can really get clothes to their stores quickly, they should use the EPOS data, as Dennis Howlett suggests. (Note: most apparel manufacturers cannot respond to demand changes quickly due to ingrained months-long lead times through the entire supply chain. If Zara has broken this problem, they are a long way toward replenishing actual consumption.)
What is interesting about relying on store manager input is the other part of the paragraph above. Get input from store managers on fashion trends that they are seeing: what things are people wearing that they get from other stores? what things are people putting together that Zara could capitalize upon. These are difficult-to-discover trends that will be more useful than expected demand.
Dear Andrew,
Thanks for an insightful analysis on organizational governance. How does human need (for control and power) affect such a decentralized decisioning process? What are the impedements and a strategy to overcome the same?
Regards,
Kishor.
Hi Andrew,
An engaging post full of fun stuff to think about!
I’ve been working in this area within a large, complex company for some years now. I recently commissioned Tim Lewens at Cambridge University to write a view of knowledge management from a philosopher’s perspective because Polanyi’s work has been mis-quoted so often in the knowledge management press; I like your split of general and specific knowledge as a way of thinking about it.
Some colleagues and I are looking at how current and emerging organisational and decision making concepts from the military could be applied to help complex companies devolve decision making, following work that two other colleagues had published last year on Mission Command. We’ve seen how this can be applied to multidisciplinary teams to good effect, enabling the kind of devolved decision-making that you’ve outlined, but it requires business leaders to understand their role in decision making as well. At some point decision-makers aren’t those with the most information, but those with the right thinking style and experience to ask for the information they really need, and enough confidence in the people providing it and doing the work that their decisions will be carried out to allow them to do so without interference.
I think the decoupling that’s occured is that access to information is no longer a barrier - the ability to assimilate it in a way that can produce new insights has become far more important. One can think of this as specific knowledge, unique understanding, experience or whatever else one might like to call it, but work that Kathy Hagen did in 1997 with me showed that understanding how people think about information and situations is as important as what they know at any particular time - I think this is also the premise of de Bono’s book Tactics.
Thanks,
Chris.
This guy is too smart for his own good! I appreciate all the work you put into your articles.
Hello Andrew, Excellent article!
I agree with you in that some loan officers and mortgage underwriters are better at assessing credit risk. However, after building out a model that other less experienced loan officers can learn from, we are building our bench strength by teaching others how to accurately assess credit risk.
Centralization of mortgage underwriting activites does make sense with low volumes of information, but for an organization to survive and thrive, we need to operate in a decentralized fashion. Too much reliance on a few key players will ultimately become a bottle neck and impede the flow of information.
Jason
LoneStarFinancing - Home Loans
Good read, I wish many would understand and accept like you do.
Justin Reed.
All In One Forum
Stumbled onto this post, but I found it quite interesting. Enough has been said about Malcolm Gladwell’s books, but there’s actually an interesting section of ‘blink’ (by gladwell), which speaks to the fact that less information can actually prove more effective than more information in predicting heart attacks - if you can identify the handfull of truly relevent indicators… Unfortunately this defies the common man’s thirst to know absolutely everything and make a decision on that pile of info!
Hey Andrew, Your articles always help me in every aspect. Thanks for sharing your thoughts.
Page 1 of 1 pages
