The ‘knowledge problem’ is an expression that is today associated with Friedrich Hayek, although the labelling seems to have first appeared in the work of Israel Kirzner in the 1980s, when writing about Hayek, i.e. not by Hayek himself. It refers to the proposition that conceived as a single entity, the economy contains a huge set of economically relevant information whose elements are dispersed among a large collection of human agents and cannot feasibly be centralised. An immediate implication is that any centralized decision makers – whether central planners or a national government – will, in practice, work with a vastly smaller set of information than that contained within the economy as a whole. And that observation has profound implications for the roles to be afforded to the state in economic matters.
The idea itself, in an unlabelled form, has existed since the dawn of political economy and appears in varying guises during the ‘classical period’ in the work of a number of prominent thinkers of that time, of whom Adam Smith was the first.
Smith’s own, first thoughts can be glimpsed in his Theory of Moral Sentiments’ (TMS) and his Lectures on Jurisprudence at Glasgow University. TMS, for example, contains the much cited chessboard analogy when characterising someone he called a ‘man of system’, meaning a would be ‘planner’ of the economic system.
“He seems to imagine that he can arrange the different members of a great society with as much ease as the hand arranges the different pieces upon a chess-board. He does not consider that in the great chess-board of human society, every single piece has a principle of motion of its own, altogether different from that which the legislature might choose to impress upon it.”
The operative words here are “every single piece”. Knowledge about the individual ‘principles of motion’ is dispersed and, whilst the ‘man of system’ should advisedly recognise their existence, it would be infeasible for him to know what they were in each, individual case.
Thus, in writing in the WoN about the effects of his suggested dialling down on the granting of special privileges to particular interest groups in society and on the tendency to impose restrictions of trade, Smith says (of a limiting case):
“All systems either of preference or of restraint, therefore, being thus completely taken away, the obvious and simple system of natural liberty establishes itself of its own accord. … The sovereign is completely discharged from a duty, in the attempting to perform which he must always be exposed to innumerable delusions, and for the proper performance of which no human wisdom or knowledge could ever be sufficient; the duty of superintending the industry of private people, and of directing it towards the employments most suitable to the interest of the society.”
Elsewhere in the book, Smith makes it clear that, faced with practical political and social realities, the prospect that all systems of preference and constraint could be “completely taken away” is utopian: the forces ranged against it were just too powerful to be completely vanquished. Nevertheless, the imagined System of Natural Liberty serves as a benchmark counterfactual that can serve to guide governments toward more productive paths.
Nor should this conclusionary position be read as a critique of the intellectual qualities of the decision makers themselves. The knowledge problem stems from the huge disparity in the sizes and natures of two distinct information sets – that of the decision makers and that of society or the economy as a whole. As J.S. Mill put it in his Principles of Political Economy, published around the end of the classical period in 1848:
“It must be remembered, besides, that even if a government were superior in intelligence and knowledge to any single individual in the nation, it must be inferior to all the individuals of the nation taken together.”
Following Mill, in the later 19th century and through the 20th century, interest in the (still unlabelled) knowledge problem was mostly confined to scholars of the Austrian School of Economics: first Karl Menger, then Ludwig von Mises, then Hayek. These were the chief critics of centralized planning in the period when socialist and communist doctrines shifted from being the imaginings of intellectuals to become policy experiments, but the critiques themselves were conducted at a high level of theoretical abstraction and some of the Smithian legacy, empirical and grounded in everyday observation, was lost.
The legacy is still available though, and with modern information technology is easier to access and explore than ever, containing insights that are as relevant today as they were in the 18th century.
As explained in an earlier blog in this series (Adam Smith’s ‘Theory of Productivity Growth’ – Regulatory Policy Institute), the WoN opens with a statement connecting productivity growth, the division of labour and the application of human capital, from which is developed a theory of growth in living standards (the ‘wealth’ of nations). Within this framework, it is easy to see how sets of economically valuable information develop in fragmentary ways. Learning (the accumulation of new knowledge/information) derives chiefly from experience and, as the division of labour deepens, the work experiences of different members of society diverge according to tasks performed. Further, within the increasingly numerous specialisations, each individual will have her/his own, particular history of experiences. Thus, we are led to the highly differentiated information sets that characterise developed economies, the vast majority of whose content will be beyond the knowledge of centralised decision makers. Put simply, the division of labour stimulated a division of knowledge/information.
Given this, it may appear to be odd that much economics teaching refers to ‘asymmetric information’ as a source of ‘market failure’ when, in the Smithian ‘gestalt’ it can be seen as a correlate of economic success. The teaching is misdirected: the big, underlying ‘knowledge problem’ lies not with the differences in information sets per se, but rather with the failure of decision-makers and analysts to recognise the severity of the limits to their own knowledge.
So, contemplation of the ‘knowledge problem’ is worthwhile, because it can both serve as an antidote to hubris and, perhaps of more practical assistance, act as a steer towards how successful policy design and development might, from the very outset, actively consider and explore constraints to effective action imposed by information limitations.