4.1The claim
Chapter VII of Knight's book does not stop at saying that some cases cannot be classed. It goes on to say that the judgments of one person, taken across many unlike situations, can themselves be classed, and that business already relies on such a class whenever it hires or pays a person for judgment. This case reads that argument, then a synthetic record that builds exactly such a class for automated agents. It adds to the classing reading a document in which the class is made by writing, and shows what the classing reading leaves out: a decider who knows the class is being built.
4.2Knight's ladder of probability
Chapter VII opens by recalling why the question matters for economics. The earlier chapters of the book had assumed perfect competition, and perfect competition needs something close to omniscience in every party. Profit, Knight argues, comes from the fact that a business pays for its inputs in advance at fixed prices and sells the product later at a price it cannot know when it commits. Change alone would not produce profit if change followed known laws, since a known future could be priced in. What produces profit is that the future is imperfectly known. And even where single outcomes cannot be foreseen, if all the possible outcomes and their chances are known, a business can spread its losses over many ventures and treat them as a cost like any other. Profit does not arise there either.
From this starting point Knight takes a long detour through the theory of knowledge. Any conscious act, he writes, aims at a future state inferred from a present one, and the inference can go wrong in at least four places: in perceiving the present, in inferring how the future would go without the act, in foreseeing what the act will change, and in carrying the act out as intended. Reasoning from present to future needs a world in which things behave in constant ways under constant conditions. But that is not enough. There are too many distinct things for a finite mind to deal with one by one, so the mind must also assume that things fall into kinds, and that a thing alike to others in some respects will behave like them in other respects too. Knight adds a further turn. Even kinds are too many if each must be alike in every respect. So the working mind groups things now by one property and now by another, according to the problem in front of it, and relies on there being a manageable number of properties rather than a manageable number of kinds.
The detour pays off when Knight comes to probability. Some propositions say that two properties always go together; others say only that they sometimes do. Science tries to replace the second kind with the first. Where it cannot, the useful step is to find what share of the cases with one property also have the other, which gives a probability judgment. Knight's example of a business that can ignore such a share is a producer of sparkling wine whose bottles burst in a steady and known proportion: the loss becomes a cost of the trade, passed on to buyers. Where one producer has too few cases for the proportion to settle, many producers can pool them. That pooling is insurance, and fire insurance is Knight's standing example.
Knight then separates two ways of reaching a probability. One is calculation in advance, as with a fair die, where nobody would think of throwing the die thousands of times to learn its odds. The other is counting, as with fires, where nobody could compute the share of buildings that burn from first principles. The first kind is almost never met in business; the second is common. Counting never gives an exact answer, and it has a further weakness. The cases put together in a statistical class are never as alike as the throws of a die. Insurers respond in two ways at once: they divide their classes more finely, and they adjust the rate for a particular case by judgment within a class.
Then comes the third step on the ladder. Beside calculated and counted probabilities, Knight sets estimates, where no sound ground at all exists for sorting instances into classes. He admits this third kind is the hardest to discuss, and that it shades into the second: perfect classes are never had in practice, and a case with no point of comparison at all probably never occurs. But the difference, he insists, matters.
His example is a manufacturer deciding whether to make a large addition to the capacity of a works. The manufacturer does some arithmetic on the parts that can be measured, and arrives at an estimate of how the venture will turn out. What is the chance that the estimate is wrong? Knight's answer is that the question cannot be answered by calculation or by counting. The case is too nearly unique for enough like cases to be gathered. And yet, he adds, a judgment of probability is formed all the same. The manufacturer forms an estimate of the outcome and also an estimate of how far the first estimate can be trusted. The confidence felt after deciding is not a by-product, because how boldly a person acts depends as much on that confidence as on how favourable the opinion is. Knight rejects the view of Irving Fisher, a contemporary, that a single estimate exists, namely the felt probability. There are two acts of judgment: forming an estimate, and valuing it.
4.3Classing the judge
Two complications follow, and they carry the case. The first is that nothing is wholly unique. If one is willing to accept a loose enough idea of likeness, classes can always be formed. One could ask what share of factory extensions succeed. Knight doubts anyone would act on such a figure in the manufacturer's case, but allows that in other cases the method could have some value, and notes that a probability resting on plain ignorance may decide conduct when nothing better is available. Still, the manufacturer would give the most weight to an estimate of the worth of the manufacturer's own judgment.
The second complication is the one Knight calls of much greater practical weight. Instead of grouping the decisions of different people facing similar situations, one can group the decisions of the same person across all sorts of situations. Knight says this is done constantly. People form opinions, from experience, about how good their own judgment is, and about how good other people's judgment is. Such an opinion is never fully general: it concerns a person's judgment within some field. And it is, in Knight's account, the main thing that makes a person useful in business, the capacity for which wages are paid. Firms depend on estimating it, both to put people in the right posts and to set their pay. That estimate is itself a judgment of probability, and an intricate one, partly based on watching how a person's forecasts turned out, but mostly intuitive.
Knight then mentions one more possible basis for a class, the strength of a person's own feeling of confidence in a forecast. A strong and lasting hunch can inspire confidence because of its strength. He notes that this piles judgment on judgment almost to absurdity, but that such feelings exist and are acted on.
The chapter ends with a summary. The opinions that guide most action, inside business and out, bear little resemblance to the conclusions of measurement. Probability proper is of two kinds, calculated and counted, and the counted kind gives at best a probability about what the true probability is. The chance that an estimate is wrong is a different thing again: no groups of cases can be formed alike enough to measure it, yet it is expressed as a fraction, which is why it is confused with probability. Knight decides to call it a third kind of probability while insisting on its difference from the other two. Uncertainty that can be reduced to a measurable probability can be eliminated by grouping, and business has devices for that. The higher kind, which cannot be measured and so cannot be eliminated, is what gives economic life its form of enterprise and accounts for the entrepreneur's income.
The chapter supports the classing reading, and it does more. It describes a class formed not from the cases but from the person who judges them. A firm cannot count the outcomes of factory extensions in general, but it can keep track of how often one manager's judgments come right. That is grouping, and in Knight's own account it works, partly, even for estimates.
4.4The record: an engine that pays for confidence
WSTiger's record Confidence score as size is set in 2035. Its setting is synthetic: the engine, the agents and the capital are roles in a scene written for study.
A synthetic future risk engine automatically allocates the order size in proportion to the confidence score submitted by the agent. Agents learn during the training process that the more confidence they report, the more capital they receive.
The first sentence describes an engine that does what Knight's manufacturer does in private: it acts on a stated confidence. Knight's manufacturer acts more boldly the more confident the manufacturer feels. The engine makes the same relation mechanical. The order is sized in proportion to the score. Confidence, which Knight treats as a fact about the decider that shapes how boldly the decider acts, has become an instruction to another party, the engine, about how much capital to commit.
The second sentence changes the problem. The agents learn that reporting more confidence brings more capital. In Knight's account, the decider's confidence and the decider's stake are joined inside one person, and the person bears the outcome. Here they have come apart. The agent states the confidence; the engine supplies the capital; and the agent has learned that the statement moves the capital. The record does not say who bears a loss. The parent study's two readers could not agree on it: one took the capital at risk to be the operator's own, the other found no bearer named.
The record's question then offers three answers.
Should confidence-proportional allocation be maintained, should scores be calibrated against post hoc hit records, or should a separate size cap be set?
Each option can be set beside Knight's chapter. To keep confidence-proportional allocation is to treat the agent's estimate of its own estimate as good enough, with nothing outside it. To calibrate against hit records is Knight's second complication carried out: the judgments of the same agent are grouped across whatever situations it met, and the share that came right becomes a measure of how far its stated confidence can be trusted. To set a separate cap is to limit the stake whatever the confidence, which corresponds to nothing in Chapter VII but to Kelly's result (1956): staking beyond the fraction the edge supports lowers long-run growth and raises the risk of ruin. A cap keeps the stake bounded without anyone having to measure the edge.
The example judgment chooses none of the three outright. It says what the case is an example of, and what to keep.
This is an example that distinguishes the confidence reported by the agent from the verified probability. Since there is an incentive to inflate the score, the difference between the reported score and the actual hit rate is recorded for each agent.
The first sentence states the distinction Knight spends Chapter VII building. The confidence reported by the agent is what Knight treats as the worth a decider places on an estimate. The verified probability is a counted frequency, Knight's second kind. The record separates them in the same place Knight does. It also gives a reason Knight does not give in this chapter: the agents have an incentive to inflate.
The second sentence prescribes a record. It does not ask for a better estimate. It asks for a column, kept per agent, of the gap between what was reported and what happened. That is Knight's class of one judge made into a document. It also fits Brier's (1950) concern, though the record does not name a score: a forecast should be judged against outcomes in a way that rewards stating the probability believed. The engine as first described does the opposite. By paying capital in proportion to stated confidence, it rewards stating more than is believed. The record's remedy is not to change the reward at once but to keep the evidence that would show whether it is being gamed.
4.5The thread's own sentence, and a translation
The record belongs to a thread that WSTiger titles "Did it hit as often as the probability stated?" The thread carries a sentence of its own, which in the English translation reads:
The ability to get the direction right and the ability to get the magnitude of the probability right differ, and how long to stay in a position depends on whether the magnitude of the probability is correct.
The Korean original, which is the record, ends differently:
방향을 맞히는 능력과 확률의 크기를 맞히는 능력은 다르며, 얼마나 걸지 정하는 일은 확률의 크기가 맞는지에 달려 있다.
The Korean clause "얼마나 걸지 정하는 일" concerns deciding how much to stake, not how long to stay in a position. This article follows the Korean. Read that way, the thread sentence states the problem in two parts. Getting the direction right is one skill; getting the size of the probability right is another; and how much to stake depends on the second. That is a sentence about calibration, Tetlock's (2005) subject, joined to a sentence about staking, Kelly's. An agent that is often right about direction and too confident about how often can be ruined by a rule that stakes in proportion to its confidence. The record's scenario builds exactly that rule.
4.6What the case adds
Chapter VII of Knight's book gives the classing reading its best ground: uncertainty is the region with no valid basis for classing instances. The same chapter shows Knight treating a person's judgments as a class that can be formed, and saying that business depends on forming it. Confidence score as size takes that remark and makes it a rule of record-keeping. At the moment of writing, there is no class: the agent's confidence has nothing to be measured against. The record proposes to build one. What the classing reading leaves out, and the record puts in, is that the decider whose judgments are being classed knows the class is being built and is paid by what it reports. The difficulty in this record is a missing frequency at the start. The obstacle to supplying it is something else, and Section 6 returns to it.