Causal versions of maximum entropy and principle of insufficient reason

The principle of insufficient reason (PIR) assigns equal probabilities to each alternative of a random experiment whenever there is no reason to prefer one over the other. The maximum entropy principle (MaxEnt) generalizes PIR to the case where statistical information like expectations are given. It is known that both principles result in paradoxical probability updates for joint distributions of cause and effect. This is because constraints on the conditional P ( effect ∣ cause ) P\left({\rm{effect}}| {\rm{cause}}) result in changes of P ( cause ) P\left({\rm{cause}}) that assign higher probability to those values of the cause that offer more options for the effect, suggesting “intentional behavior.” Earlier work therefore suggested sequentially maximizing (conditional) entropy according to the causal order, but without further justification apart from plausibility on toy examples. We justify causal modifications of PIR and MaxEnt by separating constraints into restrictions for the cause and restrictions for the mechanism that generates the effect from the cause. We further sketch why causal PIR also entails “Information Geometric Causal Inference.” We briefly discuss problems of generalizing the causal version of MaxEnt to arbitrary causal DAGs.

Why classic probability cannot be maintained

Pierre Simon Laplace defines classic probability in his “Essai philosophique sur les probabilités” in 1814. According to Laplace, probability is the correlate of partial human knowledge. Laplace’s notion of probability rests on the idea that if there is no reason to believe one event is more likely to happen than the other, then the two events should be considered equally probable. This idea is the principle of insufficient reason defined by Jacob Bernoulli, most probably a counterpoint to Leibniz’ principle of sufficient reason. The principle of insufficient reason is called the principle of indifference by Keynes and it is known under this name in criticism. Laplace defines probability as the number of useful happenings for creating an event divided by the number of events equally probable. The paper traces three presuppositions of Laplace’s definition and fasifies them one by one to show that his definition of classic probability cannot be defended. The paper claims that because i. in a determinist world there are not only epistemic probabilities, ii. epistemic probabilities are not necessarily subjective and iii. the principle of indifference does not provide sufficient basis for analysing probability, the classic definition of probability cannot be argued for.

WHAT TO EAT

Given the Babel of information and misinformation, the question emerges: Is there a rational method to decide what to eat? To seek an answer, we must go back to a two-hundred-year-old idea that is usually referred to as "La Place's Principle of Insufficient Reason" and states that events are to be assigned equal probabilities if there is no reason to think otherwise. E. T. Jaynes has recently extended the principle to state use that "in making inferences on the basis of partial information, we must use that probability distribution that has the maximum (informational) entropy . . . maximally noncommital with regard to missing information subject to what is known." Without going into a detailed computer program, we can infer that the most rational aliment will be obtained by fulfilling known nutrient requirements, minimizing intake of proven toxic substances, and randomizing everything else. In other words, the most variable diet that fulfills known constraints is the diet of choice. The clue is variation: use different foods, change brands frequently, and never use the same recipe twice. Bring on the escargot, candied grasshoppers, roast guinea pig, alfalfa sprouts, macadamia nuts, falafel, turnip greens, squid, mare's milk, and seaweed salad. Minimize the intake of packaged foods, since they contain additives of unknown toxicity, and where they are unavoidable, keep varying the suppliers according to additives.