When adding information processing into a system hurts, and when it helps (Monday seminar)

6 October 2014
11:30 am to 12:30 pm
When adding information processing into a system hurts, and when it helps (Monday seminar)
Dr. David Wolpert
Santa Fe Institute

There is a very rich relationship between how information is propagated through a system and the
highest value of their expected utility that  a decision maker embedded in that
system can achieve by properly choosing how to react to their observations. 
This relationship is a crucial factor in determining how we should design 
any control system that involves either human or computational decision makers who modify
their behavior to maximize expected utility.
In his famous paper on this relationship, Blackwell proved that
a decision maker who chooses her action based on observing nature cannot achieve higher expected utility if 
the output of her observation apparatus is post-processed before she receives it. This is true regardless of
the utility function, distribution over nature's state, or type of post-processing.

Here we show how to extend Blackwell's analysis to arbitrarily complicated Bayes nets,
with information processors inserted at arbitrary points in the net, and even arbitrary numbers of decision makers
in the net. We show that inserting extra processors into such a Bayes net $may$ increase the maximal utility,
despite Blackwell's result, finding necessary and sufficient conditions for it to do so. 
We then extend our results to find necessary and sufficient conditions for inserting processors
into a Bayes net to increase the maximal information redundancy, synergy, multi-information, transfer entropy,
gini index, local information, covariance, etc., among any subset of the variables in a Bayes net.
We discuss the implications of these results for extending the data processing inequality, to arbitrary Bayes nets,
and to concern many measures in addition to mutual information. We also show how to extend our results 
to apply to bounded rational decision makers.