Unified framework for information integration based on information geometry

• Published in: Proceedings of the National Academy of Sciences - PNAS Vol. 113; no. 51; pp. 14817 - 14822
• Main Authors: Oizumi, Masafumi, (土谷 尚嗣), Naotsugu Tsuchiya, Amari, Shun-ichi
• Format: Journal Article
• Language: English
• Published: United States National Academy of Sciences 20.12.2016
• Subjects: Biological Sciences; Causality; Complex systems; Consciousness; Geometry; Information; Neuroscience; Physical Sciences; Probability distribution; Statistics; mutual information; information geometry; consciousness; integrated information; transfer entropy
• ISSN: 0027-8424; 1091-6490; 1091-6490
• DOI: 10.1073/pnas.1603583113

Assessment of causal influences is a ubiquitous and important subject across diverse research fields. Drawn from consciousness studies, integrated information is a measure that defines integration as the degree of causal influences among elements. Whereas pairwise causal influences between elements can be quantified with existing methods, quantifying multiple influences among many elements poses two major mathematical difficulties. First, overestimation occurs due to interdependence among influences if each influence is separately quantified in a part-based manner and then simply summed over. Second, it is difficult to isolate causal influences while avoiding noncausal confounding influences. To resolve these difficulties, we propose a theoretical framework based on information geometry for the quantification of multiple causal influences with a holistic approach. We derive a measure of integrated information, which is geometrically interpreted as the divergence between the actual probability distribution of a system and an approximated probability distribution where causal influences among elements are statistically disconnected. This framework provides intuitive geometric interpretations harmonizing various information theoretic measures in a unified manner, including mutual information, transfer entropy, stochastic interaction, and integrated information, each of which is characterized by how causal influences are disconnected. In addition to the mathematical assessment of consciousness, our framework should help to analyze causal relationships in complex systems in a complete and hierarchical manner.