Introduction to Broad Information Theory (Lecture)

“Broad” as distinguished between “narrow” Information Theory, a sub-field of physics that deals with questions of information transmission and entropy
“Broad Information Theory (BIT)” presents a comprehensive view of how information is organized in systems, both physical and virtual, across many levels from the tiny to the giant, and how those systems interact with each other
Fundamental theses:
- Across different domains of human knowledge, systems can be described as an architecture of their fundamental elements and the interactions between them
- Informational patterns in a domain are evolved through competitive selection pressure to meet underlying needs in the environment.
  - When an informational pattern appears across unrelated domains, those domains may have similar underlying informational needs
- The idea that a pattern represents a general solution to defined needs can be tested by building computational game-simulation models
BIT explores domains to discover viewpoints and analytical tools and uses them to synthesizes a coherent information-theoretical view across domains.
- Synthetic field
- Ex. Behavioral economics
Domains to be discussed include:
- Physics (computational view of the universe)
- Biology (microbiology/immune system, organ interaction)
- Ecology (ecosystems, populations)
- Sociology (competing ideologies)
- Psychology (organization of information in human brain, heuristics, human free will)
- Artificial Intelligence (organization of information in artificial neural networks)
- Theology (religions as operating systems, role of divinity through information theoretical lens)
About me:
- Most importantly: I’m an expert generalist
- Undergraduate degree in Physics w/ High Distinction
- Accepted into Ohio State University PhD Physics program on a National Needs scholarship – left early
- Have worked as software engineer, software architect, data scientist
- Other fields: biology, history, law, investing, planning process (Agile/Scrum), organizational change, cognitive science/psychology, writing, music, self-defense, education
- Experience: became fluent in Spanish, lived overseas
Analytical frameworks:
- Scientific method
  - Method for falsifying models (against experiment, i.e. “the razor of reality”)
- Critical thinking
  - Structured logical reasoning
- Critical theory
  - Subjectivist sociological framework
  - Asserts that there is no better or worse analytical viewpoint (relativist)
  - Views the individual as the intersection of overlaid identities (intersectional)
- The razor of reality
  - Objectivism
Concepts from Mathematics and Machine Learning
- Finding minimum/maximum
- Non-convex functions (almost all interesting problems)

Requisite basics:

Science
- A structured process for reducing error
- Science does not deliver truth. It falsifies untruth, as measured by experiment against the observable universe (the “razor of reality”)
- “Scientific method” – a procedure for constructing theoretical models and falsifying those that can be falsified
  - Observe
  - Hypothesize
  - Construct and conduct experiments to falsify the hypothesis
  - Repeat
  - Publication, peer review, and criticism are means of sourcing collective intelligence to better hypothesize and falsify hypotheses
- All observations – and thus all theories – have inherent limits
- Scientists do not “prove” things. We find regions of measurable parameters within which we cannot disprove them.
  - Mathematics proves things
- As distinct from: “scientism”
  - Religion masquerading in the trappings of science
  - Typically involves deferral to “experts”, credentials as a stand-in for data
  - “Trust the science” ← inherently unscientific phrase
  - “In God we trust. All others bring data” – W. Edwards Demming
Computation
- A physical implementation of operator mathematics
  - State: a number or, more generally, a function describing something
    - Why a function?
    - Any function can be approximated as a sum of other more limited functions
      - Taylor expansion
      - Finite element method
    - Lowest-degree expansion of a function is a 0 degree polynomial – i.e. a number
    - You can think of digitization as an expansion of a continuous to regions so small that a 0 degree polynomial (ex. Number) in each region represents the function’s output
      - Example: digital photograph
  - Operators: transform state into new state
    - I put an apple into the food processor, transforming the apple to applesauce
    - I can apply a sequence of actions (i.e. kitchen food preparation process, manufacturing process); we can think of this as a sequence of operator

How We Came to Know the Cosmos, Dr. Helen Klus, http://www.thestargarden.co.uk/Atoms-and-waves.html
The Universe as a Computer, JH Wierenga, https://quantumoccam.net/explanations/the-universe-as-a-quantum-computer
The Universe as a Computer, Mike James, https://www.i-programmer.info/babbages-bag/356-universe-as-computer.html (re-read)

References: