Someone recommended that I read about process philosophy, so I checked it out. Here's a collection of my notes on what process philosophy is and how it relates to approaches to knowledge in Reinforcement Learning.
Traditional western philosophy is obsessed with describing reality from a set of static components. These components have dynamic features which are treated as secondary or derivative to the static individuals; they are ontologically derivative, or secondary to the actual components--i.e., their existence is secondary and dependent on the physical nature (Seibt, 2018). This core research programme is substance metaphysics, the study of the nature of being which originates from substances.
Process philosophy starts by upending this approach to metaphysics--this study of existence of reality. reality and existence are thought to be the behavior of a dynamic system (Seibt, 2018). Interactivism is a branch of process philosophy which broadly concerns itself with the interaction of agents (Bickhard, 2000); Interactivism considers it self to be a descendant of genetic epistemology (Piaget and Duckworth, 1970), also known as constructivism.
The majority of PK's contact with interactivism is through epistemology, as PK approaches do not have metaphysical commitments. PK particularly concerns itself with epistemological commitments. When we strip interactivism of it's metaphysical commitments and evaluate it from an epistemological perspective, it is not clear what the core contribution or insight of Interactivism is, as many other branches of pragmatism make the same assertions.
Two core commitments of of interactivism are pragmatism and fallibility. Under a pragmatic approach to knowledge, conceptual content is evaluated by action. A fallible and pragmatic approach to knowledge is present in many historical and modern approaches to epistemology--including, pragmatism (Pierce, ), enactivism (Noe, 2004), and inferentialism (Brandom, 2009)--to list a non-exhaustive sample of alternative epistemologies.
From an interactivist perspective, knowledge is:
"constituted as goal-oriented interactive competence, and representation is a functional aspect of such competence: interactions and interactive systems that are not appropriate to an environment, that are not sensitive to that environment and to its potentialities, will not be competent in that environment.'' - Bickhard and Richie (1983) p. 5
Again, this approach not terribly different from enactivist approaches knowledge. However, Interactivism does make the specification of knowledge being both goal-oriented and interactive: two core components of a predictive knowledge paradigm (Sutton, 2009). Their argument that competency is related to knowledgability--or judging the knowledge of an agent by its actions--is compatible with PK and more broadly pragmatism in general.
"The interactive claim is that such interactive sensitivity, such ability to take into account an environment, its potentialities, and its challenges, is the fundamental form of representation... Such [representations] will be in terms of the internal course and outcomes of some interactions, which may in tern be useful in determining the course of other interactions.''- Bickhard and Richie (1983) p. 5
Interactivism is clearly compatible with PK, as General Value Functions (GVFs)--a core method of specifying predictions used in PK--satisfy interactivism's specification of representation: anticipation of the dynamics of an agent's environment in terms of it's behaviour. This description of representing the environment in terms of anticipating the dynamics on a low level mirrors nexting (Modayil, 2014; Gilbert, 2009), and other sensorimotor approaches to knowledge, such as enactivist approaches. From an enactivist perspective, there may be high-level representations which are not in sensorimotor terms--taking a weaker stance than PK--however; these are secondary to and derivative of low-level dynamic representations.
There are a few gems in the interactivism manifesto. For instance, a criticism of the spectator of knowledge: the theory of knowledge which holds that observation is purely reception, that the mind is passive in perception, and that knowing is related to a passive beholding. These commitments are fundamentally incompatible with PK (Kearney, 2018), a fact which is evident when you consider that GVFs are explicitly encoded in terms of behaviour through the policy parameter π (White, 2015). See (Noe, 2004) for in-depth discussion.
Much of the manifesto is a dig at what I interpret as conceptual platonsim, which is often described by (Bickhard, 1983) as encodingism. Encodings are mental correspondence which captures the structure of what is being represented. Canonical examples include painting and sculptures--items which are representations of some real, physical entity, but are not exact copies. From an artificial intelligence perspective, representations could be symbolic entities with particular properties in a knowledge base, or learned kernels which represent some feature--e.g. a facial feature.
As Bickhard presents it, encodingism can be split into strong and weak versions. Strong encodingism takes encodings to be all of mental representation. Weak encodingism suggests that encodings exist, but there are are other independent forms of representation which are necessary and independent. See (Bickhard, 1983) for further discussion.
Interactivism rejects encodingism, stating that representation is functional: encodings represent only insofar as the representation has a function for an agent. In this sense, this interactivism takes a functionalist (Levin, 2018) approach to conceptual content, similar to inferentialism and interactivism.
Representations are insufficient on their own as a description of conceptual content, as they require an agent to interpret the representation. This is similiar to Piaget's assertion that if knowledge were really a copy of the world, one would have to understand the world already to construct a mental copy (Piaget, 1970), as Bickhard points out.
The second criticism of encodings is that representations must be representations of something. If an encoding which represents something is logically independent of all other encodings, answering the question "what does this encoding represent?" can only be "whatever it represents". To specify the encoding in terms of some other representation is to specify it in terms of reducible form which is prior to it, making it dependent on other encodings. In short, encodings cannot be a basic irreducible form of representation, as it is uncertain how to connect the representation to what it is representing.
Interestingly, this seems like an anti-representationalist approach which simply does not go the whole way. An alternative to representation as the origin of understanding and awareness is expression (Forester, 2018), as outlined by Brandom (Brandom, 20019). Instead of considering interactive sensitivity representation of the environment, we can think of these sensitivities or predictions in terms of their relationships. Rather than evaluating conceptual content and understanding representations in terms of what they are representing, we can evaluate conceptual content by the act; we may understand the conceptual content of an agent by their ability to apply concepts and the relation between them.
This anti-representationalist approach also supports a view of perception and awareness which mirrors the interactivist account of awareness. Interactivism asserts that perception and awareness are emergent concepts which arise from the processes of a system. If we take an expressive approach to understanding and conceptual content, then perception and awareness can be explained in terms of being able to anticipate and apply concepts.
The interactivist criticism of encodings has been applied to machine intelligence---criticising methods such as CYC for, similar to Ring's arguments criticism of the same ontology and call for grounding knowledge in sensorimotor experience (Ring, 2016). However, instead of such projects being doomed due to lack of grounded, sensorimotor experience, the encoding criticism dooms CYC for attempting to capture correspondences between mental representations and the environment, rather than the dynamics.
Similarly, these arguments can be rephrased not as arguments against encoding, but rather as arguments against representation.