"Conscious Realism" and "Multimodal User Interface" theories

I recently sent an email to Donald Hoffman, professor at the University of California, Irvine, with kudos for his book, Visual Intelligence, which has had a profound impact on my thinking about perception. Understandably, he's very busy kicking off the new school year, so I was grateful that he sent at least brief response and a reference to his latest published paper, titled Conscious Realism and the Mind-Body Problem. Naturally, I was eager to read it.

Much of the study of how human consciousness arises stems from the assumption that consciousness is a product of physical processes; that consciousness is a product of a physical processes in the brain. This paper starts from the opposite assumption: that "consciousness creates brain activity, and indeed creates all objects and properties of the physical world." When I read this in the abstract, I must have largely ignored its significance. Having read Visual Intelligence, I'm familiar with Hoffman's focus on how our minds construct the things we perceive, so I took this summary as a shorthand for this concept of construction of the contents of consciousness. It becomes apparent that this claim is far more literal than I had assumed.

Hoffman begins by explaining the ubiquity of a central assumption as follows. "A goal of perception is to match or approximate true properties of an objective physical environment. We call this the hypothesis of faithful depiction (HFD)." After giving lots of examples of this assumption and reasons why it's taken for granted, Hoffman declares his rejection of it:

    I now think HFD is false. Our perceptual systems do not try to approximate properties of an objective physical world. Moreover evolutionary considerations, properly understood, do not support HFD, but require its rejection.

Now, I'll state here that most of Hoffman's claims in this paper appear logically valid and, on the face of it, uncontested. But I would have to say that this one probably isn't logically supported: that evolutionary considerations require the rejection of HFD. By and large, however, this paper claims that it is not necessary to assume there is an objective physical world in order to study and understand consciousness, which seems acceptable.

The term "objective physical world" deserves some explanation. It identifies the view that there is a single reality that exists without regard to observers. If there is an apple on the table before two people, the apple really is there, whether either of them perceives it. Naturally, one would imagine that if one of them can see the apple, the other one probably can (barring obstructions), because both of them have access to information (e.g., light) reflected off the apple and into both their eyes. They may see different sides of the apple, but the apple is definitively there.

To be sure, one should not dismiss Hoffman as a fringe nut that claims there is no reality, per se; only people and their subjective consciousnesses. He doesn't in this paper. In fact, it's clear he does appear to accept the assumption that there really is an objective reality, but that we don't have direct "access" to it. A classic example of this distinction is a detailed treatment of the table not as a solid object with straight edged surfaces, but as a collection of atoms and, mostly, empty space and, as such, rough, continuously changing surfaces. In this sense, there really isn't a table; that's just a percept (or concept) we use to refer to the collection of atoms.

To help illustrate the distinction between what one perceives and the subject matter of perception, Hoffman introduces the analogy of deleting a computer file by "dragging" a file icon and "dropping" it onto a trash can icon. This action is intuitive and designed specifically as an analogy of the actual file delete operation, but it actually bears no resemblance to what actually goes on under the surface. In fact, even the icon is not equivalent to the file; it's merely a percept specifically designed to represent it to the end user. By analogy, Hoffman refers to the table or the apple as merely "icons" we create in our minds to represent what most people would reflexively call "real objects". In fact, to the person who says, "no, the apple is just a bunch of atoms," Hoffman would in turn say, "the atoms are themselves icons we create."

Hoffman introduces the term "multimodal user interface", or "MUI", to summarize what consciousness is. In contrast to the view that perception is all about constructing a mental model of reality that closely resembles reality, Hoffman claims perception is about constructing practical models that "get the job done". And just as computer designers might construct icon based interfaces to help make it easier for humans to understand and practically manage information, our own minds actually set out to construct "practical" percepts in order to help us simplify what we do. But the mental models, Hoffman claims need not bear any resemblance to what is being modeled.

To be sure, Hoffman may say the percepts -- mental models -- a conscious entity holds bear no resemblance to their referents, but he doesn't claim that there is no correlation to them. Hoffman says that user interfaces, including our own consciousnesses, by design have the following characteristics:

  • Friendly formatting
  • Concealed causality
  • Clued conduct
  • Ostensible objectivity

That is, a user interface's "purpose" is to distill immensely complex behaviors down to practical "icons" of objects and behaviors that stand for that underlying complexity, but don't literally mirror it. Take the file-delete example. The icon on the desktop is a sufficient stand-in for a file, even though the file, a pattern of magnetic fields on a metal platter, bears no resemblance to the icon. It's a "friendly format", in this sense. Further, the action of dragging and dropping it onto a trash can icon to "delete it" has its own causal chain, which conceals the true, deeply complex causal chain that actually happens to effect the file delete operation. Yet the drag-n-drop operation and the trash can icon give an intuitive clue of what will happen if something is dropped onto it. Finally, this drag-n-drop-to-delete operation is designed to consistently do the same thing every time, thereby engendering in the user an ostensible sense that there is an objective operation going on that will always happen, even though a moment's reflection tells us that a failure in the underlying software or hardware could cause something else to happen when one drops a file icon on the trash icon.

So far, I can see that there's a practical use for this notion to people trying to understand human perception or to engender consciousness in machines. For one, the claim is that percepts do not have to bear much resemblance to their referents in the "real world". They just have to have practical utility. An icon in a user interface just needs to be useful enough for the user to be aware of a file's existence and to do some basic stuff with it. Similarly, the mental percept an antelope has of a lion in the distance only needs to be useful enough to stay alive to be useful. It doesn't need to be a highly detailed representation of the lion beyond that basic utility. It also alludes to the view that a high fidelity representation in a computer of the "real world" doesn't make the machine that has it any more aware of what is represented. For instance, just because a self-driving car has a 3D map of the terrain out in front doesn't mean it can "see" where the road is. It's still necessary to create a practical model of how the world works that uses this 3D representation as source data, like an algorithm that seeks basically level ground, defined by a threshold of variation that separates level from non-level ground. If this were the message of the paper, I would say it adds genuine value: a set of concepts and terms to use to help steer people away from fallacious assumptions about how consciousness works and to suggest paths for further study.

But this isn't where the paper ends. It's more where it starts. In fact, this paper is less about explaining how consciousness works than about how reality works; it's metaphysics instead of epistemology. As stated earlier, it starts with the assumption that consciousness exists and that the subject of consciousness is optional. To avoid sounding like a total subjectivist, Hoffman states that:

    If your MUI functions properly, you should take its icons seriously, but not literally. The point of the icons is to inform your behavior in your niche. Creatures that don't take their well-adapted icons seriously have a pathetic habit of going extinct.

If Hoffman accepts the idea that there is a physical, objective reality, what is it composed of? "Conscious Realism asserts the following: The objective world, i.e., the world whose existence does not depend on the perceptions of a particular observer, consists entirely of conscious agents." Honestly, I would love to say that this claim is explained, but it really isn't. Hoffman claims that humans are not the only conscious agents, but doesn't say that tables, apples, and such are conscious, per se. "According to conscious realism, when I see a table, I interact with a system, or systems, of conscious agents," which really does seem to suggest that the table is conscious, but not clearly.

    Conscious realism is not panpsychism nor entails panpsychism. Panpsychism claims that all objects, from tables and chairs to the sun and moon, are themselves conscious (Hartshorne, 1937/1968; Whitehead, 1929/1979), or that many objects, such as trees and atoms, but perhaps not tables and chairs, are conscious (Griffin, 1998).Conscious realism, together with MUI theory, claims that tables and chairs are icons in the MUIs of conscious agents, and thus that they are conscious experiences of those agents. It does not claim, nor entail, that tables and chairs are conscious or conscious agents.

This is one of the problems I have with this paper, though. Although Hoffman rejects the notion of inanimate objects as conscious in a trippy, Disney cartoon sense, he doesn't really elaborate on what he does mean. Moreover, if a table is labeled as conscious in order to stick a placeholder for a physical object in the objective world, what value does this add over the simpler, more intuitive conception of the table as being a physical object? It almost seems as though, in order to come up with a rigorous, clean-cut, math-friendly theory of how consciousness constructs perceptions of the world, Hoffman throws the baby out with the bathwater by claiming that even though there is an objective world, it is not composed of actual objects.

I think if Hoffman were inclined to speak of "conscious realism" and "multimodal user interfaces" as tools and techniques for studying consciousness and guides to creating it, this could be a practical concept. He could say that our perceptions of reality really do reflect, if simplistically, abstractly, and practically, an actual, objective reality. By taking pains to say there isn't really one -- or that it is entirely disconnected from our ability to perceive it -- this paper seems to do something of a disservice to science:

    We want the same [approach] for all branches of science. For instance we want, where possible, to exhibit current laws of physics as projections of more general laws or dynamics of conscious agents. Some current laws of physics, or of other sciences, might be superseded or discarded as the science of conscious realism advances, but those that survive should be exhibited as limiting cases or projections of the more complete laws governing conscious agents and their MUIs.

While I can see that it is possible, perhaps, to express other branches of science in the terminology of MUIs, I don't see how it would advance our understanding of their subject matter. Gravity was well understood by Newton, yet expressing it in terms of the theory of General Relativity makes it possible to do more with the subject matter than was possible in the purely Newtonian framework. What new insights will the physicist have as a result of expressing gravity in terms of multimodal user interfaces and with reference to heavenly bodies as conscious entities? If anything, it sounds more like this extra layer would only add to the confusion people have in trying to understand already complex concepts and could even potentially take away certain practical conceptual tools. So I don't see the point.

All that said, the MUI concept does seem to add value to my own way of thinking of perception. The four functions of a good user interface listed above (friendly formatting, concealed causality, clued conduct, ostensible objectivity) seem to shout out how scientists trying to engender perception in machines should frame their goals and concepts. But the rest of Hoffman's paper, which dabbles in the philosophy of what reality is, seems to have little use for AI research.


Popular posts from this blog

Coherence and ambiguities in problem solving

Discovering English syntax

Neural network in C# with multicore parallelization / MNIST digits demo