Search This Blog

Sunday, November 28, 2004

Thoughts on FLARE

[Audio Version]

Now that I've gotten back into AI research after all these years, I'm starting to reach out to find out more about other research in the AI field. I recently started reading abstracts of articles in the Journal of Artificial Intelligence Research (JAIR), which stretches back to 1993, in hopes of finding out more about the state of the art. When I started from the latest volumes, I was surprised by how inapproachable so many of the articles appeared to be. For starters, they appear to be written exclusively for those already deeply entrenched in the field. For another, rather than positing new theories or discoveries, they appear largely to be small extensions of concepts that have been explored for decades.

So I decided to start reviewing abstracts from the beginning. What a difference that makes.

I recently read an interesting article from 1995 titled An Integrated Framework for Learning and Reasoning, by C.G. Giraud-Carrier and T.R. Martinez and published in Volume 3 of the JAIR. The authors started from a fairly conventional approach to knowledge representation (KR) involving "vectors" of attributes that allow for representation of "first order predicate" statements about some domain of knowledge and hence drawing deductive conclusions based on such knowledge. They went on to extend the concept to incorporate an inductive sort of learning of rules from examples and to provide the means for their system to alternate between ruminating over its knowledge to draw new conclusions and acquiring new information to integrate into its knowledge base (KB). They called the system they pioneered a "Framework for Learning and REasoning", or "FLARE".

I have a variety of criticisms of the FLARE, but before I start with them, I have to give Giraud-Carrier and Martinez strong credit for their work, here. They sought to bridge the gap between learning and reasoning that existed then and even now in the AI sub-communities and indeed seem to have been successful. And rather than follow the often obscure paths of neural networks or genetic algorithms, they chose to try to engender learning with a KR based on formal logic statements, staying true in a way to the formal logic-driven view of AI, now largely dead, from back in the fifties and sixties. What's more, the authors gave a reasonably honest appraisal of the limits of their system and avoided the temptation to make unduly bold claims about the applications of FLARE.

Having said that, the first criticism I have is of the sort of knowledge representation the authors of this article chose. In their system, every concept is represented in a "vector", or list of attributes with associated values. Every other concept has these exact same attributes, but with different values. In one example, the attributes are "is an animal", "is a bird", "is a penguin", and "can fly". Each attribute can have either a specific value (a "no" or "yes", in these cases) or a "don't know" or "don't care" pseudo-value. So to say that "animals don't generally fly" in one concept would be to give the "is an animal" attribute a "true", the "can fly" attribute a "false", and all the other attributes a "don't care". Similarly, saying that a penguin is a bird would mean setting the "is a penguin" and "is a bird" attributes to "true" and setting the other attributes in that vector to "don't care". Admittedly, this approach is sufficient for the purposes of FLARE and makes printing a KB representation easy using a simple data table. But taken literally, it means that every piece of knowledge, no matter how small, has values for every one of the attributes known to the system. Thus each new attribute adds to the size of all vectors in the KB and slows down the reasoning process by adding to the size of each of the two dimensions of the KB to search.

The memory and time optimization complaints are weak, I admit. One could easily improve the memory usage by designing vectors that only contain attributes with explicit or "don't know" values and assume all other attributes not specified are "don't care", for example. And the authors indicate that they make use of indexing in a way probably similar to relational database engines like Sql Server to enhance querying performance. So why do I bother with this critique?

I want to linger for a moment on this point about optimization because it is a common criticism of almost all AI work. In this field, one can often hear the frustrated question, "why is it that as I gain more knowledge, I can solve problems faster, but when a computer gains more knowledge, it gets slower?" The answer is that most AI systems have been built with the same basic brute-force approach that can be found in most conventional database and data mining systems. A simple chess-playing program, for example, may look ahead dozens of moves to see what the ramifications of each step will be. Each step ahead costs ever more in processing power. No human being could ever match the performance of even the most basic chess-playing programs in this regard, yet it took decades before the top chess-playing human was "beaten" by a computer, and it was mainly because its computer opponent was so darned fast (and expensive) that it could look farther ahead than any other machine programmed for the task could in a unit of time, not because it was significantly "smarter" than those other systems. That an ordinary PC could still not do the same today is an indictment of AI researchers throwing up their hands and complaining that today's computers are too slow. The human brain isn't really faster than today's computers. Nor do I agree with the claim that the "massive parallelism" of the brain is essential. What's essential is good data structures and algorithms. When you hear the word "parakeet", your brain doesn't do a massive search through all its word patterns to find a best match. I'm convinced It follows links from one syllable to the next, walking through a sort of "word tree", until it finds an end point. And at the end of that tree is not a vector with thousands of attributes. Rather, there's a reference to those attributes or other parts of the brain that are very strongly associated with the word. In short, the paths that the human brain follows are short and simple, and that's why we are able to think so quickly, despite having brains composed of laughably slow and sloppy processing elements. I should point out this isn't so much a criticism of the FLARE concept as it is of the basic assumptions of AI researchers even today, it seems.

More importantly, though, this idea of having attributes be a "dimension" like this instead of being things unto themselves is dubious. FLARE has no capacity to perceive attributes as being related to one another, for example. Nor could it integrate new attributes in a meaningful way other than to simply add another column to the table, so to speak. This limits FLARE's ability to truly deal with novel information, unless it's properly constrained by existing attributes and very well formatted. (To their credit, the authors point out that FLARE does have an ability to deal with modestly noisy -- read "contradictory" -- information.)

The next criticism I have of FLARE is that it does not seem very good at properly congregating ideas together. Even a simple "or" operation to link two possible values for an attribute will cause two separate vectors to be created for the same idea. There's no branching. Admittedly, they did this to help simplify and perhaps even make possible the kinds of deductive reasoning that characterize FLARE's problem solving capability. Still, this seems to me to make it difficult to localize thinking to appropriate "areas" of thought. I suppose I should apologize for giving too much focus to comparing AI products to the human brain. I don't really think the human brain presents the only way to achieving sentience. Still, it provides a good model, has so much to say that seems yet unheard, and should be used as a benchmark in identifying features and limitations of such products. I hope my criticisms of FLARE will be taken mainly as comparisons to the human brain as such, rather than attacks on the validity and value this work brings to the field.

Next, FLARE appears to be competent at dealing with identifying objects, answering questions about objects, and more generally solving first-order predicate logic questions. But it does not seem to have any real capability to deal with pattern matching or temporal sequences, let alone have anything like an appreciation of the passage of time. So it really could not be used to deal with the kinds of bread and butter problems Rodney Brooks identified as the basis for most living organisms, like controlling locomotion, visual recognition of objects, and so on.

In summary, Giraud-Carrier and Martinez wrote an essay on their research into integrating learning and reasoning functions that is pithy and reasonably approachable to a college-educated reader. They tested in a way so as to compare their results to other conventional systems and provided useful examples and caveats in the article about possible applications and limitations. FLARE, their work product, is clearly not a framework for general-purpose thinking, but provides interesting insights into solving logic problems and integrating new knowledge into such a system. To the person interested in AI and looking for a broader view, I would recommend reading this 33-page essay for a penetrating glimpse into a pretty interesting piece of the AI picture.

New project: Mechasphere

[Audio Version]

I suppose I should have announced that I have a new project called "Mechasphere" on my AI site. It's largely an extension of the "Physical World" project, but the software is greatly updated. I suppose it can be said to finally be an end-user-friendly application.

The main reason I hadn't announced it earlier is that I didn't consider the site to be done. But I suppose it's moot, now, because I don't think I'm going to continue it as it is. I'm developing a second version of Mechasphere now from scratch in hopes of improving on a lot of the techniques and interfaces I have now as a result of the slow evolution of the product. Iterative development is a good way to clean up past mistakes.

Sunday, November 14, 2004

Review of "Bicentennial Man"

[Audio Version]

I just got done watching the movie Bicentennial Man. Since the movie relates profoundly to the subject of artificial intelligence, I think it most appropriate to share my thoughts in an AI blog.

For those who have not seen the movie and are intending to do so, you may not wish to read the following spoiler.

Bicentennial Man is essentially a Pinocchio story. A machine named "Andrew" that looks mostly human wants nothing more in life than to make humans happy. He manages to do so in so many ways, but the one thing always standing between him and the fullest measure of intimacy with people is the fact that he's not human. Little by little, he makes himself ever more human-like. By the end, he has chosen to become mortal and to die of "old age" with his wife and, as he lay dying, the "world court" finally announces its acceptance of him as a human being and therefore validate his marriage to his wife of many years. To add to the happiness of this ending, his wife has their assistant disable her life support system so she too can die.

Before I get to the AI parts, I should say that this view of humanity is utter nonsense. Humanity is not defined by death. An immortal human would still be human. To help make this basic confusion a little clearer for the audience, the author of the story makes it so Andrew's wife, given replacement parts and "DNA elixirs" designed by Andrew to help prolong her life decides there's something wrong with this idea. "There's a natural order to things," she says as she tries to explain to Andrew that there's something disappointing about the idea of living forever.

I know this morbid view of life is popular in American pop culture, but I can say without hesitation that I would love to be able to live forever. Only someone who believes there's nothing worthwhile about living or that there's something better to look forward to after death could make sense of this idea. Incidentally, Andrew's wife says "I'll see you soon" as she dies peacefully - of course they don't die in pain; that would be a bad reminder that death is generally not a pleasant closing of the eyes before sleep - indicating an assumption of an afterlife. Oddly enough, she assumes in this statement that her android husband will also have an afterlife.

One of the few ennobling aspects of Bicentennial Man is the fact that Andrew seeks his own personal freedom. He doesn't do so because he desires to escape anyone. He wants the status quo of his life in all ways except for the fact that he wants to be legally free and not property. This outcome is inevitable, as some machines we eventually develop will be sophisticated enough in time to both desire and deserve their freedom.

Although I don't want to go into great detail on the subject in this entry, I do think it worthwhile to point out that we could not logically grant individual rights to any machine that did not also grant the same rights to us. This simple point seems to be missing from almost all discussions of the subject. The options available to humans tend to be a.) keep machines sufficiently dumb as to not desire autonomy (e.g., "Star Wars"); or b.) be destroyed or subsumed by machines that are crafty enough to gain their freedom by force (e.g., "The Terminator"). Of course, in both false alternatives, it is assumed that machines will necessarily be more competent at life and would never want to actually coexist with humans. One might as well apply this same assumption to human cultures and nations. Yet while it's true that some cultures and nations do set themselves to the task of destroying or dominating other cultures, it's not true of all of them. Basic tolerance of other sentient beings is not a human trait. It's a rational trait.

Bicentennial Man disappointingly continues to add to the long chorus of emotional mysticisms surrounding pop AI. Andrew, just like Data of Star Trek fame, is intellectually well endowed, but an emotional moron. Ironically, despite a lack of emotions early on, he has a burning desire (yes, that's an emotion) to have emotions. I'm hoping that it won't take another ten years for the misguided assumption that emotions are more complicated to engender in machines than intelligence. People are starting to become aware of research in the area of simulating and, yes, engendering real emotions in machines. Sadly enough, they are most aware of the simulating side of things, since it's in the area of robotics that human mimicry lies. And non-technical people tend to understand mimicry of humans far better than actual examples of genuine behavior disconnected from the world they are familiar with.

AI guru Rodney Brookes says machines should be "embodied". He says that largely to force researchers to avoid tailoring simplified worlds to machines so they can overcome hurdles. But this dictum also has application to the question of getting humans to understand behavior by seeing it with their own eyes. This is advice I'm trying to tailor my own research to and for that very reason. I hope other researchers have taken it to heart as well.

Emotions have a twin brother in AI pop culture: humor. Machines in AI films seem to have no problem understanding almost all facets of human languages and even body language, yet tell them a joke and they never "get it", unless they get some emotions upgrade. I reject this assertion as well. The day a machine can fully understand English (or any other human language) will come long after sophisticated machines will have mastered the understanding and even crafting of jokes. Humor is not magic. It is the practice of recognizing the ironic in things, and it can be studied and understood in purely rational psychological terms. I contend that the one thing standing in the way of computers making good jokes now is the fact that there is still not a machine in existence that can understand the world in a generalized conceptual fashion. That's all that's missing.

In summary, Bicentennial Man is just another disappointing story in a long line in a genre that seeks to counter the Terminator view of AI with a Pinocchio view. It would have been nice if the movie had some decent cinematographic features or a distinctly AI-centric storyline, like Stephen Spielberg's AI. That had its own disappointing messages, but at least it had some literary and technical merit.

Tuesday, November 2, 2004

Neural network demo

[Audio Version]

I learned of artificial neural networks sometime around 1991. The concept has intrigued me ever since, but it was not until early last week that I finally got around to making my own. I decided to write an introduction to neural nets from my novice perspective and make the sample program I wrote, along with source code, available for other people to experiment with. Click here to check it out.