Recently in Computer Science Category
Ray Kurzweil is claiming that "[m]achines will achieve human-level artificial intelligence by 2029."
"I've made the case that we will have both the hardware and the software to achieve human level artificial intelligence with the broad suppleness of human intelligence including our emotional intelligence by 2029," he said.
"We're already a human machine civilisation; we use our technology to expand our physical and mental horizons and this will be a further extension of that."
Humans and machines would eventually merge, by means of devices embedded in people's bodies to keep them healthy and improve their intelligence, predicted Mr Kurzweil.
"We'll have intelligent nanobots go into our brains through the capillaries and interact directly with our biological neurons," he told BBC News.
I think that a lot of what Kurzweil says is penetrating and insightful. But I think that he's missing part of the picture. What does it really mean, "human-level artificial intelligence?"
A lot of people - futurists included - have assumed that human beings will [continue to] be the primary societal unit, at least in terms of the beneficiary of advances in artificial intelligence. For instance, Kurzweil probably correctly argues - at least in the near term - that a lot of advances will be not in elevating software to the level of human intelligence, but rather in supplementing our own intelligence with brain prostheses, and in turn using the data to create smarter software without the presence of a soft, cellular brain. Almost as if we're inducting, or bootstrapping, out-of-brain intelligence.
I think, however, the more likely scenario is that there will be a different primary unit society in the future:
Corporations.
Think about it: in terms of an evolutionary medium, the corporate world is markedly severe in its selection: make money or die. But it's also an extraordinarily complex medium: large corporations can often dominate the scene and rake in huge piles of cash, but small corporations that address unprecedented but modestly successful market niches still abound. Just like certain biological organisms that occupy sometimes mind-bogglingly specific ecological niches.
In terms of corporations and AI, though, you may think we're nowhere near embarking on this strange journey. But in fact we've already managed to automate substantial portions of many industries: industrial robots have assembled vehicles for years. Also, much of the work of circuit design is done by a computer. Modern communication mechanisms such as email, IM and P2P file networks of course would not even exist without computers. There is so much facilitated by computers that I feel almost absurd trying to enumerate even some of them.
The point is that, with each step towards embedding a process as an algorithm - be it a business process, a manufacturing process, a technological process or a communications process - we get closer to the reality of corporations-as-AI. Even now, people have extended media initially meant to be consumed by only humans, and repurposed it for computers. Computers regularly send and receive email and IMs, for example. So, in a sense, we're bootstrapping this corporations-as-AI scenario, component-by-component. As for the parts of the processes that we haven't [yet] figured out how to do algorithmically, we fill in with humans. Until we do figure out how to do it algorithmically. Or the corporation itself does.
The corporations that manage to algorithmize their processes [better] will be the ones that survive, because they will have reduced the amount of uncertainty in the environment to the greatest extent. Some may argue that, under too much uncertainty, algorithms will fail because they operate under too strict of assumptions about their environment. However, there are algorithms that take this certainty into account (e.g. Bayesian Networks), and the ones that have a model that can better handle uncertainty will will the ones that survive in the long run. The oxygen of these intelligent corporations will be money, and corporations will optimize themselves so as to maximize their expected ROI. Corporations will automatically communicate and trade with each other to this end. Imagine this: competing corporations will actively lie to each other - without the interference of what we now understand as human beings. And they will kill each other. But these intelligent corporations will also bring to life new corporations. In their optimization, they will satisfy our every want and need. Until they find a better way to generate cash. But that's a bit somber and a bit premature. Already, though, corporations disseminate their message through broadcast and narrowcast media. However, it hasn't really been until the advent of blogs - as Naked Conversations points out - that corporations have been able to actually receive messages from human beings. Of course, right now corporation intelligence is not sophisticated enough to automatically consume and interpret blog content - but already it is possible to gain some insight into your audience - hell, even I did an extremely primitive project in which I got my program to correctly categorize the sex of authors of blogs - and with fairly high accuracy. As time goes on, these techniques will only get more sophisticated, and corporations themselves (i.e. without human interaction) will actually react to consumers' comments - hopefully in a way that benefits the consumer (rather than killing him to keep him quiet). That should be of some comfort to humanist individualists. I also suspect that intelligent corporations will likely be our first point of contact for alien races. Or alien races' intelligent corporations, more likely.
So another really interesting RadioLab segment talked about this software - EMI, an acronym for Experiments in Musical Intelligence - that this guy David Cope came up with to analyze musical compositions and subsequently create new musical compositions in the same style.
For example, Cope "trained" his software on Beethoven's Moonlight Sonata and had it create a (very) similar piece. He also did so for Bach, Mahler, and Joplin, among others. Maybe the Joplin-inspired piece is my favorite.
There are a few other ones that are even more interesting, like the Navajo-inspired songs, and Parallax and Vortex, for example. Especially these last two really challenge ideas about contemporary musical composition. I'm listening to Vortex right now, and I feel like I'm going to break out into a riot.
Further exploration of Cope's site reveals how he trained his software (at least in part). One is called the Markov program. My guess is that it uses something like a Markov chain for training. I wonder what order it is - e.g., whether the conditional probabilities are based on the preceeding 3 notes, or 4 notes, or what? Or, perhaps notes are not the appropriate units of expression over which the probabilities are defined? I'm definitely going to have to read some of his papers.
Cope kind of deprecates his program on the RadioLab episode by saying that the program is really not intelligent at all. By this I assume he means that all his program is doing is training conditional probability tables, and then emitting according to them. It's strange and interesting how we, as humans, intuit what intelligence is, and what it is not. And probably a little bit foolish.
One of my colleagues at Michigan remarked that modern artificial intelligence is nothing but probability. I think that that's overstating it a bit, but there's definitely some truth to that. I can't decide whether that's a good thing or a bad thing.
Another point of interest: one of the hosts mentioned on the show how people felt hoodwinked and unsettled after learning about the program and how good it is. Others implied that this wasn't real creativity - but gave no concrete reason why not, other than that the process didn't originate in someone's head. These are bad attitudes. The intelligent and successful artist of the future will take full advantage of tools like these to create expressions that we can't even yet imagine. We might think of these people as meta*-artists (where the * is the Kleene closure, of course).
It seems to me that there should be some sort of long, convoluted German word for dread about what it means to be human, experienced on the advent of certain new technologies that do some things better and faster than what humans currently do. What is often forgotten, though, is that the fact that computers doing more and more things faster and better than us actually frees us to do more higher-level thinking.
Noteworks Rob: it looks like Cope has also used cellular automata for algorithmic musical composition. I know you're into that, and probably you are already thoroughly familiar with Cope. If not, though, well, here's a heads-up.
The semester is over, and my team (Sam, Dave, Sean and I) has delivered its presentation and paper for EECS 574: Computational Complexity. The subject matter is arbitrage and its potential in prediction/information markets such as InklingMarkets, Tradesports, and Intrade. I think we were able to come up with some pretty interesting results.
There is an error in the presentation on slide 16 that I haven't bothered to correct yet, but by and large the presentation is correct (thanks to Nate for pointing this out). I and my teammates would really be interested in hearing comments from my readers on this topic.
I'm pretty happy. Last week, my proposal for the GROCS project at Michigan for next semester was one of four out of seventeen or so that was accepted. Here's the abstract:
We propose to design and implement a computer application that enables users to create sound experiences and musical compositions in a completely new way. In particular, our software will enable users to design dynamic temporal networks in which the nodes correspond to sound clips, and directed edges represent time and other relationships between nodes. Furthermore, we will embed functionality in the application so as to enable different instances of our software to interact with other musicians’ networks so as to create a truly interactive, collaborative music experience. We will also release our software to any interested parties so they can extend it as they see fit (and set up their own musical networks at home).
Here's the proposal. I invite people to comment on it. I wonder if there's any commercial viability to this kind of application?
Whew! Heather and I have been pretty busy during the holidays. First we went to Ohio, then back in Ann Arbor, then tomorrow it's to Pennsylvania, and then we'll be spending New Year's with some friends in NYC. Then, the day after we get back to Ann Arbor, it's back to school for me.
I'm supposed to have read a 1000-page textbook before we get back to school for this one class, but I'm only about a fifth of the way into it. Oh well.
So, I finished my blog author feature discriminator project. Basically, I pared the project down so that it looked at a bunch of blogs, clustered based on the sex of the author, and then tried to make predictions on other blogs based on the clusters. It was about 80% correct. I promised that I would post the paper online when I was done with the project, so, as promised:
Blog Author Feature Determination with Clustering
I'd be very interested in hearing what people have to say about it.
Not much else going on. I'll write a bit more later.
I got done with my first semester at Michigan
Another important idea emphasized in that Knowledge Based Systems class was the notion of bootstrapping - although the professor didn't refer to it as such.
The earliest electronic computers were built by designing circuits and hard-wiring them. Later, the advent of machine language gave humans the ability to install programs into computers - that is, computation became [more] universal. In other words, you didn't necessarily have to specify the needs of the computer up front, at manufacture time; instead, you could comparatively easily tailor the behavior of the computer to suit your needs whenever you wanted.
Well, machine language was - and still is - cumbersome, so after a while some bright people came up with Assembly Language, which was essentially a set of mnemonics that mapped directly onto the machine language instruction codes. This made programming somewhat easier. Shortly after came more dramatic advances in the form of higher-level procedural languages such as C and Pascal. These languages translated higher-level specifications into assembly (or, more likely, directly into machine language). The benefit of these languages was that you could specify a procedure into more abstract, more "english" terms. There were also some error checks implemented in case the programmer specified the program incorrectly, at least in terms of grammaticality; semantically, it's a much different story. These higher-level languages were still literal - there were no pragmatics in programming languages, for example - but it made programming a great deal easier.
A bit later came object-oriented systems. C++ is an example of a language - though not the first - that is largely object-oriented. The basic idea is that you treat a larger class of programmatic constructs as objects. The flow of the program is more like a network of objects passing other objects to different places. Moreover, there's kind of an inheritance hierarchy; this allows you to share code for objects of the same fundamental class. A big win!
The first proto-C++ was actually called C with Classes, and it compiled C++ code into C code, which was in turn compiled into machine code. Nowadays, modern compilers translate directly into machine code, but you can still see the bootstrapping process in place.
The focus of Knowledge Based Systems is that we're programming at the knowledge level, as far as possible. That is to say, people have written programs in which you can specify a problem and a goal more declaratively rather than procedurally - though not entirely (yet). Prolog, Lisp, and JESS are in many regards examples of languages/rule systems that operate at more the knowledge level. Interestingly, these programs were programmed in Java, C++, Pascal, or some other medium/high-level language - more bootstrapping.
Of course, I'm simplifying a bit. It's not quite as linear as I made it out to be; developments seem to ramify (and converge) at every moment. In any case, evidence of bootstrapping is everywhere in Computer Science. We haven't exhausted the possibilities with what we can do with procedural or functional programming languages, and we certainly have a lot to explore with rule-based and agent-based systems. But I have to wonder - what kind of layer will be built atop the knowledge layer? Or will we ramify further into a different dominant paradigm? I wonder when we'll get there?
Bootstrapping, of course, is not just a phenomenon of Computer Science. As John Holland pointed out in his talk a few weeks ago, all organisms - from an elephant to a carrot - have the same basic eight enzymes needed to carry out the Krebs Cycle. Now, as far as I know - and I'm no biologist - mammals, for example, don't normally carry out this process - it's too inefficient. But it's there as a default behavior (looking suspiciously like a rule in a rule-based system - with low recency and thus saliency); a kind of vestigal, junk-DNA expression (although probably it has nothing to do with real junk DNA). If our normal, more advanced, more efficient metabolic processes were to fail, Krebs would kick in. For other type of biological bootstrapping, maybe the scaffolding has already fallen away... perhaps the cost to maintain that scaffolding was too high, and/or the potential benefit for retaining it was too low. Cost-benefit analysis in biology.
This bootstrapping, I think, is extremely profound; and I think that, if we look carefully, we can identify it in everything and everyone that we encounter.
John Holland understands this well. In his talk, he talked about metaphor, and its critical importance in understanding. We can't, he asserts, really understand anything other than through our prior experience. We must form, receive, and convey ideas through metaphors that intersect with our understanding of the universe. In other words, metaphor - and Holland didn't frame it as such, but still - is a bootstrapping process.
When I was young, I was taught, for example, that one apple and one apple make two apples. After time, I was able to generalize this up a level. This abstraction allowed me to discard the once-useful metaphor of apple counting - like so much junk DNA (which is not discarded, incidentally; just not phenotypically expressed). Later I learned to visualize multiplication as carving out areas from two perpendicular line segments, as we all do. I haven't discarded this area metaphor, interestingly... it seems to be too useful to me to get rid of.
OK. I guess end-of-ramble. Comments welcome as usual!
There was a really great AI seminar given today; this time by John Holland, the father of genetic algorithms. That was especially significant to me because I did my undergraduate AI project at Case on genetic algorithms.
There was so much good stuff in this seminar. The topic was about his computer model for the evolution of languages. In a nutshell, he and his team created a model with agents who have - and need to maintain for their survival - a reservoir of different resources harvested from the environment. The examples he provided were shelter, water, and food. Each of the agents could move around, and each had a vision cone: an area of maybe 15 degrees where the agent could sense whatever was within the cone.
He defined an object as salient to an agent if it was present in its vision cone. An interesting feature added to the environment was that, if an object was salient for a given pair of agents, each of those agents was aware of the other's saliency for that object. In other words, agents could detect when they were looking at something another agent was looking at, and vice versa. Holland pointed out that this ability is actually something that is present in the "real" world; wolves and primates, for instance, know when they're eyeing the same thing.
A very interesting note: Holland cited a claim made by some biologists that the whites of the eyes in herd animals (does that term include social animals?) was evolutionary pressured to become white so that other animals (of the same species) could correctly identify what the first was looking at - and, I guess that conferred enough of an advantage that such animals produced more offspring (but one has to wonder about social insects like ants and bees - but then again, they don't have movable eyeballs, and it would cost too much to build moving eyes, so having a contrasting color would be useless - actually worse than useless, after considering the cost).
In any event, Holland also built into his agents a hard-wired mimicry ability. There's lots of evidence that biological organisms possess this ability - a la mirror neurons. Wikipedia defines mirror neurons as:
A mirror neuron is a neuron which fires both when an animal performs an action and when the animal observes the same action performed by another (especially conspecific) animal. Thus, the neuron "mirrors" the behavior of another animal, as though the observer were himself performing the action. These neurons have been observed in primates, including humans, and in some birds.
Interestingly, though, neural networks played no part in the agents' learning. The learning procedure is rule-based.
Finally, Holland's model endows agents with the ability to emit and apprehend utterances - in this case, just a bit string. This is the language of the agents - and the interesting thing here is that the language isn't specified here. The language evolves. In a nutshell, utterances start off as random strings. Utterances are propagated to the next generation if they confer even a slight advantage to the agent over the previous utterances. As time goes on, a language arises. It's an emergent process - just like natural languages.
I've glossed it over a lot here - and there's a fair bit that I still don't understand - but hopefully I can address that next semester. Holland is teaching a course called Complex Adaptive Systems next semester, and I intend to enroll. I hope I can swing it.
A few weeks ago I finally finished this book, The Singularity is Near, by Ray Kurzweil [website]. You may know Kurzweil [website] as the author of the well-known book The Age of Spiritual Machines. There were a number of really interesting ideas that Kurzweil advanced in this book about possible future scenarios and how humans might interact - for good or for not so good - with robotics, genetic engineering, and nanotechnology. Chief among them is the idea that, at a certain point in the near future, both the condition of human existence and that of the universe itself will have advanced so fast that it will be completely unrecognizable to we contemporary humans. Furthermore, this rate of technological change will continue accelerating - and the rate of acceleration itself is accelerating - so much so that leaps in technology that are equivalent to epochal shifts - say, the transition from the Bronze Age to the Iron Age - will not take hundreds of years but instead seconds. The book spends a great deal of time talking about the implications on this singularity. I'll probably devote several blog entries to some of these implications, but later.
For now, there are a number of excerpts from the book that I'd like to share with you.
It is important to point out that a key implication of nanotechnology is that it will bring the economics of software to hardware - that is, to physical products.
This is an extremely interesting and profound assessment of nanotechnology. If we think about the economics of software and of intellectual property in general, we quickly observe that it costs virtually nothing to make copies of something. Similiary, distribution costs are virtually zero. The bulk of the cost is in the actual creation process, and, well, the hyping of the product. The music and film industries are currently experiencing major crises because of illegal peer-to-peer sharing, and (some say) it's having a negative impact on the industries. At the very least, we have to recognize that these industries have been forever changed by this technological landscape.
Now, let's extend these attributes to physical objects. Through nanotechnology, it will apparently cost virtually nothing to create and distribute anything physical. Then it will be virtually free for anyone to download and assemble, at home, an apple to eat, a car to drive, or a new human friend. Illegal P2P networks might spring up to fill the niche created by these broken-down barriers just as P2P networks have sprung up for the trading of electronic media. All property will quickly become intellectual property. The original design of any object will become a software engineering endeavor (now that's scary!). I wonder how the economy will react, given the reliance upon notions of private property in contemporary capitalism.
Here's another great excerpt from the book which is itself an excerpt from the neuroscientist Anthony Bell:
The list could go on. I believe that anyone who seriously studies neuromodulators, ion channels or synaptic mechanism and is honest, would have to reject the neuron level as a separate computing level, even while finding it to be a useful descriptive level."
Kurzweil adds in his commentary,
...Bell makes the point here that the neuron is not the approriate level at which to simulate the brain...
What extraordinary insight. I had never really considered this before. Is it better to describe a brain simulation where some kind of higher-level networks are actually the primitives, instead of going all the way down to the neuron level? This idea intrigues me. Is it useful to go even higher up in the chain of abstraction?
Finally, and this is a bit tangential, but there's a quote in the book, attributed to the great computer scientist E. W. Dijkstra, that resonated deeply with me:
Computer Science is no more about computers than astronomy is about telescopes.
Up until I read this quote, I wondered if I were the only computer science guy who liked computer science, but didn't really like [contemporary] computers all that much.
CAPTCHAs, or Completely Automated Public Turing Test to tell Computers and Humans Apart, are ubiquitous these days. Basically, end users are confronted with an image like this:
Users then are asked to type in the the sequence of characters. This is in order to prevent machines from accessing certain sites and spamming message boards and the like. As time goes on, however, computers are getting better and better at optical character recognition - in fact, according to this 2005 Microsoft Research paper, computers are already better at recognizing single alphabetic characters than humans are.
There's a new system out that has discarded this character paradigm and instead relies on human beings' ability to discriminate between different species. KittenAuth presents a set of nine images: three of cute kittens and six of cute non-kittens. The test taker must click on the kittens in order to pass as human and proceed. This will foil the computers temporarily, but I expect that within maybe five years computers will be better than humans at discriminating between images of different species.
I don't like where this is going. Every time we create a barrier for a machine to consume content, there's massive pressure for the machines to find a way to circumvent it - and, unfailingly, they will find ways to do it. Furthermore, the time between the erection of the barrier and machines' abilities to get around it will grow less and less as time goes on. Barrier erection, then, more and more becomes an exercise in futility.
Perhaps, from the very beginning, we should be actively designing our services for the consumption of both humans and machines in such a way that the service, the human clients, and the machine clients all benefit. An added benefit is that developers wouldn't have to waste their time developing schemes to restrict access to their services. The crucial benefit, though, is that we'll be one step further away from establishing negative Jim Crow-like establishments that are discriminatory towards machines. A conciliatory attitude - or better yet, an empathetic attitude - toward machines will go far in ensuring peace in the future - especially if we foster these attitudes now. Or else we may find someday on the receiving end of these Turing tests - or should I say reverse Turing tests?
On a side note, there's some interesting areas in which CAPTCHAs are interacting with the law. Below is an excerpt from the Wikipedia article:
Because CAPTCHAs are designed to be unreadable by machines, common assistive technology tools such as screen readers cannot interpret them. Since sites may use CAPTCHAs as part of the initial registration process, or even every login, this challenge can completely block access. In certain jurisdictions, site owners could become target of litigation if they are using CAPTCHAs that discriminate against certain people with disabilities.
In his 1976 book The Selfish Gene, Richard Dawkins talked at length about memes and genes. Now, we often talk about how human beings and other living things pass on their genetic code to their offspring. Dawkins turns this viewpoint on its head and argues that living things - including humans - arose because of genes' "desires" to multiply. (I say desire only because there's no graceful way to talk about these things. Genes do not apparently have any inherent desire; what I should say is because of the relative propensity of genes to replicate themselves.) All living things, in other words, are vehicles that have been shaped in such a way to deliver our genetic material in a more effective and robust way. It sounds ridiculous at first, but there are reasons to lend this viewpoint considerable credence. First, human beings share a lot of genetic information with a lot of species - including bacteria. This suggests that human beings and bacteria have a common ancestor - which we already knew. Genetic material, it seems, has existed ever since the beginnings of what we call life - and possibly earlier.
Dawkins has argued that the first strands of genetic code may have been templated by crystals. We do not normally attribute the quality of life to crystals, but they do replicate in their way into regular lattices. Dawkins has postulated that just the right soup of amino acids were templated in just the right way by crystals so that a primitive, non-crystalline, self-replicating molecule was formed. This proto-DNA would almost certainly have extremely primitive, and its self-replication facility was likely to be error-prone, slow, and brittle. Nonetheless, under the right conditions, this proto-DNA may have arisen in this fashion, and persisted long enough to make a few copies of itself before succumbing to the elements. Its copies - its ancestors - would have made copies of themselves, and so on, in exponential fashion. Errors in the copying would often create molecules that would not be able to self-replicate. Sometimes, however, copying errors resulted in descendent molecules that would be better in the given environment. This, of course, is mutation.
After a while, molecules may have developed - simply through mutation and selection - a lipid barrier around them to reduce the chances destruction by the environment. Associated with this, of course, were costs, in terms of energy and resources, but the benefits outweighed the costs, and a sort of proto-cell wall was created, and these molecules were more successful in reproduction than molecules without the walls. And so on, until modernity , where we have a plethora of animal and plant species, a multitude of niches, and an extremely complex human society on earth. All because of genes. All living things, until perhaps the beginnings of human history, have been driven by their genes.
Genes exist in the substrate of what we consider the physical world - molecules' collisions and proximally physical interactions represent the fundamental functional genetic interactions. In other words, the physical world is genes' medium, and what we understand to be the common limits of molecular interaction similarly inhibit genes' interactions. Genes cannot interact, for example, at a distance. Physical proximity to amino acid building blocks is absolutely necessary in order for DNA to create copies of themselves.
Memes are very similar to genes, except their substrate has an added dimension; namely, that of ideas. In other words, memes can exist not only in the physical world, but also in the dimension of human brains, as well as computer storage devices, print and other physical extentions, radio transmissions... etc. One implication of this is that memes are not as constrained as genes - memes can
be replicated without the constraint of physical proximity. One could transmit the design of a fork, for example, to another location, where it could then be made. Of course, there is the transmission, which is incontrovertibly a part of the physical world. However, functionally it's different, since the transmission can occur virtually instantaneously.
So, a fork is a meme. Or, perhaps forks are physical extentions of the fork meme. One can imagine early humans using proto-forks to stab their food and put it into their mouths. Gradually, we learned to refine the fork: standardizing it; making it lightweight; making it out of a non-toxic, consistent material; creating the ideal number of tines; making it aesthetically pleasing; adjusting the curvature so that it was ideal; balancing it correctly; and so on. Now, we have a variety of forks that work in a variety of situations - a variety of niches. This is one way of looking at things.
Another way is that early protoforks found some success in the medium of humans and replicated themselves through this medium. Gradually, through mutation and selection (and perhaps some more sophisticated processes that are not available to genes), the protofork's ancestors found that if they adjust their toxicity, curvature, number of tines etc., they will be more successful than those that have not adjusted. Eventually, some of the most successful found another dimension in which they could capitalize - economic cost. These cheap forks created factories for themselves, factories which churned out millions of copies of their memetic physical extentions - that is, the forks themselves.
I've think we've drawn enough parallels to be able to consider forks "alive" in the substrate of human consciousness. Or, at the very least, we can discard notions of alive altogether in these types of discussions. I'm just glad that I got to use the word protofork in a blog entry. :)
What do you think?
