Language
July 23rd, 2008 Posted in Artificial Intelligence, PsychologyEvery time I try to write some software to deal with language, I’m always amazed by how difficult it is. Humans rarely appreciate exactly how complex our language skills really are. For example, today I had to write code to compare strings of text to each other. The trick is to say “are these two strings representing the same thing?” It may seem simple to compare two simple sentences to each other, but it actually requires a huge amount of knowledge. For example, how can you deal with spelling mistakes? (“My nmae is” vs. “My name is”) and how can you understand synonyms (“I’m annoyed” vs. “I’m angry”) etc?
Of course, some of this can be done just by gathering huge collections of grammar rules together and going through them all methodically, but that’s nowhere near enough. As an example, think about the following problem: You are given a body of text and somewhere in that text is an address. Your job is to find that address with no extra information. How do you do it? Well, you could use postcodes as a marker, but that’s not perfect, and you can’t rely on there being one. You can spot words like “house”, or “flat”, but that’s not going to work if you find those words used in a grammatical sense such as “I saw a lovely house today”, or in a different context such as “I am lying flat on the floor”.
So how is it that a human can look at a body of text and identify special components like addresses so easily? It’s not even difficult for us, because we’re bringing so many different skills to bear on the problem. Not only do we have an intuitive understanding of grammar, but we also have a huge database of words that are commonly associated with buildings, and we understand the near-infinite possible permutations of language that could be used to introduce an address within a block of text. And we have many more skills to add to those.
The best way to learn to appreciate the human brain is, without a doubt, to try to copy it. Once you undretand the complexity of the tasks that our grey matter so effortlessly solves, I guarantee you’ll be utterly amazed.
