Thinking Nets
review by Hans Moravec

How scientists are moving beyond computers to create a rival to the human brain
by David H. Freedman
Simon&Schuster, April 10, 1994 (200? pages)

World War II gestated two kinds of brain-like machine, analog computers to direct fire at airplanes and bombs from them, and digital computers to break codes, prepare tables and simulate atomic explosions. Analog computers acted like small bits of nervous system, digital computers mirrored mental abstractions. The approaches spawned independent post-war academic disciplines explicitly aimed at artificial thought, named by Cybernetics by Norbert Wiener, and Artificial Intelligence by John McCarthy. By 1960, two former classmates had become the rival spokesmen for the two approaches. Frank Rosenblatt at Cornell championed pattern-recognizing electronic nerve nets called the Perceptrons, while Marvin Minsky at MIT encouraged students to write thought-simulating programs for digital computers. In 1957 Sputnik, implying intercontinental nuclear missiles, put fear of technological surprise into the US defense community and encouraged it to substantially support far out university research. Thinking machines were a major beneficiary, but fixed funding totals created serious competition between research groups. Already a bit stagnant after a promising start, neural net research screeched to a stop with publication of Minsky's and Seymour Papert's Perceptrons, which proved fundamental limitations in two-layer nets, and convinced DOD research managers to divert funding to reasoning programs--notably at Minsky's lab.

Reasoning programs themselves stagnated in the 1970s, after promising initial results and inflated promises. In 1983 John Hopfield showed how to train three-layer nets, which overcame many Perceptron limitations, and by the late 1980s neural nets again had funding and a following. Reversing the situation of twenty years earlier, some young enthusiasts declared the reasoning program approach dead. Some reporters believed them.

Freedman, a science reporter, tells a good historical and technical tale. A few errors in names and events are confined mostly to the introductory chapter, written after the main text and apparently insufficiently fact checked. We learn about logic programs and expert systems as well as Perceptrons and the neural net "connectionism" of the 1980s. The narrow successes of the many expert systems in business are contrasted against the few applications and many promises of neural nets. Reasoning programs follow chains of inference along if-then rules extracted in tedious interrogations of human experts, while neural nets automatically learn to classify imponderable input patterns from examples. In detecting credit card fraud, for example, nets trained from millions of past transactions beat expert systems distilled from experienced credit analysts. Echoing the connectionist enthusiasts, Freedman argues that expert systems, and computers in general, are a dead end for machine intelligence, but that nets are up to the job. The book presents Doug Lenat's decade-long "Cyc" effort, to encode common sense in as many as 100 million expert system rules, as the last forlorn hope for reasoning programs.

I was not convinced, for reasons evident in my own field of robotics, as well as many commercial "intelligent" programs for writing and speech recognition, symbolic mathematics, language translation, game playing, industrial vision and more, which use pragmatic mixtures of numerical, statistical, inference and learning methods that are rarely either expert systems or neural nets. Expert systems and neural nets are nonspecific programming techniques for encoding decision criteria and learning input-output relationships, that are often inferior to more problem-specific encoding, inference and learning algorithms. It is highly improbable that future fully intelligent machines will be built with either technique alone, though both may be used in places. Blind biological evolution may be stuck with solutions once chosen, but intelligence-guided technology is not so limited. A weak versions of the book's position is justified. Early thinking programs were so small that it was possible to hand tailor every parameter and decision point. As computers and problem sizes have grown, it has been increasingly necessary and worthwhile to entrust the details to automatic search and learning processes, of all sorts. The amount of program construction and tuning delegated to the machine is sure to continue to grow.

Freedman relays the strange claim that nets will make computers obsolete, suggesting an image of soft, lifelike nets versus rigid, mechanical computers. In the 1960s, when computers were fabulously expensive, Perceptrons were analog devices, but in 1980s connectionists used digital computers grown cheap and powerful. Computers are, after all, universal machines, and can represent any degree of "softness" in fine resolution numbers. Most commonly, "nets" are simply matrices of interconnection weights adjusted by a training algorithm. Other many-parameter learning programs have similar structures--hidden Markov models learn matrices of transition probabilities, and high degree curve fitters tune arrays of coefficients. The few nets implemented as special chips don't alter this observation: other learning algorithms can likewise be committed to special hardware. It rarely pays however, because, in a world of rapidly accelerating universal computers and improving algorithms, speed advantage is temporary, but algorithm inflexibility costs permanently.

Freedman is a young reporter swept away by young researchers intoxicated by a hot idea. No harm done, but things are bound to look differently in daylight, after the party ends and the hangovers subside.


Hans Moravec is a Principal Research Scientist with the Robotics Institute of Carnegie Mellon University. He has been developing spatial perception for mobile robots for two decades, and promises practical results by the end of the third. He is author of Mind Children: The Future of Robot and Human Intelligence (Harvard 1988) and the forthcoming Mind Age: Transcendence through Robots (Bantam 1995).