Walter Pitts and Neural Nets

Pitts is one of the least studied geniuses of the early information age. See also Wikipedia, Nautil.us.

Cabinet Magazine: There are no biographies of Walter Pitts, and any honest discussion of him resists conventional biography. Pitts was among the original participants in the mid-century cybernetics conferences, though he began his association with that group of scientists when he was only a teenager. His intellectual strengths contributed to some central cybernetic theories reliant on logical structures and universal elements. Yet the sketchy details of his life provide the kind of evidence that resists the structure of those theories.

The mid-century cybernetics conferences, or Macy Conferences (1946-53), theorized about automatic or self-balancing systems in biological and technological ecologies. Warren McCullough, Norbert Weiner, John von Neumann, Margaret Mead, and Gregory Bateson were among those invited, and their discussions mixed methodologies and evidence from a broad range of disciplines including anthropology, neurophysiology, mathematics, logic, and computational networks. ...

Pitts entered into the company of this group as mysteriously as he left it. None of his colleagues knew very much about him. He came from a reportedly troubled working class family in Detroit and entered McCullough's home as a live-in collaborator when he was only seventeen. Even at this early age, he had already worked with prominent scientists in logic and mathematical biology. Three years later in 1943, he co-authored, with Warren McCullough, a theory of neurophysiological organization that would provoke sustained debate throughout the years of the cybernetics conferences. The paper proposed a logical structure for neural nets as coded circuitry, thus supporting machinic theories of mind as well as ambitions involving artificial intelligence. Pitts was only in his 20s during the cybernetics conferences of the 1940s and though he was an autodidact, he had already mastered or could almost instantly absorb the subject matter from the several fields of study engaged by the conferences. Not only did he circulate among prominent scientists, but those scientists either rivaled for his collaboration or deeply valued his critique and commentary. Pitts could, without fail, identify errors in logical thinking and he simply did not pursue information or discourse outside of the realm of logic. He considered the search for universal elements of mental structure to be not just a species of speculation, but a hard science of the brain and the mind. ...

Several of the scientists and psychiatrists of the group thought Pitts was schizophrenic and potentially very ill. The prominent psychiatrists who moved in his circles were more and more baffled by his reclusive shyness and his apparent personal discomfort. Later Pitts began to live on his own in Cambridge where he may have experimented with homemade drugs. No one seems to know much about his later life. He died in 1969 and some have speculated that he committed suicide. ...

McCulloch–Pitts (MCP) neuron:

Overview: In 1943 Warren S. McCulloch, a neuroscientist, and Walter Pitts, a logician, published "A logical calculus of the ideas immanent in nervous activity" in the Bulletin of Mathematical Biophysics 5:115-133. In this paper McCulloch and Pitts tried to understand how the brain could produce highly complex patterns by using many basic cells that are connected together. These basic brain cells are called neurons, and McCulloch and Pitts gave a highly simplified model of a neuron in their paper. The McCulloch and Pitts model of a neuron, which we will call an MCP neuron for short, has made an important contribution to the development of artificial neural networks -- which model key features of biological neurons.

The original MCP Neurons had limitations. Additional features were added which allowed them to "learn." The next major development in neural networks was the concept of a perceptron which was introduced by Frank Rosenblatt in 1958. Essentially the perceptron is an MCP neuron where the inputs are first passed through some "preprocessors," which are called association units. These association units detect the presence of certain specific features in the inputs. In fact, as the name suggests, a perceptron was intended to be a pattern recognition device, and the association units correspond to feature or pattern detectors.

This is an old interview with McCulloch. Nature vs Nurture and IQ starting at ~4min ;-)


Warren Sturgis McCulloch Interview from paul pangaro on Vimeo.

Nautil.us: ... Thus formed the beginnings of the group who would become known as the cyberneticians, with Wiener, Pitts, McCulloch, Lettvin, and von Neumann its core. And among this rarified group, the formerly homeless runaway stood out. “None of us would think of publishing a paper without his corrections and approval,” McCulloch wrote. “[Pitts] was in no uncertain terms the genius of our group,” said Lettvin. “He was absolutely incomparable in the scholarship of chemistry, physics, of everything you could talk about history, botany, etc. When you asked him a question, you would get back a whole textbook … To him, the world was connected in a very complex and wonderful fashion.”

... In a letter to the philosopher Rudolf Carnap, McCulloch catalogued Pitts’ achievements. “He is the most omniverous of scientists and scholars. He has become an excellent dye chemist, a good mammalogist, he knows the sedges, mushrooms and the birds of New England. He knows neuroanatomy and neurophysiology from their original sources in Greek, Latin, Italian, Spanish, Portuguese, and German for he learns any language he needs as soon as he needs it. Things like electrical circuit theory and the practical soldering in of power, lighting, and radio circuits he does himself. In my long life, I have never seen a man so erudite or so really practical.” Even the media took notice. In June 1954, Fortune magazine ran an article featuring the 20 most talented scientists under 40; Pitts was featured, next to Claude Shannon and James Watson. Against all odds, Walter Pitts had skyrocketed into scientific stardom.