AI Trailblazers:
EXPLORING THE LEGACY OF ALLEN NEWELL AND HERBERT SIMON
NIKI KAPSAMBELIS
At a commencement ceremony for doctoral candidates in the 1990s, Manuela Veloso remembers vividly the keynote speaker: Herbert Simon, Nobel laureate, Turing Award winner, artificial intelligence pioneer and one of her most influential mentors at Carnegie Mellon University.
She recalls how Simon, a giant in the field, exhorted the new Ph.D.s: “Do not live your life as a zero-sum game.”
Researchers, he explained, do not need to win at someone else’s expense. In other words, “Everybody has some value,” Veloso paraphrased.
At the time, she was a young faculty member; today, she is head of AI research for J.P. Morgan Chase and Herbert A. Simon University Professor Emerita. And she credits the influence of Simon and his frequent collaborator, Allen Newell, with shaping her approach to problem solving.
Veloso is not alone, but quite possibly, many AI students and researchers who carry the torch lit by Simon and Newell are not fully aware of the breadth of their influence. At the northwest corner of the Carnegie Mellon University campus stands a buff brick building, Newell Simon Hall, that is the physical embodiment of their legacy; the work undertaken within its walls, and by the people who have passed through its doors, stands on the foundation of ideas that Simon and Newell championed.
Yet Raj Reddy hypothesizes that most of the people who now work in that building, which greets visitors with a robotic receptionist, have only a vague idea about who either man truly was. Reddy, the Moza Bint Nasser University Professor, counts himself among those who not only worked with Simon and Newell, but was directly impacted as they built their legacy.
“Basically, both of them had one foot in cognitive science and the other foot in AI,” Reddy said. “They were trying to build models that would explain how a human being thinks and acts.”
Reddy arrived at Carnegie Mellon 54 years ago, long before there was a buff brick building, or a robot receptionist or a Robotics Institute (of which he would become the founding director). There was no School of Computer Science, for which Reddy later would become dean; there was just a small computer science department with a handful of professors and a few more associates.
Within the span of a decade, all of that would change dramatically, and Allen Newell and Herb Simon were the catalysts. Exuberant in their collaboration, excited about ideas, curious not only about their own research but also about the world, they would together help create an entirely new discipline that would touch every corner of our culture — while also asking the critical question of how it would impact human lives.
Manuela Veloso, Herbert A. Simon Professor Emerita
Raj Reddy
“Both of them had one foot in cognitive science and the other foot in AI.”
THE THINKING MACHINE
Newell met Simon, a political scientist 11 years his senior, at the RAND Corporation, a nonprofit think tank dedicated to research and global policy. At the time, they were focused on air defense systems. And in the early 1950s, many considered computers to be glorified calculators, meant to crunch numbers and little else.
Their epiphany happened when they realized that computers could represent and manipulate symbols and weren’t just limited to numbers. In fact, computers could interpret patterns based on prior experience, which humans also do. From there, Newell and Simon hypothesized that computers could simulate decision-making.
“That seemed, to me, a tremendous breakthrough,” Simon told the Pittsburgh Post-Gazette in 2000. “And one of the first rules of science is if somebody delivers a secret weapon to you, you better use it.”
“One of the first rules of science is if somebody delivers a secret weapon to you, you better use it.”
Through the fall of 1955, they created a program that would allow a computer to “discover” the proofs of geometry theorems; by the end of December, they got it to work, prompting Simon to tell his students in January 1956: “Over Christmas, Allen Newell and I invented a thinking machine.”
In the years that followed, their collaboration would generate a field that expanded well beyond the world of geometry problems, bringing to bear concepts that were the stuff of science fiction when they started; but always, in the DNA of each new initiative of artificial intelligence, lies a germ of the seed they planted.
Nearly a half-century and a Nobel Prize later, Simon would continue to frame their AI work, characteristically, through the lens of what it meant for people; he still didn’t want research to be a zero-sum game. He wanted to give people the tools to harness the broader world of knowledge that surrounds them: “Human knowledge has been changing from the word go,” he told the Post-Gazette. “One of my big interests has been to see how we can give computers those capabilities. Because I don’t care how big and fast computers are, they’re not as big and fast as the world.”
AN ENDURING LEGACY
For Tom Mitchell, Founders University Professor, the application of human cognition to machine learning has long been a hallmark of Carnegie Mellon — one that still influences his own work. For example, one line of his research compares brain scans of people looking at sentences that also have been shown to an artificial intelligence program, similar to an early version of ChatGPT. The computer was able to predict the brain activity of the person reading the sentence.
He recalls a course that he team-taught with Newell and former SCS professor Geoffrey Hinton (who would later win the 2018 Turing Award) that focused on architectures for intelligence. The concept was to build integrated AI systems that incorporated things that humans do: seeing, hearing, planning and playing games, all while improving automatically over time from experience.
“It was one of the most fun things I had done, academically, to that point,” Mitchell said, adding that it was one reason why he decided to stay at Carnegie Mellon after first arriving as a visiting professor.
And today, that same work is reflected in the Robotics Institute, he noted. “Robotics ends up being a great driver for looking at how to pull together all these different aspects of intelligence,” said Mitchell.
Newell and Simon’s work in vision and perception provided the foundation for Martial Hebert’s research in object recognition, scene understanding and perception for autonomous systems, enabling robots to perceive and interpret their environment. Takeo Kanade’s development of algorithms that assist in object recognition, image understanding and autonomous navigation align with Simon and Newell’s emphasis on creating perception and cognition in machines.
Their legacy touches virtually every corner of the School of Computer Science: that robot who greets visitors at the entrance? It uses research drawing from their emphasis on understanding human behavior. Chris Atkeson, professor in the RI and HCII, focuses on human-robot interaction, building on the principles of cognitive architectures and human behavior that Simon and Newell emphasized. Decision-making and bounded rationality, two other hallmarks of Atkeson’s research, inform assistant teaching professor Stephanie Rosenthal’s development of algorithms that allow for planning and scheduling in changing environments.
Tom Mitchell (right) pictured with Marcel Just, D.O. Hebb Professor of Psychology, University Professor of Psychology.
“Robotics ends up being a great driver for looking at how to pull together all these different aspects of intelligence.”
MARY SHAW
AI INNOVATOR
Mary Shaw, the A.J. Perlis Professor of Computer Science, has spent her career pioneering the fields of software engineering, formal methods and computer science education. With a distinguished career spanning decades, Shaw has been an innovator in the field through her research, teaching and leadership.
Born in 1943, Shaw earned a bachelor’s degree in mathematics in 1965, followed by a master’s degree in computer science in 1968.
Mary Shaw was a driving force in the development of formal methods, which involve mathematical techniques to ensure the correctness of software systems. Her work improved software reliability and quality through rigorous analysis and verification methods. She played a crucial role in the development of the Software Engineering Institute (SEI) at Carnegie Mellon University, where she led the effort to create the Capability Maturity Model (CMM). This model became an industry standard for evaluating and improving software development processes, promoting best practices and organizational maturity.
Beyond her research, Shaw has been deeply involved in reshaping computer science education. She has advocated for a holistic approach that combines theory and practice, emphasizing the importance of both technical skills and problem-solving abilities. Her educational initiatives have influenced curricula worldwide, ensuring that future generations of computer scientists are well-prepared to tackle real-world challenges. Her emphasis on rigorous methodologies and formal analysis has had implications for AI systems’ design, verification and safety. As AI technologies continue to advance, Shaw’s work remains relevant in ensuring the reliability of AI applications.
Shaw was named an IEEE Fellow in 2010 and is an ACM Fellow. She received the 2011 ACM SIGSOFT Distinguished Service Award for her outstanding leadership in software engineering. Shaw was awarded the National Medal of Technology and Innovation in 2014, the highest honor for technical progress in the U.S.
WHAT IS RESEARCH?
Veloso remembers distinctly a lecture Newell gave in December 1991 called “Desires and Diversions,” given seven months before his death, when he talked about his life in research: how he was driven by the overarching goal of understanding the human mind, but a handful of diversions from that goal produced major scientific achievements on their own.
“My research was a product of that lecture,” Veloso recalled. “I never read science fiction; I never cared about robots.”
But the idea of exploring and integrating different functions intrigued her, and she was off on a career of her own, developing intelligent robots capable of learning and interacting with humans and their environment.
She credits Simon with helping her frame problems. From her arrival at Carnegie Mellon in 1986 until Simon’s death in 2000, Veloso estimates she sat in 14 of his lectures that asked the question: What is research?
“He was very consistent in saying that basically the research needs to be something that was novel, in the sense that other people had not done this. People had to care about it. And you had to publish. You had to let other people know what you were doing.”
Veloso still applies these principles to her work at JP Morgan Chase, and she believes that Simon, who died 18 years before she took her position there, would have been proud of her. She would have loved to discuss the role of AI in organizations with him.
She met with Simon often as a young faculty member. She remembered an occasion when she was upset — a paper was rejected, or some other aspect of her career had not gone as she’d hoped. Simon asked her what was wrong.
“Herb, you have a Nobel Prize. You won’t understand,” she said.
But Simon did understand. He told her she was mistaken to assess the worth of her work based on outside opinion; instead, she should know herself if her work was good, independent of whether it was accepted or rejected somewhere.
Years later, not long before he died, Simon was asked whether people would become expendable if computers could think for themselves. His answer hewed closely to what he told Veloso about knowing her own worth.
“Computers thinking don’t make you expendable,” he told the Post-Gazette. “Maybe we ought to think of worth in terms of our ability to get along as a part of nature, rather than being the lords over nature.”
Furthermore, he added: “Technology may create a condition, but the questions are, what do we do about ourselves? We better understand ourselves pretty clearly, and we better find ways to like ourselves.”
“Both Simon and Newell embraced an open-mindedness that continues today.”
A THOUSAND FLOWERS
Both Simon and Newell embraced an open-mindedness that continues today, according to Raj Reddy: “It is the culture of the whole school; how we act, and behave, and think and empower people.”
Newell got up early every morning, working at home before coming to the office at lunchtime. For the rest of the day, all he did was meet with people and give them advice: “It didn’t have to be AI. It could be anything,” Reddy said. “That made a major difference to all the younger faculty members in the department, me especially.”
Mitchell recalls how Newell and his wife bought him housewarming gifts when he moved to Pittsburgh and made sure he felt comfortable. A tall man with an engaging smile, Newell was always happy to see people, always asking what they were interested in or working on.
“He cared what was going on in your life, and he wanted all that to be good,” Mitchell said. “He wanted to find places where you and he didn’t think the same thing. He was always searching for where you didn’t agree with him. And he’d say, ‘Well, that’s interesting. Let’s dig into that.’”
In “Desires and Diversions,” Newell gave his audience several maxims that he lived by. One of them still resonates today, made even more poignant by the fact that he died a few months later: “Choose a final project that will outlast you,” though he tempered it with another maxim: “Everything must wait until its time.”
For Newell, science was the art of the possible, and the role of the scientist was to pass the torch so the idea could expand to its next incarnation, when the time was right.
“They were eclectic. They were broad enough in their thinking,” Reddy said. “They were saying: ‘Let’s spread our wings and let a thousand flowers bloom.’”
To this day, the multidisciplinary approach might be the most enduring legacy of Simon and Newell, because it extends across Carnegie Mellon’s campus, not just in the halls of the building that bears their name. And the principles that guided their early “thinking machine” are infused in every corner of society around the globe.
It is, for both men, the project that outlived them. ■
Allen Newell and Herbert Simon