THE GLOBAL REACH OF CMU AI

SCS to Lead Four National Science Foundation AI Institutes

Chris Quirk

Global health care, the future of transportation, food security, consumer privacy in a networked world — as intractable problems accrue and grow, artificial intelligence is increasingly being called upon as part of the solution. Carnegie Mellon AI researchers have stepped up to help surmount these obstacles where large data sets must be analyzed and patterns discovered to find answers.

Now, the National Science Foundation has teamed with the U.S. Department of Agriculture, the U.S. Department of Homeland Security, as well as corporate sponsors Accenture, Amazon, Google and Intel to provide $220 million in grants to create 11 new institutes specifically dedicated to AI research across a wide range of sectors.

“I am delighted to announce the establishment of new NSF National AI Research Institutes as we look to expand into all 50 states,” said National Science Foundation Director Sethuraman Panchanathan. “These institutes are hubs for academia, industry and government to accelerate discovery and innovation in AI. Inspiring talent and ideas everywhere in this important area will lead to new capabilities that improve our lives from medicine to entertainment to transportation and cybersecurity, and position us in the vanguard of competitiveness and prosperity.”

Carnegie Mellon School of Computer Science and School of Engineering faculty have taken on leadership roles in four research concentrations of these new AI Institutes.

AI Institute for Resilient Agriculture (AIIRA)

George Kantor, a research professor at the Robotics Institute, will be working with the AI Institute for Resilient Agriculture (AIIRA), based at Iowa State

University. “I’m really excited about bringing this broad range of AI expertise in the Robotics Institute and the Machine Learning Department here into this new domain,” said Kantor. As with all the institutes, conducting world-changing research in each particular field by applying AI techniques and tools, ultimately results in the goal of solving pressing issues that affect human lives worldwide.

One of the main objectives of AIIRA is to build digital replicas of plants for research purposes, with the goal that the replicas will enable farmers to find ideal cultivars that can withstand increasing climate uncertainty and increase yields to help address world hunger. Robots developed in the project will speed complex jobs like selective pollination while providing on-the-ground updates on the health of crops.

The task of modeling an entire plant from the get-go is enormous, said Kantor, so the strategy is to first model individual plant systems separately. He offers photosynthesis as an example. “For this preliminary model, look at the sunlight, look at properties of the plant that you’ve measured, look at the temperature, and then predict how much the plant is photosynthesizing at any given time,” he explained. “There will be a lot of models like that, and we’ll improve them bit by bit. The main contribution of our research will be to glue the individual systems together with machine learning, and you can use that to simulate everything about the plant.”

Eventually, Kantor hopes to produce what he calls digital twins – a virtual copy of a plant that could be used for experiments through simulation.

“This will allow us to analyze how the plant grows, and then expand that to see how the plants grow at the field scale,” Kantor said.

Kantor will also employ robotics to do first-hand observation of crops. In addition to doing arduous and labor-intensive tasks like pollinating particular plants for optimal pairings and inspecting the physical condition of the plants, the robots will simultaneously collect data used to advance knowledge of the crops, creating an expanding information loop. Eventually, the modeling could be predictive of how plants might respond to new weather patterns of the future.

“This is extremely important in terms of creating more resilient plants for the climate changes that are coming,” said Kantor.

George Kantor, Research Professor and Senior Systems Scientist, Robotics Institute

Srinivasa Narasimhan , Professor, Robotics Institute

Kantor will partner with Srinivasa Narasimhan, professor at the Robotics Institute, who has been doing breakthrough research with non-line-of-sight imaging. Narasimhan will “be developing cameras that can look into plants, and actually see below the surface,” Kantor said.

“It will be very exciting to take that information and incorporate it with these plant models.”

While the AI and other tools Kantor and his fellow researchers develop stand at the vanguard of AI’s evolution, he emphasizes the years of sweat equity that have made the project possible. “Our team has been working in agricultural robotics for over a decade now, and we’ve had a whole series of projects — like using intelligent robots in the orchard — that will help us bring artificial intelligence out into a field and do things with it,” Kantor said.

Institute for Collaborative Assistance and Responsive Interaction for Networked Groups (AI-CARING)

Reid Simmons, a research professor at the Robotics Institute and in the Computer Science Department, is working with the Institute for Collaborative

Assistance and Responsive Interaction for Networked Groups (AI-CARING), which will reside at the Georgia Institute of Technology. With the growing numbers of older Americans in need of daily assistance, and the workforce increasingly unable to keep up with the need, AI-CARING is devoted to finding ways to help aging adults remain independent through the use of coordinated and collaborative AI and robotics. “Our findings will provide fundamental research for human-AI collaboration applications,” said Simmons.

The home health care environment is rich with physical and informational complexity and creating ways to provide assistance is a major challenge. Simmons gives the example of a support unit gauging changes in a person experiencing physical or mental decline over time. “It may be gradual or sudden, like from a slip or fall, but we have to be able to track that and adapt,” he said.

Reid Simmons, Research Professor, Robotics Institute, Computer Science Department

The AI will need to work with a network of family, friends and clinicians to be successful. Each person the AI interacts with in its assignment of caring for an elderly individual will have a different role and will require different information. A son-in-law may not need medical data but would need to know about scheduled appointments with doctors or clinics to arrange transportation. Likewise, an in-home caregiver may need important information from the prior caregiver’s shift to provide necessary assistance, which the AI could deliver reliably.

“Everyone in the network is going to be different, and that means their actions will be unpredictable,” said Simmons. “That’s a huge issue.”

Paramount to the AI-CARING’s mission are important ethical questions and the challenges to meet them. The privacy of the medical information the AI works with will have to be preserved, while also making critical data available to those caregivers who need it, said Simmons. “There are things about someone’s condition you may tell a doctor, but you would not be willing to tell a neighbor, or even the person themself for fear how they might handle it at a given moment. So, the AI will have to be able to understand what information needs to be conveyed, to whom and at what level.”

AI Institute for Future Edge Networks and Distributed Intelligence (AI-EDGE)

Ethics also drives the work that will be done at the AI Institute for Future Edge Networks and Distributed Intelligence (AI-EDGE). While it may simplify matters for engineers, it’s no longer preferred practice to store large amounts of potentially identifiable personal information used to improve networks and devices on cloud servers. Gauri Joshi, an assistant professor in the Electrical and Computer Engineering Department, and Ameet Talwalkar, assistant professor in the Machine Learning Department, will head Carnegie Mellon’s work with AI-EDGE to develop new tools to maximize the learning capacities on individual devices to protect consumer data while improving network efficiencies. The institute will be based at Ohio State University.

AI that can keep the data on users’ phones and devices private while still channeling small packets of critical information to algorithms based in the cloud will vastly improve the performance of the network and the predictive capacities of devices and other things, such as self-driving vehicles. “Instead of taking all the data from where it’s being generated — on smartphones or smart devices — and moving it to a central server, doing all the processing and moving the final model back, the idea is to do as much as possible directly on the smartphone, car or device,” Talwalkar explained.

 

Gauri Joshi, Assistant Professor, Electrical and Computer Engineering

Ameet Talwaker, Assistant Professor, Machine Learning

The technique, called federated learning, employs the computing resources on a remote device to teach the AI on the device itself, and upload an anonymized summary of that information to a server. There, the server aggregates the information with other user updates. “This is going to be the default in all small devices,” said Joshi. “Currently we don’t have the scalability in the algorithms to handle so many devices because it’s been deployed maybe for thousands or at most millions of devices, and the scale of some of these systems is a billion devices.”

 

USDA-NIFA Institute for Agricultural AI for Transforming the Workforce and Decision Support

At the USDA-NIFA Institute for Agricultural AI for Transforming the Workforce and Decision Support, Wenzhen Yuan, an assistant professor in the Robotics Institute, will work with researchers at Washington State University to improve agricultural processes, thus maintaining healthier, hardier crops, and bigger harvests. Yuan, whose research involves AI, robotics and sophisticated sensor devices, is developing robotic devices that could inspect and report information for analysis. Yuan’s research will extend from examining the readiness of fruit for picking to providing information on the prime moment to harvest crops. “Humans are not very good at quantitative tasks, and they can be inconsistent,” said Yuan. “The AI will help us be more rigorous in our analyses and make better harvesting and other kinds of agricultural decisions.”

With four of the 11 AI institutes having recruited SCS faculty to spearhead research, the school again demonstrates its capacity for attacking some of the biggest global challenges out there. “We like to work on important things, and we all have to eat, so that’s pretty important,” Kantor said. “Automated tractors were one of the things that Red Whittaker worked on more than 30 years ago, so agriculture has always been a target application in our field, and there’s a historical legacy for it here at Carnegie Mellon.” ■

Wenzhen Yuan, Assistant Professor, Robotics Institute