A University of Alberta professor who literally wrote the book on reinforcement learning has been awarded the world’s most prestigious prize in computing science.
Richard Sutton, a University of Alberta computing science professor and one of the founders of modern computational reinforcement learning, has been honoured as co-recipient of the 2024 Association for Computing Machinery A.M. Turing Award, often referred to as the “Nobel Prize in computing.”
Sutton, who has been instrumental in transforming Alberta into a world-renowned artificial intelligence hub since arriving at the University of Alberta nearly 25 years ago, was honoured alongside longtime collaborator Andrew Barto. The award, named for British mathematician Alan M. Turing, carries a US$1-million prize funded by Google.
Reinforcement learning, a branch of artificial intelligence, has countless applications, from global supply chain optimization to improving the reasoning capabilities of ever-popular chatbots. The wide-reaching impact of the field is likely no surprise to Sutton, who claims that at its core, artificial intelligence is simply about “understanding humanity.”
“It’s the desire to understand the world, the desire to make tools to make ourselves better. It’s really totally human — the natural continuation of what humans are.”
It’s a fitting perspective for a scientist who got his start as a Stanford undergraduate studying behavioural psychology. Sutton went on to cultivate his fascination with the way both humans and systems think at the University of Massachusetts Amherst, where he met Barto.
Sutton and Barto, influenced by the ideas of late AI pioneer A. Harry Klopf, began exploring the concept of “a learning system that wants something, that adapts its behavior in order to maximize a special signal from its environment.” They went on to co-author Reinforcement Learning: An Introduction. First published in 1998, it continues to be one of the pivotal texts in the field.
“Anyone who’s gone anywhere in reinforcement learning has read that book, and it’s shaped how they think about it,” says computing science professor Michael Bowling.
“All major areas of reinforcement learning have hundreds of researchers and hundreds of papers being published in them every year — all of that is sitting on top of Rich’s foundational work,” Bowling adds.
Sutton’s impact on Alberta’s AI legacy began in 2003 when he moved to Edmonton to teach at the University of Alberta’s Department of Computing Science. From there he went on to serve as Chair of Reinforcement Learning and Artificial Intelligence at iCORE/AITF until 2018, and founded the Reinforcement Learning and Artificial Intelligence Lab, where he now serves as a principal investigator. Alongside his current fellowship, Sutton is chief scientific advisor at Amii (Alberta Machine Intelligence Institute) and a Canada CIFAR AI Chair.
In 2017, he co-founded Google DeepMind Alberta, the company’s first international research lab. That same year, he announced a partnership with celebrated video game engineer John Carmack and took on the role of research scientist at Carmack’s Keen Technologies.
Adding to his impact on reinforcement learning, Sutton’s mentorship of emerging researchers continues to leave an indelible mark on the field of AI. It was his former doctoral student David Silver who, alongside Sutton and Martin Müller, developed AlphaGo, a computer program that defeated the best human Go players in 2016 and 2017.
In 2018, the Canadian Artificial Intelligence Association recognized Sutton with a Lifetime Achievement Award, and he has also received an Outstanding Achievement in Research Award from the University of Massachusetts at Amherst. He is a fellow of the Royal Society of Canada, the Royal Society of London and the Association for the Advancement of Artificial Intelligence. Sutton’s scientific publications have been cited approximately 150,000 times, a testament to his continued impact on the field.
“Barto and Sutton’s work is not a stepping stone that we have now moved on from. Reinforcement learning continues to grow and offers great potential for further advances in computing and many other disciplines,” says ACM president Yannis Ioannidis. “It is fitting that we are honouring them with the most prestigious award in our field.”
Over Sutton’s time with the University of Alberta, the institution has established its reputation as one of the world’s leaders in the field, and he’s not finished yet. One of Sutton’s personal slogans, “work on ideas, not software,” encapsulates his multifaceted career, and he has no shortage of further avenues for exploration in mind.
Alongside two U of A collaborators, Amii fellows Michael Bowling and Patrick Pilarski, Sutton penned The Alberta Plan for AI Research, which details a roadmap towards the next “grand scientific prize” of understanding intelligence, including some of the research questions and projects that will be pursued in the decade to come.