Ready, Set, Algorithms! Teams Learn AI by Racing Cars

Anyone with an Amazon Web Services account can participate in the league. Teams or individuals can compete online in “virtual” races or in person at events world-wide.

Teams build and train AI algorithms using Amazon SageMaker software, deploy them to self-driving cars measuring about 10 inches, then race them around a track of roughly 17 feet by 26 feet. The fastest car wins. A DeepRacer car in Morningstar’s Chicago office. Photo: Morningstar Inc. “It’s actually having practical applications,” said James Rhodes, chief technology officer of investment research firm Morningstar. Thanks to the training, the company expects to have dozens of projects based on reinforcement learning and other machine-learning techniques in deployment by the end of 2020, he said.

Besides training autonomous vehicles, reinforcement learning can be used to help robots walk faster or to develop security systems that can automatically adapt to different environments, experts say. “[It’s] a pretty complicated technology and there’s a pretty steep learning curve,” said Mike Miller, general manager of AI devices at Amazon Web Services.

AWS developed the DeepRacer program as a way to teach software developers about machine learning in a more engaging way than reading scientific articles, Mr. Miller said.

The algorithms are complex because they gather data on their own, instead of being fed millions of images to learn from, said Peter Stone, professor of computer science at the University of Texas at Austin, who isn’t involved with DeepRacer. Programmers write code to “reward” the algorithms when they do something right, such as winning a race or avoiding an obstacle. In the case of algorithm-powered cars, that could include such tasks as staying close to the center line on the track, minimizing wide steering angles and turning to avoid barriers and crashes.

At Morningstar, more than 450 software developers, equity analysts and quantitative researchers have formed nearly 100 racing teams in 10 countries since January, when Mr. Rhodes began allowing employees to use the technology.

Morningstar has invested “north of tens of thousands of dollars” on the miniature cars and training software to date, Mr. Rhodes said. “It’s bringing the virtual [world] into the physical space, especially for individuals who are not necessarily computer scientists,” he said.

Earlier this year, one of the Morningstar teams came up with an idea for a tool based on reinforcement learning that looks for patterns in regulatory filings to more accurately identify various information. The tool was deployed in June. Another team came up with an idea for a tool that uses reinforcement learning to automatically find and fix broken links to financial institutions’ websites, Mr. Rhodes said. That tool is still in development.

Insurer Liberty Mutual, meanwhile, has about 270 employees, including software engineers and data scientists, participating in DeepRacer.

“It’s a fun way for people to get practical exposure for what are very important algorithms in a safe environment where they’re not going to mess up any core business applications,” said James McGlennon, the company’s chief information officer.

The company is already using other machine-learning techniques to tweak prices for auto insurance based on risk factors and to look for anomalies in operations. The goal of the DeepRacer program is for employees to think about ways the company can use reinforcement learning to help the business, Mr. McGlennon said.

Reinforcement learning is one of many machine-learning methods that will ultimately be used by companies, said Dario Gil, director of research at International Business Machines Corp. It is challenging to rely on that technique alone when training a real-world autonomous vehicle, he said, because so much of the method relies on trial and error. “There’s a reason why reinforcement learning gets stuck in the world of games,” he added.

Source: www.wsj.com