Grand Theft Auto V might be a fun anarchy simulator, but it’s proving to have some practical uses in the real world, too. Some AI developers are using it to prototype their self-driving car algorithms and now you can watch one of them learning as it goes, live on Twitch.
Last year, the Darmstadt University of Technology in Germany and Intel Labs collaborated on a method of pulling visual information from Grand Theft Auto V, according to a report from Bloomberg. The game has now been modified such that it can be used in research projects designed to perfect self-driving programs.
The AI currently trying its luck at driving around Los Santos is called Charles. It uses pixel-level-analysis of on-screen information to make decisions about how to drive around the fictional city. In essence, it uses the same visual data that a human driver would. For an idea of what Charles “sees,” the developer has put together a short image library to explain it.
Crafted in Python — hence the Stream name, “Python Plays” — Charles currently doesn’t have any short-term memory, so doesn’t have much of an ability to learn. That’s why its driving is so poor right now and doesn’t seem to get much better no matter how long you view the stream. It does occasionally make it a few blocks without hitting anything and long-term viewers claim it is getting better, but it is far from being a safe or capable self-driving system.
Despite its obvious problems though, for a homebrew project by a single developer, the fact that it’s able to get around without any specialized information is reasonably impressive. You can imagine with time and further development, how an AI could become more proficient at driving by using a virtual toolkit like GTA V.
This kind of testing will hopefully help teach automated driving systems how to respond to unexpected obstacles with the same alertness and awareness as a skilled human driver. Anything can happen when you’re on the road, and it’s essential that an AI is able to respond appropriately, no matter what the eventuality.
At some point, self-driving technology will be advanced enough to stand up to trials on the same roads as every other human motorist. However, until we reach a certain level of sophistication, it will be the pedestrians roaming the streets in Grand Theft Auto V who bear the brunt of any unexpected occurrences.
Updated on 05-19-2017 by Jon Martindale: Added Twitch stream and information on it.
Apple won the PC market’s worst quarter, but that’s not the entire story. It won the race by stumbling less than everyone else.
PC makers shipped just 57 million computers worldwide in Q3, which might look good, but it’s a 21% year-over-year plunge (via Counterpoint Research). And despite Apple’s Mac shipments falling by 11%, to 5.6 million, its market share rose from 8.7% to 9.8%. How did that happen?
Update (October 22 23:54 PT): In response to our query, GM clarified that offline recording support is already available on eligible vehicles. Like Tesla's Sentry Mode, it requires owners to connect a USB drive to save incident recordings, which can be viewed on the vehicle's infotainment screen. The upcoming Connected Cameras feature will build on this by giving owners remote access to these recordings via the revamped OnStar app, allowing them to check footage without walking out to their car.
General Motors has announced plans to add a new Connected Cameras feature to its OnStar service starting next year, giving drivers a way to monitor parked vehicles remotely and capture footage of certain incidents.
Missing out on the self-driving future is a tough pill to swallow, especially as a car enthusiast. But that is what Tesla handed us on Wednesday, without a hint of mercy.
During its October 21, 2026 earnings call, Elon Musk told investors the Cybercab is meant mainly as a ride service. Further, nearly all of them will be owned and managed by Tesla.