A.I. teaches Minecraft players about architecture | AI
Minecraft is a popular 3D video game where players build and navigate their own digital environments. The modification will tell players whether their buildings fit into certain architectural styles and offer ideas for how the structures could be improved.
“One of the things that’s important to learn when you’re a kid and throughout life is creativity, abstraction—how to envision what you want and then create it,” says senior author Ross Knepper, assistant professor of computer science at Cornell University. “This is a tool that helps people not get discouraged, maybe if they’re beginning at Minecraft and don’t know how to use their imagination right off the bat.”
A.I. teaching players
Based on buildings Minecraft players created and uploaded for others to use, the researchers created a deep neural network—a kind of machine learning trained to predict whether data belongs in a certain category. Through that network, players could learn whether their building is medieval, modern, Asian, or classical—four especially popular tags Minecraft players use. Once the building is classified, another algorithm can show the users similar buildings to inspire them to make improvements to their own.
“People are really interested in having more design spaces in Minecraft, and being able to build certain types of architecture, but there weren’t any design tools as far as we were aware that could teach them,” says first author Irene (Euisun) Yoon.
Yoon curated the data set to make sure the buildings were labeled correctly, since their algorithm was less accurate than they would have liked because it was trained with fewer than 1,000 player-created buildings. Ideally, they could train such an algorithm with tens or hundreds of thousands of pieces of data.
“If you ask an architect to tell you what a building’s style is, the architect will say, ‘OK, it’s one-and-a-half stories, it has dormers, it’s a Cape Cod.’ Deep learning is doing that but it’s doing it in a black box way (hidden from view). It learns patterns, but not necessarily the same patterns an architect would say are the key things,” Knepper says. For example, if all the modern-style houses in a data set have pools on the roof, the computer could assume that rooftop pools are a requirement for modern houses.
Players teaching A.I.
For Knepper, a roboticist by training, the Minecraft project helped answer questions about how a robot might follow a human’s instructions.
“If I say, ‘Build a house,’ today a robot is going to say, ‘I don’t know what that means.’ ‘Which brick should I put where?’ is the level at which robots need instruction,” Knepper says. “We’d like humans to be able to interface with robots more like we interface with each other. So if I tell it to build a medieval house or an ancient house and give some of the high-level details, it would know at that point how to turn it into a plausible thing that does everything you want. We’re not there yet, but this is the first step towards that goal.”
Coauthor Bharath Hariharan, assistant professor of computer science, approached the research from the perspective of his own work in computer vision. In trying to interpret an image, scientists can train a computer to pick up cues such as shape and solidity, but may have trouble processing perspective or scale. Using people’s intelligence through their Minecraft structures and tags can help computers learn to solve those problems.
“When you’re working with images, it’s really hard to actually get at the essence of what something is,” Hariharan says. “A machine observing how people build can actually learn quite a bit about what shape is, what structure is, what buildings are.”
The researchers will present a paper on their work at the Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence and Interactive Digital Entertainment in Canada.
The paper is based on work supported by the National Science Foundation.
Source: Melanie Lefkowitz for Cornell University