#Xdream evolving images for visual series

'This approach allows you to use artificial intelligence to figure out what triggers neurons best. 'In this way we have evolved a super-stimulus that drives the cell better than any natural stimulus we could guess at,' says Livingstone. 'The big advantage of this approach is that it allows the neuron to build its own preferred images from scratch, using a tool that is not limited by much, that can create anything in the world or even things that don't exist in the world,' says Ponce. The algorithm was developed by Will Xiao in the laboratory of Gabriel Kreiman at Children's Hospital and tested on real neurons at Harvard Medical School. At first, the images looked like noise, but gradually they changed into shapes that resembled faces or something recognizable in the animal's environment, like the food hopper in the animals' room or familiar people wearing surgical scrubs.

#Xdream evolving images for visual series

It goes through a series of images over the course of minutes, mutates them, combines them, and then shows a new series of images.

The XDREAM algorithm uses the firing rate of a neuron to guide the evolution of a novel, synthetic image. However, this approach is limited by the fact that one cannot present all possible images to understand what exactly will best stimulate the cell. Earlier studies on neuronal preference used many natural images to see which images caused neurons to fire most. Researchers have known that neurons in the visual cortex of primate brains respond to complex images, like faces, and that most neurons are quite selective in their image preference. 'We were seeing something that was more like the language cells use with each other.' 'What started to emerge during each experiment were pictures that were reminiscent of shapes in the world but were not actual objects in the world,' he says. 'When given this tool, cells began to increase their firing rate beyond levels we have seen before, even with normal images pre-selected to elicit the highest firing rates,' explains co-first author Carlos Ponce, then a post-doctoral fellow in the laboratory of senior author Margaret Livingstone at Harvard Medical School and now a faculty member at Washington University in St.

The work appears May 2 in the journal Cell. As the images evolved, they started to look like distorted versions of real-world stimuli. To find out which sights specific neurons in monkeys 'like' best, researchers designed an algorithm, called XDREAM, that generated images that made neurons fire more than any natural images the researchers tested. view moreĬredit: Ponce, Xiao, and Schade et al./Cell XDream is implemented in Python, released under the MIT License, and works on Linux, Windows, and MacOS.Image: This figure shows natural images (right) and images evolved by neurons in the inferotemporal cortex of a monkey (left). Overall, XDream is an efficient, general, and robust algorithm for uncovering neuronal tuning preferences using a vast and diverse stimulus space. These results establish expectations and provide practical recommendations for using XDream to investigate neural coding in biological preparations. Lastly, we found no significant advantage to problem-specific parameter tuning. Furthermore, XDream is robust to choices of multiple image generators, optimization algorithms, and hyperparameters, suggesting that its performance is locally near-optimal. XDream extrapolates to different layers, architectures, and developmental regimes, performing better than brute-force search, and often better than exhaustive sampling of >1 million images. XDream can efficiently find preferred features for visual units without any prior knowledge about them. We also explored design and parameter choices. We evaluated how the method compares to brute-force search, and how well the method generalizes to different neurons and processing stages. We use ConvNet units as in silico models of neurons, enabling experiments that would be prohibitive with biological neurons. Here we extensively and systematically evaluate the performance of XDream. A new method termed XDream (EXtending DeepDream with real-time evolution for activation maximization) combined a generative neural network and a genetic algorithm in a closed loop to create strong stimuli for neurons in the macaque visual cortex. The characterization of effective stimuli has traditionally been based on a combination of intuition, insights from previous studies, and luck.

Abstract:A longstanding question in sensory neuroscience is what types of stimuli drive neurons to fire.