"Star Trek holodeck" for freely moving animals

A space that can simulate virtual worlds perfectly - the fictional "holodeck" of the science fiction series "Star Trek" is famous. Biologists have now developed a similar system for the study of the orientation and the spatial thought of living beings: a kind of holodeck for the model animals of neurobiology - flies, fish and mice. The system allows experiments that would not be possible in the real world, the researchers explain.

How do people orient themselves when they are traveling in a new environment and how do they estimate the distance they have traveled? Such basic questions about the spatial thinking ability neuroscientists usually do not fathom directly at humans, but at modeling animals. In this research, however, there is a challenge: animal and human beings use sensory impressions to update their mental map, whereby the movements are inextricably linked to the sensations. In order to understand how the brain processes the different information, it would therefore be ideal to be able to solve the individual aspects of each other.

Separation of perception and behavior

In this context, we have envied the science fiction world of the Star Trek TV series with an invention: the Holodeck, says Andrew Straw from the Albert Ludwigs University of Freiburg.

But this is now over, because research space has now really changed: Straw and an international research team have developed a holodeck-like system for the creation of virtual worlds.

We have created an all-encompassing, three-dimensional virtual reality in which test animals could move freely, explains Straw. This allows us to combine a visual landscape prepared by us with its actions and perceptions.

The visual landscapes include, for example, vertical columns, various plants, or swarms of so-called Space Invader figures from a computer game. In the system, multiple cameras capture the exact three-dimensional position of fish, fly, or mouse. A computer program registers every movement of the animal within milliseconds, so that an updated image of the virtual environment can always be projected onto the wall. As it turned out, the animals actually perceived the simulated objects as genuine and changed their behavior in the different visual environments, the researchers report.

This shows a real fish swimming in a swarm of video game space invaders, showing the abilities of the system to produce highly artificial visual scenery within an immersive environment.

Virtual heights and swarm behavior

The researchers, for example, confronted mice with virtual abysses by the system, which enabled them to document their height in the artificial environment. In the flies, however, they changed the direction of flight through the visual stimuli in the holodeck. In the case of the fishes, the focus was on the swarm behavior. The scientists simulated experimentally a swarm of Space Invader figures, in which a test fish moved. The virtual swarm was programmed to treat the real fish as a part of itself. This again had a clear effect on the behavior of the animal.

For the first time, the researchers were able to manipulate the interaction between several individuals in a complex manner. They developed a photorealistic model of a swimming fish that could be computer controlled. It was found that fish follow this virtual specimen, which in turn turned the swimming direction towards another real fish. In short: exciting possibilities are emerging and the researchers are enthusiastic about their futuristic "toys". We should, therefore, be curious about what they will discover about the foundations of orientation.

This shows a virtual fish interacting with a real fish. The experimenters digitized the appearance and movement patterns of a real fish and thus could create a visually realistic virtual fish, which they then programmed to interact with a real fish.

Source:

• Nature Methods, DOI: 10.1038/nmeth.4399
• strawlab.org