3D solutions for science

A collaboration between researchers at Goldsmiths, University of London and the Francis Crick Institute is examining how Virtual Reality can help scientists.

The rise of cheap computing power and storage has seen scientists accumulate huge amounts of data on everything from DNA to cancer cells and drug molecules. One of the big challenges facing scientists is how to get to grips with this mountain of data.

At the Crick last month technologists and scientists met for a workshop to discuss how Virtual Reality (VR) could potentially offer a solution to how to understand and interact with scientific data.

I talked to one of the workshop organisers Dr Sylvia Xueni Pan, Lecturer in Graphics at Goldsmiths, about advances in VR and whether the technology has finally found its ‘killer app’…

Pete Wilton: Why has VR been a ‘solution in search of a problem’ for so long?

Sylvia Xueni Pan: Essentially, Virtual Reality is about recreating reality – realistic looking graphics with which we can interact with in a naturalistic way, so that we can “react as if it were real” as we apply our everyday skills in this digital space. The vision of VR is that we can use it for things we cannot do in real life.

The basic version of VR technology – 3D graphics, 3D display, and 3D interaction – have existed for decades, in fact as early as 1965. But we haven’t seen many applications partially because the hardware was expensive and the end result was not that great. But recently technical progress in computational power (so we can render better graphics very quickly) and tracking technology (so we can better capture users’ head and body movement which enables richer 3D interaction) have lowered the entry barrier for VR.

We are able to produce very realistic experiences on a much smaller budget. This opens doors for experts from all disciplines to get involved in VR or at least start thinking about it as a potential solution for their problems. In a way VR has finally become a good solution, and ready for whatever problem it was originally designed for.

PW: What scientific problems do you think VR is uniquely qualified to help solve?

SXP: Any problem that involves high dimensional data could benefit from VR. We all know that data visualisation facilitates understanding and communication by reducing cognitive load – in fact, we rarely see any scientific publication without figures or plots.

Scientific data is, by definition, 3D or even higher dimensional. With VR you can visualise these high dimensional data and interact with them naturally with your body and your hands: you can walk around the visualisation of a molecule and really understand its 3D shape; you can grab two 3D molecules, one in each hand, and try to fit them together.

PW: What can researchers learn from how VR is used for outreach?

SXP: During our workshop I learnt that many existing collaborations between Goldsmiths and bioscientists from various institutes started when those bioscientists tried Professor William Latham’s Mutator VR application. It’s a generative VR application producing organic-looking “creatures” in VR with which the users could also interact with. So, after going through the Mutator VR experience, many scientists were intrigued by the richness of the experience and were then inspired to use this mechanism to visualise and interact with their own “organic” data, and a lot of the applications shown in our workshop were a result of this.

PW: Which scientific uses for VR have most surprised you?

SXP: As part of the BioBlox VR demo at the workshop, my task was to try to put two molecules together in the correct way. When I put them the wrong way, I remember I could feel that the two molecules were pushing against each other. Afterwards when I thought about that I realised how amazing this was as there was no real force applied to my hands, I was just holding the controllers in the air!

So what happened was that the programmers have cleverly used vibration feedback when the two molecules touch each other, and my brain just combined the visual and vibration feedback together and “concluded” that there must be a force pushing the two molecules away! So the force feedback was a nothing but an illusion, I thought that was great!

PW: What do you hope people will take away from this workshop?

SXP: My biggest hope is that bioscientists who have never tried VR start thinking about using it. In this workshop I met many people who tried a high-end VR headset for the first time, had their questions addressed, socialised with those who have already been using it. I think, as a result of the workshop, people are definitely a lot less sceptical about it!

PW: How do you see VR technology developing over the next five years?

SXP: Going back to the three important technical features of VR: 3D models, 3D display, and 3D interaction, I think the biggest room for improvement in the next five years will be in the space of 3D interaction – and I expect both new software and hardware.

There will be higher resolution and more robust tracking technology combined with cleverer machine learning algorithms to track and interpret our gestures, which will enable us to interact with objects like we do in real life. And more interesting haptic feedback will also be integrated as part of this experience. I really look forward to playing with those new 3D interaction devices!