Tom Goddard
August 7, 2019
For Tom Ferrin's ACS talk August 26
Outline
Types of 3D biology data visualized with VR
Video of routine VR use at UCSF
Example biology problems where VR seems useful
Preferred VR hardware: Vive Pro including WiGig wireless, Oculus Rift S with inside out tracking.
Other 3D visualization technologies
Types of 3D biology data visualized with VR
We are trying VR in ChimeraX for 3D biology data at various scales:
atomic models, cryoEM, light microscopy time series, medical imaging.
Opiods bound to opioid receptor, computational docking.
Crawling neutrophils, light microscopy.
Translation initiation factor inhibitor, cryoEM.
Lung CT scan.
Jacobson lab weekly VR group meeting
Video illustration of VR discussion of drug docking.
Matt Jacoboson's UCSF drug design lab uses VR for group meetings.
Usually two group members use VR while others participate via projector.
Projector shows what one of the VR users sees.
One VR user is on a wireless connection, the other is wired. (Up to 3 wireless can run in a single room).
Each VR headset runs on a separate computer (a requirement of the VR driver software).
Participants can connect to VR session remotely.
We use remote VR meetings at UCSF with NIH and Benaroya Institue in Seattle.
Firewalls can block connections.
What problems benefit from VR?
VR can be used for visualization and analysis (mutating residues, interactive molecular dynamics).
Good VR uses often need precise 3D spatial perception, such as drug interactions (hydrogen bonds, hydrophobic contacst, ...) within a protein binding site.
Immersive viewpoints benefit from VR, e.g. view from within a drug binding site,
Example where VR helped understand drug resistance
Figuring out a mechanism of
anti-cancer drug resistance to clinical drug avapritinib which binds to a mutant receptor tyrosine kinase
(KIT) protein found in some cancers.
KIT transmembrane protein initiates regulatory signaling cascade and is a kinase.
Some cancers commonly exhibit a KIT mutation causing over-activation allowing cancer cell proliferation.
Drugs midostaurin and avapritinib block activation.
These drugs are competitive inhibitors acting at the ATP binding site.
Patients often acquire secondary KIT mutations that make midostaurin or avapritinib ineffective.
Matt Jacobson's lab at UCSF hypthesized the drug resistance mechanism based on observations using VR.
Several mutations that result in drug resistance were distant from the binding site yet disrupted drug binding.
Distant mutations were found to rigidify part of the protein which lead to different dynamics of the flexible
P-loop region (coordinates phosphate transfer) in the ATP binding site.
Receptor tyrosine kinase (KIT) with docked midostaurin and avapritinib drugs.
KIT mutations in circles inhibit drug avapritinib (blue) although through an indirect interaction that moves P-loop (green ribbon) which blocks binding.
cryoEM model refinement can benefit from VR
Challenging model building at 3 Angstroms resolution typical of cryoEM.
Automated structure building produces many errors at this resolution where individual atoms are not resolved.
Weeks or months of manual human interaction is used to correct the structure errors.
VR allows precise observation of mismatches between atomic models and cryoEM maps.
6 degree-of-freedom hand controllers make it simpler to correct problems compared to 2 degree-of-freedom mouse.
ChimeraX ISOLDE plugin developed by Tristan Croll at Univ Cambridge uses interactive
molecular dynamics while fixing structure errors.
Example: Photosynthesis supercomplex
Diatom light harvesting complex with 200 ligands (chlorophylls and pigments).
20% of earth's primary energy production is from photosynthesis using this molecular machine.
VR is not currently used for building atomic models of large structures at 3 Angstrom resolution.
This structure illustrates the challenges, but was built by conventional non-VR methods.
Chlorophyll and other ligands showing cryoEM density at ~3 Angstrom resolution.
Photosystem II light harvesting supercomplex (ligands in yellow)
Ligands with proteins hidden: chlorophyll and fucoxanthin pigment (green), lipids (orange), ion-like (magenta).
VR equipment
We use Vive Pro and Vive headsets on Windows, Mac and Linux, and simpler Oculus Rift S on Windows.
Also use wireless with Vive Pro. Might mention resolution, refresh rate, field of view specs.
Vive Pro
HTC wireless adapter
Wireless transmitter
Oculus Rift S
VR graphics card, Nvidia RTX 2080
Other 3D visualization technologies
Hololens 2
Magic Leap One
Oculus Quest
Looking Glass
Some new technologies are less ready for research use than VR tethered to a PC.
Augmented reality: Microsoft Hololens 2 and Magic Leap
Limited field of view, 40-50 degrees horizontal field for AR versus 100 degrees for VR
Standalone headsets with low power computation and graphics.
Biology applications not compatible with custom operating systems.
Standalone VR, Oculus Quest
Advantage: Does not use a desktop or laptop computer.
Android operating system, incompatible with biology apps on Windows, Linux, Mac operating systems.
Insufficient graphics power (compare to size of typical VR graphics card).
LCD stereo glasses
Looking Glass, stereoscopic display not requiring glasses.
Requires custom software support to render dozens of different view points.
Uses hand gestures for interaction with no hand controllers -- less mature technology.
Older 3D technologies are being discontinued by manufacturers
Conventional LCD stereo glasses, hard to obtain.
NVidia 3D Vision support in graphics driver discontinued April 2019.
