The importance of having a good 3D body representation of the patient

How does the brain represent our body? We see it as a stable and unchanging image, however, several studies have induced in different people the illusion of body ownership over external objects that were not part of the human body. This is telling us that the way the brain represents our body can be in deed very plastic.

What is important for us here, is that this illusion can also be induced for the entire body, and here a first person perceptive of the artificial body is the key. This is described as the sensation of possessing an artificial body in which the real body is substituted as the origin of the perspective sensation. Nowadays, this can be induced with immersive virtual reality and users have the sensation of being embodied in that virtual body.

IVR has many different and potential uses, it is widely use in physical and mental health research and rehabilitation. For instance, this is the case of the work done in Virtual BodyWorks (VBW). We use break thought in neurosciences and Virtual Reality (VR) research to change society in a wide range of areas: pain and mobility, emotional health, diversity and inclusion.

We also apply IVR to rehabilitate gender violence aggressors. Changing aggressive people perspective with IVR has been proved that can modify emotion recognition, and it creates a substantial change to the offender.

And now, within the Socrates project, we wanted to use all this previous knowledge from neuroscience and VR with the objective to treat obesity with self-conversations inside a VR experience. We use embodiment to treat obesity to target three aspects that are inherent with severe obesity: recognition, self stigmatization and self-determination.

Here, one of the key problems we had since the beginning of the development of this project was the importance of having a good self body representation.

In order to overcome this necessity, we have been researching and developing a good scan technology, and we created an easy way of getting a patient avatar. We are still ending some parts of the pipeline in order to use it automatically and by external users, but with just a front and side image of the patient it generates a good 3d body representation of the person, and it also creates a morphable avatar that can change user body shape at run time inside the conversation experience.

We constructed a cost-effective and accurate model to infer the 3d human body shape from a front and side image of a person taken with a smartphone using CNN. (See below some models). Convolutional Neural Networks (CNN) are widely used for image recognition and classification. They have the ability to learn complex connections between their inputs and outputs.

3D Human body shape using CNN

About the author

Elena Álvarez de la Campa Crespo has a PhD in Computational Biology at the Universitat de Barcelona (2019). She is graduated in Biotechnology (2013) and with a Master in Genetics and Genomics (2014). Her doctoral research in the field of computational biomedicine, carried out at Vall d’Hebrón Research Institute (VHIR) and at Centro de Investigaciones Científicas (CSIC), has revolved around understanding the molecular bases of specific systems and characterizing the impact of genomic sequence variants, with a strong focus on knowledge, determination and computational prediction of protein structures and their molecular modelling. A central axis of her research has been the creation of machine learning models to improve the annotation of genomic sequence variants in sequencing projects, increasing the scope of in silico tools in the clinical environment. All the knowledge gained during her research, a rigorous scientific approach and her first-level analytical and problem-solving skills, have contributed to become part of Virtual Bodyworks. Her role in the company is to be an analyst and data scientist, specialist in Artificial Intelligence, and to manage leading projects in the field, adapted to the creation of Immersive Virtual Reality (IVR) applications for medical and social rehabilitation purposes. Both professional careers have endowed her with a great capacity for working on collaborative projects with higher research institutions and also with a considerable experience in grant management, especially European Grants (Horizon 2020, EIT, Interreg-POCTEFA).