Machine Learning Methods for 3D Digitization of Garments and Avatar Interactions

Abstract

From the moment we are born, interaction with our surroundings is essential for survival and integration into society. We sense and navigate our world, pick up objects like toys or food, wear clothes, learn to use tools and communicate with our peers. As social beings, this is a core part of our existence, so any virtual world that aims to feel realistic and immersive needs to correctly model and replicate natural human interaction. However, this is a challenging task, as human interaction is complex and nu anced. It can be virtualized by animators, but animated avatars are limited to what has been manually done, and animations are usually full of artistic decisions. Another way of representing interactions is by capturing them from the real world, but this process is usually expensive and, while it can produce much more output than animating, it is still limited to what exists in the data. Physical simulations are less restricted to the data, as they are driven by mathematical formulations, but these formulations are difficult to implement and usually very slow to execute, making them unfit for real time applications. The recent advancements in machine learning offer a groundbreaking approach to addressing highly complex problems, such as the virtualization of humans and human interaction. Instead of modeling and implementing the problem explicitly, massive amounts of data are fed to the model which can automatically discover and replicate the complexity and nuance of the problem. By analyzing vast datasets, machine learning models can automatically capture and replicate the subtleties of these interactions, avoiding the constraints of traditional methods. This the sis leverages this innovative perspective to explore two pivotal aspects of human interaction: the interplay between humans and garments, and the dynamics of human-to-human interactions.