AI can transform a photo of your dog into a VR avatar
Almost four years have passed since Facebook has officially changed its meta business name, in the midst of the promises of the founder Mark Zuckerberg that a fully -made digital “metavese” was just at the corner of the street. Since then, the adoption of users of virtual reality spaces has flat, and Zuckerberg himself has apparently moved accent on AI companions and podcast rays prohibitions. For many, simply sit on the sofa at home with a dog by their side remains more attractive than sliding in VR.
But what happens if your friend in fur could join you?
Researchers from the National Institute of Science and Technology in Ulsan in South Korea have developed an AI system that could make it possible. Using a system they call Dogrecon, the team was able to take a single photo of a dog and convert it into a realistic 3D digital model that could be used in virtual or augmented reality applications. Although it is not entirely ready for prime time, the unique Dogrecon weighting mechanisms helped him generate sharper and more realistic renderings compared to similar AI models, which often produced blurred and distorted images with an inaccurate body geometry. The researchers shared their results in the International IT Vision Journal.
The video shows some first examples of how these 3D dogs generated by AI could appear in virtual environments. As part of the left, a pack of puppies (including a Shiba Inu in green dog boots) slowly walk in a backyard garden. On the right, two dogs can be seen walking in a living room. Once the 3D dog avatars are generated, researchers can control them using simple text prompts.
“With more than a quarter of households with pets, the expansion of 3D reconstruction technology – people focused on humans – to include pets was a goal,” said the first author Gyeongsu Cho in a statement sent to Tech Xplore. “Dogrecon offers a tool that allows anyone to create and animate a digital version of its pets.”
The researchers did not immediately respond Popular sciences Comment request.
[ Related: A furry antelope robot is keeping tabs on its organic cousins. ]
Transform the 2D photo into 3D avatars
Although substantial progress has been made in recent years to improve the realism of human avatars in virtual spaces, much less attention has been paid to our furry companions. According to the team, one of the reasons for this is that pets (dogs in particular) are more complex anatomically than us. Dogs are available in hundreds of breeds, each with unique shapes, sizes and physical characteristics. Fido four -legged position also leads to more frequent joint occlusion compared to humans, which makes 3D modeling precise more difficult.
All these additional variables create more opportunities so that the model produces errors during the generation of dogs in a digital space. However, researchers argue that the lack of pets properly modeled in virtual reality limits the overall credibility of any so-called “metavese”.
“Without a doubt, human -specific approaches are important, but humans are only a small part of real biodiversity,” write researchers.
The team attempted to fight against these challenges by first using models of statistics specific to the breed to capture the variations of the forms and postures of the body. They then used a second generative AI model to create multi-sex images that remain geometrically aligned on the shape of the dog’s body. By combining these two stages, the system could take a single image of a dog (in this case, real images drawn from the Internet) and allow the models of AI to predict essentially what the dog would look like and would move from different angles.
Not all predictions of AI are precise reflections on what a dog should look like. To remedy this, the researchers applied a weighting mechanism to their model, prioritizing the outings that seemed more realistic and filtering those with distortions or other unusual visuals that are commonly found in the images of poorly generated AI. The weighting, in this context, helps to define parameters for the AI outputs for the IMER the desired results. When they compared Dogrecon to other AI models capable of generating videos from images, they found that their system systematically produced more anatomically precise results.
This does not mean that the results were perfect. The 3D models created by Dogrecon generally resemble the breeds of dogs they are supposed to represent, but in the few examples shown in the VR and AR parameters, they always seem somewhat disconnected and shocking visually. Their movement is less like the effortless agility of a border collie and the more the stiff animation of a beginning Southern park episode.
However, rudimentary that they may seem, researchers say that tools like this could be welcomed by users of virtual reality who wish to bring their puppy in their next digital adventure. As for real dogs – they still hang out on the sofa.
More offers, reviews and purchasing guides
