Light field technology promises lifelike digital twins

The technology has huge potential for remote education, medicine, and virtual tourism

A global expert on light field technologies, Jingyi Yu, says it is now possible to capture all aspects of a person’s appearance including skin reflection, light, depth, colour, clothing textures and aspect, to provide an ultra-realistic and finely detailed reproduction of anyone in real time.

Yu is assistant professor in the Department of Computer and Information Science at University of Delaware and the director of the Virtual Reality and Visual Computing Centre at ShanghaiTech University.

Yu also, in 2015, founded Plex-VR, a startup on light field technologies that has received venture funds from Alibaba and VC firms including GSR and SAIL. He will present at New Zealand’s AI Day 2019 conference in Auckland on 27-28 March.

He says the exact replica imaging is the result of multiple cameras — with up to 80 being used to capture constant images of every angle, with the person then reconstructed in 3D instantly. The technology is powered by accelerated computer and algorithm speeds, and has huge potential for remote education, medicine, and virtual tourism.

“We have lived in a 2D image world for decades, but we are close to when 3D imaging will become normal, and bring a new dimension and more reality to our digital experience,” Yu says.

According to the organisers of AI Day, light field technology works by capturing all light emitting from an object and using advanced machine learning and image processing technology to enable the creation of a high resolution and super realistic dynamic models of that object.

The executive director of the AI Forum of New Zealand, Ben Reid, says recent advances in machine learning technology are enabling the rapid development of these new augmented reality capabilities.

“We are all getting used to images of ‘virtual humans’ that are realistic, but you can still see they are computer generated. But with this technology you won’t be able to tell the difference.

“Managed correctly, this will open up enormous potential in any industry or service where digital communication plays a role, and increasingly this is just about everywhere.”

Reid says China is one of the world’s leading centres of AI investment and New Zealand’s AI innovators must build strong and lasting links with the Chinese AI market.

Prof Yu’s visit is supported by Asia New Zealand Foundation. Details of AI day are available online.


Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

More about

Show Comments