AI can turn spoken language into photorealistic sign language videos

By Chris Stokel-Walker

FluxFactory/Getty Images

An AI that can produce photorealistic videos of sign language interpreters from speech could improve accessibility by removing the need for humans.

Ben Saunders at the University of Surrey, UK, and his colleagues used a neural network that converts spoken language into sign language. The system, called SignGAN, then maps these signs on to a 3D model of the human skeleton.

The team also trained the AI on videos of real sign language interpreters, teaching it how to create a photorealistic video of anyone signing based off an image of …