The creation of algorithms for working with images has always been a rather difficult, but promising task. When I was still writing my graduation project in 1999, the topic of "pattern recognition" was very relevant in automatic control and management systems.
That's what they can do today. Indian developers have presented a system that can create short videos from blurry images. The algorithm works on the basis of convolutional and recurrent neural networks and allows you to turn motion artifacts in images into short (up to ten frames) video.
More details …
When viewing a blurred image, a person can mentally complete a picture of what is happening. For example, seeing a photograph of a bird with fuzzy wings suggests that the blurring of the image is due to artifacts in the movement of the wings during the acquisition. For computer vision systems, however, this task is more difficult, and most of the known methods are aimed only at removing motion artifacts and smoothing frames.
Scientists at the Indian Institute of Technology, led by AN Rajagopalan, suggested that a single blurry image could be used to create a whole short video: that is, restore the original movement from its artifacts in the image. To do this, they developed an algorithm based on convolutional neural networks, which are actively used for tasks related to automatic image recognition, as well as recurrent neural networks.
The model is trained on a large number of videos, which are split into frames. After that, the neural network looks for such a frame, the artifacts on which most closely match the artifacts of the training sample frame. After that, the decoder "restores" the training sample frame artifacts into motion captured on video. Thus, the model stores data on possible recovered motions from each blurred frame available in the training sample.
Promotional video:
As a result of the work, the neural network produces video, reconstructed from the blurred image, consisting of ten frames. The developed algorithm, according to the creators, will be able to help in the future to improve not only the restoration of blurred images, but also the videos themselves.
Removing motion artifacts in individual frames can also improve video streaming. So far, for this purpose, mainly algorithms for adapting the bitrate depending on the video speed and its buffering are used.
Elizaveta Ivtushok