BrainCode: Brain-inspired Intelligence for Semantic Video Compression
Dates: 01/2025 – 12/2027
Funded by: European Commission, Horizon Europe, HORIZON-MSCA-2023-PF-01
Project Coordinator: P. Tsakalides and I. Katsavounidis (Meta)
PI for FORTH: E. Doutsi
Funding: total cost/funding: € 279,227
Summary:
The BrainCode project proposes novel compression techniques for extended reality (XR) data which are energy efficient while ensuring
a reconstruction quality that satisfies the human visual semantic perception. There are several challenges concerning the complexity
and the power consumption of the latest video compression standards which have not yet taken into account by the signal processing
community. We propose that these challenges can be addressed by machine learning based architectures in order to avoid the exhaustive
comparisons between sequential frames. We aim at releasing a semantic video compression algorithm that uses Convolutional Neural
Networks (CNNs) and drives the bit allocation with respect to the content of the visual scene. Another goal of BrainCode is to mimic the
visual system as an intelligent mechanism that processes the visual stimulus. This can be claimed as it consumes low power, it deals with
high resolution dynamic signals and the dynamic way it transforms and encodes the visual stimulus is beyond the current compression
standards. During the last decades, a lot of effort has been made to understand how the visual system works, what is the structure and role
of each layer and individual cell that lies along the visual pathway, and how the huge visual information is propagated and compacted
through the nerve cells before it reaches the visual cortex. There are very interesting mathematical models which approximate the neural
behaviour and they have been widely used for image processing applications including compression. The BrainCode project searches
the latest neuroscience models for the design of a groundbreaking XR video compression architecture. The efficiency of the above
approaches is expected to improve several image processing applications like computer vision, virtual reality, and video compression
among other, where the real-time processing of the visual scene plays a substantial role.
Visit Website