DeepMind Demonstrates Image and Audio Compression Capabilities of LLMs

Source:

Venturebeat
on
September 25, 2023
Curated on

September 29, 2023

LLMs (Large Language Models), mostly recognized for their capabilities in predicting words, are now viewed from a new perspective as strong data compressors, according to a recent research paper by Google's DeepMind. The authors showed that with some modifications, LLMs can compress data as effectively or even better than popular compression algorithms currently in use. This fresh viewpoint can steer the development and evaluation of LLMs in a new direction. LLMs learn to transform inputs such as images or text into a 'latent space' that encapsulates key features of the data. This 'latent space' typically has fewer dimensions than the input space, which allows LLMs to compress the data into a smaller size. In their study, DeepMind researchers tested LLMs' data compression capabilities using vanilla transformers and Chinchilla models on text, image, and audio data. Despite originally being designed for text compression, these models accomplished impressive compression rates on image and audio data, even surpassing domain-specific compression algorithms like PNG and FLAC. However, despite these promising results, LLMs are currently not practical tools for data compression compared to existing high-speed models like gzip. One profound realisation from the research was that bigger LLMs are not necessarily better. Although larger models achieved superior compression rates on larger datasets, their effectiveness decreased on smaller datasets. With these findings, compression can serve as an indicator for evaluating the learning capabilities of these models, potentially impacting the future assessment of LLMs. Yet despite these impressive results, conventional compressors like gzip are not becoming obsolete anytime soon due to their superior speed and size advantages over LLMs.

Ready to Transform Your Organization?

Take the first step toward harnessing the power of AI for your organization. Get in touch with our experts, and let's embark on a transformative journey together.

Contact Us today