Beyond Text: LLMs Show Superior Image and Audio Compression

Source:

VentureBeat
on
September 25, 2023
Curated on

October 11, 2023

Large Language Models (LLMs) are utilized by Google’s AI subsidiary DeepMind, not only for predicting the next component of a word but also for data compression. DeepMind's researchers believe that LLMs should also be considered powerful data compressors. By viewing these models through the lens of compression, LLMs were seen to compress data as competently, if not more than, prevalent compression algorithms. Their findings suggested that these models can be developed and evaluated in novel ways through this outlook. In the experiment, the researchers utilized open-source LLMs and adjusted them for arithmetic coding, a form of lossless compression. When tested on text, image, and audio data using basic transformers and Chinchilla models, LLMs demonstrated striking proficiency. For example, the Chinchilla model successfully compressed data to 8.3% of its original size. Despite being primarily trained on text, they also achieved impressive compression rates on image and audio data, outstripping domain-specific compression algorithms. Even though LLMs have shown promising abilities, they are not viable alternatives to existing compression models due to their size and operational speed. For instance, one LLM model with 3.2 million parameters required an hour to compress the same amount of data, which gzip could achieve in less than a minute. Therefore, the researchers concluded that while larger LLMs do achieve higher compression rates on larger datasets, their efficacy diminishes on smaller datasets. This insight may significantly impact the future evaluation of LLMs.

Ready to Transform Your Organization?

Take the first step toward harnessing the power of AI for your organization. Get in touch with our experts, and let's embark on a transformative journey together.

Contact Us today