Mabaloo - Lossless Video Compression Techniques: PART 1

Lossless Video Compression Techniques: PART 1

Written by Anshul Saxena

Compression Techniques are ever evolving and improving themseleves to reduce size requirement of multimedia graphics to as little as possible. This article exaplains the basic techniquies of lossless compression.

Modern business requirements for capturing, creating, editing and processing moving images employ a wide range of techniques for reducing the amount of data to be stored and transmitted. Significant advances are occurring in the reduction of bit-rates for end-use distribution and consumer applications such as Internet video streaming, the DCI (Digital Cinema Initiative), Blu-Ray DVD, and high-definition TV. Audiovisual content is typically compressed at every point possible during the production and delivery process, since the human eye is notoriously forgiving of compression techniques in the delivery process. Of course, there is an economic driver in keeping the size of files as small as possible, for both transmission and storage.

For archivists, however, an important distinction needs to be maintained between lossy and true lossless encoding and decoding systems. Perhaps the most basic question facing archivists is how to maintain the essence of audiovisual content in the smallest and most manageable package possible. Typically, compression is performed when an input video stream is analyzed and information that is indiscernible to the viewer is discarded, based largely on the human-perceptual qualities of the content. The perception of the video content is the main driver of video compression technologies, allowing for the perception of a complete audiovisual experience, while benefiting from data compression of the data stream to the extent that compression of the video signal can be achieved without an impact on the audiovisual experience for the end user. Lossless Compression:

Ideally, any kind of compression process on the audiovisual content should produce a kind of lossless video compression that is imperceptible to the end user of the audiovisual content while simultaneously maintaining the benefits of keeping all of the information of the source and also the benefits of compression during the production process. Lossless compression offers the best of both worlds, with mathematically identical content to an uncompressed file while simultaneously enjoying the benefits of a smaller file size in increased file storage versus uncompressed files, and the benefits of faster transportation of the compressed file.

"Lossless" means that the output from the decompressor is bit-for-bit identical with the original input to the compressor. The decompressed video stream should be completely identical to original. In a non-video context, there exist a number of examples of lossless formats.

Most computer users are familiar with compressed archive file formats such as arc, Pkzip, Alladin’s StuffIt, or UNIX TAR files which find redundancy in data or patterns which can be described using less bits. They often reduce the size of documents and spreadsheets by 4- or 5-fold. Typically, the tape drives used in enterprise data storage systems will permit optional data reduction schemes which process arithmetical patterns in source data, into smaller data sets prior to storage. These data sets are encoded and extracted in less than real-time, so that the streaming rates are greater than for writing and reading uncompressed data. These general-purpose algorithms are designed to compress a wide range of business data including office applications, e-mail, databases and binary data. Sony claims a data-reduction ration of up to 2.6:1 with its Adaptive Lossless Data Compression (ALDC) used in AIT data drives, while a ratio of 2:1 is claimed for LTO. While data extraction has to be 100% accurate, these algorithms are not optimized for images and audio data. These different levels of data compression cannot be cascaded for greater effect. Often, a previously compressed AV file [MPEG, for example] will occupy the same or even greater space on the storage device, when general-purpose data-reduction is applied1.
Wavelet compression is one of the most effective methods of image compression. The algorithm is based on multi-resolutional analysis, and this mathematical analysis has been the frontier of lossless compression over the last several decades. Similar to traditional JPEG compression, a wavelet compression algorithm presents an image as sets of real coefficients, though JPEG uses discrete cosines, instead of wavelets. Most of the wavelet coefficients of a typical image are nearly zero, and the image thus is well-approximated with a small number of large wavelet coefficients. The advantage of wavelet compression is that, in contrast to traditional JPEG’s discrete cosine algorithms, the wavelet algorithm does not divide image into blocks, but analyzes the whole image. The characteristic of wavelet compression allows to get best compression ratio, while maintaining the quality of the original audiovisual signal.

Benefits of Compression in Archives One important requirement in archives is that data is not corrupted in storage or during migration over long periods of time, because content may be re-used and re-purposed over multiple life-cycles. Traditionally, two options have been proposed:

1. moving and storing uncompressed files or streams which are absolutely identical to the original, but may require large amounts of bandwidth and storage space;

2. compressing moving image files or bit-streams in order to reduce storage requirements and transmission times

While most compression algorithms or data-reduction schemes involve irreversible losses of visual or audio information, the JPEG-2000 schema enables significant levels of data reduction for storage and transmission of both still- and moving-images, and guarantees completely faithful reconstruction of content after decoding.

Intra-frame Compression

The basis for traditional JPEG and MPEG encoding of data within individual frames of video is the application of a Discrete Cosine Transform [DCT] to shape individual blocks of image data into similar patterns or sequences of pixels, followed by entropy encoding, which describes these areas in mathematical short-hand. This intra-frame compression saves large amounts of data when compared to the task of storing or transmitting a full description of every pixel in the original image. MPEG-2 and MPEG-4 standards have enabled increasingly adaptive borders of macroblock boundaries to match more closely the transitions in image complexity.

MPEG-4 Part 10/H.264 AVC includes an additional post-filtering stage to smooth the edges of macroblocks which are visible typically at higher compression ratios. JPEG-2000 includes a more advanced intra-frame compression technique known as wavelet, which minimises blocky artefacts when compared to DCT compression.

Inter-frame Compression

MPEG video compression supports a hybrid encoding regime, in which not only individual frames may be compressed, but entire sequences of frames with similarities [usually within a shot or scene] may be encoded with data specifying only the differences between adjacent frames. Under certain conditions, typically at lower bit-rates, these differences over time, known as temporal or inter-frame encoding techniques, require less data for a given picture quality, than does encoding every frame. Inter-frame encoding typically maps groups of pixels within macroblocks which stay the same from one frame to the next [i.e. fixed backgrounds] or which move in the same direction [e.g. moving objects or panned backgrounds]. Rather than recoding these image regions, their relative positions are tracked using motion vectors, which require much less coding.