adamwilt.com > EBU's 1998 Evaluation of DV Formats
copyright 1998-2009 Adam J. Wilt  
 
DV FAQ - technical - EBU
search
 

updated 2009.02.22 - new link to the entire Final Report; formatting changes.

The SMPTE/EBU Task Force for Harmonized Standards for the Exchange of Program Material as Bitstreams Final Report: Analyses and Results was published in July 1998 and released at IBC '98. This seminal 192 page report provides a roadmap for digital video development, implementation, interchange, and usage. If you're involved in the selection, specification, design, or implementation of digital video systems, you owe it to yourself, your company, and your customers to read this report. If you're just curious about the future of digital video, you will also benefit from perusing its pages. The report is reprinted in the September 1998 issue of the SMPTE Journal, and is also available for download as a PDF from the EBU website.

The following excerpts from Annex C: Networked Television Production - Compression Issues discuss carefully-controlled subjective quality testing of 25 MBit/second (DVCPRO) and 50 MBit/second (Digital-S) DV  compression through multiple generations. As far as I know there is no more impartial, evenhanded evaluation of these formats available. As such, I know these tests will be waved around as "proof" that "DV is perfect" or that "DV is utter crap" depending on which side of the fence one happens to be on. People will use test results the way a drunk uses a lamppost -- for support, rather than illumination! Don't fall into that trap; instead, read these tests as another data point, and integrate them into your understanding of how DV formats fit into the continuum of digital video choices available. Or, as the Final Report itself says in section 3.2 (emphasis added),

With those caveats, let me make a couple of comments:
The following information is copyright (c) 1998 the European Broadcasting Union (EBU) and the Society of Motion Picture and Television Engineers, Inc. (SMPTE). All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means - electronic, mechanical, photocopying, recording, or otherwise - without the prior written permission of one of the copyright holders.

Reproduced by permission of both the EBU and the SMPTE. Thanks to Horst Schachlbauer (EBU), S. Merrill Weiss (SMPTE), and William C. Miller (SMPTE).


C.2.    Compression families for networked television production

For core applications in production and post-production for Standard Definition Television, two different compression families on the market are currently advocated as preferred candidates for future networked television production:

The EBU strongly recommends that future networked television production should focus on compression families based on DV and MPEG-2 4:2:2P@ML which have been identified as being appropriate for television production operations.

The EBU has issued Statement D-82: "M-JPEG in Networked Television Production", to discourage its future use14.

The discussions also revealed that the co-existence of different compression families15 in their native form within both local and remote networked production environments would require the implementation of hardware-based, common agile decoders16. In many instances, such decoders must allow "glitchless switching" and can therefore realistically be implemented within one compression family only. Manufacturers have stated that, within the foreseeable future, the coexistence and interoperation of different compression families requiring a "common agile decoder" within a networked television plant will pose a number of operational problems and will therefore be the exception and not the rule.

The positioning of the above compression families within a future networked digital production scenario requires careful analysis and differentiated weighting of the current and future potential influence of various technical constituents on that scenario.
 

C.4.    Television production based on DV compression

C.4.1.    Format stability

C.4.1.1.    DV compression chip-set

The DV chip-set has been developed for consumer applications. It provides a broad application base with resultant economies of scale in commercial production. The chip-set can be configured for either processing a 4:1:1 sampling raster ("525-1ine countries") or a 4:2:0 sampling raster ("625-1ine countries''). This chip-set is used within camcorders for domestic and industrial use, designed by different manufacturers but increasingly used in professional ENG and studio applications. The 4:2:0 sampling raster requires additional vertical pre-filtering of the color-difference channels to avoid aliasing. However, considerations of the cost and size of the vertical filter result in sub-optimum performance for professional applications.

25 Mbit/s compression is based on a DV compression chip-set, processing the video with a sampling raster of 4:1:1. The prefiltering applied to the luminance and color-difference signals is fixed and does not comply with the figures derived from a "real" 4:1:1 template. Details can be found in the EBU Report: "Tests on Panasonic DVCPRO / EBU Project Group P/DTR."

50 Mbit/s compression is based on a combination of DV compression chip-sets. The chip-set processes standard 4:2:2 digital Video signals without additional pre-filtering. The chip required for pre-shuffling 4:2:2 DV-based 50 Mbit/s is manufactured by JVC exclusively. Details of compression performance can be found in the EBU Report: "Tests on JVC Digital-S / EBU Project Group P/DTR".
 
 

Chip-set: (Note 1) Available
Cost: Consumer Oriented
Application base: Consumer and Professional, Video and PC Market
Source DV @ 25 MBit/s Matsushita, Sony (Note 2), Toshiba, JVC
Source Shuffling @ 50 Mbit/s JVC
Independent source: Next Wave Technology
Application base: Consumer, PC
Standards: (Notes 3, 4) DV: IEC 61834
DV-based 25 Mbit/s, DV-based 50 Mbit/s: Draft SMPTE Standard (PT20.03)
Note 1: Panasonic and JVC have publicly stated their commitment to make the chip-set and appertaining documentation available to all interested parties on an equitable and non-discriminatory basis. This is the reason why DV chip-sets can already be found in a variety of different NLE and PC-based applications.
Note 2: DV consumer and DVCAM only
Note 3: The SMPTE is currently finalizing the details of the Draft Standard for the DVCPRO recording format, (D-7). Details of the mapping of DV macroblocks as well as mapping of digital audio and video data into the SDTI transport stream have recently been submitted to the SMPTE for standardization. The 4:1:1 filtering characteristic is an inextricable part of the Standard which allows broadcasters to retain a degree of predictability of resultant subjective picture quality after cascading. The DV chip-set does allow a degree of fine tuning for motion adaptation as a manufacturers option. In 50 Mbit/s configurations, the shuffling chip further allows a degree of flexibility to handle DCT coefficients.
Note 4: The SMPTE is currently finalizing the details of the Draft Standard for the DigitaI-S recording format, (D-9). Details of the mapping of DV macroblocks as well as mapping of digital audio and video data into the SDTI transport stream have recently been submitted to the SMPTE for standardization.

 

C.4.2    25 Mbit/s intra-frame DV-based compression - basic characteristics for News and Sport
 

A/V Data-rate-Net Storage capacity / 90 min: (Note 1) ca. 28 Mbit/s - ca. 19 Gbyte
Sampling raster / Net Video data-rate: (Note 2) 4:1:1 / 25 Mbit/s
Compression scheme: (Note 3) DCT Transform, VRL with Macroblock pre-shuffling
Editing granularity: (Note 4) One TV-frame
Quality at 1st Generation  Good, comparable with Betacam SP
Quality at 4th Generation: Good, comparable with Betacam SP
Quality at 7th Generation: Still acceptable, better than Betacam SP
Post-processing margin: (Note 5) Small
Error concealment: (Note 6 ) Acceptable
 
Note 1: The net A/V data-rate and the storage capacity required for a 90 min programme are within the data transfer- and storage volume capabilities of modern tape and hard-disk-based mass data-storage devices. The integration of DV-based compression transport streams into fully networked, robot-driven hierarchical storage-management systems, operating within a broad application base is therefore feasible. 
Note 2: The picture quality achievable with the 4:1:1 sampling raster is inferior to the one defined for the 4:2:2 studio and is more closely related to best-possible decoded PAL-I quality. Although this has been obvious to the experts participating in the EBU tests, there was agreement however that, on average, the resultant resolution was still adequate for the applications envisaged.
Note 3: All DV-based compression formats feature special pre-sorting of the macroblocks prior to DCT and VRL encoding. With that exception, DV compression can be considered a member of frame-bound, conventional compression systems. The achievable signal quality of such a system has been tested by the EBU Project Group P/DTR.
Note 4: DV-based compression is frame-bound and allows simple assemble and insert edits of the compressed signal on tape and disk, thus avoiding Iossy decompression and re-compression. However, for edits requiring access to individual pixel elements (wipes, re-sizing, amplitude adjustments), the signals have to be decoded.
Note 5: Post-production potential with 4:1:1 DV-based 25 Mbit/s compression is limited, due to the combined effects of reduced chroma-signal bandwidth and the progressive accumulation of compression artifacts.
Note 6: Compressed video signals require elaborate Forward Error Correction schemes to guarantee data integrity if routed through noisy channels. An overload of the Forward Error Correction system results in the loss of complete macroblocks. Concealment is the obvious solution to cope with such situations; completely erroneous macroblocks can be substituted with spatially-adjacent ones although this will achieve only limited results. The DV-based 25 Mbit/s compression format allows for the substitution of erroneous macroblocks by spatially-coinciding macroblocks from the preceding frame with acceptable results. Frame-bound compression prevents error propagation in this case.

C.4.3.    50 Mbit/s, 4:2:2 intra-frame DV-based compression - basic characteristics for mainstream broadcast production
 

A/V Data-rate-Net Storage capacity / 90 min: (Note 1) ca. 58 Mbit/s - ca. 39 Gbyte
Sampling raster: 4:2:2
Compression scheme: (Note 2) DCT, VRL with Macroblock pre-shuffling
Editing granularity: (Note 3) One TV-frame
Quality 1st Generation: (Note 4) Identical to Digital Betacam
Quality 4th Generation: (Note 4) Similar to Digital Betacam
Quality 7th Generation: (Note 4) Comparable, slightly worse than Digital Betacam
Post-processing margin: (Note 5) Adequate
Error concealment: (Note 6 ) Acceptable
 
Note 1: The net A/V data-rate and a storage capacity required for a 90 min programme are within the data transfer- and storage volume capabilities of modern tape- and hard-disk-based mass data-storage devices. The integration of DV-based 50 Mbit/s compression transport streams into fully networked, robot-driven hierarchical storage-management systems, operating within a broad application base is therefore feasible.
Note 2: All DV compression formats feature special pre-sorting of the macroblocks prior to DCT and VRL encoding. With that exception, DV compression can be considered a member of frame-bound, conventional compression systems.
Note 3: DV-based compression is frame-bound and allows simple assemble and insert edits of the compressed signal on tape and disk, thus avoiding Iossy decompression and re-compression. However, for edits requiring access to individual pixel elements (wipes, re-sizing, amplitude adjustments), the compressed signals have to be decoded.
Note 4: At normal viewing distance, the picture quality of 1st generation DV-based 50 Mbit/s was practically indistinguishable from the 4:2:2 source. At normal viewing distance, experts had difficulty to identify differences between the performance of DV-based 50 Mbit/s through all generations for non-critical sequences. No significant decrease of picture quality was observed up to the 7th generation. In direct comparison with the source, critical sequences processed by DV-based 50 Mbit/s showed a certain softening of sub-areas containing high picture detail. This effect could be observed with a slight increase through each generation. In general, the level of impairment of 7th generation does not compromise picture quality.
Note 5: DV-based 50 Mbit/s compression does not employ pre-filtering. Post-processing margin up to the 7th generation has been rated as adequate for mainstream broadcasting applications.
Note 6: Compressed video signals require elaborate Forward Error Correction schemes to guarantee data integrity if routed through noisy channels. An overload of the Forward Error Correction system results in the loss of complete macroblocks. Concealment is the obvious solution to cope with such situations by substituting complete erroneous macroblocks with other ones. These can be spatially and / or temporally adjacent macroblocks. Concealment is independent of DV-based 50 Mbit/s compression and can be implemented in different ways depending on the application in actual products. DV-based compression processes in segments of five macroblocks, thus, preventing error propagation beyond one video segment of five macroblocks.

C.4.4.    Subjective test results when following ITU-R Recommendation BT.500-7

Picture quality of 4:1:1 DV-based 25 Mbit/s and 4:2:2 DV-based 50 Mbit/s compression schemes have been evaluated subjectively and objectively within a variety of different operating scenarios. The sequences below were presented in the test in 4:2:2 quality (absolute reference) and in Betacam SP 18 quality (relative reference).

The subjective tests were performed in accordance with the rules given in ITU-R BT 500-7 for the application of the "Double Stimulus Continuous Quality Scale, (DSCQS)" method which entails two different viewing distances: four times picture height (4H) for the critical viewing distance, and six times picture height (6H) for the normal viewing distance. The range of quality ratings extends from bad - poor - fair - good - excellent within a linear scale. The difference between the perceived quality of the reference and the system under test is subsequently evaluated and presented on a scale ranging from 0 to 100%. The 12.5% border is defined as the Quasi Transparent Threshold (QTT) of visibility. The processed subjective quality results do not scale linearly. In pictures rated 30%, degradation is quite visible.
 

C.4.5.    Picture quality of a 4:1:1 DV-based 25 Mbit/s compression scheme

The proposed operating scenarios range from acquisition-only to hard news and magazine production. The picture Content of the sequences represents actions that are frequently encountered in both News and Sport.
 

C.4.5.1. Results obtained for sequences subjected to 1st generation post-processing
 

DV 25Mb/s 4:1:1, 1st generation, 4H viewing distance
C.4.5.1.1.    Comments for viewing distance 4H
DV 25Mb/s 4:1:1, 1st generation, 6H viewing distance

C. 4.5.1.2.    Comments for viewing distance 6H

C.4.5.2.    Results obtained for sequences subjected to 4th generation post-processing

The post-production scenario encompassed four generations of 4:1:1 DV-based 25 Mbit/s processing, two of which involved one temporal shift and one spatial shift each.
 

DV 25Mb/s 4:1:1, 4th generation, 4H viewing distance

C.4.5.2.1.    Comments for viewing distance 4H

DV 25Mb/s 4:1:1, 4th generation, 6H viewing distance

C.4.5.2.2.Comments for viewing distance 6H

C.4.5.3.    Results obtained for sequences subjected to 7th generation post-processing

The post-production scenario encompassed seven generations of 4:1:1 DV-based 25 Mbit/s processing, three of which involved one temporal shift and two spatial shifts each.
 

DV 25Mb/s 4:1:1, 7th generation, 4H viewing distance

C.4.5.3.1.    Comments for viewing distance 4H

DV 25Mb/s 4:1:1, 7th generation, 6H viewing distance

C.4.5.3.2.    Comments for viewing distance 6H

C.4.6.    Picture quality of a 4:2:2 DV-based 50 Mbit/s compression scheme

The proposed operating scenario is that of networked mainstream broadcast operations. To assess the picture quality and post-processing ceiling obtainable with 4:2:2 DV-based 50 Mbit/s compression, Digital Betacam was included in the test as an established high-end compression system.

The results given below were obtained for viewing distances at 4H (34 observers) and 6H (26 observers) from a subjective test carried out by the RAI and the IRT on a 4:2:2 DV-based 50 Mbit/s compression scheme.
 

C.4.6.1.    Results obtained for sequences subjected to 7th generation post-processing and pixel shift

The picture sequences were subjected to 7th generation post-processing with the pixel shift characteristics given in the table below.
 

Processing Horizontal Shift (Pixel)
+1 = 2 Y pixel shift right 
-1 = 2 Y pixel shift left
Vertical Shift (Line)
+1 = 1 line shift down 
-1 = 1 line shift up
1st generation to 2nd generation. no shift +1
2nd generation to 3rd generation. no shift +1
3rd generation to 4th generation. no shift +1
4tht generation to 5th generation. +1 no shift
5th generation to 6th generation. no shift -1
6th generation to 7th generation. -1 -2
 
Note: The "Diva with Noise" sequence was originally included in the test. This sequence is an extreme test for all compression systems. The "General" result, expressed as numerical values on the histograms above, represents the average over the sequences tested without inclusion of the "Diva with Noise" test sequence.

 
DV 50Mb/s 4:2:2, 7th generation, 4H viewing distance
DV 50Mb/s 4:2:2, 7th generation, 6H viewing distance
C.4.6.1.1. Multi-generation performance of Digital-S (DV~based 50 Mbit/s) compression C.4.6.1.2.    Multi-generation performance of Digital-S (DV-based 50 Mbit/s) compression and the compression used in Digital Betacam

12. For specific applications, this also includes MPEG-2 MP @ ML if decodable with a single agile decoder.

13. For recording on a VTR, a fixed bit-rate must be agreed for each family member.

14. At the time of writing (July 1998), M-JPEG implementations had no defined structure to aid interoperability at the bitstream level. In order for M-JPEG to become acceptable for use in programme exchange, the following requirements have been identified:

15. A compression family is defined by its ease of intra-family bitstream transcoding and the availability of an "agile decoder" in integrated form.

16. Software-based agile decoding is currently not considered to be a practical option. It is still undefined how an agile decoder will output the audio and Metadata part of the bitstream.

18. Different Betacam SP recorders were used for the 4:1:1 DV-based 25 Mbit/s and 4:2:2 DV-based 50 Mbit/s tests. In both cases, the recorders were in current post-production use and were not specially selected or realigned. The histograms for the 7th generation performance in both test series clearly show the variance of test results achieved with analogue equipment.


Copyright (c) 1998 the European Broadcasting Union (EBU) and the Society of Motion Picture and Television Engineers, Inc. (SMPTE). All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means - electronic, mechanical, photocopying, recording, or otherwise - without the prior written permission of one of the copyright holders.

Reproduced by permission of both the EBU and the SMPTE. Thanks to Horst Schachlbauer (EBU), S. Merrill Weiss (SMPTE), and William C. Miller (SMPTE).


Contact me via email

Posted 30 November 1998; link & formatting updated 2009.02.22.