 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
| DV Pix - Multigeneration tests with different codecs |
|
[updated 11 June 2000]
Jack used the Mac version of Premiere (whether 4.2 or 5.0 is not known) and the FireMAX DVSoft codec. For each generation, he superimposed a small graphic in a corner of the screen, causing Premiere to decompress the clip, super the new graphic, and recompress it (essentially the same as Processing Inside the Editor).
I performed the same test on a PC using Premiere 4.2 and the Adaptec DVSoft codec (same basic code as the FireMAX DVSoft codec on the Mac) supplied with DPS Spark. I also more recently ran the test on a Mac using Final Cut Pro 1.2 and both the Apple DV - NTSC codec supplied with FCP 1.2 (and to included in QT 4.1) and the DV Plus codec in the ProMax DV Toolkit 3.0. DV Plus is derived from the DVSoft codec.
I also went five generations of making a DV-compressed clip, exporting one frame as a still, and then importing that still to make the next generation clip (see Import/Export), since that behaves quite differently from internal processing-induced decompression/recompression. In Import/Export (as you'll see below), the Premiere/DVSoft combo performs very well, whereas in internal processing the chroma information shows a left shift (about 2 pixels per generation). Codec guru Brad Pillow tells me this is a result of the simple 4-tap filter used for speed in the DVSoft/DV Plus codecs, and that a much better, but slower 9-tap filter exists in the code but has never been turned on. Perhaps in a future release of DV Plus... the DVSoft codec is no longer actively being developed, so it's unlikely we'd ever see the fix there.
Perry Mitchell ran a similar test using the Radius codec in AfterEffects on the Mac to generate his multigeneration tests. He repeatedly made an AE composition, forcing an internal decompress/recompress cycle each time.
Kent Borg ran multigeneration tests on the same image using AfterEffects with the DVSoft, Apple, and Radius codecs, which he comments on in detail in his DV Compression Experiments article at the Multimedia Workshop.
Dmitry Gromov has also tested
a
variety of DV codecs. Have a look at his page, then tell me: is it
just me, or does anyone else think it odd that some of the MMX-enabled
codecs cause a green shift? DVSoft and Microsoft both do, as does SoftDV
(not yet posted on my site, but's noticeable, whereas the Mac version,
like DVSoft without MMX, has no such color shift). Are we seeing a bug
in the accuracy of the MMX engine? Or a bug in the reference
implementation
of DV on MMX that the codecs slavishly follow? The MainConcepts and
Canopus
codecs are clean, though... hmm...
All the images here are "high" quality JPEGs output with PhotoShop; there are very few visible artifacts added by this process and the JPEGs shown here are almost identical to the TIFF versions of the same pix.
The images labeled "1st" (2nd) and "5th" (6th) generations are really "half a generation more": these are generations following compression into a DV file on the PC using DVSoft, whereupon the AVI clip was imported into Mac QuickTime for testing. They show a bit of lateral chroma shift from that first compression, and are not directly comparable to the other clips as a result. The full images from those tests (including luma ramps and other details) are available here.
Likewise, Jack did not specify the location within the original frame of his original image (which is a 295x97 pixel chunk of the original 720x480 frame); Perry centered the image for rendering. My PC tests have it in the lower left corner of the frame; my FCP tests were run with the Stress Test image. This matters because differing positions will cause different alignments of image details with DCT block boundaries, so the comparisons below should be taken as general outcomes, not exactly comparable results. The exact positioning and extent of mosquito noise or chroma offsetting, for example, will differ depending on where the nasty edge is positioned within a DCT block.
Nonetheless, it's interesting to see how FireMAX (Mac DVSoft), PC DVSoft, and DV Plus all cause left chroma shifts. Also, the DVSoft codec shows no such shift when the export/import multigen testing is performed. What's the difference between cycling the codec by exporting/importing stills, and cycling the codec by decompressing/recompressing internally? As Click 'n' Clack would say, "How do it know?"
DV Plus performs just like DVSoft in the "clamped" mode; in the "full range" mode used to cope with white clip, it's very close, though there are minor errors in some luma / chroma values over multiple generations (some of which may be fixed in a later release, according to codec coder Brad Pillow).
The Radius (now Digital Origin, and a division of Media100, at that) SoftDV codec shows no chroma shift when performing internal processing in AfterEffects (Perry reports that occasionally he'll see a very slight vertical chroma shift when working in 4:2:0 with EditDV, but this may simply be the 4:2:0 sampling and decimation coming into play). There does seem to be a tiny fraction more mosquito noise buildup with SoftDV than with DVSoft/DV Plus. SoftDV, like DV Plus, offers a full range mode; it's not known if that mode was used in these images.
The Apple DV - NTSC codec supplied with QuickTime 4.0 (including the updated codec supplied with Final Cut Pro 1.2 and slated for general release in QuickTime 4.1) is a poor performer. Prior to FCP 1.2, the codec also showed odd striping on solid fields when played back through a Sony hardware codec, and many folks were rendering with the "Medium" setting instead of the "Best" setting to work around this (the FCP 1.2 release's update works around the striping by adding a slight dither pattern to solid areas). As the tests show, the Medium setting is much worse than the Best setting.
Apple is aware of the problems with the current codec and are working on an improved version. The QuickTime 4.1.2 and Final Cut Pro 1.2.5 releases (June 2000) use a codec that allows YUV processing, thus avoiding white clip issues and the rounding errors of the YUV-RGB conversion, but reports indicate that poor compression quality is still an issue.
I still need to test the Canopus codec (Yamada-san, want to lend me an eval unit, grin?), which is reportedly very good.
You can also display this page split into two windows side-by-side or one above the other so you can scroll the pix next to each other for comparison purposes. Your browser must support frames for this to work.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
See Kent Borg's DV Compression Experiments for additional tests on this same image.
Dmitry Gromov has also tested a variety of DV codecs using naturalistic scenes.
Two other tests look at multigeneration performance using the hardware codecs in DVCPRO/DVCAM VTRs and compare them against other formats:
The European Broadcasting Union (EBU) ran a series of multigeneration tests, rating subjective quality of DV compared to uncompressed 601 digital video and to Betacam SP. It makes for interesting reading.
The May 1999 issue of DV Magazine used the Tektronix Picture Quality Analyzer to compare a variety of formats, including DV. You can download PDF files of their testing methodology and their results.
Copyright (c) 1998, 2000 by Adam J. Wilt.
Images from Jack Hegen and Perry Mitchell reproduced
by
permission.
The MasterCard and VISA names and logos are trademarks
of their respective copyright holders.
You are granted a nonexclusive right to duplicate,
print,
link to, frame, or otherwise repurpose this material,
as long as all authorship, ownership and copyright
information
is preserved, and a link to this site is displayed.
Last updated 11 June 2000.