|adamwilt.com > HD > Four Affordable HD Camcorders Compared, part 2||copyright © 2006-2007 Adam J. Wilt|
|HVR-Z1, GY-HD100, XL H1, and HVX200 native recording compared, with PDW-F350 and Varicam||search|
Green's four-camera tests in Burbank ("Four
Affordable HD Camcorders Compared"), Chris Hurd (dvinfo.net),
Mike Curtis (hdforindies.com)
and I arranged a four-day test in April 2006 near Austin, TX--the
"Texas Shootout". We explored the four cameras--Canon XL H1, JVC
GY-HD100, Panasonic AG-HVX200, and Sony HVR-Z1--more thoroughly,
recording to their native formats, using test charts and static
scenes, dynamic motion, indoor and outdoor setups, and location work.
We also went handheld, to roughly evaluate the cameras for run'n'gun
work. We shot some scenes using the high-end Panasonic AJ-HDC27H
Varicam as a reference, and took the opportunity to get a first look
at the new Sony PDW-F350 XDCAM HD camcorder, but this report focuses
on the four affordable (under US$10,000) camcorders.
Omega Broadcast Group (omegaaustin.com) provided facilities, monitors, lighting, tripods, cameras, and Macs. Seven camera operators came from across the country, bringing gear with them. A five-person capture crew worked with Mike to record live signals and playback from each camera's native format; one person was fully occupied logging camera setups and shots (see sidebar for credits). We spent four feverish days working to capture almost 60 separate setups, about half of them recorded uncompressed to hard disk in Mike's array of Macs and all recorded to native formats: HDV and DVCPROHD tapes, P2 DVCPROHD cards, and XDCAM HD optical disks. Where possible, natively-recorded files were imported into FCP for analysis (the P2 recordings and 60i/50i HDV), while other clips (24f Canon, 24p JVC, the XDCAM HD shots) were captured via SDI or analog component to uncompressed files in FCP, from which Mike made JPEG2000 or DVCPROHD files to send me via FTP.
In the following, I base my analysis on both on-set playback and discussions we held at the time, as well as detailed examination of footage after the fact. In the former case, we used the 1280x768 Panasonic BT-LH1700W monitor, a 1440x1050 Canon Realis LCOS projector, and a 42" plasma of unknown resolution. Afterwards, I played back footage using Final Cut Pro 5.0.4 using FCP's Digital Cinema Desktop Preview playback to a 1920x1200 HP LCD and analog component playback to the same display using an AJA Kona LH card, for detail, and HD-SDI playback from the Kona to a BT-LH1700W, for highlight handling and dynamic range.
Please read the earlier article for caveats relating to camera tests; I don't have space to reprise them. In the Burbank tests, we inadvertently handicapped the Canon XL H1; in Austin, we ran the HVX200 at a disadvantage: keep that in mind. Also read reviews of the individual cameras (DV Magazine: May 2005, May 2006, June 2006; possibly online at http://www.dv.com/reviews/cameras.php, although their archiving of older articles is spotty). Check http://dvinfo.net and http://www.hdforindies.com for other perspectives and more details: we did more tests (24p, 50i, uncompressed; detailed study of Varicam and F350 images) than I have space to cover.
We shot test charts in standard video gamma and standard color matrices. While this duplicated Barry Green's tests, Mike Curtis wanted fresh baselines, and I had new charts (a DSC Labs Chroma du Monde with seven-step resolution trumpets, the better to see aliasing with, and a MultiBurst Square Wave chart in place of the Combi 4.1). We confirmed the findings of the Burbank tests, and added data on the F350, on the XL H1 with the 3x wide-angle and 16x manual lenses, and on the GY-HD100 with the 13x wide-angle zoom. But we also recorded natively, and my report is solely on the native-recording tests, complete with luma and chroma subsampling, so we're measuring system performance, not just raw camera performance.
We shot charts (indeed, most footage) at apertures from f2 to f4, to avoid diffraction-limited resolution losses. Charts were shot from about five feet away (except for the 3x wide angle lens on the Canon, for which the chart had to be brought to within a foot) with the lenses zoomed, for the most part, a bit past the halfway point towards telephoto. Our intent was not to do a comprehensive test of the extremes of lens performance (unlike Tim Sassoon’s detailed comparison of the Canon 16x manual and 20x servo lenses in the July 2006 DV Magazine). Instead, we wanted to pick a relative “sweet spot” in lens performance so we could get a rough baseline on system performance, with more of a focus—pun intended—on limiting resolution and aliasing from each camera’s sensors as recorded on each camera’s native recording format.
All cameras were shot with sharpness minimized, except the Z1. The Z1 was set to 7, on a scale of 0 to 15, because it gets so soft at 0 we all agreed it needed help. (The Z1's CCDs are 960x1080 native.) In retrospect, we should have done the same with the HVX200, setting its sharpness between -4 and 0 on a scale of -7 to 7; the HVX's 960x540 CCDs are coarse enough that it needs help, too.
The F350 performed admirably in 24p mode with shutter off: 800 TVl/ph horizontal resolution and a solid 800+ lines vertically. However, when the shutter was turned on at 1/48 second or higher, vertical resolution dropped to 540 lines, and the image showed signs of field-doubling, like Cineframe modes on the FX1 and Z1 HDV cameras.
F350 images were crisp and contrasty, and made the XL H1 seem soft by comparison. The F350 shows clear aliasing at 1000 and 1200 TVl/ph on the MultiBurst chart; the XL H1's image is much flatter there. The camera and its lens showed little chromatic aberration (C.A.) on the charts, which undoubtedly contributes to the crispness of the F350's pictures.
We also looked at the Canon XL H1 with the 16x manual lens, and the 3x wide angle. Surprisingly, while they showed considerable barrel distortion, they didn't suffer vastly in terms of overall contrast and sharpness. The stock 20x and the 3x both showed considerable C. A., the well-regarded 16x much less so. Indeed, the 16x looked to equal the 20x overall; what it may lack in detail it makes up for with reduced C.A.
The JVC's optional 13x lens looked about as good on the test charts as the 16x in terms of sharpness, but I wonder how carefully I squared off the charts and focused: different parts of different charts are in focus while other parts are softer. The 13x showed less C.A. than the 16x on both charts, for a visually crisper picture.
The HVX200 shoots both 1080- and 720-line formats, and it's sharper in 1080. While its resolution is only 540x540 (TV lines per picture height horizontally x TV lines vertically), the 1080-line recording preserves more of that resolution.720p recording uses 960 samples/scanline, so filtering for recording causes detail near 540 TVl/ph to be diminished, whereas 1280-sample recording in 1080-line modes has a cutoff at 720 TVl/ph: 1080-line images show no graying-out of detail at all: the images retain considerable contrast at 540 lines, simply switching into aliasing at that point. Displayed on a 1080p monitor, scaled-up 720 images look a bit softer--although on a 720p display, there's little difference aside from the grayed-out area at 540 lines on the vertical resolution trumpets.
It's amusing to compare 720p charts: the JVC HD100, the least expensive of the 720p cameras, records the most detail, since its 1280x720 HDV recording matches the square-pixel 720p format. Both DVCPROHD cameras cut off at 540 TLl/ph, the limits of DVCPROHD's 720p sampling. However, the HVX200's charts show aliased detail well past that point, whereas the higher-resolution--but better filtered--Varicam shows more uniform gray areas in both resolution trumpets and multiburst patches beyond 540 lines. A naive observer might think that the HVX is able to render higher resolutions, based on visible chart detail, but that detail is really a spurious lower-frequency aliasing artifact ("Aliasing in Detail", DV Magazine, June 2006). Look at the small text and other edges on the ChromaDuMonde chart: it's crisper on the Varicam's image.
|camera||CCD pixels, HxV||luma sampling, HxV||H res xV res|
|camera||CCD pixels, HxV||luma sampling, HxV||H res xV res||comment|
|Canon XL H1||1440x1080||1440x1080||800x700+||(800x540 in frame modes)|
|Panasonic AG-HVX200||960x540||1280x1080||540x540||(MTF high at 540 TVl/ph)|
|Sony PDW-F350||1440x1080||1440x1080||800x700+||(800x800+ in 24p; 800x540 in 24p with shutter on)|
|Sony HVR-Z1||960x1080||1440x1080||540x700+||(540x540 in CineFrame modes)|
Notes: Despite the similar numbers for the XL H1 and the PDW-F350, the F350's image were crisper, contrastier, and showed less C.A.
We left the AG-HVX200 in 1080/24PA mode for the rest of the tests.
We shot a high-dynamic-range (HDR) scene with each camera at the same location, so there would be no angular differences between the shots (although the tripod did get moved slightly at one point). The scene, showing Chris Hurd reading Roger Goodman's Guide to the Varicam, is intended to show how each camera reacts to extremes of light and shadow, and how each camera's color sampling affects the image.
We white-balanced on tungsten, and gelled one background light with amber and the other's bottom half with blue, so we could see how the cameras handle not just overexposure, but overexposed color as well. Since there is no motion, the codecs are not particularly stressed.
I took reflected-light readings using a Spectra Cine Pro IV-A light meter across the black and white backdrops; compared to the gray area of the chip chart, the black cloth is 4, 5, and 6 stops darker, while the row of six circles in the patterned white cloth running across to the blue-gelled lamp are 1.5, 2, 2.5, 3, 4.5, and 5.5 stops brighter, an overall range of 11.5 stops.
For this and subsequent tests, we tried to set up each camera for the "best" image visually, usually using nonstandard gammas and black stretch. We set each camera up to expose the crossover point on the CDM's grayscales at 50% in standard gamma, locked exposure, then dialed in the gamma and other settings we desired. We left color matrices and gains alone: if we tried to match the cameras, we'd still be there today, and besides, we wanted to compare the basic color rendering of the cameras.
Canon XL H1: Cinegamma 2, black stretch on, low manual knee, sharpness -7. Gray rendered at 37%
JVC GY-HD100: Cinegamma, black stretch 3 (maximum), manual knee at 80%, sharpness MIN (not off). Gray rendered at 43%
Panasonic AG-HVX200: Cinelike D gamma (knee and black stretch not separately controllable), sharpness -7. Gray rendered at 40%.
Sony HVR-Z1: Cinegamma 1, black stretch on, knee not controllable, sharpness 7. Gray rendered at 37%
Still frames show the results. However, because we tried to set each camera up optimally (and arguably we could have done better on the Panasonic; I'll be doing follow-up tests on the HVX200) and we left color rendering alone, these shots cannot be directly compared for which one is "best"--only for how they differently rendered the scene.
Dynamic range: despite differences in tonal scale rendering, all four cameras were roughly equal in handling extremes of light and darkness. The Canon did the best job of shadow rendering, but (depending on how you look at things) may sacrifice the highlights slightly at this exposure setting. The JVC crushed blacks slightly more than the others, though it nicely separated shadows (as you might expect from its 43% gray, the highest of the bunch). The flatness of the HVX's cinegamma-D curve, without separate black stretch, left the shadows compressed, though the detail is still there.
Color rendering: As the stills show, the cameras do different things with color. To my eye, the Panasonic and JVC are the most natural, most closely matching actual scene colors. The Canon and Sony are more saturated and "Kodachrome"-like; they match each other fairly well, with the Sony tending more towards orange in the reds.
The stills also show differences in color sampling: The HVX200's 4:2:2 color pretty much reproduces the scene as it was, albeit a bit softer horizontally than the luma signal. The JVC's 4:2:0 progressive chroma shows some coarsening in the vertical direction, most noticeably on the red cloth against the green backdrop, and in the cover of Goodman's Guide. The Canon and Sony, using 4:2:0 interlaced images, show pathologically evil chroma "sawtooths" in the red cloth, Goodman's Guide, the orange pitcher, and the test chart--an artifact of 4:2:0 interlaced color.
This sawtoothing can be ameliorated with chroma blur, but it's even more of a nuisance than DV25's 4:1:1 sampling. In fairness, FCP's codecs show what's there without chroma smoothing (see Codec Differences), whereas playback hardware and other software codecs often provide some sort of smoothing or filtering: 24f images captured from the XL H1's tape via a Kona card to uncompressed 4:2:2 files on the Mac do not show sawtoothing: the Canon smooths the image upon output, arguably with a slight drop in chroma resolution.
The XL H1 stands apart in highlight handling. Observe the blue wash on the backdrop, for example. Most of the cameras simply let color components saturate, with hue shifts in areas of incipient overexposure; the blue wash goes cyan as both the blue and green channels reach maximum values. The Canon desaturates these bright areas, preventing hue shift. While there may be a slight loss of detail in some overexposed areas as a result, many find the resulting image more naturalistic. Different cameras, both SD and HD, handle the abruptness of hue shift in different ways (most probably due to differences in knee processing), but the XL H1is the first low-cost camera I've seen that eliminates it entirely.
Noise: All the cameras are clean in the highlights, but show different noise levels in the shadows, mostly noticeable in the wall and the floor at the left side of the picture. The Z1's image is cleanest, followed closely by the XL H1's. The JVC is noticeably noisier, and the HVX200 is slightly noisier than the JVC. The HVX's noise is characteristically colorful, while in the others it's mostly luma noise. Perhaps the higher chroma resolution of DVCPROHD is partly responsible, but even in E-E mode, the HVX is visibly noisier.
We set up the cameras side-by-side, and introduced two models, one blond with pale skin and the other dark-skinned and dark-haired. We shot in wide, medium, and closeup framings; we shot close up at +1 and +2 stops to look at overexposed skintones; we shot medium-wide at -2 stops for shadow handling, then switched off key and fill for more underexposure. We also shot medium-wide at +6dB and +12dB.
Both the HD100 and the XL H1 handled overexposure the best, showing the least blowout and less hue distortion just prior to blowout. Their +2 stop shots were only slightly worse than the HVX200's and Z1's +1 stop shots--but recall that exposures were set in normal gamma on the gray card and aren't directly comparable: the HVX is nominally 1 stop overexposed in this setup. The HVX200 handled hue in overexposure a bit better than the Z1 but not as well as the JVC or Canon.
In underexposure, the JVC fell below the others in shadow detail, but on the WFM it appears its master black level was being crushed at 0% (and the HVX200's is just riding above it) while the other cameras have their master blacks set a few percent above 0%, so this test is inconclusive (it also explains the crushed blacks in scene #9).
At high gains, the Z1 was the cleanest: its +12 dB pix were better than the others at +6 dB. The Z1 also held its chroma the best as gain was boosted, while the others tended to desaturate unevenly. The XL H1 was a close second in noise levels, while the HVX and HD100 got the worst marks: at +12 dB, they looked like fast 16mm film from the 1970s.
We had our models stand in front of a greenscreen and blew their hair with a fan. They walked at varying speeds past the set, and danced in it, in medium and closeup framings. They shook patterned blankets to stress codecs. Martial artists sparred in both normal and low-key, side-lit scenes.
For the most part, all the cameras behaved perfectly well. The HVX showed no motion degradation; with its intraframe DVCPROHD codec, moving frames look as good as static frames. However, in normal playback, the long-GOP formats handled these scenes just as well as the HVX, with no noticeable compression artifacts. Only on the "shaky blanket" tests did the long-GOP cameras show a bit of degradation in the fast-moving detail, with the XL H1 and the F350 doing the best and the HD100 and Z1 showing noticeably more compression artifacts on still frames--but all clips, played back at 1x speed, looked fine: it required single-framing or slo-mo playback to notice defects.
Our quick 'n' dirty greenscreen setup didn't provide adequate separation between our blonde model and the background, making keying difficult, at least at my level of experience. I found it easier to get a clean key with less background spill on the DVCPROHD footage than on any of the HDV clips, but I don't know whether to attribute this to the HVX's 4:2:2 chroma sampling or its different color rendering overall.
We took the cameras outside and had each operator hand-hold each camera while shooting a martial artist practicing his fighting-staff moves, so we could compare handling, image stability, and ease of focusing.
As might be expected, the shoulder-mounted HD100 and XL H1 gave the most stable images, with the JVC capturing the smoothest moves despite its lack of image stabilization. The HVR-Z1 did very well for a handheld; Sony's excellent optical Steadyshot soaked up whatever tremors the operators imparted to the camera. The HVX fell behind the others; its weight and off-center handgrip clearly reduced everyone's ability to handhold it smoothly. Panasonic's optical image stabilization is less aggressive than Canon's and Sony's, and it appears to operate over a smaller angular deflection, so it was less effective in removing handheld jitters.
None of the four cameras excelled in on-the-fly focusing, with every one showing at least a couple of out-of-focus shots. The JVC and Canon appear to have been shot at smaller apertures (deeper depth of field) so it's hard to make conclusive statements about focusability.
All operators expressed a preference for the crisp CRT finders on the F350 and Varicam--their larger sensors and corresponding shallower DoF may have influenced this somewhat, but no one could deny the advantage that crisp, 500+ line CRT finders with variable peaking gave over the coarse LCDs on the low-end cameras.
I took each camera in turn and tracked Mike Curtis as he walked and ran twice around me over the course of about 40 seconds, passing areas of shadow and highlight, to get roughly comparable footage of fast motion and typical sunlit-scene contrasts. I also held a medium telephoto lockdown in the middle of the shot to see how stably I could hold the camera. I ran the F350 at both 25 Mbit/sec and 18 Mbit/sec to compare it to the other cameras (I had meant to run it at 35, not 25, but mistakes were made...).
As in the previous series of shots, all my lockdowns were acceptable aside from the HVX, which I wasn't able to hold sufficiently still. In motion, though, the HVX footage wasn't substantially worse than the footage from the other cameras.
This scene was the most revealing in terms of compression quality, as Mike ran quickly past the shadowed front of Omega's facility. While the HVX's pix were boringly consistent no matter what was going on, the long-GOP cameras all showed increased noise during the most stressful parts of the shot.
|camera||bitrate/GOP||still quality||1x quality|
These rankings are on arbitrary scales. The quality degradation seen in normal 1x playback is an increase in noise in stressful scenes; it's only in still-frame or slo-mo playback that compression artifacts show up as distinct defects.
What impressed me what how good Canon's HDV codec held up, even better than XDCAM HD's 25Mbit VBR codec (although the difference in the shots makes such comparisons suspect). And while XDCAM HD's 18Mbit VBR image was visibly worse than the 25Mbit image, it wasn't unusably so. JVC's codec also fell short of the mark here, even though it had an easier job (30 progressive frames/sec @ 1Mpixel/frame vs. 60 interlaced fields/sec at 1.5 Mpixels/frame), but again, the pix were perfectly usable in 1x playback.
Back in the studio, we measured each camera's MOD (minimum objective distance, or near focusing limit) at full wide and full telephoto. We also found the "breakpoint" where the MOD starts moving out as one zooms in.
|camera||lens||WA MOD||breakpoint||tele MOD|
- "Z" scales run from 00 to 99; could not correlate Canon's Z scale with focal length due to lack of a lens scale.
- Canon specifications give the XL H1's 20x WA MOD as 20mm (.79"), tele MOD as 1m (39.4")
- HVR-Z1 breakpoint measured at Z51 in Austin, but we didn't write down its focal length; I measured Z56 and 12mm on my own Z1 after the fact.
The cameras were hauled out to Auditorium Shores, on the south bank of the Colorado River across from downtown Austin. There we shot the statue of Stevie Ray Vaughan, trees blowing in the wind, and a slow pullback from a downtown skyscraper to reveal downtown. Stuart English's daughters posed for a telephoto shot by the water, and our two models sat on a blanket for two shots. We photographed a dragon boat (in Austin!) practicing on the river, and closeups of rocks and sticks being tossed into the water--all to get some real-world footage with sparkling water, big skies, people, and both natural and man-made backgrounds.
The HD100 showed "sticky details" in high-complexity, gently moving areas of foliage: some elements of the image would freeze for six frames, then jump slightly, instead of moving continuously. I shot foliage specifically to invoke this, but it was also seen in reeds, grasses, and leaves on the far riverbank behind the dragon boat. It's a subtle defect, but on the big screen I wasn't the only person to notice it. None of the other cameras showed "sticky details".
The removable-lens JVC and Canon both had noticeable color fringing on contrasty edges of buildings, telephone poles, and the like, while the Z1 and HVX were less affected. Surprisingly, the JVC's 13x lens was no better on location than the 16x, despite its superior showing on the test charts.
The models on the blanket both had light-blasted highlights on their skin. My preference here is for the JVC's rendering, followed closely by the HVX: the blown-out areas hold a bit of chroma and look a bit more natural than the Canon's white highlights--which only goes to show that what looked good in the studio didn't look quite the same in the field. Conversely, the JVC's highlight rendering on water splashes was the least pleasing; its green-tinged watery highlights looked less natural than the desaturated renderings of the other cameras.
At no time did our HD100 show the "splitscreen effect", regardless of subject, light levels, or gain settings.
At the conclusion of these articles one is supposed to pick "the best camera" and cast the others into darkness and damnation.
All these sub-$10,000 cameras would have been deemed miraculous two years ago. At the same time, compared to their more expensive brethren (from the Panasonic HDX900 and Sony PDW-F350 to the Varicam and the CineAlta), they all have severe limitations. Each is a study in compromise; each manufacturer chooses some aspects of performance and handling to optimize, invariably at the expense of others.
Canon XL H1: The sharpest of the bunch, the best-looking 25Mbit codec, impressively low noise (and that's without engaging any of the noise-reduction options), and no hue shifts in highlights. Shoulder mounted configuration aids stability, but it's still a handful. Highlight handling looks better in some shots than in others. Interchangeable lenses are a plus, but the stock servo zooms, while responsive in run'n'gun situations, are frustrating when precise, repeatable moves are called for. Vivid colorimetry is a bit much, but it's almost infinitely adjustable. 24f and 30f modes compromise vertical resolution.
JVC GY-HD100: Sharpest 720p recording and very pleasing, naturalistic image rendering with excellent highlight handling. A shoulder-mounted HD100 makes stable, steady pictures, and it's an ergonomic delight. Best focusing aids of the bunch. Interchangeable lenses with calibrated zoom and focus scales. Its codec suffers the most degradation under stress, and long-GOP "sticky details" detract from subtle motion rendering. It's a bit noisy, too.
Panasonic AG-HVX200: DVCPROHD recording with consistent, surprise-free rendering of simple and complex scenes alike. Pleasing colorimetry, but lots of noise. Least stable handheld, and softest image.
I think we discriminated against the HVX by using CINELIKE D gamma and minimum sharpness; HD NORM or HIGH gamma with low manual knee and midrange sharpness setting might be better choices. I'll be doing follow-up tests with different gammas and sharpness settings. Also bear in mind the HVX's unparalleled flexibility: 720p and 1080i/p, variable frame rates, timelapse, and single-frame. This camera does things the others can't, something not explored in this sort of test.
Sony HVR-Z1: It has the cleanest image, but was otherwise undistinguished. It has no true 24p option. It is however the cheapest of the bunch, has in-camera downconversion with letterboxing, full 50Hz/60Hz switchability in both SD and HD, superb optical stabilization, and arguably the best servo zoom available.
Which one would I pick? I want the ergonomics and tonal scale rendering of the JVC along with the sharpness and image processing of the Canon, combined with the frame-rate and format flexibility, intraframe recording, and color of the Panasonic, and the low cost, low noise, optical stabilization, and international compatibility of the Sony. And I want better glass than any of them had. No camera wins hands-down; none of them lose. Different tools for different needs.
Side-by-side tests like this overemphasize differences rather than noting similarities; you can take any one of these cameras and shoot stunning material with it. Which one you pick depends on your aesthetic preferences, working style, and postproduction workflow.
Just choose one that works best for you and start shooting: remember, talent trumps technology every time.
Okay, lemme see what I can remember... (Please keep in mind this is all coming from the perspective of a narrative and music video cinematographer/tv and doc camera operator- All of these opinions are based on operating only since I haven't seen the footage comparisons yet)
Canon XL HI:
All materials on this page
copyright © 2006-2007 by Adam J. Wilt.
You are granted a nonexclusive right to reprint, link to, or frame this material for educational purposes,
as long as all authorship, ownership and copyright information is preserved and a link to this site is
retained. Copying this content to another website (instead of linking it) is expressly forbidden.
last updated 2007.06.30