|
COMPUTER PLAYING OF STEREOSCOPIC VIDEOS Prof. John Hart Program in Atmospheric and Oceanic Sciences University of Colorado Boulder, CO 80302 August 2007 An interesting question is: How does one play stereoscopic side-by-side, or over-under, or separate Left and Right videos out of a PC and into to a pair of monitors or projectors with high image quality and smooth motion (no skipping, flickering, hitches, sync-errors, slo-play, etc.)? I did some "Player" tests using a 3D video I had made for the National Stereographic Association Convention (NSA2007) called Wicked Liquid. It is a video about extreme kayaking and surely tests the mettle of any player with a) fast action sequences with flying water all over the place, b) imbedded HD movie clips that move about in windows, c) high-resolution highly-detailed stills that are both fixed and animated with zooms, pans, rotations, and scrolls, d) complex transitions between scenes, and e) all the previous combined in multiple windows moving within each other, etc., to a tightly synchronized soundtrack. This program runs for 8 minutes and was projected at 1400x2100, or 1400x1050 per side. The goal, especially when showing on a big screen, is for the computer to supply a clean and accurate rendering of the original material. Some common ailments in amateur stereo projection using a PC are: Skips, where the show appears to jump from one segment into the middle of the next, Flickering, where, for example, a smooth pan appears to pulse or flicker uniformly in time, Hitches, where the playing appears to stop momentarily on a frame then jump into motion a fraction of a second later, Sync errors, where one side (L or R) gets out of step with the other, causing painful retinal rivalry, and Slow-Play, where the video can't keep, has dropped frames, runs slow, and perhaps gets out of sync with the audio. A good projection won't have any of these diseases. It will be super smooth. It will also, ideally, retain essentially ALL the fine resolution of the original material. This is especially difficult in the case of stills having smooth Ken Burns style animations without "motion blur" or softening artifacts. Images obtained from high-resolution digital still cameras are inherently more detailed than most continuous video, and hence, counter to intuition, provide a pretty good test, when animated, of digital stereo projection. Note: Motion Blur is a technique designed to make fast moving objects, like a rapidly moving car, appear as a human would see it - blurred or smeared. Motion blur, along with can help reduce flickering when projection is done at 30fps, well below the time response of the human eye, which tapers off above 60fps. However, as shown below, it's hard enough to run a high resolution projection at 30fps, let alone 60fps. Two software systems were tested. 1) The readily available Stereoscopic Player (www.3dtv.at ), and 2) a GPU (fast graphics card processing unit) accelerated program called Depth Q (http://www.depthq.com/server.html). The former was used with the XVID MPeg4 based encoded/decoder (www.xvid.org ), while the latter was used with its own motion-Jpeg encoder/decoder. I chose XVid at the urging of Roger Maddy (www.puppetkites.net ) , who encoded some intial short test clips for me, and provided a lot of helpful advice. The XVid encoder was run in single pass mode with a range of "target bit rates" from 14 to 120 megabits per second (Mbps) using VirtualDub to encode the sources. DepthQ movies were encoded using its own "DQ3D" encoder set to quality 90 or 97 (these are high values, 100 is uncompressed) and 4:4:4 sampling (the best). Two computers were tried: Computer1 : Shuttle Core2Duo 2.67GHz, 2GB DDR2 Ram @ 800MHz, SATA3 Raid0 (2 drives), NVidia Quadro FX1500 Computer2 *** : Shuttle Core2Duo 2.67GHz, 2GB DDR2 Ram @ 667MHz, SATA3 (1 drive), NVidia GT7900 HOW DO THEY PLAY? Two methods of using Stereoscopic Player were tested. a) Single Stream playing of Over-Under combined video. The advantage of single stream is that it is much less likely that the left and right sides will get out of sync. The over-under files were made using Virtual Dub to compress into a final avi, and AVISYNTH to stack the L and R uncompressed avi's into an over-under format. b) Dual Stream playing using separate Left and Right files (generated by running uncompressed L and R avi's through Virtual Dub The table below shows statistics from Stereoscopic Player's ViewProperties/Filters menu. The FILE name shows the bitrate (e.g. WL20 = 20 mega bits per second). The last two lines of the table indicate that the DepthQ (DQ) encoded/decoded movies played fine on computer 1. Computer 2 was not tested with DQ because DQ requires the Quadro graphics card. Rows labeled ** are with Computer 2. All others are for Computer 1. Files labeled DS are dual stream playback. For DS tests the total data rate is shown. Each file in DS mode was encoded at half the rate shown. All files without the DS extension are Single Stream. The file WL80DSNR was a dual stream test, at 80Mbs total (40Mbps per side) on Computer1 with no Raid (i.e. using 1 SATA3 drive only).
Conclusions on Playability: 1) For image quality Dual Stream is BETTER than Over-Under single stream playing (less flicker, higher bitrates possible before hitches and such). Dual Stream is SUBJECT TO SYNC ERRORS (more than over-under). However, for animated slide shows, minor sync errors are not significant. For videos it can be fleetingly annoying. Major sync errors where one side has a totally different image than the other (e.g.WL50DS**, WL80DS**, WL120LR) are not good. With Dual Stream you can get a good play at 50 Mbps, but with over-under, only 30Mbps. I don't know the reason for this, but maybe its associated caching overflows and page faults with the bigger images? 2) Raid0 helps significantly. For example, compare the statistics for WL80DS and WL80DSNR. 3) The Quadro board of Computer 1 helps with fine (1/60 sec) sync errors, because both L and R output channels are phase locked. The Quadro is necessary for Depth Q, since it uses special buffering and GPU acceleration. It is not known if the FX1500 helps with flicker reduction in Stereoscopic Player since I didn't try Computer 1 with a lower end graphics card. 4) With Computer 1, WL50DS (dual stream L and R files, each encoded at 25 Mbps, played in Stereoscopic Player's Dual Monitor mode) gave the best output using Stereoscopic Player. 5) CPU Usage: The windows performance monitor indicates CPU usage is about the same on both Cores, and increases with the bit-rate (no surprise). For example Computer 1 in Dual Stream mode scales like: 25Mbps = 65%,68% average, peak 82%. 40Mbps = 75%,78%, peak 90%. 60Mbps = 80%,84% peak 92%. 6) DepthQ played super-smooth without any flicker or significant flaws at Q90 4:4:4, Q95 4:4:4, and with only minor stumbles at Q97 4:4:4. In a class by itself (in terms of playable data rates). It's Q90 images compare with 80Mbps XVid compressions (see below).
HOW DO THEY LOOK? The other side of the coin involves the degradation of images (particularly stills, in animated or transitional settings) by the compression. Obviously a low bit-rate is going to look worse than a high one. Below are some crops from stills in the midst of being animated. These were obtained by locking onto a frame in Virtual Dub and doing a screen capture (1600x1200 monitor, pixel per pixel view). DQ90 is indistinguishable from DQ97.
My take on this is that 30 Mbps loses quite a bit, especially in areas of low-contrast texture, like the grass and high mountain slopes. Losses are also evident in the water above and left of the Kayaker's head. 50 Mbps is pretty darn good. Clearly, 14 and 20 Mbps are unacceptable. For reference, the full image of the scene (which is zoomed and panned in the show) is shown below:
Downloads of full frame test images (right click SAVE AS and download): Scene DQ97, 14 Mbps, 20 Mbps, 30 Mbps, 50 Mbps, 80 Mbps Kayaker: DQ97, 14 Mbps, 20 Mbps, 30 Mbps, 50 Mbps, 80 Mbps
CONCLUSIONS Using the computer equipment listed above, Stereoscopic Player and the XVid encoder/decoder can play a complex 1400x1050 (per side) stereo movie with good quality. In my opinion, the experience of playing 2800x1050 ProShow-animated slideshows is better using video than by using ProShow Executables - which generate giant hitches and skips at this resolution and nonetheless contain a lot of motion blur. 50 Mbps runs OK, as described above, in the Dual Stream mode. Roger and I agree that 40Mbps is a reasonable bit rate to aim for in playing these sorts of things nicely at this resolution, and 50 gives a little headroom. The above conclusion is probably scalable. To go to 1920x1080 from 1400x1050, increase the bit rate from 50Mbps to about 68 Mbps. Perhaps a hotter Core2 (e.g. a 3.0 GHz Extreme with a bigger cache) could play 68 Mbps with just the same fidelity as Computer 1 plays 1400x1050. FWIW, 2 x 1600x1200 (about the same pixel count as 2 x HD) plays fine at Q90 4:4:4 using DepthQ on Computer 1. My Suggestions for Making Animated Slide Shows at 1400x1050: 1) Use ProShow Producer (www.photodex.com) to make Right and Left uncompressed avi's of animated slide shows. In order to do this load all the right files into ProShow's image directory and make a script. Ouput the R .avi at 1400x1050 and 30fps. Remove all the right files, and then copy all the left files (with exactly the same names as the R files) into ProShow's image directory. Open the script. Output the L .avi as before. In this way the script acts on the L files just as it did on the R files. If you want Z-axis variations, you must change the motions of chosen slides in the L script to contain different translations (pans). 2) Use VirtualDub and XVid to compress the L and R avi's independently. Under VDub's video button, set the frame rate to 30.0 and compression/configure to a target rate of 25000 (type it in). 3) Install Stereoscopic Player (www.3dtv.at) on your computer. When using Stereoscopic Player, "Open Left and Right Files" to play the compressed avi's and use the Dual Monitor viewing mode. Set Monitor1 to Left Channel and Monitor 2 to Right Channel (or vice versa). Play full screen un-scaled to a 1400x1050 projector or monitor. 4) The computer should have: a) Core2Duo 2.67GHz or greater, 1066 FSB b) 2GB 800 MHz Ram c) Raid0 SATA3 2 drive array d) Start with a cheap (e.g. NVidia GT7800) PCIE graphics card, migrate to Quadro FX1400 if necessary. Note that the above is a computer that works. I can't say what slight variants will do. Have fun out there! | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
