Testing Axis Zipstream
By Ethan Ace, Published Apr 13, 2015, 12:00am EDTHas Axis found its breakthrough differentiator? Can they demand a premium for this?
Axis claims Zipstream can "lower bandwidth and storage requirements by an average 50% or more."
We upgraded the firmware on the Axis Q1615 we first tested last September to test Zipstream.
Here is a 1 minute preview of the Zipstream test:
Undoubtedly, Zipstream can drop bandwidth consumption dramatically but here are the big questions:
- What type of image quality degradation exists and when did it occur?
- How much bandwidth consumption savings exists?
- What mostly drives the bandwidth consumption savings - the dynamic RoI compression or the dynamic GOP variation?
- How does Axis Zipstream compare to the most efficient bandwidth cameras on the market and to other bandwidth savings modes (like Arecont and Vivotek)?
- How might it compare to H.265?
- What type of issues occurred with VMSes?
Key Findings
Here are the key findings from the test:
Dynamic GOP variation drove most of the bandwidth savings (50% reduction in bandwidth common with this alone) and appears to be the big innovation here. This is the ability to adjust the I frame interval on demand, long when there is minimal activity (saves bandwidth) but adding an I frame interval on demand when, e.g., something major changes in the scene (ensures image quality does not suffer. In our tests, with Dynamic GOP enabled, we saw I frame intervals commonly range from 7 to 15 seconds.
Minor image quality degradation with no noticeable degradation on faces or license plates but with some clear degradation, especially when dynamic GOP was enabled on foilage or grass (examples provided inside).
Clearly far more meaningful than other bandwidth saving features we have tested before which either significantly reduced quality (e.g., Arecont) or did not make much much of a difference / came off a very high number (e.g., Vivotek).
Better but mixed results against top bandwidth performing cameras. We tested models from Bosch, Hikvision and Samsung. With Zipstream off, Axis was not very competitive. With Zipstream 'maxed' out, only the Bosch was close. In between, especially with Dynamic GOP disabled, results were mixed.
More potential than H.265 with Zipstream Dynamic GOP: While H.265 may reduce bandwidth up to 50% compared to H.264, Dynamic GOP does reduce bandwidth by 50%+ in many / most surveillance scenarios with the added benefit that it requires no changes on the VMS side (which H.265 does) and the potential to work with H.265 for more savings.
No VMS integration issues across all Zipstream settings with Avigilon, Exacq, Genetec and Milestone.
Market Impact
Zipstream is a legitimate improvement and may have a significant market impact. However, remember Zipstream is limited to ARTPEC-5 cameras which only a few existing Axis cameras support. Most new cameras being released going forward will support ARTPEC-5 and therefore Zipstream.
The key question is whether other manufacturers can enable / develop / source dynamic GOP to match Axis.
Hikvision is releasing a 'Smart H.264' which they claim is similar to Zipstream and many manufacturers, including Hikvision, are announcing H.265. All of this will make bandwidth a much more sizeable differentiator for the foreseeable future.
Zipstream Configuration
Zipstream enabled cameras add a new tab to the Video Stream Settings page in the camera's web interface, seen below.
This tab contains two options:
- "H.264 bitrate reduction" dynamically adjusts compression levels in different areas of the scene, with moving objects less compressed, and static background objects more compressed. Settings range from Off to High, each with different levels of bitrate reduction and visible effect.
- "Dynamic GOP" allows the camera to manage the I-frame interval, in a range from whatever is set in the H.264 tab, as a minimum, to a maximum of 1200. I-frame interval is decreased when moving objects are present, and increased when the scene is static to reduce bandwidth.
In this video we review the configuration options for Zipstream, showing visible quality differences (or lack thereof) at off, high, and dynamic GOP levels.
Quantization Effects
Strangely, quantization was reported as a steady 28, with 0 deviation, by AVInaptic, regardless of Zipstream being on, off, low, medium, or high. Technically, this can not be correct. With high, clearly, significant parts of the image are going to be more highly compressed. Nonetheless, it appears that Axis is only reporting the default compression level set.
The image below shows the AVInaptic analysis of two clips, one with Zipstream turned off, on the left, and another with it set to High, with Dynamic GOP on, on the right. Both show a steady quantization graph and zero deviation from Q28.
Dynamic GOP Analysis
The image below shows a comparison of standard fixed GOP (10 in this case) versus the dynamic GOP used by Zipstream (set to 1200 max), as analyzed by AVInaptic.
While I-frames on the left are steady, every 10 frames, interval seen on the right varies, from 74, to 71, to 47, as the camera adjusts interval according to what is in the scene.
Bitrate Impact
Zipstream reduced bitrate in all scenes tested, with best results in low motion scenes and nighttime scenes.
We review Zipstream's effects on a static scene and one with a moving subject in full light in this video. Bandwidth decreased from 1.3 Mb/s when the scene was static to about 80 Kb/s (0.08 Mb/s) when using Zipstream High plus Dynamic GOP.
And in this video example we review bandwidth reductions in the same scene at about 0.5 lux, with more drastic effects, reducing bitrate from nearly 17 Mb/s to ~2.7 Mb/s, an ~84% reduction.
Bitrate Measurements
We first measured bitrates in an indoor low motion scene. In full light, ~160lx, Zipstream without Dynamic GOP was moderately effective, reducing bitrates by 14% on low and 25% on high compared to measurements with Zipstream off. However, turning Dynamic GOP on in this still scene reduced bitrates by over 90% to just over 100 Kb/s.
At 1 lux, the effects of Zipstream's dynamic compression were more pronounced, reducing bitrate by 71-85% Turning Dynamic GOP on in this scene again resulted in an over 90% decrease, though not as great a gain as in full light.
In outdoor daytime scenes, Zipstream was similarly effective to our indoor daytime scene, with up to 26% reduction using dynamic ROI (without dynamic GOP). Turning dynamic GOP on increased reductions to 67 and 75%, depending on the scene. These reductions are significant, but reduced compared to indoor tests due to higher amounts of motion in the scene, from passing vehicles, foliage, and other movements.
Reductions in bandwidth due to Zipstream were very similar when using both high frame rates and low. In our indoor still scene, bitrate reductions due to Zipstream (without dynamic GOP) were slightly higher when using 30 FPS than 10 FPS. However, reductions due to dynamic GOP were slightly greater when using 10 FPS.
Image Quality Impact
In our tests, we saw limited impact on image quality caused by Zipstream. Moving objects and text remained clearly visible, without issues, while background objects (foliage, grass, walls and other finishes, etc.), less important in the scene, displayed some artifacting.
We review these effects in multiple common scenes below.
Human Subjects/Text
The image below compares images of our subject and test chart taken with Zipstream off (on the left) and a fixed GOP of 10 vs. Zipstream set to High, with Dynamic GOP turned on (maximum of 1200).
Visible quality in both is extremely similar, with no notable compression artifacts from the increase compression or longer I-frame interval used by Zipstream. Similar details of the subject are visible, and even the fine text of the test chart in line 8 (OHVCK) is similarly legible in both.
Moving Vehicles
Details were also similar when capturing a moving vehicle, with Off, High, and Dynamic GOP settings all able to capture a license plate at ~15-20 mph in a ~20' field of view without issue.
Outdoor Scenes
Visible quality differences were most noticeable in outdoor areas with moving foliage, such as the scene below. Here, swaying branches and leaves which were clear with Zipstream off appeared blurred or smeared when it was turned on, visible in the Zipstream High image below. When Dynamic GOP was added, these effects were most pronounced, appearing hazy, seen on the right below.
Versus Other Manufacturers
Zipstream is highly effective at reducing bitrate in supported cameras. However, other manufacturer's cameras still provide lower bitrate in some scenes.
For each of the scenes we tested, we did the same exact test and measurement for the Bosch NBN-932V, Hikvision 4132 and Samsung SNB-6004. Below are the results:
VMS Integration
We saw no issues streaming or playing back Zipstream video in Avigilon Control Center, ExacqVision, Genetec Security Center, or Milestone XProtect. Regardless of Zipstream or Dynamic GOP settings, cameras connected and streamed as normal, and playback was smooth, without large gaps, both forward and reverse.
Note that when streaming or playing back clips in VLC Media Player or Quicktime, errors occur, with the programs displaying a blank screen with only P-frame changes visible until the next I-frame. This may be problematic when viewing non-proprietary (AVI, MKV, WMV, etc.) clips in some programs.
Genetec has integrated Zipstream settings into their Config Tool, allowing users to set these parameters from that client, instead of using the web interface, shown here:
Test Procedure
Q1615 firmware version 5.75.2 was used for this test. Note that this firmware is not yet publicly available, but was supplied by Axis for testing.
All camera settings were left defaulted except for WDR, which was turned off in all models, and exposure, standardized to 1/30s.
VMS versions used:
- Avigilon Control Center 5.4.0.30
- ExacqVision 6.8.4.76415
- Milestone XProtect Corporate 2014 9.0c
- Genetec 5.2 SR9
36 reports cite this report:
