VMD Shootout II - Hikvision, Dahua, SamsungBy: Derek Ward, Published on Jan 06, 2014
Our first VMD Camera Shootout included Avigilon, Axis, Bosch and Sony. Members enjoyed it and asked for more manufacturers to be tested. This test includes 3 new manufacturers - Dahua, Hikvision and Samsung, building on our original camera test reports:
We followed the same testing approach as our first test series, starting with a simple indoor conference room:
Moving on to a more complex open office area:
And finally, an outdoor area with moving vegetation and headlights on a nearby road:
Our tests looked to answer the following key questions:
- What issues most commonly cause false positives?
- Whose VMD was best at rejecting these events?
- How did detection range vary from manufacturer to manufacturer?
- Can settings be optimized for better detection without increasing false alerts and vice versa?
Users should note that these tests were performed using approximately a typical 60° horizontal field of view with top of the line box cameras from Dahua, Hikvision, and Samsung. We performed some preliminary testing with minidome models and found performance was close in most scenes, but it should not be assumed that all cameras in the manufacturer's line perform similarly to the tested models.
- Dahua's performance was approximately the same as Axis (standard VMD), Bosch, and Sony during the day in both range and false activation rejection. At night, it outperformed Axis standard VMD and Sony, with range about equal to the Bosch NBN-733V, with very few false activations.
- Hikvision outperformed Axis (standard VMD), Bosch, and Sony during the day, at 80' range with few false positives. Detection range at night was better than all four manufacturers in the original test, at 60', without notable false positive issues.
- Samsung's range (~145' outdoors) was greater than all cameras, including Avigilon (~120'), the previous top performer, with few false activations. At night, detection range and accuracy was also better (~80' vs. 50' for Avigilon).
- Compared to Axis VMD 2.1, performance varied depending on the scene. All three manufacturers in this test outperformed Axis VMD 2.1 in low light, where it missed numerous detections at even short range. During the day, Samsung outperformed VMD 2.1 by ~30-40', Hikvision was approximately equal in range (~80-90'), while Dahua was ~30' shorter. Axis VMD 2.1's false positive rejection rate was nearly perfect, however, better than any camera in this test.
These key findings reflect individual camera performance and issues:
- Though Dahua's range was on par with many other cameras, and false activations were not an issue, we found their motion detection to be the least reliable, even quirky. It failed to activate consistently at times despite the scene and settings remaining unchanged.
- The Samsung SNB-5004 had the best detection range in this test, both indoors and out, in full light and dark. However, it was most prone to false activations outdoors, from shadows and passing car lights, or flashing LEDs in the open office test scene.
- Hikvision's performance was near that of Samsung in interior scenes, at all light levels tested. Outdoors, detection range was shorter than Samsung, though better than Dahua. False activations were uncommon, with the exception of shadows moving in the scene, which consistently activated motion detection.
Based on our tests, these are our recommendations on how to best optimize each camera:
Hikvision cameras should use sensitivity 3 (default) or 4. Below this, valid events are frequently missed, while false activations increase drastically above 4.
- We recommend keeping Dahua's default sensitivity of 55, with a threshold of 1-5. Increasing sensitivity above this results in numerous false activactions. In some scenes, using multiple zones to cover the area, instead of a single large zone performed better, but we did not find consistent patterns as to when this is preferable.
- For Samsung, we found that leaving the sensitivity to the default "normal" setting had the best ratio of true detections to false alerts, and lowering or raising sensitivity will either directly reduce detection range or increase the likelyhood of false alerts.
The following videos review configuration of each manufacturer's VMD.
Daytime Detection Range
At ~145', the Samsung SNB-5004 is the only camera to detect our subject, giving it the longest detection range of all the cameras tested.
At ~80', the Hikvision DS-2CD864FWD begins to detect our subject consistently.
Finally, at ~55', the Dahua HF3101N is able to detect our subject.
Nighttime Detection Range
All of the cameras tested had a reduction in detection range, with the Dahua going from ~55' in the daytime to ~45' at night, Hikvision going from ~80' to ~60', and Samsung still having the furthest detection range at night, but dropping from ~145' in the day to ~80' during night.
Indoor Conference Room
In our conference room in full light, all cameras were able to reliably detect a human subject at all ranges up to 20'.
In low light (~1lx), Samsung's VMD was the only camera tested to reliably detect our subject at all ranges up to 20' in our conference room scene. The Hikvision cameras had an 85% accuracy rating with our subject at ~20', and 100% detection when our subject was ~18' or closer. Dahua cameras performed poorly in this scene, offering 60% detection with our subject ~20' away, and attained 100% detection accuracy with our subject ~12' from the camera.
The Hikvision cameras had a 75% false-alarm rate with a shadow moving in the scene, while the Samsung SNB-5004 had a 60% false-alarm rate. Both Dahua cameras did not trigger.
When we waved a flashlight through the room, the Dahua HF3101N had a 10% false-alarm rate, while the Hikvision and Samsung cameras were unaffected.
Opening the door changed the light in the scene and created a false alarm rate ofHikvision 75% and Samsung 90%. Dahua remained unaffected.
Moving the cameras to an open office scene, about 40' long, and our subject ~40 from the cameras, full light performance of Hikvision and Samsung are nominal, with Samsung detecting our subject as soon as he goes around the corner, and the Hikvision detecting him at the same instant ~75% of the time. Both the Dahua box and minidome cameras could not reliably detect our subject from this distance.
However, once our subject begins to move down the aisle, all cameras are able to detect our subject, with increased accuracy the closer he walked to the cameras.
In the same scene at low light, the Hikvision and Samsung cameras retain their similar detection rates as a full light scene, but Dahua is unable to detect our subject.
The Samsung SNB-5004 had a high frequency of overnight alerts provided by a flashing LED. There were roughly 20 alerts every hour. The Hikvision and Dahua cameras did not have false alerts from the flashing LED.
The following cameras and firmware versions were used in the test:
- Dahua HF3101N: 2.210.General 01.0.R
- Hikvision DS-2CD864FWD: V5.0.2
- Samsung SNB-5004: 1.11_131011