Testing: ioimage wdc100dn Video Analytics

By: Benros Emata, Published on Sep 14, 2010

In early 2010, one of the most well known analytic developers, ioimage, was acquired by DVTel. This triggered a debate about its meaning and the state of the overall video analytics market. No doubt, the initial wave of enthusiasm for video analytics in the security industry has long since passed, supplanted with well-founded skepticism.

In this report, we test ioimage's wdc100dn camera [link no longer available] to better understand its performance. We successfully integrated the ioimage camera with the ExacqVision VMS to record video and monitor alerts from our test.

This report continues our video analytics testing series. We recommend readers compare to our other test: Agent Vi, VideoIQ, VitaminD, and Archerfish Solo. VideoIQ is probably the best comparison as they are both 'smart cameras' aimed at the professional market. Agent Vi is another good comparison as their analytic software can be loaded on 3rd party cameras.

We examined the following issues/aspects:

  • How well does detection work in narrow or wide FoVs? Inside we demonstrate a test of detection capabilities from 10ft. to 300ft. horizontal FoVs.
  • How easy or difficult is the system to setup? Inside we examine how to calibrate the system and the key lessons we learned.
  • How well does the system perform in low-light conditions? Inside we provide a comparison of daytime versus nighttime performance
  • Can the system really eliminate nuisance alarms? Inside we challenge the system with scenes containing plenty of troublesome vegetation and lighting effects
  • How is the system optimized to suit various applications? Inside we discuss the advanced settings available inside the system
  • ioimage's analytics performed well in a variety of environments with minimal false alerts due to vegetation and stationary lighting effects. Reflected light from objects in low light scenes was the most notable cause of false positives. Low-light scenes produced a modest increase in false negatives, especially at outer reaches of the scene and lower contrast areas.

    The analytics detected humans quite consistently from very tight field of views (as narrow as 10 feet) to very wide field of views (as wide as 300 feet).

    ioimage analytics need calibration that require domain specific (not IT) skill and experimentation. While the time per camera calibration is minimal (generally 15 minutes or less for a two man team), becoming proficient can take days of training/practice. Failure to properly calibrate scenes generally resulted in increased false negatives (missing intruders) but minimal false positives (nuisance alerts).

    Product Overview

    The wdc100dn [link no longer available] (MSRP $1795) is one of four ioimage intelligent IP cameras featuring built-in analytics.  The ioicam family of IP cameras [link no longer available] are designed for stand-alone or VMS/DVR integrated operation and feature PoE capabilities (excluding the PTZ model), an IR cut-filter, and an integrated web-server for simplifying management tasks.

    The ioicam intelligent IP camera line [link no longer available] also includes:

    Get Notified of Video Surveillance Breaking News
    Get Notified of Video Surveillance Breaking News

    • mmp100dn (3 megapixel)
    • xptz100dn (PTZ - subset of detection types)
    • sc1dn (entry-level VGA - subset of detection types)

    Among the ioicam IP cameras the mmp100dn and wdc100dn support the broadest variety of detection rule types such as intrusion region, tripwire, fence tresspass, unattended baggage, object removal, stopped vehicle, loitering and camera tampering. The mmp100dn provides an additional ability to track objects in a Picture-in-Picture (PIP) or split view format.

    Architecturally, ioimage IP cameras support stand-alone or VMS/DVR integrated operation. Stand-alone operation is accomplished via IP or analog connectivity (the ioicam features an analog output). 

    In an IP stand-alone environment, configuration, live viewing and alarm management are all conveniently performed via a web browser connection to the camera's internal web server. However, in this scenario there is neither support for on-board storage nor local manual/scheduled recording.

    In contrast, the analog method uses an on-screen display for alarm monitoring. Management, configuration and optimization still requires an occassional IP connection to the camera's web server application, and requires an analog recording device to store video.

    Recommendations

    Prospective adopters of the ioicam technology should carefully consider the complexity of the scene to be monitored and the specific detection type(s) that will be required for the deployment.  More difficult scenes require more expertise to deploy, and in some applications the system may not perform well at all.  In more challenging applications the installation will require a strong level of conceptual knowledge to find elegant strategies to circumvent problem areas.

    The support for autonomous stand-alone and VMS/DVR integrated operation provides more deployment options architecturally and thus delivers flexibility in cost for implementing a system.  However, for a smaller than 4 to 8 channel ioicam system, there exists many free to low-cost VMS/DVR solutions that can provide video storage, management, and alarm monitoring capabilities.

    End-users should strongly consider a recording platform for the ioicam system if only to use as a means to gain insights into the system's performance over time, and as a reference for potential optimizations. For example, we successfully integrated the ioicam with a free one (1) channel ExacqVision VMS.  

    Summary of Test Results

    • Only the very simplest of low-complexity scenes with simple detection types can be configured expeditiously with little knowledge and forethought
    • Procedures to calibrate the camera for video analytics operation are clearly described in a step-wise manner in the user interface
    • Following the procedures properly consistently produced calibrations that withstood the internal verification tests
    • Includes some very helpful aids in calibrating analytics, such as a digital zoom PIP feature, unknown camera height calculation, horizon estimation, warning on poor calibrations, "show human size" feature
    • Calibrations challenging on extreme overhead 'shots' and extremely deep/far 'shots' with shallow down-tilts
    • Analytics minimized nuisance alarms due to vegetation exceptionally well during both daytime and low-light conditions
    • Analytics minimized nuisance alarms due to flickering light sources, and other challenging lighting effects very well during daytime conditions.  Under low-light conditions reflective surfaces triggered false alarms when moving vehicle light sources mimicked movements on the reflective surfaces, themselves.  Also, stationary light sources did not produce any noticable frequency of nuisance alarms during low-light conditions.
    • Analytics consistently exhibited poor discrimmination between human and vehicle subjects within human intrusion detection regions. I.e., vehicles trigger a high number of false positives
    • Analytics demonstrated an exceptionally high probability of detection at various horizontal FoVs and depths in a low complexity daytime testing scenario
    • Analytics exhibited high frequency of false negatives on extreme overhead mounting location (80+ feet above detection plane) within a complex scene (high vehicular and human traffic)
    • Optimization settings in the 'advanced' rules definition are intuitive and not overly complex to understand theoretically. We did not extensively test the effectiveness of 'advanced' optimization options

    Physical Overview

    The ioimage wdc100dn integrates analytics inside the IP camera form factor. The ioimage camera does not feature on-board storage, HDMI out, and USB support. In comparison, VideoIQ's ICVR includes a hard disk drive, solid state drive, SD card slot, and USB port within the camera itself. In the following screencast, we cover the physical attributes of the wdc100dn, and point out a few subtle caveats

    Key points include:

    •  The 1/3" CMOS sensor is made by Pixim
    •  The I/O terminal block supports 1 alarm input and 1 relay output
    • Power input supports 12-24VDC or 24VAC
    • BNC connector is for analog video out
    • By default, the analog video out is disabled, but can be enabled in the web interface settings
    • The 3.5mm jack is for two way audio support
    • Two way audio can't be enabled from the web interface - only through third party VMS (if supported)
    • Camera supports CS mount lenses (lenses are not included with the camera)
    • Accessories include a documentation & utilities CD, quick install guide, and audio splitter plug
    • The analog signal provides additional video overlays and on-screen display information that is  not available on the IP video feed

    Installation, Setup, Calibration and Detection Rules

    There are some key concepts to be mindful of when selecting a vantage point for the camera in relation to the scene of interest. When object size is more critical for the application, a shallower camera down-tilt provides more discrimmination of size in relation to varying depths within the scene. Conversely, if accurate distance measurements along the detection plane are of greater priority, a steeper down-tilt provides better resolution or pixels/foot throughout the depth of scene. Theoretically, an example of where distance resolution would be a priority is when detection rules are to rely on speed measurements of objects to optimize detection behaviors.

    Establishing the browser based connection to the ioicam requires Active-X controls and as such Microsoft Internet Explorer is recommended.

    For analytics functionality, the setup of the ioicam system consists of a calibration phase and a rules phase.  The calibration phase requires the application of four (4) human markers and (2) ground markers.  After the calibration markers are provisioned the next step optionally allows the administrator to provide an empirical value for the camera's height above the detection plane and set the horizon line.  However, the manufacturer suggests that you skip straight to the last step of calibration to verify what the system has automatically calculated for camera height and the horizon line.  If your calibration markers were provisioned properly the system's estimates of camera height and horizon line should come fairly close to their actual values.  If the provisioning of markers are done poorly enough then the administrator may recieve a 'warning' message suggesting that calibrations be adjusted.

    Simulating a poor implementation with purposefully inaccurate height and ground marker provisioning produced some interesting effects on system performance. Note that in both the following scenarios, the system provided a warning that the calibrations were inaccurate  (we simply ignored these messages for purposes of testing). 

    Our first simulation of 'poor' calibration defined the human and ground markers such that the system would 'think' objects were smaller than in actuality (more zoomed in than actual). As a result human subjects within the scene appeared much smaller than the detection size range.  This resulted in roughly a one-third to one-half decrease in the probability of detection of human subjects, continued false alarms on vehicles, and no change to the normally observed low nuisance alarm rate on vegetation. 

    The second simulation, erred in the opposite direction where human subjects were made to appear much larger to the system than in actuality.  During this trial, absolutely no detections took place. 

    In the following screencast, we provide a demonstration of the setup procedures required to calibrate the ioicam system and apply a basic human intrusion detection rule.

    Test Scenarios, Methodology and Discussion

    In nearly all test cases we configured a human intrusion detection region 'blanketing' the entire scene. Our methodology does not aim to exclude potential nuisance causing objects (e.g., vegetation, reflective surfaces, etc) from the detection region, but aims to include them in order to test the technology's native capacity to resolve such issues.

    We started tests with the default 'advanced' settings to test baseline performance.  Some adjustments to 'advanced' settings were made to test the system's ability to tune detection behaviors.  In all scenarios the human intrusion detection rule exhibited difficulty in discrimminating against vehicles. And, in nearly all test cases nuisance alarms due to vegetation were minimal.

    In our initial Detection vs. FoV tests we mounted the camera only ~6.5 feet above the detection plane level.  This is below the manufacturer guideline of ~15 feet for detection out to approximately 150 ft depths using a 2-5mm lens.  In this relatively 'controlled' environment we still achieved solid detection results.  Additionally, we achieved reliable detection out to depths greater than 200 feet.

    In an extreme test case where the camera was mounted approximately 80 feet above the detection plane, 'looking' down into a busy intersection, we experienced a clear increase in the level of false negatives on human subjects. This deficiency seemed to disproportionately affect human subjects moving longitudinally across the scene.  Results for this particular trial seem to suggest a limitation for high mounting locations with steep camera down-tilts (overhead shots), such as on roof-tops 'looking' directly or steeply downward.

    In another test case we zoom in to a distant roof-top parking lot.  The camera is situated approximately 35 feet above the detection plane. Due to the length of the 'shot' the camera down-tilt is shallower than the manufacturer guidelines. The FoV did not yield as many pixels per longitudinal distance as a steeper down-tilt would have provided. This created some issues with calibrating ground markers effectively. We use markers of approximately 100ft to compensate for the lack of pixels/foot in the longitudinal direction. Despite some challenges in calibration, the detection capabilities performed nearly equal to the Detection vs. FoV test - low false negatives, low nuisance alarms due to vegetation, and high false positives on vehicles.

    In the low-light (nighttime) test scenario, our vantage point provided an inherently manageable balance between camera height, camera down-tilt, and depth of scene. However, the scene was considerably busier than the more 'controlled' scene in an earlier test (Detection vs. Horizontal FoV). Calibration tasks proceeded without notable issue. We performed a daytime control of the same scene and setup to establish a baseline. The daytime performance was in alignment with expectations based on our previous findings, with a slight increase in false negatives in this fairly busy scene.

    As the scene transitioned to night, the low-light test case exhibited low frequency of nuisance alarms due to vegetation and stationary (at times flickering) light sources. There were some reflective surfaces that consistently triggered nuisance alarms when light from approaching vehicles simulated movement on those surfaces.  The low-light scenario provided a couple of extremely low contrast areas, within which consistent false negatives on human subjects resulted. In comparison with the daytime control, the nighttime trial produced a slightly higher frequency of false negatives, especially at the fringes of the scene and within low contrast areas.

    While there are some clear strengths in the ioicam intelligent IP camera products' usability and performance, the application, implementation and optimization of these types of systems still require expert 'hands' for all but the simplest detection needs. With 'plain' video surveillance applications the behaviors are relatively simple to predict and estimate as the output of the system is mostly optical and visual.  Our own senses can tell us whether a camera at a particular height, azimuth, down-tilt, lens angle, etc. will produce the results the end-user desires.  In analytics the output or performance is not so obvious. The optical/visual output represents information that is only useful in the context of 'human' senses and not the 'analytics' senses.  Thus, the non-intuitive nature of designing intelligent video systems, requires significant conceptual and empirical knowledge to properly design, deploy and optimize an effective video analytics solution.

     

    Detection vs. FoV

    Our first test was designed to determine ioicam's ability to detect human subjects at varying horizontal FoVs. We minimize the complexity of scene by limiting excessive vegetation, human traffic, and vehicles in order to produce focused results. Our test subject enters and leaves the FOV in a lateral progression from one edge to the other.  We repeat this procedure at horizontal widths starting as narrow as 10 feet and ending to as wide as 250+ feet. Note that the lens angle of ~52 degrees results in a near 1:1 ratio of depth to horizontal FoV. For example, at a depth of 100 feet from the camera origin the horizontal FoV is roughly 100 feet, as well.

    Test Parameters:

    • Camera Height: ~6.5 feet above ground level
    • Focal Length: 5mm
    • Horizontal Lens Angle: ~52 degrees (empirical)

    In this low complexity scene there were little to no false negatives at all tested horizontal field of views (10ft to 300ft).  We experienced no nuisance alarms originating from the vegetation near the horizon. Also, some challenging lighting conditions didn't appear to adversely affect the analytics' perfromance.

    In this screencast we examine the results obtained from the Detection vs. Horizontal FoV testing scenarios.

    Performance Issues and Theoretical Optimizations

    In our tests the ioimage product performed well in mitigating nuisance alarms due to vegetation, flickering light sources, small animals, and reflective surfaces.  However, in this video, we demonstrate some of the nuisance alarms discovered during our tests and also some issues with false negatives that appeared more frequently during the nighttime/low-light conditions. In addition, we'll discuss some potential fixes for these types of performance issues.

    • When deploying detection regions, try to place the detection regions in the middle of the scene.  The analytics appear to require some 'breathing' room to monitor and analyze a subject before a detection is triggered. A disproportionate number of false negatives seemed to occur near the fringes in our test scenarios.
    • For human intrusion regions, make an effort to avoid including areas with any level of vehicle traffic. Theoretically, false positives on vehicles can be minimized by decreasing the object 'speed' limit, but this optimization was not tested sufficiently to make any judgements on its efficacy.  Regardless, traffic speed can vary throughout the course of a day and trigger nuisance alarms as a result.
    • For troublesome vegetation, elect to remove them from the detection region. Alternatively, enclose the vegetation in its own detection region and define parameters to lower the sensitivity of the region. I.e, increase the required minimum distance traversed inside region (advanced setting).

1 report cite this report:

New HD Video Analytic Cameras (ioimage / DVTel) on Sep 17, 2014
DVTel is releasing their first HD video analytic cameras, nearly 5 years after acquiring ioimage. With Avigilon EOL'ing VideoIQ products, this is...

Related Reports

Bosch Budget 3000i Cameras Tested on Dec 05, 2019
Bosch has long had a hole in its lineup for, as it describes, "competitively-priced cameras". Now, Bosch has released its 3000i series cameras...
AVTech ~$70 IP Cameras Tested Vs Dahua and Hikvision on Dec 04, 2019
Taiwanese manufacturer Avtech is taking direct aim at low cost leaders Dahua and Hikvision with ~$70 starlight and white light illuminator...
Vunetrix Health Monitoring Company Profile on Nov 26, 2019
Vunetrix boasts that they make the integrators the 'HERO' by using Vunetrix's monitoring. We spoke to Vunetrix to better understand their...
Wyze AI Analytics Tested - Beats Axis and Hikvision, Now Improved, Now Being Terminated on Nov 25, 2019
When we originally tested Wyze's free person detection deep learning analytics in July 2019, they performed well, with few false alarms, but a long...
Top 2020 Trend - AI Analytics on Nov 22, 2019
170+ Integrators answered: What do you think will be the top industry trend in 2020? Why? For the 4th year in a row, AI/video analytics was...
Avigilon H4 Intercom Tested on Nov 20, 2019
Avigilon is well-known for video surveillance and access, but how well does the company's intercom work? We purchased and tested Avigilon's H4...
Open vs End-to-End Systems: Integrator Statistics 2019 on Nov 11, 2019
Preference for open systems is on the decline, according to new IPVM statistics. We asked integrators: For video surveillance systems, do you...
Rhombus Cameras, VMS and Analytics Tested on Nov 06, 2019
Rhombus boasts they have created "the new standard in Enterprise, cloud-managed video security" and told IPVM in January 2019 they offer twice the...
100+ Companies Profile Directory on Nov 06, 2019
While IPVM covers the largest companies in the industry regularly (like Axis, Dahua, Hikvision, etc.), IPVM strives to do a profile post on each...
Aiphone Video Intercom Tested (IX Series 2) on Nov 05, 2019
Aiphone was one of integrator's favorite intercom manufacturers but how well do their products work? The company's newest offering, the IX Series 2...

Most Recent Industry Reports

Disruptor Wyze Releases Undisruptive Smartlock on Dec 06, 2019
While Wyze has disrupted the consumer IP camera market with ~$20 cameras, its entrance into smart locks is entirely undisruptive. We have...
Bosch Budget 3000i Cameras Tested on Dec 05, 2019
Bosch has long had a hole in its lineup for, as it describes, "competitively-priced cameras". Now, Bosch has released its 3000i series cameras...
Anixter Resisting Takeover From Competitor on Dec 05, 2019
Mega distributor Anixter is going to be acquired but by whom? Initially, Anixter planned to go private, being bought by a private equity firm....
Security Sales Course 2020 - Last Chance Save $50 on Dec 05, 2019
This sales course is customized for the current needs and challenges specific to professionals selling video surveillance and access control...
Ireland National Children's Hospital Chooses Hikvision End-to-End With Facial Recognition on Dec 05, 2019
The world's most expensive hospital project ever, the New Children's Hospital in Ireland, has chosen an all-Hikvision surveillance system including...
AVTech ~$70 IP Cameras Tested Vs Dahua and Hikvision on Dec 04, 2019
Taiwanese manufacturer Avtech is taking direct aim at low cost leaders Dahua and Hikvision with ~$70 starlight and white light illuminator...
Ultinous European Analytics Startup Company Profile on Dec 04, 2019
European analytics-startup Ultinous pitches customers to "Have your own video analysis service!" We spoke to Ultinous to better understand their...
Access Startup Multi-Mount Aims To Streamline Reader Installs on Dec 03, 2019
Startup Multi-Mount claims it makes installing access readers 'Fast', 'Secure,' and fit 'any size frame.' The company states its bracket 'fits most...
Resideo CEO To Step Down on Dec 03, 2019
Resideo's CEO, Mike Nefkins, is stepping down, just 18 months after being brought in to lead the now plagued spin-out. Inside this note, we...
Arcules CEO Retracts False GDPR Claim + Dahua and Milestone Claims Examined on Dec 03, 2019
Arcules CEO has retracted a false claim about his organization being a "fully compliant GDPR company" after IPVM reporting (Arcules CEO Threatens...