Motion Detection Performance Tested

By: Benros Emata, Published on Sep 05, 2011

Motion detection is an important element of many, if not, most surveillance systems. It plays a central role in both storage search time reduction. Storage is routinely reduced by 30% - 80% by using motion based rather than continuous recording. Likewise, an investigator can often much faster find a relevant event by simply scanning through areas of motion rather than watching through all video.

At the same time there are a number of challenges associated with using motion detection:

  • Scene Conditions: The accuracy of motion detection and the amount of times motion is detected can vary depending on what's in the scene - people, cars, trees, leaves, etc. - and the time of day - night time with lots of noise, sunrise and sunset with direct sunlight into a camera, etc.
  • Performance of Detector: Motion detetion is built into many surveillance products - from DVRs to VMS systems and now IP cameras. As such, how well each one works can vary significantly.

Our Test

In this report, we share our results from a series of tests we performed to better understand motion detection performance. 

We did a series of tests in different locations:

  • Indoor well light scene to simulate the simplest scene possible
  • Indoor dark scene (<1 lux) to examine what problems low light caused
  • Outdoor parking lot to see how a complex scene with trees, cars and people would perform
  • Roadway to see how a moderately complex scene with periodic cars would perform

Three IP cameras were used with their motion detection enabled to see differences in performance:

With these tests, we answered the following questions:

  • How can one estimate motion percentage accurately?
  • Does motion estimation vary significantly by scene?
  • How accurate was motion detection in each scene?
  • Did certain cameras exhibit greater false motion detection than others? What scenes or conditions drove those problems?
  • Here are the key findings:

    • In simple scenes (e.g. good lighting and narrow FoV), estimating percentage of motion is relatively straightforward and has a low margin for error.
    • In complex scenes (e.g. outdoor, wide FoV, difficult lighting, and vegetation) estimating perentage of motion is difficult and can vary greatly.
    • Variance in performance across cameras increases significantly as the complexity of the scene increases.
    • All three cameras performed ideally (no false positives/negatives) in the very simple daytime indoor test
    • In the parking lot test all three cameras tended to record on motion in near continous fashion, due to vegetation and lighting effects
    • The Arecont Vision camera had significant false positives in low-light/nighttime scenes likely due to noise/gain as well as when sun is setting into the camera's FoV. This resulted in near continuous recording.

    In light of these findings the following is recommended:

    • Unless the scene is very simple, motion detection settings (e.g. masking, sensitivity, and object size) will likely need to be optimized in order to achieve worthwhile benefits
    • For low-light scenes, consider reducing maximum gain. Also, experiment with different camera makes/models as performance can vary considerably in low-light/nighttime scenarios
    • Be careful about estimating motion in complex scenes as one's estimates can be way off

    Daytime Indoor (Artificial Motion)

    Get Notified of Video Surveillance Breaking News
    Get Notified of Video Surveillance Breaking News

    In this scenario, motion detection is configured via the VMS interface on all cameras. The cameras are left to motion record for a period of ten minutes. A subject enters the scene to introduce motion at three evenly spaced intervals during the ten minute span. A snapshot from the motion recorded video and playback timeline is featured below. Note that on the timeline 'blue' represents motion recorded video and 'gray' indicates no video recording.

    [link no longer available]

    Observing the timeline reveals that all cameras reliably detected motion during each of the three subject entries into the scene. Also, just as importantly, no false positives were triggered during times when the subject was out of the scene. The findings are not surprising given the simplicity of the environment (narrow FoV, strong and consistent lighting, no vegetation, and large subject/object).

    Nighttime Indoor (Artificial Motion)

    In this scenario, motion detection is configured via the VMS interface on all cameras. The cameras are left to motion record for a period of ten minutes. A subject enters the scene to introduce motion at three evenly spaced intervals during the ten minute span. A snapshot from the motion recorded video and playback timeline is featured below. Note that on the timeline 'blue' represents motion recorded video and 'gray' indicates no video recording.

    [link no longer available]

    In this low-light environment, motion detection/record behavior now varies distinctly across the three cameras. The Arecont due to noise and lighting effects continually 'believes' motion is occuring and as such effectively records in continuous fashion (high false positive rate). In direct contrast the Panasonic never 'believes' there is motion present throughout the ten minute scenario and as a result has produced no recording at all (high false negative rate). The Axis performs the same as it had in the daytime indoor situation, sensing motion/non-motion appropriately.

    Daytime Outdoor (Natural Motion)

    In this scenario, motion detection is configured via the VMS interface on all cameras. The cameras are left to motion record for a period of 30 minutes. All motion is organic and representative of a real world surveillance scenario. A snapshot from the motion recorded video and playback timeline is featured below. Note that on the timeline 'blue' represents motion recorded video and 'gray' indicates no video recording.

    [link no longer available]

    The frequency of motion detection/recording as seen on the timeline is inordinately high for all three cameras (~80 - 90% motion). The more complex environment (e.g., trees, shadows and other lighting effects) causes quite a number of false positives across the board. Note that the Panasonic nearly continously recorded throughout the 30 minute test, and that the Axis was just slightly less hyper sensitive. The Arecont was the least sensitive among the three but still considerably over sensitive when considering just conventionally relevant or important motion (e.g. human and vehicle subjects).

    24 Hour Roadway (Natural Motion)

    The following is a sequence of four timeline snapshots of two cameras (AV1315DN and P1344) configured to motion record throughout a 24 hour period. The cameras view the same roadway scene with the same FoV/lens angle. Their comparison provides some insights into differences in motion detection/recording sensitivities across different camera makes/models. Note that the VMS is used to initialize the motion detection settings from the camera and are left at the VMS prescribed defaults.

    Late Afternoon (3:30pm - 5:30pm)

    [link no longer available]

    During the ~2 hours of daylight depicted on the timeline the Arecont camera exhibits considerably greater sensitivity than the Axis camera despite near identical FoVs and default (VMS prescribed) motion settings. As a result, the Arecont produced considerable false positives, whereas the Axis is characterized by quite a number of false negatives.

    Evening (6:00pm - 8:00pm)

    [link no longer available]

    The playback is queued up on an Axis false negative, as evidenced by the presence of a vehicle entering the FoV captured by the overly sensitive Arecont camera. Also note how the Arecont begins to nearly continuously record after sunset around 7:15pm (red arrow), while the Axis tends to maintain the same frequency of motion detection/recording.

    The high false positive rate (near continuous record) exhibited by the Arecont after sunset is somewhat predictable given that the Arecont produced similar results in the Nighttime Indoor test.

    Nighttime (8:00pm - 10:00pm)

    [link no longer available]

    Here we see the trend of the Arecont's hyper sensitivity and near continuous recording continue during the nighttime/low-light hours. The playback is queued up on an incident where both the Arecont and Axis detected motion.

    Early Morning (5:00am - 7:00am)

    [link no longer available]

    In this timeline snapshot, the Arecont begins to revert back to a less sensitive detection behavior at sunrise (red line). However, it is still clearly over sensitive and still produces considerable false positives and a fairly heavy duty cycle of recording to non-recording (~1:1 or 50%). In contrast the Axis continues to motion record at roughly the same frequency as in previous time periods.

    Other Natural Motion Scenarios

    The following timeline snapshot depicts the motion detection/recording behavior of two scenes: (1) small parking lot, and (2) indoor office. Motion was organic/natural to get a sense of the expected recording percentages of some typical surveillance scenes.

    Small Parking Lot & Indoor Living Room/Office Space (3:30pm - 5:30pm)

    [link no longer available]

    The parking lot motion detection/recording frequency as depicted on the timeline was generally reliable and performed as expected. Most incidents were of legitimate human and vehicle traffic. Note that the FoV is of only moderate width and is well constrained to the overall area of interest.

    In the indoor office scene, the majority of the incidents were due to legitimate human activity/motion.

    Methodology

    Here are the three (3) cameras used in the 'Motion Based Recording' study:

    • Arecont AV1315DN (online $460) - 1.3MP D/N; 1/2.7" CMOS; MPL4-10; 0.1/0 (Color/BW)
    • Axis P1344 (online $759) - 720p D/N; 1/4" CMOS; F1.2 Computar ; 0.05 Lux (BW)
    • Panasonic WV-SP306 (online $550) - 1.3MP D/N; 1/3" MOS; WV-LZA62/2 lens; 0.3/0.05 lux (Color/BW)

    Here are the firmware versions for each of the cameras:

    • Arecont AV1315DN - fw 65197
    • Axis P1344 - fw 5.22.2
    • Panasonic WV-SP306 - fw 1.30
Comments : Members only. Login. or Join.

Related Reports

Dahua Smart Motion Detection Camera Tested on Mar 03, 2020
Dahua has introduced Smart Motion Detection, AI-based VMD, claiming to use an...
Avigilon Appearance Search Tested on Oct 30, 2019
Avigilon Appearance Search claims that it "sorts through hours of video with...
Integrated IR Camera Shootout 2020 - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Panasonic, Uniview, Vivotek on Jan 30, 2020
The best and worst cameras tested in this IPVM shootout showed major...
Vivotek "Neural Network-Powered Detection Engine" Analytics Tested on Sep 17, 2019
Vivotek has released "a neural network-powered detection engine", named Smart...
Hanwha AI Object Detection Tested on Sep 28, 2020
Hanwha has added detection and classification of people, cars, clothing...
Hidden Camera Detectors Tested on Nov 18, 2019
Hidden cameras are a growing problem as cameras become smaller, cheaper and...
Avigilon H5A Analytic Cameras Tested on Oct 07, 2019
Avigilon has released its H5A analytic cameras, claiming to "detect more...
Facial Recognition 101 on Mar 18, 2020
Facial recognition interest, use and fear is increasing. This guide aims to...
Mobotix Thermal Detection Camera Tested on Jun 09, 2020
For years Mobotix has struggled but now sales are surging driven by Mobotix's...
Access Control Levels and Schedules Tutorial on Sep 29, 2020
Configuring access levels and setting up schedules is central to maintaining...
Avigilon Elevated Temperature Detection Camera Tested on Sep 17, 2020
Avigilon has entered the temperature screening market with the release of...
Injes Tiny Temperature Terminal Tested on Jul 17, 2020
While temperature terminals have trended bigger, the Injes DFace801 is...
Avigilon Face Mask Detection Tested on Jun 24, 2020
Face mask detection or, more specifically not wearing a face mask, is an...
Face Detection Shootout - Dahua, Hanwha, Hikvision, Uniview, Vivotek on Jul 30, 2020
Face detection analytics are available from a number of manufactures...
Camera Field of View (FoV) Guide on Nov 13, 2019
Field of View (FoV) and Angle of View (AoV), are deceptively complex. At...

Recent Reports

New Products Show Fall 2020 continues tomorrow with Genetec, Milestone, Avigilon, Microsoft and more! on Sep 29, 2020
IPVM's sixth online show continues tomorrow and will feature New Products...
Avigilon / Motorola VS Virtual ISC West on Sep 29, 2020
ISC West has historically been so dominant that no player would think of...
Dartmouth College Deploys K3 Temperature Screening on Sep 29, 2020
While Dartmouth College has a $6+ billion endowment, the College has bought...
Hanwha AI Object Detection Tested on Sep 28, 2020
Hanwha has added detection and classification of people, cars, clothing...
Favorite Access Control Manufacturers 2020 on Sep 28, 2020
200+ Integrators told IPVM "What is your favorite access control management...
OnTech Smart Services Partners With Google and Amazon To Compete With Integrators on Sep 25, 2020
A pain point for many homeowners to use consumer security and surveillance is...
The Future of Metalens For Video Surveillance Cameras - MIT / UMass / Immervision on Sep 25, 2020
Panoramic cameras using 'fisheye' lens have become commonplace in video...
Hikvision Sues Over Brazilian Airport Loss on Sep 24, 2020
Hikvision was excluded from a Brazilian airport project because it is owned...
China General Chamber of Commerce Calls Out US Politics on Sep 24, 2020
While US-China relations are at an all-time low, optimism about relations...
Verkada Disruptive Embedded Live Help on Sep 24, 2020
Call up your integrator? Have someone come by the next day? Verkada is...
IP Networking Course Fall 2020 - Last Chance - Register Now on Sep 23, 2020
Today is the last chance to register for the only IP networking course...
Drain Wire For Access Control Reader Tutorial on Sep 23, 2020
An easy-to-miss cabling specification plays a key role in access control, yet...
Norway Council of Ethics Finds Hikvision Human Rights Abuses "Ongoing" on Sep 23, 2020
Hikvision's involvement in "serious human rights abuse" in Xinjiang is...
IPVM Camera Calculator User Manual / Guide on Sep 23, 2020
Learn how to use the IPVM Camera Calculator (updated for Version 3.1). The...