Axis Camera Shootout 2011

Author: Benros Emata, Published on Jun 19, 2011

With so many cameras from each manufacturer to choose from, determining quality differences can be difficult. This challenge is increasing as IP camera manufacturers offer more resolution options (SD, HD, 3MP, 5MP, etc.) and different product tiers (entry level, mid tier, premium, etc.).

To provide guidance on tradeoffs, we are starting a new test series of 'Intra-Manufacturer' shootouts. The purpose is to compare and analyze camera models and technology within one manufacturer's portfolio to get a practical sense of the differences in real world surveillance applications.

In this first test, we examined differences among Axis Communications cameras. We took four Axis cameras:

  • M1101 - an entry level SD camera
  • M1104 - an entry level HD camera
  • P1344- a premium level HD camera
  • P1347 - a premium level 5 MP camera

We tested in several scenarios, encompassing a wide variety of real-world surveillance environments (e.g. indoors, outdoors, evenly-lit, low-light, and WDR scenes).

Analyzing Video Quality

We analyzed a series of scenes to see how performance varied across the cameras. We digitally zoomed each camera to the same level to show differences. See an example scene comparison from our parking lot test.

We conducted our analysis across 10 scenes. They are:

  • Indoor Even Artificial Lighting (Far)
  • Indoor Even Artificial Lighting (Near)
  • Indoor Low-Light (Default)
  • Indoor Low-Light (Normalized)
  • Outdoor Daytime Even Lighting (Far)
  • Outdoor Daytime Even Lighting (Near)
  • Outdoor Nighttime (Far)
  • Outdoor Nighttime (Near)
  • WDR Scene (Bright)
  • WDR Scene (Dark)

Analyzing Bandwidth Consumption

Not only do we examine which cameras provide the best quality in each scene, we now break down the bandwidth consumption, showing who's the best and worse for the amount of network bandwidth and storage consumed.

Our analysis covers 8 metrics:

Get Video Surveillance News In Your Inbox
Get Video Surveillance News In Your Inbox

  • Even Artificial Lighting Bandwidth Consumption Per Camera
  • Low-Light (Default Settings) Bandwidth Consumption Per Camera
  • Low-Light (Normalized Settings) Bandwidth Consumption Per Camera
  • Outdoor Even Lighting Bandwidth Consumption Per Camera
  • Outdoor Nighttime Bandwidth Consumption Per Camera
  • WDR Scene Bandwidth Consumption Per Camera
  • Average Bandwidth Consumption Per Scene
  • Average Bandwidth Consumption Per Camera Overall

Our focus was to understand how image quality differed across this broad range, answering questions such as:

  • How much better is an Axis HD than an inexpensive SD camera?
  • What quality differences exist between an Axis entry level and premium HD camera?
  • How much better is an Axis 5MP compared to their HD and SD cameras?
  • When is it worth it to spend more a higher resolution or feature set camera?

Inside the Pro section, we answer of each these questions.

Key Findings for Each Camera's Performance

First, let's summarize the relative performance of each camera:

  • The SD M1011: Significantly worse details captured than either HD camera in essentially every scene, even at fairly modest FoVs (~16 feet wide) that made a practical difference in capturing key details. Bandwidth consumption was relatively high in a majority of scenes. Least expensive camera by far but significance relative deficiencies in performance.
  • The HD M1114: Relative to the premium P1344 HD camera, the M1114 captured roughly the same image details in most daytime and WDR scenes. At night, the visual quality of the M1114, which lacks a mechanical cut filter (i.e., true D/N functionality), tended to be modestly worse than the P1344.
  • The HD P1344: Image quality compared to M1114 as noted above. In most of the scenes, the P1344's bandwidth consumption was lower than the M114.
  • The 5MP P1347: Relative to the HD cameras, modest increases in visual details captured in most daytime scenes, material differences in WDR scenes and hardly any improvements in night time scenes. Additionally, the 5MP tended to drop frame rate to 6fps more aggressively than HD cameras, creating motion blur for moving subjects. While frame rate was about 60% less in most scenes (12fps vs 30fps), bandwidth consumption tended to be similar to HD.

Recommendations

When assessing image quality tradeoffs within these Axis camera offerings, we recommend the following:

  • General Daytime use: The HD cameras are likely the best option as they capture more details than the SD and almost the same as the 5MP. Of the HDs, the P series (P1347) provides lower bandwidth consumption than the M1104 and more features. However, the P series costs ~$200-300 more.
  • WDR use: The 5MP deliver notably superior performance. However, the price is significantly more expensive than either of the 2 HD cameras.
  • Night time use: The P1344 provided the best overall image quality as the color only M1114 was grainy and consumed high amounts of bandwidth while the P1347 did not provide any more details and increased motion blur as the frame rate dropped to compensate for lower light sensitivity.

Indoor Even Artificial Lighting (Far Subject)

In this scenario, cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. The scene is illuminated via artificial lighting and is evenly distributed (~200 lux).

Below is the composite snapshot (download full video clip):

The results are congruent with expectations, as the highest resolution 5MP provides the greatest level of details and a high certainty of identification. In contrast, the SD lacks significant detail and the subject appears blob-like. Both the P and M series 720p's outperform the SD, while only slightly underperforming the 5MP in terms of detail level. Indeed, the certainty of identification in the 720p's remains fairly high. Between the P and M series 720p's however, the P1344 tends to provide a slight edge in clarity and crispness over the M1114.

Indoor Even Artificial Lighting (Near Subject)

In this scenario, cameras are digitally zoomed into a human subject ~17ft out, corresponding to a HFoV of ~16ft. The scene is illuminated via artificial lighting and is evenly distributed (~200 lux).

Below is the composite snapshot (download full video clip):

There are a couple of interesting findings revealed at this narrower FoV. Firstly, within the 720p's (P1344 vs. M1114) the P1344 demonstrates greater separation from the M1114 despite equal resolutions. Secondly, the 720p P1344 has closed the gap considerably versus the 5MP P1347. Indeed, both the P series cameras (5MP P1347 and 720p P1344) deliver strong details and a near absolute certainty of subject identification.

Below is the bandwidth utilization graph:

The bandwidth consumptions are fairly low across the board (~1mbps), with little variation between cameras as well. This is likely attributed to the even, non-challenging lighting characteristics. Under these circumstances the 5MP P1347 presents a solid quality to bandwidth ratio.

Indoor Low-Light (Default)

In this scenario, cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. The scene is 'dark' with essentially no lighting (~0.1 lux). All cameras are set to default settings. Note that the M1114 and M1011 are color only.

Below is the composite snapshot (download full video clip):

The P1344 stands out as the only camera without considerable noise present. Yet, the overall investigative value is no greater than what the P1347 and M1114 provide. In fact, it can be argued that the color only M1114 provides greater utility due the indication of color (e.g. blue shirt). Interestingly, among the color only cameras (M1114 and M1011), the higher resolution 720p fairs reasonably better than the SD, which appears quite under-exposed and 'dark'.

Below is the bandwidth utilization graph:

Most remarkable is the extreme spike in bandwidth utilization of the 5MP P1347 (10.2 Mbps). Perhaps equally remarkable is the extreme drop in bandwidth observed for the 720p P1344 (300 Kbps). Comparing the two images from the composite, it is clear that gain and noise is a significant factor in the P1347's drop in compression efficiency. In contrast the P1344 produces a relatively noise-free image and as such does not experience any real efficiency loss. With little noise present the overall scene has a low complexity level (low potential bandwidth), from which the P1344 is able to 'benefit'.

Indoor Low-Light (Normalized)

In this scenario, cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. The scene is 'dark' with essentially no lighting (~0.1 lux). Cameras have been normalized to a 1/30s shutter speed. Note that the M1114 and M1011 are color only.

Below is the composite snapshot (download full video clip):

The very limited lighting conditions has rendered the color only cameras 'useless' - having completely 'blacked-out' images. Similarly, the 5MP P1347 also produces a 'blacked-out' image with no investigative utility. The 720p P1344 is the only camera providing at least a modicum of scene details. Despite it being the 'best' performer among the field, its image production is still of marginal value to the investigator (e.g. crude detection of motion).

Below is the bandwidth utilization graph:

The uncharacteristically high bandwidth (8.59 Mbps) utilization of the SD M1011 results from the excessive noise being generated as the camera attempts to compensate for low-light via increased gain. Interestingly, the other cameras do not appear to experience this phenomenon. The P1347 (50kbps) and M1114 (80kbps) seem to reach a threshold where gain is either no longer triggered or where such gain does not impact the image (and subsequently bandwidth/compression).

Outdoor Even Midday Lighting (Far)

In this scenario cameras are digitally zoomed into a human subject ~75ft out, corresponding to a HFoV of ~70ft. The scene is illuminated via natural sun light and is evenly distributed (~10,000 to 20,000 lux).

Below is the composite snapshot (download full video clip):

The 5MP P1347 provides the greatest level of facial and license plate details among the four cameras. While likelihood of identification is not absolute, a fair degree of certainty exists. Also, notice that only the 5MP P1347 provides enough details on the license plate for making a reasonable guess as to its specific characters. The SD M1011, in contrast, delivers the least level of detail. As evidenced in the SD image, the human subject appears as a 'blob' and the license plate can be conceivably mistaken for a logo or design on the shirt itself. The 720p's deliver essentially equal images of basically equal utility. There are some useful details and indications, e.g. hair style, complexion, build and gender. However, identification is only slightly likely at best and there is absolutely no chance at capturing plate characters.

Outdoor Even Midday Lighting (Near)

In this scenario cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. The scene is illuminated via natural sun light and is evenly distributed (~10,000 to 20,000 lux).

Below is the composite snapshot (download full video clip):

At this narrower FoV both 720p's deliver strong facial details and license plate details. Their overall utility is no less than the 5MP P1347 under these conditions. The SD M1011 continues to lag in details, especially with respect to license plate readability, but facial details, body contours, and overall subject clarity have improved significantly compared to the 'far' scenario.

Below is the bandwidth utilization graph:

Aside from the M1011's noticeably low bandwidth utilization, bandwidths are relatively low across the board and vary only to a slight or moderate extent. It is interesting, perhaps to point out the disparity in bandwidths between the two 720p cameras (P1344 at 1.2Mbps and M1114 at 1.9Mbps), given that their picture qualities were essentially equal.

Outdoor Nighttime (Far)

In this scenario cameras are digitally zoomed into a human subject ~75ft out, corresponding to a HFoV of ~70ft. The scene is illuminated via artificial lights (~0.5 to 10 lux).

Below is the composite snapshot (download full video clip):

Somewhat unexpectedly the 720p color only M1114 performs relatively well in this scenario. The notion of an advantage for b/w mode under low light conditions is somewhat challenged by these results as the P1347 and P1344 D/Ns demonstrate no additional utility here. Indeed, it is the M1114 which actually provides added richness (e.g. complexion, shirt and pant color, and overall better contrast that helps determine or rule out hair style). However, note the SD color only does not provide the same level of 'quality' as the 720p color only. The SD is quite noisy and under-saturated.

Outdoor Nighttime (Near)

In this scenario cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. The scene is illuminated via artificial lights (~0.5 to 10 lux).

Below is the composite snapshot (download full video clip):

The MP cameras provide fairly equal levels of human subject detail with a subsequent strong likelihood of identification. However, the color only M1114 is shown to slightly lag the two P series D/N cameras in terms of license plate clarity. Overall, both the M1011 and M1114 color only images do not appear compromised due to low-light and provide decent images despite a degree more noise detected.

Below is the bandwidth utilization graph:

Noise levels present in the images appear to account for the bandwidth disparity between D/N and color-only cameras. Given that the D/Ns provided overall greater detail levels than the color-only cameras, their quality to bandwidth ratios are quite impressive under these conditions.

WDR Performance (Bright)

In this scenario cameras are digitally zoomed into a human subject ~40ft out, corresponding to a HFoV of ~37ft. The scene's WDR character is produced by a ~2000 lux bright area and a ~200 lux interior 'dark' area. The subject is positioned in the bright area.

Below is the composite snapshot (download full video clip):

No clear differences can be declared in terms of WDR specific capabilities. Overall, the field performs modestly (e.g. not able to resolve any details in the back-lit area, but yielding some body contours and some facial details). However, overall utility tends to favor the higher megapixel P1347. Indeed, resolution appears to be the separating factor in overall image utility. Note that between the two 720p models, the 'lower' tier M series provides slightly better facial details than the P1344.

WDR Performance (Dark)

In this scenario cameras are digitally zoomed into a human subject ~40ft out, corresponding to a HFoV of ~37ft. The scene's WDR character is produced by a ~2000 lux bright area and a ~200 lux interior 'dark' area. The subject is positioned in the 'dark' area.

Below is the composite snapshot (download full video clip):

The general expectation in the 'dark' area of a WDR scene is an under-exposed subject. However, the apparent illumination of the subject is adequate across the board. Again, as in the 'bright' WDR test, resolution is the distinguishing factor with the 5MP P1347 providing the highest level of details amongst the field.

Below is the bandwidth utilization graph:

Bandwidths are fairly low with little variation between cameras. In some sense the scene is analogous to the 'Indoor Even-Lit' scenario. It is interesting to note that the WDR scene exhibited the lowest overall bandwidth utilization amongst all scenarios.

Motion Analysis

Our findings indicate that under low-light conditions the 5MP D/N P1347 has the greatest tendency to extend exposure length and drop its frame-rate. The SD color only M1011, exhibits a similar tendency as well. However, the effect is more profound in the P1347 given that its max frame rate is only 12fps, while the M1011's is 30fps. The resulting motion blur can be an issue when objects in motion must be analyzed. The following brief video clips illustrate the phenomenon.

Daytime

In this short clip lighting conditions are near ideal and both cameras operate at full frame-rate. For the 5MP P1347 the full frame rate is 12fps, and the M1114's is 30fps. As the subject moves forward from the back to the front of the scene no detectable motion blur occurs. The two video panes on the right show the same cameras fully zoomed out.

At the speed of objects in this scene, 12fps is evidently adequate to maintain smooth motion video under these conditions. We'll see in the next clip that 6fps and a lengthened exposure has a definite impact on the quality of motion.

Nighttime

The following clip features all four cameras, but this time under low-light conditions. As a result, the P1347 in the top left corner now operates at a reduced frame-rate of 6fps, and motion blur is apparent. The M1011, featured bottom left, also operates at a diminished frame-rate of 12fps (30fps max). The two 720p cameras (P1344 and M1114) on the right side, despite low-light conditions, maintained a full 30fps throughout the test scene.

The P1347 at 6fps and maximum 1/6s exposure has a definite negative impact on the motion quality. The SD M1011, which operates at a diminished 12fps also exhibits a degree of motion blur. Its impact is not as pronounced as in the P1347, however.

Bandwidth Analysis

Average Bandwidth by Scene

The low-light (default) scene produced the highest average bandwidth consumption (4.26 Mbps). During this test all cameras were at default settings, and as such adapted to the low-light environment via increased exposure times and increased gain. Indeed, looking at the composite reveals quite a bit of noise present in the majority of the images. The result was a loss of compression efficiency. The remaining two low-light scenes maintained decent compression levels, as they averaged roughly only half (~2.4 Mbps) that of the low-light (default) scene. Scenes with better lighting characteristics fared well with bandwidths around the 1 Mbps mark.

Overall Average by Camera

The color-only cameras, and the 5MP P1347 tended to produce the most noise during low-light scenes. This is congruent with expectations, given that, in general, color and higher MP (even in D/N mode) cameras tend to require more light than D/N and lower resolution cameras. The 720p D/N P1344 exhibited surprisingly low noise characteristics during low-light/nighttime scenes. As a result its bandwidth profile reflects its consistent compression efficiency and economical performance.

Methodology

The following are the four (4) cameras used in the 'Axis Intra-Shootout':

  • Axis P1347 (online $1350) - 5MP D/N; 1/2.5" CMOS; F1.6 Kowa;0.08 Lux (BW)
  • Axis P1344 (online $759) - 720p D/N; 1/4" CMOS; F1.2 Computar ; 0.05 Lux (BW)
  • Axis M1114 (online $475) - 720p D/N; 1/4" CMOS; F1.2 Fujinon; 0.6 Lux (Color)
  • Axis M1011 (online $160) - 4CIF Color-only; 1/4" CMOS; Integrated F2.0; 1.0 Lux (Color)

Each of the four (4) cameras were simultaneously recorded to an Exacqvision VMS during six (6) key scenarios. All cameras were set to their defaults during testing, except during the low-light tests. During low-light tests D/N cameras were 'forced' into B/W to ensure consistency of comparison. Furthermore, cameras were normalized to 1/30s shutter speed for one of the tests. All camera lenses were adjusted to a uniform lens angle (~50 degrees) and recorded/analyzed at their - VMS supported - maximum resolutions.

Here are the key default settings and firmware versions for each camera:

  • Axis M1011 - 1/6s max exposure w/ AGC; fw 5.00.1
  • Axis M1114 - 1/6s max exposure w/ AGC; fw 5.09
  • Axis P1344 - 1/6s max exposure w/ AGW; fw 5.06.1
  • Axis P1347 - 1/6s max exposure w/ AGC; fw 5.20

Here are the five (5) key scenarios:

  • Indoor Even Artificial Lighting - ~200 lux
  • Indoor Low-Light - ~0.1 lux
  • Outdoor Even Midday Lighting - ~ 10,000 to 20,000 lux
  • Outdoor Nighttime - ~0.5 to ~10 lux
  • WDR Scene - ~2000 lux (bright area) and ~200 lux (dark area)

1 report cite this report:

Directory of Camera Shootout Series on Dec 27, 2011
The following directory lists all of our camera shootouts. These shootouts pit 4 - 8 different surveillance cameras in simultaneous tests on real...
Comments : PRO Members only. Login. or Join.

Related Reports

Dahua Dual Imager Dome Camera Tested (HDBW4231FN-E2-M) on Nov 07, 2018
Dahua has introduced a dual-imager dome model, the HDBW4231FN-E2-M, with two independently positionable sensors including integrated IR, not found...
Winter 2019 IP Networking Course on Nov 05, 2018
This is the only networking course designed specifically for video surveillance professionals.  Lots of network training exists but none of it...
Cloud Video Storage Usage 2018 on Oct 31, 2018
Storing email and documents in the cloud have long been common, with on-site email or file servers increasingly eliminated. However, what about...
Axxon Face Search Tested on Oct 26, 2018
AxxonSoft has brought facial recognition to their Axxon Next VMS for free with the simply named Face Search, claiming to allow users to find...
Video Quality / Compression Tutorial on Oct 17, 2018
While CODECs, like H.264, H.265, and MJPEG, get a lot of attention, a camera's 'quality' or compression setting has a big impact on overall...
Dahua Face Recognition Camera Tested on Oct 15, 2018
Dahua has been one of the industry's most vocal proponents of the value that AI creates: As part of this, Dahua has released a facial...
Last Chance - October 2018 Camera Course on Oct 04, 2018
Today is the last day to register for the October 2018 Camera Course, register now. This is the only independent surveillance camera course,...
Axis 4K vs. 5MP Camera Shootout (Q3518-LVE + Q3517-LVE) on Oct 04, 2018
Axis has released the 4K counterpart to their 5MP Q3517-LVE (see our test). Axis claims of both these cameras: Simply put, AXIS Q3518(17)-LVE...
Hikvision 4MP Camera Tested (DS-2CD2345FWD-I) on Sep 27, 2018
Hikvision's latest Performance Series / EasyIP 3.0 4MP model, the DS-2CD2345FWD-I, was the top performer in our 4MP shootout, besting rivals from...
4MP Camera Shootout - Axis, Dahua, DW, Hanwha, Hikvision, Uniview, Vivotek on Sep 24, 2018
4MP usage continues to climb, especially for low cost fixed lens models. To see who was best, we bought and tested seven 4MP models from Axis,...

Most Recent Industry Reports

'Sticker' Surveillance Camera Developed (CSEM Witness) on Nov 16, 2018
The Swiss Center for Electronics and Microtechnology (CSEM) has announced what it calls the: world’s first fully autonomous camera that can be...
ISC East 2018 Mini-Show Final Report on Nov 16, 2018
This is our second (updated) and final show report from ISC East. ISC East, by its own admission, is not a national or international show, billed...
Facial Detection Tested on Nov 16, 2018
Facial detection and recognition are increasingly offered by video surveillance manufacturers. Facial detection detects faces in an image/video...
Throughtek P2P/Cloud Solution Profile on Nov 15, 2018
Many IoT manufacturers either do not have the capabilities or the interest to develop their own cloud management software for their devices....
ASIS Offering Custom Research For Manufacturers on Nov 15, 2018
Manufacturers often want to know what industry people think about trends and, in particular, the segments and product they offer.  ASIS and its...
Hikvision Silent on "Bad Architectural Practices" Cybersecurity Report on Nov 14, 2018
A 'significant vulnerability was found in Hikvision cameras' by VDOO, a startup cybersecurity specialist. Hikvision has fixed the specific...
French Government Threatens School with $1.7M Fine For “Excessive Video Surveillance” on Nov 14, 2018
The French government has notified a high-profile Paris coding academy that it risks a fine of up to 1.5 million euros (about $1.7m) if it...
Integrator Credit Card Alternative Divvy on Nov 13, 2018
Most security integrators are small businesses but large enough that they have various employees that need to be able to expense various charges as...
Directory of Video Intercoms on Nov 13, 2018
Video Intercoms, also known as Video Door-Phones or Video Entry Systems, have been growing in the past decade as more and more IP camera...
Beware Amazon Go Store Hype (Tested) on Nov 13, 2018
IPVM's trip to and testing of Amazon Go's San Francisco store shows a number of significant operational and economic issues that undermine the...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact