Axis MP Pinhole Shootout

By: Ethan Ace, Published on Jun 18, 2013

For covert surveillance, pinhole lenses / cameras are a common choice because they can be hidden behind walls, or in ordinary objects. Until recently, the only options were analog / SD. However, Axis launched the P12 series, bringing 720p MP to pinhole. How much of a difference does that make and what tradeoffs did it bring?

We bought two Axis P12s plus 3 analog alternatives (two Marshall Electronics pinhole lenses and a Supercircuits [link no longer available] 'snake camera' [link no longer available]). We then tested them head to head in a custom test harness, installing each camera side by side:

Here's the view inside, from left to right, the Axis P1214, the Marshall lenses with Sony box cameras and the Axis P1204. In the top center, is the Supercircuits snake camera:

Here are our key findings from this test:

  • In strong, even lighting, there is no little benefit to megapixel resolution if the camera's intent is to capture close up images of a subject, such as typical ATM machine applications.
  • At farther ranges, even 10' from the camera, the benefits of higher resolution become apparent as details are quickly lost in the analog Supercircuits PC229XP.
  • However, in lowering light (~2 lux), noise in the megapixel models begins to narrow the performance gap, making the Supercircuits PC229XP more competitive.
  • In low light, <1 lux, only the Marshall 2.8mm lens provided usable images. Other cameras were extremely dim and noisy at all ranges. 
  • The 2.8mm Marshall pinhole lens offers significant advantages over the 6.0mm version at close range. The wider angle allows viewing the area to the sides of the main subject, providing better overall awareness in the scene. The lower F stop (1.0 vs. 1.8) also provides facial details in our <1 lx scene, which the 6.0mm does not.
  • HD recording provided increased details over VGA, despite Marshall lenses not being rated for megapixel cameras.
  • In scenes with strong backlighting, the CS mount Marshall pinhole lenses are a clear benefit, as a WDR cameras may be used for best performance.
  • The Supercircuits PC229XP performed well in WDR scenes with the subject close to the camera, outperforming the HD Axis P1204. However, at range, it was by far the worst in our WDR test.

Note: For the purposes of this test, we used Sony G6 cameras with the CS mount Marshall lenses, in order to gauge performance gains when using high performance low light/WDR cameras. Using lower cost or lower quality cameras will affect performance, potentially bringing the lenses to the Axis P12's level of performance.

Price Comparisons

Pricing compares as follows for the models in our test.

  • P1204: $450-500 USD
  • Marshall V-PL60CS: $239 + cost of box camera
  • Marshall V-PL28CS: $329 + cost of box camera
  • Supercircuits PC229XP: $250 [link no longer available] + cost of encoder

Taking the cost of box cameras (in the case of Marshall lenses) or encoders (in the case of the PC229XP) into account, the P1204 is likely the lowest cost and least complex pinhole option. Budget cameras in the $200-300 range could be used with Marshall lenses for a similar price point, but require more difficult installation and likely no real advantage. 

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

Recommendations

There are three key recommendations to take away from this test:

  • The Axis P12 series performs better than analog options in scenes requiring more than simply close up video. It lags behind high performance low light/WDR cameras with add-on lenses, but is in a much smaller form factor allowing it to be installed where these cameras would not fit. Finally, it is far less expensive than WDR/low light box plus lens options.
  • When absolute best performance is needed, users should consider Marshall's add-on lenses with a quality camera. Assuming size and cost allow, this package will provide the best imaging in low light and WDR applications. In this case, we recommend the wider-angle models, such as the 2.8mm lens in this test, as they offer advantages in situational awareness in the scene, where narrower lenses show only the subject's face at close range.
  • If close up video is the main intent of the camera, the Supercircuits PC229XP may be a reasonable choice, especially when connected to a DVR. The additional cost of an encoder brings the total price close to the Axis P12 series, a better performing camera.

Physical Overviews

In this video we review the construction of each of the cameras in our test and compare form factors. Especially of note:

  • Relative size varies widely, with the analog remote head camera being smallest by far, though it requires a separate encoder.
  • The Axis P12 series is the smallest "all-in-one" model, with very small camera heads suitable for tight locations.
  • Box cameras equipped with pinhole lenses are extremely large, likely to be a limiting factor in many covert installations.

Installation Issues 

In this video we demonstrate mounting the cameras into our pinhole test rig. Users should note the following:

  • Remote head cameras are often mounted using self-adhesive mounts. The P12 and analog cameras both used this type of mount.
  • Box cameras with pinhole lenses require more mounting consideration due to their weight and size. Users should beware of this when planning installation to make sure the camera can be properly supported.

Similar Field of View

First, we compare the three cameras in our test with similar fields of view, all between 50 and 60 degrees: Axis P1204, Marshall V-PL60CS with a Sony VB600, and the analog PC229XP.

Full Light

At close range, the HD/megapixel models provide more even tone and color, without the overexposed areas seen in the PC229XP. However, this does not result in additional details revealed, as details of the subject, even eye color, are clearly visible in the analog camera.

At 10' range, the HD cameras show clear advantages over SD, as the chart is easily readable, with more detail of the subject provided. The PC229XP provides details that may identify a known subject, but some clarity is lost. The chart is also harder to read, due to overexposure and lower resolution.

Finally, at 24' range, the PC229XP shows little detail, enough to specify light or dark hair or clothing, and only the first line of our chart is legible. The P1204 and V-PL60CS provide similar levels of detail, with a slight edge to Axis, which provides better color information

2 Lux

Lowering the light level to 2 lux, the HD cameras show advantages over the analog model. The PC229XP has become so desaturated that it nearly looks black and white, though the subject's face is still clearly detailed. The P1204 and VB600 with Marshall lens, however, still show color information, though with increasing noise.

At 10' range, due to increased noise in the HD cameras, the performance gap between them and the PC229XP narrow somewhat. The chart is still more readable in the HD models, however, due to their higher resolution.


At 24' range, performance is similar to 10', with noise obscuring details in both the P1204 and PL60CS. Again, the chart is more readable in the HD models.

Dark

At close range in our dark scene, below 1 lux, none of the cameras produced usable details of the subject's face. 

Moving to 10', the test chart becomes visible, but the subject is still obscured in all cameras.

Finally, at 24', neither the chart nor the subject are visible in any of the cameras.

Standard vs. Wide Angle

Next, we tested two Marshall pinhole lenses side-by-side, to demonstrate the tradeoffs between the standard 6.0mm lens and the wide angle 2.8mm model.

Full Light Comparison

At close range, ~2', the wide angle lens' benefits are most clear, with much more area to the left and right of the subject visible, which could show details of an assailant approaching from behind. The narrower 6mm model would be unlikely to capture this.

However, at 24', we can see the key drawback to the 2.8mm model's wider FOV. At this distance, details of the subject aside from general color and clothing style information are lost, and the chart becomes unreadable after line 2. The 6.0mm model does not suffer from these issues.

Low Light Performance

The 2.8mm lens has distinct advantages over the 6.0mm model in low light due to its lower F number (1.0 vs. 1.8). This is obvious at near range below 1 lux, where the subject's features are visible in the 2.8mm lens, but not the 6.0.

At 24', the subject is detectable in the 2.8mm lens, but not in the 6.0mm model:

 

HD vs. SD

Though the Marshall lenses have advantages over other pinhole models, allowing the user's choice of camera for specific scenes (low light, WDR), they are not megapixel rated. In order to determine whether the lens was a limiting factor when using HD cameras, we tested the lenses in the same scenes, with the same cameras, at both 1.3MP and VGA resolution. At 2' range, 1.3MP offers little advantage, since pixel count on target is so high, even using VGA cameras. However, we can see fewer details in the background of the wider angle 2.8mm lens.

At 24', results are similar in the 6.0mm model, with the subject and chart easily visible. However, in the 2.8mm lens, the performance gap is narrower. While 1.3MP makes one additional line of the chart readable, and the subject's flace is more discernable, it does not otherwise provide more details. 

Low Light Performance

In low light scenes, below 1 lux, again there is little difference at 2'.

At 24', neither resolution reveals usable details in either camera.

 

WDR

Finally, we set up the cameras in a scene with strong backlighting, against an open overhead door on a sunny day, in order to test their WDR handling.

At close range, with the subject squarely in front of the door, the P1204 provides the dimmest imaging of the subject's face, though details are still visible. The PC229XP handles the subject well, but the background is overexposed, obscuring details outside.

With the subject outside, about 15' from the camera in strong sunlight, the PC229XP is clearly the worst performer, with overexposure obscuring the subject and the entirety of the chart. The P1204 suffers as well, with the chart unreadable past line 1, but the subject still identifiable. The Marshall lenses/Sony cameras perform best, revealing the most details of the subject/chart, though the 2.8mm model suffers somewhat due to lower resolution

Finally, with the subject in the dark area inside, next to the door, at about 10' range, once again the P1204 displays the dimmest image. The PC229XP and Marshall lenses are brighter, with details easily visible.

Test Parameters

The following cameras were used in this test:

All cameras were run using the latest firmware, with default settings with some exceptions:

  • Resolution was set to the camera's maximum, at 30 FPS
  • Encoding was set to VBR with no cap where possible (All cameras aside from the PC229XP, as the SNT-EP104 only supports CBR)
  • Shutter speed was normalized at 1/30s maximum

3 reports cite this report:

Camera Form Factor Guide on Mar 16, 2018
When selecting surveillance cameras, users may choose from a number of different form factors, each with its own unique strengths and weaknesses,...
Samsung Covert WDR Camera Tested on Oct 07, 2015
IPVM member Steve Beck asked us in IPVM chat to test the Samsung covert WDR camera (the SNB-6011) so we did it. Covert cameras are often used in...
Testing the Smallest HD Cameras on Jul 09, 2014
Miniature HD IP cameras are a growing trend. It started with cameras that had small 'heads' but large 'base' units. Increasingly, though, the bases...
Comments (16) : PRO Members only. Login. or Join.

Related Reports

Camera Field of View (FoV) Guide on Nov 13, 2019
Field of View (FoV) and Angle of View (AoV), are deceptively complex. At their most basic, they simply describe what the camera can "see" and seem...
Color Low Light Mega Camera Shootout - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Panasonic, Speco, Sony, Vivotek on Nov 12, 2019
This is the biggest color low light shootout ever, testing 20+ super low light models from 10 manufacturers: Increasingly, each manufacturer...
Biggest Low Light Problems 2019 on Nov 08, 2019
Over 150 integrators responded to our survey question: "What are the biggest problems you face getting good low-light images?" Inside, we share...
Rhombus Cameras, VMS and Analytics Tested on Nov 06, 2019
Rhombus boasts they have created "the new standard in Enterprise, cloud-managed video security" and told IPVM in January 2019 they offer twice the...
IP Camera Mount Shootout - Axis, Bosch, Dahua, Hanwha, Hikvision, Vivotek on Nov 04, 2019
Which manufacturers offer the best or worst mounts? We bought and tested ~50 IP camera mounts and related components from Axis, Bosch, Dahua,...
Milestone XProtect 2019 R3 Tested on Oct 30, 2019
Milestone has had problems over the last few years releasing significant new software. Now, in XProtect 2019 R3, Milestone is touting "one search...
Avigilon Appearance Search Tested on Oct 30, 2019
Avigilon Appearance Search claims that it "sorts through hours of video with ease, to quickly locate a specific person or vehicle of interest...
Covert Elevator Face Recognition on Oct 24, 2019
Covert elevator facial recognition has the potential to solve the cost and complexity of elevator surveillance while engendering immense privacy...
Illustra Pro Gen3 4K Camera Tested on Oct 23, 2019
Johnson Controls has released the latest generation in their Illustra Pro line, the Pro Mini-Dome Gen 3, a non-OEM model claiming to "optimize...
Integrated IR Camera Usage Statistics 2019 on Oct 21, 2019
Virtually every IP camera now comes with integrated IR but how many actually make use of IR or choose 'super' low light cameras without IR? In...

Most Recent Industry Reports

Hikvision CEO And Vice-Chair Under PRC Government Investigation on Nov 14, 2019
In a surprising and globally covered move, Hikvision CEO Hu Yangzhong and Vice-Chairman Gong Hongjia are being investigated by China's securities...
Camera Field of View (FoV) Guide on Nov 13, 2019
Field of View (FoV) and Angle of View (AoV), are deceptively complex. At their most basic, they simply describe what the camera can "see" and seem...
UK Big Brother Watch: Hikvision Is 'Morally Bankrupt' on Nov 13, 2019
UK civil liberties advocate Big Brother Watch has condemned Hikvision as being 'morally bankrupt' following IPVM exposing Hikvision marketing...
Color Low Light Mega Camera Shootout - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Panasonic, Speco, Sony, Vivotek on Nov 12, 2019
This is the biggest color low light shootout ever, testing 20+ super low light models from 10 manufacturers: Increasingly, each manufacturer...
Wireless / WiFi Access Lock Guide on Nov 12, 2019
For some access openings, running wires can add thousands in cost, and wireless alternatives that avoid it becomes appealing. But using wireless...
Hikvision Markets Uyghur Ethnicity Analytics, Now Covers Up on Nov 11, 2019
Hikvision has marketed an AI camera that automatically identifies Uyghurs, on its China website, only covering it up days ago after IPVM questioned...
Open vs End-to-End Systems: Integrator Statistics 2019 on Nov 11, 2019
Preference for open systems is on the decline, according to new IPVM statistics. We asked integrators: For video surveillance systems, do you...
Biggest Low Light Problems 2019 on Nov 08, 2019
Over 150 integrators responded to our survey question: "What are the biggest problems you face getting good low-light images?" Inside, we share...
US Issues Criminal Charges For Fraudulently Selling Hikvision And Other China Products on Nov 07, 2019
The US government has made an unprecedented move on the video surveillance supply chain, charging a US company, Aventura for "having conspired with...
The Access Control Codes Guide: IBC, NFPA 72, 80 & 101 on Nov 07, 2019
For access, there is one basic maxim: Life safety above all else. But how do you know if all applicable codes are being followed? While the...