Axis MP Pinhole Shootout

By Ethan Ace, Published on Jun 18, 2013

For covert surveillance, pinhole lenses / cameras are a common choice because they can be hidden behind walls, or in ordinary objects. Until recently, the only options were analog / SD. However, Axis launched the P12 series, bringing 720p MP to pinhole. How much of a difference does that make and what tradeoffs did it bring?

We bought two Axis P12s plus 3 analog alternatives (two Marshall Electronics pinhole lenses and a Supercircuits [link no longer available] 'snake camera' [link no longer available]). We then tested them head to head in a custom test harness, installing each camera side by side:

Here's the view inside, from left to right, the Axis P1214, the Marshall lenses with Sony box cameras and the Axis P1204. In the top center, is the Supercircuits snake camera:

Here are our key findings from this test:

  • In strong, even lighting, there is no little benefit to megapixel resolution if the camera's intent is to capture close up images of a subject, such as typical ATM machine applications.
  • At farther ranges, even 10' from the camera, the benefits of higher resolution become apparent as details are quickly lost in the analog Supercircuits PC229XP.
  • However, in lowering light (~2 lux), noise in the megapixel models begins to narrow the performance gap, making the Supercircuits PC229XP more competitive.
  • In low light, <1 lux, only the Marshall 2.8mm lens provided usable images. Other cameras were extremely dim and noisy at all ranges. 
  • The 2.8mm Marshall pinhole lens offers significant advantages over the 6.0mm version at close range. The wider angle allows viewing the area to the sides of the main subject, providing better overall awareness in the scene. The lower F stop (1.0 vs. 1.8) also provides facial details in our <1 lx scene, which the 6.0mm does not.
  • HD recording provided increased details over VGA, despite Marshall lenses not being rated for megapixel cameras.
  • In scenes with strong backlighting, the CS mount Marshall pinhole lenses are a clear benefit, as a WDR cameras may be used for best performance.
  • The Supercircuits PC229XP performed well in WDR scenes with the subject close to the camera, outperforming the HD Axis P1204. However, at range, it was by far the worst in our WDR test.

Note: For the purposes of this test, we used Sony G6 cameras with the CS mount Marshall lenses, in order to gauge performance gains when using high performance low light/WDR cameras. Using lower cost or lower quality cameras will affect performance, potentially bringing the lenses to the Axis P12's level of performance.

Price Comparisons

Pricing compares as follows for the models in our test.

  • P1204: $450-500 USD
  • Marshall V-PL60CS: $239 + cost of box camera
  • Marshall V-PL28CS: $329 + cost of box camera
  • Supercircuits PC229XP: $250 [link no longer available] + cost of encoder

Taking the cost of box cameras (in the case of Marshall lenses) or encoders (in the case of the PC229XP) into account, the P1204 is likely the lowest cost and least complex pinhole option. Budget cameras in the $200-300 range could be used with Marshall lenses for a similar price point, but require more difficult installation and likely no real advantage. 

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

Recommendations

There are three key recommendations to take away from this test:

  • The Axis P12 series performs better than analog options in scenes requiring more than simply close up video. It lags behind high performance low light/WDR cameras with add-on lenses, but is in a much smaller form factor allowing it to be installed where these cameras would not fit. Finally, it is far less expensive than WDR/low light box plus lens options.
  • When absolute best performance is needed, users should consider Marshall's add-on lenses with a quality camera. Assuming size and cost allow, this package will provide the best imaging in low light and WDR applications. In this case, we recommend the wider-angle models, such as the 2.8mm lens in this test, as they offer advantages in situational awareness in the scene, where narrower lenses show only the subject's face at close range.
  • If close up video is the main intent of the camera, the Supercircuits PC229XP may be a reasonable choice, especially when connected to a DVR. The additional cost of an encoder brings the total price close to the Axis P12 series, a better performing camera.

Physical Overviews

In this video we review the construction of each of the cameras in our test and compare form factors. Especially of note:

  • Relative size varies widely, with the analog remote head camera being smallest by far, though it requires a separate encoder.
  • The Axis P12 series is the smallest "all-in-one" model, with very small camera heads suitable for tight locations.
  • Box cameras equipped with pinhole lenses are extremely large, likely to be a limiting factor in many covert installations.

Installation Issues 

In this video we demonstrate mounting the cameras into our pinhole test rig. Users should note the following:

  • Remote head cameras are often mounted using self-adhesive mounts. The P12 and analog cameras both used this type of mount.
  • Box cameras with pinhole lenses require more mounting consideration due to their weight and size. Users should beware of this when planning installation to make sure the camera can be properly supported.

Similar Field of View

First, we compare the three cameras in our test with similar fields of view, all between 50 and 60 degrees: Axis P1204, Marshall V-PL60CS with a Sony VB600, and the analog PC229XP.

Full Light

At close range, the HD/megapixel models provide more even tone and color, without the overexposed areas seen in the PC229XP. However, this does not result in additional details revealed, as details of the subject, even eye color, are clearly visible in the analog camera.

At 10' range, the HD cameras show clear advantages over SD, as the chart is easily readable, with more detail of the subject provided. The PC229XP provides details that may identify a known subject, but some clarity is lost. The chart is also harder to read, due to overexposure and lower resolution.

Finally, at 24' range, the PC229XP shows little detail, enough to specify light or dark hair or clothing, and only the first line of our chart is legible. The P1204 and V-PL60CS provide similar levels of detail, with a slight edge to Axis, which provides better color information

2 Lux

Lowering the light level to 2 lux, the HD cameras show advantages over the analog model. The PC229XP has become so desaturated that it nearly looks black and white, though the subject's face is still clearly detailed. The P1204 and VB600 with Marshall lens, however, still show color information, though with increasing noise.

At 10' range, due to increased noise in the HD cameras, the performance gap between them and the PC229XP narrow somewhat. The chart is still more readable in the HD models, however, due to their higher resolution.


At 24' range, performance is similar to 10', with noise obscuring details in both the P1204 and PL60CS. Again, the chart is more readable in the HD models.

Dark

At close range in our dark scene, below 1 lux, none of the cameras produced usable details of the subject's face. 

Moving to 10', the test chart becomes visible, but the subject is still obscured in all cameras.

Finally, at 24', neither the chart nor the subject are visible in any of the cameras.

Standard vs. Wide Angle

Next, we tested two Marshall pinhole lenses side-by-side, to demonstrate the tradeoffs between the standard 6.0mm lens and the wide angle 2.8mm model.

Full Light Comparison

At close range, ~2', the wide angle lens' benefits are most clear, with much more area to the left and right of the subject visible, which could show details of an assailant approaching from behind. The narrower 6mm model would be unlikely to capture this.

However, at 24', we can see the key drawback to the 2.8mm model's wider FOV. At this distance, details of the subject aside from general color and clothing style information are lost, and the chart becomes unreadable after line 2. The 6.0mm model does not suffer from these issues.

Low Light Performance

The 2.8mm lens has distinct advantages over the 6.0mm model in low light due to its lower F number (1.0 vs. 1.8). This is obvious at near range below 1 lux, where the subject's features are visible in the 2.8mm lens, but not the 6.0.

At 24', the subject is detectable in the 2.8mm lens, but not in the 6.0mm model:

 

HD vs. SD

Though the Marshall lenses have advantages over other pinhole models, allowing the user's choice of camera for specific scenes (low light, WDR), they are not megapixel rated. In order to determine whether the lens was a limiting factor when using HD cameras, we tested the lenses in the same scenes, with the same cameras, at both 1.3MP and VGA resolution. At 2' range, 1.3MP offers little advantage, since pixel count on target is so high, even using VGA cameras. However, we can see fewer details in the background of the wider angle 2.8mm lens.

At 24', results are similar in the 6.0mm model, with the subject and chart easily visible. However, in the 2.8mm lens, the performance gap is narrower. While 1.3MP makes one additional line of the chart readable, and the subject's flace is more discernable, it does not otherwise provide more details. 

Low Light Performance

In low light scenes, below 1 lux, again there is little difference at 2'.

At 24', neither resolution reveals usable details in either camera.

 

WDR

Finally, we set up the cameras in a scene with strong backlighting, against an open overhead door on a sunny day, in order to test their WDR handling.

At close range, with the subject squarely in front of the door, the P1204 provides the dimmest imaging of the subject's face, though details are still visible. The PC229XP handles the subject well, but the background is overexposed, obscuring details outside.

With the subject outside, about 15' from the camera in strong sunlight, the PC229XP is clearly the worst performer, with overexposure obscuring the subject and the entirety of the chart. The P1204 suffers as well, with the chart unreadable past line 1, but the subject still identifiable. The Marshall lenses/Sony cameras perform best, revealing the most details of the subject/chart, though the 2.8mm model suffers somewhat due to lower resolution

Finally, with the subject in the dark area inside, next to the door, at about 10' range, once again the P1204 displays the dimmest image. The PC229XP and Marshall lenses are brighter, with details easily visible.

Test Parameters

The following cameras were used in this test:

All cameras were run using the latest firmware, with default settings with some exceptions:

  • Resolution was set to the camera's maximum, at 30 FPS
  • Encoding was set to VBR with no cap where possible (All cameras aside from the PC229XP, as the SNT-EP104 only supports CBR)
  • Shutter speed was normalized at 1/30s maximum

3 reports cite this report:

Camera Form Factor Guide on Mar 16, 2018
When selecting surveillance cameras, users may choose from a number of...
Samsung Covert WDR Camera Tested on Oct 07, 2015
IPVM member Steve Beck asked us in IPVM chat to test the Samsung covert WDR...
Testing the Smallest HD Cameras on Jul 09, 2014
Miniature HD IP cameras are a growing trend. It started with cameras that had...
Comments (16) : Members only. Login. or Join.

Related Reports

Vivotek LPR Camera Tested on Apr 15, 2020
Vivotek has historically sold license plate capture cameras but not LPR. Now,...
Camera Analytics Shootout 2020 - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Uniview, Vivotek on Jan 22, 2020
Analytics are hot again, thanks to a slew of AI-powered cameras, but whose...
Dahua Smart Motion Detection Camera Tested on Mar 03, 2020
Dahua has introduced Smart Motion Detection, AI-based VMD, claiming to use an...
Dahua, Hikvision, ZKTeco Face Mask Detection Shootout on Jun 19, 2020
Temperature tablets with face mask detection are one of the hottest trends in...
New Axis M30 Cameras Tested on Mar 26, 2020
Axis has released a new generation of, for them, relatively low cost M30...
Hanwha AI Analytics Camera Tested on Aug 11, 2020
Hanwha has released their Wisenet P AI camera, adding person and vehicle...
Mobotix 7 Line Camera Tested on Mar 12, 2020
Mobotix is attempting a turn-around, struggling for years, then releasing the...
Face Detection Shootout - Dahua, Hanwha, Hikvision, Uniview, Vivotek on Jul 30, 2020
Face detection analytics are available from a number of manufactures...
Motorola Solutions Acquires Pelco on Aug 03, 2020
Motorola Solutions has acquired Pelco, pledging to bring blue back and make...
Video Surveillance Cameras 101 on Feb 25, 2020
Cameras come in many shapes, sizes and specifications. This 101 examines the...
Tiandy Super Starlight Cameras Tested on Apr 30, 2020
Tiandy is taking aim at China competitors Dahua and Hikvision, with a new...
Sony Launches AI Camera Sensors on May 18, 2020
Weeks after exiting the branded video surveillance business, Sony is making a...
Avigilon Aggressive Trade-In Program Takes Aim At Competitors on Oct 20, 2020
Avigilon has launched one of the most aggressive trade-in programs the video...
Uniview Deep Learning Camera Tested on Jul 14, 2020
Uniview's intrusion analytics have performed poorly in our shootouts. Now,...
Verkada 2020 Cameras Image Quality Test on Oct 06, 2020
Verkada's first-generation cameras suffered from numerous video quality...

Recent Reports

Bedside Cough and Sneeze Detector (Sound Intelligence and CLB) on Oct 28, 2020
Coronavirus has increased interest in detecting symptoms such as fever and...
Fever Tablet Thermal Sensors Examined (Melexis) on Oct 28, 2020
Fever tablet suppliers heavily rely on the accuracy and specs of...
Verkada Fires 3 on Oct 28, 2020
Verkada has fired three employees over an incident where female colleagues...
Recruiters Online Show LIVE Thursday! on Oct 27, 2020
IPVM's 7th online show resumes Thursday with 12 recruiters presenting...
Eagle Eye Networks Raises $40 Million on Oct 27, 2020
Eagle Eye has raised $40 million aiming to "reinvent video...
Hikvision Q3 2020 Global Revenue Rises, US Revenue Falls on Oct 27, 2020
While Hikvision's global revenue rises driven by domestic recovery, its US...
VICE Investigates Verkada's Harassing "RawVerkadawgz" on Oct 26, 2020
This month, IPVM investigated Verkada's sexism, discrimination, and cultural...
Six Flags' FDA Violating Outdoor Dahua Fever Cameras on Oct 26, 2020
As Six Flags scrambled to reopen parks amid plummeting revenues caused by the...
ISC Brasil Digital Experience 2020 Report on Oct 23, 2020
ISC Brasil 2020 rebranded itself to ISC Digital Experience and, like its...
Top Video Surveillance Service Call Problems 2020 on Oct 23, 2020
3 primary and 4 secondary issues stood out as causing the most problems when...
GDPR Impact On Temperature / Fever Screening Explained on Oct 22, 2020
What impact does GDPR have on temperature screening? Do you risk a GDPR fine...
Security And Safety Things (S&ST) Tested on Oct 22, 2020
S&amp;ST, a Bosch spinout, is spending tens of millions of dollars aiming to...
Nokia Fever Screening Claims To "Advance Fight Against COVID-19" on Oct 22, 2020
First IBM, then briefly Clorox, and now Nokia becomes the latest Fortune 500...
Deceptive Meridian Temperature Tablets Endanger Public Safety on Oct 21, 2020
IPVM's testing of and investigation into Meridian Kiosk's temperature...
Honeywell 30 Series and Vivotek NVRs Tested on Oct 21, 2020
The NDAA ban has driven many users to look for low-cost NVRs not made by...