Axis MP Pinhole Shootout

Avatar
Ethan Ace
•Published Jun 18, 2013 04:00 AM

For covert surveillance, pinhole lenses / cameras are a common choice because they can be hidden behind walls, or in ordinary objects. Until recently, the only options were analog / SD. However, Axis launched the P12 series, bringing 720p MP to pinhole. How much of a difference does that make and what tradeoffs did it bring?

We bought two Axis P12s plus 3 analog alternatives (two Marshall Electronics pinhole lenses and a Supercircuits [link no longer available] 'snake camera' [link no longer available]). We then tested them head to head in a custom test harness, installing each camera side by side:

Here's the view inside, from left to right, the Axis P1214, the Marshall lenses with Sony box cameras and the Axis P1204. In the top center, is the Supercircuits snake camera:

Here are our key findings from this test:

  • In strong, even lighting, there is no little benefit to megapixel resolution if the camera's intent is to capture close up images of a subject, such as typical ATM machine applications.
  • At farther ranges, even 10' from the camera, the benefits of higher resolution become apparent as details are quickly lost in the analog Supercircuits PC229XP.
  • However, in lowering light (~2 lux), noise in the megapixel models begins to narrow the performance gap, making the Supercircuits PC229XP more competitive.
  • In low light, <1 lux, only the Marshall 2.8mm lens provided usable images. Other cameras were extremely dim and noisy at all ranges. 
  • The 2.8mm Marshall pinhole lens offers significant advantages over the 6.0mm version at close range. The wider angle allows viewing the area to the sides of the main subject, providing better overall awareness in the scene. The lower F stop (1.0 vs. 1.8) also provides facial details in our <1 lx scene, which the 6.0mm does not.
  • HD recording provided increased details over VGA, despite Marshall lenses not being rated for megapixel cameras.
  • In scenes with strong backlighting, the CS mount Marshall pinhole lenses are a clear benefit, as a WDR cameras may be used for best performance.
  • The Supercircuits PC229XP performed well in WDR scenes with the subject close to the camera, outperforming the HD Axis P1204. However, at range, it was by far the worst in our WDR test.

Note: For the purposes of this test, we used Sony G6 cameras with the CS mount Marshall lenses, in order to gauge performance gains when using high performance low light/WDR cameras. Using lower cost or lower quality cameras will affect performance, potentially bringing the lenses to the Axis P12's level of performance.

Price Comparisons

Pricing compares as follows for the models in our test.

  • P1204: $450-500 USD
  • Marshall V-PL60CS: $239 + cost of box camera
  • Marshall V-PL28CS: $329 + cost of box camera
  • Supercircuits PC229XP: $250 [link no longer available] + cost of encoder

Taking the cost of box cameras (in the case of Marshall lenses) or encoders (in the case of the PC229XP) into account, the P1204 is likely the lowest cost and least complex pinhole option. Budget cameras in the $200-300 range could be used with Marshall lenses for a similar price point, but require more difficult installation and likely no real advantage. 

Recommendations

There are three key recommendations to take away from this test:

  • The Axis P12 series performs better than analog options in scenes requiring more than simply close up video. It lags behind high performance low light/WDR cameras with add-on lenses, but is in a much smaller form factor allowing it to be installed where these cameras would not fit. Finally, it is far less expensive than WDR/low light box plus lens options.
  • When absolute best performance is needed, users should consider Marshall's add-on lenses with a quality camera. Assuming size and cost allow, this package will provide the best imaging in low light and WDR applications. In this case, we recommend the wider-angle models, such as the 2.8mm lens in this test, as they offer advantages in situational awareness in the scene, where narrower lenses show only the subject's face at close range.
  • If close up video is the main intent of the camera, the Supercircuits PC229XP may be a reasonable choice, especially when connected to a DVR. The additional cost of an encoder brings the total price close to the Axis P12 series, a better performing camera.

Physical Overviews

In this video we review the construction of each of the cameras in our test and compare form factors. Especially of note:

  • Relative size varies widely, with the analog remote head camera being smallest by far, though it requires a separate encoder.
  • The Axis P12 series is the smallest "all-in-one" model, with very small camera heads suitable for tight locations.
  • Box cameras equipped with pinhole lenses are extremely large, likely to be a limiting factor in many covert installations.

Installation Issues 

In this video we demonstrate mounting the cameras into our pinhole test rig. Users should note the following:

  • Remote head cameras are often mounted using self-adhesive mounts. The P12 and analog cameras both used this type of mount.
  • Box cameras with pinhole lenses require more mounting consideration due to their weight and size. Users should beware of this when planning installation to make sure the camera can be properly supported.

Similar Field of View

First, we compare the three cameras in our test with similar fields of view, all between 50 and 60 degrees: Axis P1204, Marshall V-PL60CS with a Sony VB600, and the analog PC229XP.

Full Light

At close range, the HD/megapixel models provide more even tone and color, without the overexposed areas seen in the PC229XP. However, this does not result in additional details revealed, as details of the subject, even eye color, are clearly visible in the analog camera.

At 10' range, the HD cameras show clear advantages over SD, as the chart is easily readable, with more detail of the subject provided. The PC229XP provides details that may identify a known subject, but some clarity is lost. The chart is also harder to read, due to overexposure and lower resolution.

Finally, at 24' range, the PC229XP shows little detail, enough to specify light or dark hair or clothing, and only the first line of our chart is legible. The P1204 and V-PL60CS provide similar levels of detail, with a slight edge to Axis, which provides better color information

2 Lux

Lowering the light level to 2 lux, the HD cameras show advantages over the analog model. The PC229XP has become so desaturated that it nearly looks black and white, though the subject's face is still clearly detailed. The P1204 and VB600 with Marshall lens, however, still show color information, though with increasing noise.

At 10' range, due to increased noise in the HD cameras, the performance gap between them and the PC229XP narrow somewhat. The chart is still more readable in the HD models, however, due to their higher resolution.


At 24' range, performance is similar to 10', with noise obscuring details in both the P1204 and PL60CS. Again, the chart is more readable in the HD models.

Dark

At close range in our dark scene, below 1 lux, none of the cameras produced usable details of the subject's face. 

Moving to 10', the test chart becomes visible, but the subject is still obscured in all cameras.

Finally, at 24', neither the chart nor the subject are visible in any of the cameras.

Standard vs. Wide Angle

Next, we tested two Marshall pinhole lenses side-by-side, to demonstrate the tradeoffs between the standard 6.0mm lens and the wide angle 2.8mm model.

Full Light Comparison

At close range, ~2', the wide angle lens' benefits are most clear, with much more area to the left and right of the subject visible, which could show details of an assailant approaching from behind. The narrower 6mm model would be unlikely to capture this.

However, at 24', we can see the key drawback to the 2.8mm model's wider FOV. At this distance, details of the subject aside from general color and clothing style information are lost, and the chart becomes unreadable after line 2. The 6.0mm model does not suffer from these issues.

Low Light Performance

The 2.8mm lens has distinct advantages over the 6.0mm model in low light due to its lower F number (1.0 vs. 1.8). This is obvious at near range below 1 lux, where the subject's features are visible in the 2.8mm lens, but not the 6.0.

At 24', the subject is detectable in the 2.8mm lens, but not in the 6.0mm model:

 

HD vs. SD

Though the Marshall lenses have advantages over other pinhole models, allowing the user's choice of camera for specific scenes (low light, WDR), they are not megapixel rated. In order to determine whether the lens was a limiting factor when using HD cameras, we tested the lenses in the same scenes, with the same cameras, at both 1.3MP and VGA resolution. At 2' range, 1.3MP offers little advantage, since pixel count on target is so high, even using VGA cameras. However, we can see fewer details in the background of the wider angle 2.8mm lens.

At 24', results are similar in the 6.0mm model, with the subject and chart easily visible. However, in the 2.8mm lens, the performance gap is narrower. While 1.3MP makes one additional line of the chart readable, and the subject's flace is more discernable, it does not otherwise provide more details. 

Low Light Performance

In low light scenes, below 1 lux, again there is little difference at 2'.

At 24', neither resolution reveals usable details in either camera.

 

WDR

Finally, we set up the cameras in a scene with strong backlighting, against an open overhead door on a sunny day, in order to test their WDR handling.

At close range, with the subject squarely in front of the door, the P1204 provides the dimmest imaging of the subject's face, though details are still visible. The PC229XP handles the subject well, but the background is overexposed, obscuring details outside.

With the subject outside, about 15' from the camera in strong sunlight, the PC229XP is clearly the worst performer, with overexposure obscuring the subject and the entirety of the chart. The P1204 suffers as well, with the chart unreadable past line 1, but the subject still identifiable. The Marshall lenses/Sony cameras perform best, revealing the most details of the subject/chart, though the 2.8mm model suffers somewhat due to lower resolution

Finally, with the subject in the dark area inside, next to the door, at about 10' range, once again the P1204 displays the dimmest image. The PC229XP and Marshall lenses are brighter, with details easily visible.

Test Parameters

The following cameras were used in this test:

All cameras were run using the latest firmware, with default settings with some exceptions:

  • Resolution was set to the camera's maximum, at 30 FPS
  • Encoding was set to VBR with no cap where possible (All cameras aside from the PC229XP, as the SNT-EP104 only supports CBR)
  • Shutter speed was normalized at 1/30s maximum
Comments are shown for subscribers only. Login or Join