Testing The Real Resolution of MP Cameras (Line Count)

By: Ethan Ace, Published on Dec 11, 2013

Resolution has problems.

IP cameras equate resolution with pixels, ignoring that pixels are simply mechanical components of a camera, not the full performance of said camera. Indeed, it has gotten so bad, we have recommended that we Ban Resolution.

Classic Resolution Metric

With analog cameras, resolution was always defined by line counts (aka TV line counts or horizontal TV lines), which measured the actual details that the full camera could deliver.

In fairness, line counts were far from perfect. They only considered ideal lighting conditions and ignored digital recording. Plus, manufacturers went crazy with them, leading to analog insanity of spiraling, but pointless, TV line counts of 600 to 700 to 800 to 900 to 1000.

However, at least line counts attempted to measure real world performance, not just count parts on a chip (i.e., number of pixels = resolution).

The IP TV Line Test

IPVM is bringing back the TV line test and introducing it to IP / MP cameras.

The setup is an ISO 12233 chart with a camera positioned to capture the area of the chart. The image below shows an IP camera being tested:

We then measured the number of TV lines different cameras delivered in various conditions. Here is an example of two test charts:

Notice that in chart 1, the number of TV lines visible is ~450-500 while chart 2 on the right is much higher, 1,000-1,200.

Questions Answered

This test answers the following key questions:

  • How does pixel count relate to line count?
  • How much are line counts reduced in lower light?
  • How much does compression impact line count?
  • What other factors impact line count?

********** *** ********.

** ******* ****** ********** with ******, ******** **** pixels *** ****** ********** components ** * ******, not *** **** *********** of **** ******. ******, it *** ****** ** bad, ** **** *********** that ** *** **********.

Classic ********** ******

**** ****** *******, ********** was ****** ******* ** line ****** (*** ** line ****** ** ********** TV *****), ***** ******** the ****** ******* **** the **** ****** ***** deliver.

** ********, **** ****** were *** **** *******. They **** ********** ***** lighting ********** *** ******* digital *********. ****, ************* went ***** **** ****, leading ** ****** ******** of *********, *** *********, TV **** ****** ** 600 ** *** ** 800 ***** ** ****.

*******, ** ***** **** counts ********* ** ******* real ***** ***********, *** just ***** ***** ** a **** (*.*., ****** of ****** = **********).

The ** ** **** ****

**** ** ******** **** the ** **** **** and *********** ** ** IP / ** *******.

*** ***** ** ** ISO ***** ***** **** a ****** ********** ** capture *** **** ** the *****. *** ***** below ***** ** ** camera ***** ******:

** **** ******** *** number ** ** ***** different ******* ********* ** various **********. **** ** an ******* ** *** test ******:

****** **** ** ***** 1, *** ****** ** TV ***** ******* ** ~450-500 ***** ***** * on *** ***** ** much ******, *,***-*,***.

Questions ********

**** **** ******* *** following *** *********:

  • *** **** ***** ***** relate ** **** *****?
  • *** **** *** **** counts ******* ** ***** light?
  • *** **** **** *********** ****** **** count?
  • **** ***** ******* ****** line *****?

[***************]

Key ********

**** *** *** *** findings **** **** ****:

  • ***** ***** *** *** largest ****** ** ** lines ********, ** ******** of ******** ** *** for **** **** ** resolution (**** ** *****, 1080p ** ***, ***.).
  • ******** ****** ************* ******** cameras ** *** ***********, on *** ***** ** hundreds ** *** **** lights **** ****** ** 3 *** *** ***** 1 ***.
  • *********** ************ *** ** practical ******. ***** ** large ******* ** *** chart ********************** *** **** ***** used ** ******* *** were ***** *************** **** at ******* ***********.
  • **** ********* ************** ******, **** *********** of ***-*** *** ******** moving **** ** ** megapixel **********************.
  • ********* ********* ******** ******, ********* **** reducing *** ***** **** lowering ** ** ******* levels. ********** ***** ******** had ** ********* ******.
  • ***** ***** *********** **** as ******** *** ********** had ******* ******, **** no ********** ** *** captured.

******* ****** **** **** we **** ******** ****** of *** ** **** report, ** ******* ** exact ************, ***** ************** of ***** ******* *** be ********** *** **** somewhat **** ****** ** person.

Test *****

**** ***** ***** ******* our ******** ***** ** the **** *** *** of *** *****.

Summary ********

**** ***** ***** *** TVL ** ***** *** each ****** ** ********* lighting **********:

***** ** *** ***** numbers, ***** ***** ******* to *** ** * ratio ** ************* *:*.** in **** *****. ** for *******, ***** * 5MP ****** **** * horizontal ********** ** ****, we *** ******** *** captured ** ** ~*,*** (2560*0.41=1,049). 

Pixel ***** **********

** * ********, ** set ** *** ******* in * ******** *** room ******* ******** ********. Cameras **** ****** *** at * **** ** properly ****** *** *** mounting ***** ** *** chart, *** ******* ******** were ****.

** **** ** *** comparison *****, *** ****** from ~***-*** ** **** to ~*,***-*,*** ** *** 10MP ******.

(***** *** **** **** version)

Lowering *****

****, ** ******* *** lights ** * *** to **** ********* *** captured ** ***** ****. Cameras **** ************ ** 1/30s ******** ** ***** low ***** ******.

*** ******* ************* ** all *******. *** ***** camera **** ********** *********** in **** ***** *** to * *********** ** low ***** *********** *** resolution. 

(***** *** *** **** size *******)

**** *** ****** ***, the ***** ***** ***** to ***** *.* ***. Only *** **** *** 1080p ******* ******* ****** images ** **** *****, again ** * ******* performance *****, **** *** 720p ****** ******** ** ~200-300 ***, *** ***** camera ** ~***-***. 

(***** *** **** **** version)

***********

** ***** ** **** compression's ******* ** *** captured, ** ****** ******** from ******* ** ******* (0-100, **** * ******* of ** ** *** Axis ****** **** *****). These ******* *** ****** effect ** ***. **** compression ********* **** **** on *** ***** ** some ****** ******* ** the *****, *** *** fine ***** **** ** measure *** ******** ****** unchanged ********** ** ***********.

*******, ** ******* *** differences ******* *.*** *** MJPEG, ***** **** *** no ********* ******, ******* proof **** *.*** ** ********** *** lower ******* **** *****.

Lens ****

** ***** ** *** what ****** **** ********* had ** *** ********, we *** *** **** using * **** ****** with **** ********* ******: one ******** **********, *** megapixel *****, *** *** "ultra **** **********" *****-********* rated ****. ** ***** that **** ********* *** the ****** ******* ****** on *** ********, ***** pixel *****, **** * variance ** ***+ *** possible ****** **** ******** definition ****** ** *****-*********. Additionally, **** *** *** megapixel ***** ****** (******** and *******) ******** ** different *** ******* (***-*** vs. ***-***). 

(***** *** **** **** version)

****** **** **** (*-*):

Image ********

*******, ** ******* *** impact ** ***** *********** such ** ********, *********, and ********** ** ***.

***** *****, **** ********* had********** ******, ******** ******** TVL **** ** ** lowered ** ******* **** default. ********** **** ******* to ******* ******** ** gain ** ***.

 

******** *** **********, ** the ***** ****, *** no ********** ****** ********** of *******.

Comments (25)

Members,

What should we do with testing tv line counts? Should we adopt this as standard practice?

Or should we adopt eye chart testing?

Or neither?

It is good to have a numerical metric to compare performance but it is important that the metric be meaningful and accurate.

I think this is useful and interesting data. As you point out, there is much more to getting detailed images than just simple pixel count.

For the cameras you test, it would be helpful if you had a summary of a few key components in a standard format:

Max TVL Measurement (though I wish the TVL term would die)

Max sustainable framerate at full res

Color accuracy

Power consumption

Key protocol supported.

WDR range

Other people might have additional suggestions for data, but the idea would be to summarize all tests with a set of consistent measurements that allows people to make semi-informed comparisons between various units.

Both have good uses in tests. This test reminds me of how dpreview does their camera and lens tests. These tests may be better on long lens distortion perhaps.

By far, I found the TVL comparison and testing more beneficial. I could image a benchmark being established, such as a Pixel count/TVL ratio.

If an product under test performed dramatically different than the de facto standard, that seems to me to be very relevant information. In other words does the camera ultimately deliver on it’s pixel rating?

John, have you considered the Rotakin test chart? It was designed specifically for CCTV performance testing. However I am not sure whether they have updates it to handle megapixel cameras or not.

Jason, we have considered the Rotakin but I have never understood its value. It's ~$1000 for a cardboard cutout and, outside the UK, it has no standing.

For quantitative measurement, I am not sure why that would be better than an ISO 12233 or a Snelling eye chart for that matter.

I would agree that it seems a bit pricey, but it does add some further dimensions to a camera test - movement, the motor is capable of spinning the figure continuously which may be an added criteria for your testing (motion blur and frame integration impact) - and the number plate characters may be UK standard but are very relevant for a lot of users who want to "read number plates".

Of course you could always design your own combined chart...

It's not the price to us, but the value. You can get real used license plates for $20 online. Plus, there's lot of others ways to check for blur, most realistically simple using a real subject and vehicle.

Interesting, in that it would appear that for most purposes 1080p is the optimal bang for the buck, performance wise. Nearly double that of 720p and only slightly less than 5mpxl. Also has the best low light performance.

Great Job Ethan

What Cameras were used in the test with which len's ?

Ok to respond privately . send me an email , John has my email.

We test cameras this way and use a test chart that Irlabs.net hands out. It has resolution on one side and colors on the other. We have been suprised how many subtle color varitions the different cameras produce. Many cannot tell the difference between dark purple and black. The reds also show a great variety of results. Your comments about the lenses differences are important also. Better glass, better picture.

As a comparison, I would like to see a higher end analog camera that is spec'd at 650 TVL in your test. According to what you show here, a 720P camera is worse than one of the analog cameras rated at 650 TVL (assuming thay they really have that spec.)

Hi Peter,

Thanks. We will put this on the list of things to test. I've never heard anyone say that 700TVL or 800TVL made much of a practical difference so I am not bullish on the results but worth a look.

Certainly I found this testing method beneficial but for my needs, I find your subject/eye-chart/field- enviornment method of testing to be of much greater value. Thanks to your team btw.

This software is freeware provided free of charge by the Olympus Corporation to which the creator of the software belong as a service to allow users to make more effective use of the ISO standard ISO12233

:)

This meshes well with our own internal testing, that the current all-around best value to performance is still around 2MP or 1080P. Average, across multiple lighting conditions. Obviously, in an area with very good lighting day and night, such as under a service station canopy, a higher MP could be beneficial.

I am an IPVM eye chart fan but only when I can actually see your test results. Life would be much simpler if if you gave us a number.....whether you call it TVL or something else. I think if the TVL method got me narrowed down to 2 or 3 cameras then I could go look at the eye chart stills to finish the decision making process, it would be perfect. I know that some of us want everything. Thanks for all you guys do!

A single number would be great but:

  • There needs to be at least a few numbers - e.g., day vs night performance. A 10MP camera that scores a 1300 in the day may score a 100 at night vs a 2MP camera that might score a 900 in the day and 400 at night, etc. And then there's WDR which would be a third.
  • How will people use the numbers? If we say Camera A is an 1100 score and Camera B is a 1000 score, what does that actually mean? How much better will Camera A be or will it be distorted as an absolute metric?
  • These numbers might vary somewhat depending on lens focus, compression level, firmware level, etc. I am a little nervous with a single number. A small range might be more appropriate - like 1000 - 1100 vs 1050.

Overall, trust me, I see the value in a single number because there is real demand for it and it would make us look more 'scientific' (because people wrongly assume a precise number means a precise method). But I am concerned about the downsides.

Is there a way to introduce the effect of motion on your test? You mentioned there was no significant difference using various compression settings, but I would assume that was looking at a static scene. Start moving the chart in front of the camera, and let's see what happens. I assume that is where a 730p 60 IPS cameras will perform much better than a 10 MP camera, etc.

Greg, we have done various tests with motion, like 60fps vs 30fps one and the H.264 I vs P frame.

I am not sure what you are looking for here. Are you interested in seeing if there is going to be compression artifacts or?

For instance, in a test of 720p/60 vs a 10MP camera with motion, I am quite certain that neither will have any blurring or smearing but the 10MP cameras (typically maxed out at ~6fps) will miss very fast motion (like counting money).

Hi

This test is nice, but it's a B/W only testing,so we need additionnal stuff again.

The international EN50132 target is going to be an international test reference

It does combine B/W , Color and details ( Pixel density Inspection / Identification / Recognition / Detection ) and it's very cheap A3 format plastified sold here round 3 for 60$

Marc,

Thanks. Were on the chart is the TV Line count / pixel count / resolution portion? Is it the D / C / B / A portion and if so, what ranges does it cover?

In fact with the European Norm EN 50132-7 includs 76 pages explanations, and also the 9 UK faces ... and how we can test Licence plate reco ..so I include just small parts (in French sorry)

We are working inFrance on the the futur way to measure performance of installations and also maintenance process. Because the technical level is quite low (in general) we try to push the most simple way to test

Bellow kind of classification .

In France 90x60 pixels are required to reco a face (25x15) so +/- 400 PPM

"Compression

In order to test compression's effects on TVL captured, we varied settings from minimum to maximum (0-100, with a default of 30 in the Axis camera seen below). These changes had little effect on TVL. "

It seems you have changed compression with enough light.
Try to change compression on low light or reduce contrast of the test chart image then increase compression. I suppose in these conditions compression will have big influence on resolution.

TVL testing is what led me to IPVM.  I see that this is from more than 3 years ago but not seeing any recent.  Any plans to actually do TVL testing?

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Lenses

Camera Focusing Tutorial on Oct 14, 2019
Camera focus is fundamental to quality imaging. Mistakes can significantly reduce details, making cameras less effective. In this guide, we...
Lens Iris Tutorial on Aug 08, 2019
Cameras, like humans, have irises, controlling the light which hits the imager and impacting image details. However, cameras have multiple types of...
Lens Focal Length Tutorial on Jul 10, 2019
3mm, 6mm, 2.8 - 9mm, 5 - 50mm, etc. Camera specifications often list lens lengths but what do they mean? These metrics are important in...
Repositionable Multi-Imager Camera Shootout - Avigilon, Axis, Dahua, Hanwha, Hikvision, Panasonic, Vivotek on Jun 19, 2019
Repositionable multi-imager cameras are one of the fastest growing segments in video surveillance, with a slew of new offerings being recently...
Panasonic 32MP Multi Imager Camera Tested (WV-X8570N) on May 16, 2019
Panasonic has released their first multi imager models including the 32MP (4x4K) WV-X8570N, claiming "Extreme image quality for evidence capturing...
Axis IR Multi Imager Camera Tested (P3717-PLE) on Feb 18, 2019
Axis has released their first IR multi imager, the P3717-PLE, a repositionable model listing 360° IR illumination and flexible positioning,...
Hanwha Smaller Multi Imager Tested (PNM-9000VQ) on Feb 14, 2019
Hanwha's first repositionable multi imager PNM-9081VQ tested well, but was huge, over 12" wide and weighing in at over 10 pounds. Now, they have...
Avigilon H4 Multi-Sensor Adds 32MP, H.265, Analytics on Jun 13, 2018
Avigilon has announced the H4 Multisensor, the successor to their repositionable multi imager line, adding features like H.265, integrated IR,...
Top Features For Easy Camera Installation (Statistics) on Jun 05, 2018
Camera installation is the most fundamental and common task for video security technicians. Because of this, camera manufacturers market their...
Installing Box Cameras Indoors Guide on May 22, 2018
This guide starts our physical installation for video surveillance series, starting with Box Cameras, one of the oldest and most basic types....

Most Recent Industry Reports

Alarm Veteran "Demands A Criminal Investigation" Of UL on Oct 18, 2019
The Interceptor's Project pressure against UL continues to rise. Following Keith Jentoft's allegation that "UL Has Blood On Their Hands", Jentoft...
Camect "Worlds Smartest Camera Hub" Tested on Oct 18, 2019
Camect is a Silicon Valley startup that claims the "Smartest AI Object Detection On The Market", detecting not only people and vehicles, but...
Hikvision Global News Reports Directory on Oct 17, 2019
Hikvision has received the most global news reporting of any video surveillance company, ever, ranging from the WSJ, the Financial Times, Reuters,...
Camera Calculator V3.1 Release Improves User Experience on Oct 17, 2019
IPVM has released a new version of our Camera Calculator, V3.1, with significant user experience improvements, a new development plan, and an...
Securing Access Control Installations Tutorial on Oct 17, 2019
The physical security of access control components is critical to ensuring that a facility is truly secure. Otherwise, the entire system can be...
Access Control Course Fall 2019 - Last Chance on Oct 17, 2019
Register Now - Fall 2019 Access Control Course. Thursday, October 17th is the last day to register. IPVM offers the most comprehensive access...
US DoD Comments on Huawei, Hikvision, Dahua Cyber Security Concerns on Oct 16, 2019
A senior DoD official said the US is "concerned" with the cybersecurity of Hikvision, Dahua, and Huawei due to "CCP" (China Communist Party)...
Pelco Sarix Pro3 Camera Tested on Oct 16, 2019
Pelco has released their Sarix Professional Series 3 cameras, claiming "more security detail in challenging scenes with excellent low light and...
IPVM Camera Calculator User Manual / Guide on Oct 16, 2019
Learn how to use the IPVM Camera Calculator. The guide below includes instructions, images, gifs, and videos demonstrating and explaining the...
Altronix Claims Tango 'Eliminates Electricians' on Oct 15, 2019
Power supply provider Altronix claims its new Tango power supply 'eliminates the need for an electrician, dedicated conduit and wire runs'. In...