Testing The Real Resolution of MP Cameras (Line Count)

Author: Ethan Ace, Published on Dec 11, 2013

Resolution has problems.

IP cameras equate resolution with pixels, ignoring that pixels are simply mechanical components of a camera, not the full performance of said camera. Indeed, it has gotten so bad, we have recommended that we Ban Resolution.

Classic Resolution Metric

With analog cameras, resolution was always defined by line counts (aka TV line counts or horizontal TV lines), which measured the actual details that the full camera could deliver.

In fairness, line counts were far from perfect. They only considered ideal lighting conditions and ignored digital recording. Plus, manufacturers went crazy with them, leading to analog insanity of spiraling, but pointless, TV line counts of 600 to 700 to 800 to 900 to 1000.

However, at least line counts attempted to measure real world performance, not just count parts on a chip (i.e., number of pixels = resolution).

The IP TV Line Test

IPVM is bringing back the TV line test and introducing it to IP / MP cameras.

The setup is an ISO 12233 chart with a camera positioned to capture the area of the chart. The image below shows an IP camera being tested:

We then measured the number of TV lines different cameras delivered in various conditions. Here is an example of two test charts:

Notice that in chart 1, the number of TV lines visible is ~450-500 while chart 2 on the right is much higher, 1,000-1,200.

Questions Answered

This test answers the following key questions:

  • How does pixel count relate to line count?
  • How much are line counts reduced in lower light?
  • How much does compression impact line count?
  • What other factors impact line count?

********** *** ********.

** ******* ****** ********** **** ******, ******** **** ****** *** simply ********** ********** ** * ******, *** *** **** *********** of **** ******. ******, ** *** ****** ** ***, ** have *********** **** ** *** **********.

Classic ********** ******

**** ****** *******, ********** *** ****** ******* ** **** ****** (aka ** **** ****** ** ********** ** *****), ***** ******** the ****** ******* **** *** **** ****** ***** *******.

** ********, **** ****** **** *** **** *******. **** **** considered ***** ******** ********** *** ******* ******* *********. ****, ************* went ***** **** ****, ******* ** ****** ******** ** *********, but *********, ** **** ****** ** *** ** *** ** 800 ***** ** ****.

*******, ** ***** **** ****** ********* ** ******* **** ***** performance, *** **** ***** ***** ** * **** (*.*., ****** of ****** = **********).

The ** ** **** ****

**** ** ******** **** *** ** **** **** *** *********** it ** ** / ** *******.

*** ***** ** ** *** ***** ***** **** * ****** positioned ** ******* *** **** ** *** *****. *** ***** below ***** ** ** ****** ***** ******:

** **** ******** *** ****** ** ** ***** ********* ******* delivered ** ******* **********. **** ** ** ******* ** *** test ******:

****** **** ** ***** *, *** ****** ** ** ***** visible ** ~***-*** ***** ***** * ** *** ***** ** much ******, *,***-*,***.

Questions ********

**** **** ******* *** ********* *** *********:

  • *** **** ***** ***** ****** ** **** *****?
  • *** **** *** **** ****** ******* ** ***** *****?
  • *** **** **** *********** ****** **** *****?
  • **** ***** ******* ****** **** *****?

[***************]

Key ********

**** *** *** *** ******** **** **** ****:

  • ***** ***** *** *** ******* ****** ** ** ***** ********, an ******** ** ******** ** *** *** **** **** ** resolution (**** ** *****, ***** ** ***, ***.).
  • ******** ****** ************* ******** ******* ** *** ***********, ** *** order ** ******** ** *** **** ****** **** ****** ** 3 *** *** ***** * ***.
  • *********** ************ *** ** ********* ******. ***** ** ***** ******* of *** ***** ********************** *** **** ***** **** ** ******* *** **** ***** distinguishable **** ** ******* ***********.
  • **** ********* ************** ******, **** *********** ** ***-*** *** ******** ****** **** SD ** ********* **********************.
  • ********* ********* ******** ******, ********* **** ******** *** ***** **** ******** ** to ******* ******. ********** ***** ******** *** ** ********* ******.
  • ***** ***** *********** **** ** ******** *** ********** *** ******* effect, **** ** ********** ** *** ********.

******* ****** **** **** ** **** ******** ****** ** *** in **** ******, ** ******* ** ***** ************, ***** ************** of ***** ******* *** ** ********** *** **** ******** **** person ** ******.

Test *****

**** ***** ***** ******* *** ******** ***** ** *** **** and *** ** *** *****.

Summary ********

**** ***** ***** *** *** ** ***** *** **** ****** in ********* ******** **********:

***** ** *** ***** *******, ***** ***** ******* ** *** in * ***** ** ************* *:*.** ** **** *****. ** for *******, ***** * *** ****** **** * ********** ********** of ****, ** *** ******** *** ******** ** ** ~*,*** (2560*0.41=1,049). 

Pixel ***** **********

** * ********, ** *** ** *** ******* ** * brightly *** **** ******* ******** ********. ******* **** ****** *** at * **** ** ******** ****** *** *** ******** ***** to *** *****, *** ******* ******** **** ****.

** **** ** *** ********** *****, *** ****** **** ~***-*** in **** ** ~*,***-*,*** ** *** **** ******.

(***** *** **** **** *******)

Lowering *****

****, ** ******* *** ****** ** * *** ** **** effective *** ******** ** ***** ****. ******* **** ************ ** 1/30s ******** ** ***** *** ***** ******.

*** ******* ************* ** *** *******. *** ***** ****** **** maintained *********** ** **** ***** *** ** * *********** ** low ***** *********** *** **********. 

(***** *** *** **** **** *******)

**** *** ****** ***, *** ***** ***** ***** ** ***** 0.1 ***. **** *** **** *** ***** ******* ******* ****** images ** **** *****, ***** ** * ******* *********** *****, with *** **** ****** ******** ** ~***-*** ***, *** ***** camera ** ~***-***. 

(***** *** **** **** *******)

***********

** ***** ** **** ***********'* ******* ** *** ********, ** varied ******** **** ******* ** ******* (*-***, **** * ******* of ** ** *** **** ****** **** *****). ***** ******* had ****** ****** ** ***. **** *********** ********* **** **** on *** ***** ** **** ****** ******* ** *** *****, but *** **** ***** **** ** ******* *** ******** ****** unchanged ********** ** ***********.

*******, ** ******* *** *********** ******* *.*** *** *****, ***** also *** ** ********* ******, ******* ***** **** *.*** ** ********** *** ***** ******* **** *****.

Lens ****

** ***** ** *** **** ****** **** ********* *** ** TVL ********, ** *** *** **** ***** * **** ****** with **** ********* ******: *** ******** **********, *** ********* *****, and *** "***** **** **********" *****-********* ***** ****. ** ***** that **** ********* *** *** ****** ******* ****** ** *** captured, ***** ***** *****, **** * ******** ** ***+ *** possible ****** **** ******** ********** ****** ** *****-*********. ************, **** our *** ********* ***** ****** (******** *** *******) ******** ** different *** ******* (***-*** **. ***-***). 

(***** *** **** **** *******)

****** **** **** (*-*):

Image ********

*******, ** ******* *** ****** ** ***** *********** **** ** contrast, *********, *** ********** ** ***.

***** *****, **** ********* ************* ******, ******** ******** *** **** ** ** ******* ** minimum **** *******. ********** **** ******* ** ******* ******** ** gain ** ***.

 

******** *** **********, ** *** ***** ****, *** ** ********** effect ********** ** *******.

Comments (25)

Members,

What should we do with testing tv line counts? Should we adopt this as standard practice?

Or should we adopt eye chart testing?

Or neither?

It is good to have a numerical metric to compare performance but it is important that the metric be meaningful and accurate.

I think this is useful and interesting data. As you point out, there is much more to getting detailed images than just simple pixel count.

For the cameras you test, it would be helpful if you had a summary of a few key components in a standard format:

Max TVL Measurement (though I wish the TVL term would die)

Max sustainable framerate at full res

Color accuracy

Power consumption

Key protocol supported.

WDR range

Other people might have additional suggestions for data, but the idea would be to summarize all tests with a set of consistent measurements that allows people to make semi-informed comparisons between various units.

Both have good uses in tests. This test reminds me of how dpreview does their camera and lens tests. These tests may be better on long lens distortion perhaps.

By far, I found the TVL comparison and testing more beneficial. I could image a benchmark being established, such as a Pixel count/TVL ratio.

If an product under test performed dramatically different than the de facto standard, that seems to me to be very relevant information. In other words does the camera ultimately deliver on it’s pixel rating?

John, have you considered the Rotakin test chart? It was designed specifically for CCTV performance testing. However I am not sure whether they have updates it to handle megapixel cameras or not.

Jason, we have considered the Rotakin but I have never understood its value. It's ~$1000 for a cardboard cutout and, outside the UK, it has no standing.

For quantitative measurement, I am not sure why that would be better than an ISO 12233 or a Snelling eye chart for that matter.

I would agree that it seems a bit pricey, but it does add some further dimensions to a camera test - movement, the motor is capable of spinning the figure continuously which may be an added criteria for your testing (motion blur and frame integration impact) - and the number plate characters may be UK standard but are very relevant for a lot of users who want to "read number plates".

Of course you could always design your own combined chart...

It's not the price to us, but the value. You can get real used license plates for $20 online. Plus, there's lot of others ways to check for blur, most realistically simple using a real subject and vehicle.

Interesting, in that it would appear that for most purposes 1080p is the optimal bang for the buck, performance wise. Nearly double that of 720p and only slightly less than 5mpxl. Also has the best low light performance.

Great Job Ethan

What Cameras were used in the test with which len's ?

Ok to respond privately . send me an email , John has my email.

We test cameras this way and use a test chart that Irlabs.net hands out. It has resolution on one side and colors on the other. We have been suprised how many subtle color varitions the different cameras produce. Many cannot tell the difference between dark purple and black. The reds also show a great variety of results. Your comments about the lenses differences are important also. Better glass, better picture.

As a comparison, I would like to see a higher end analog camera that is spec'd at 650 TVL in your test. According to what you show here, a 720P camera is worse than one of the analog cameras rated at 650 TVL (assuming thay they really have that spec.)

Hi Peter,

Thanks. We will put this on the list of things to test. I've never heard anyone say that 700TVL or 800TVL made much of a practical difference so I am not bullish on the results but worth a look.

Certainly I found this testing method beneficial but for my needs, I find your subject/eye-chart/field- enviornment method of testing to be of much greater value. Thanks to your team btw.

This software is freeware provided free of charge by the Olympus Corporation to which the creator of the software belong as a service to allow users to make more effective use of the ISO standard ISO12233

:)

This meshes well with our own internal testing, that the current all-around best value to performance is still around 2MP or 1080P. Average, across multiple lighting conditions. Obviously, in an area with very good lighting day and night, such as under a service station canopy, a higher MP could be beneficial.

I am an IPVM eye chart fan but only when I can actually see your test results. Life would be much simpler if if you gave us a number.....whether you call it TVL or something else. I think if the TVL method got me narrowed down to 2 or 3 cameras then I could go look at the eye chart stills to finish the decision making process, it would be perfect. I know that some of us want everything. Thanks for all you guys do!

A single number would be great but:

  • There needs to be at least a few numbers - e.g., day vs night performance. A 10MP camera that scores a 1300 in the day may score a 100 at night vs a 2MP camera that might score a 900 in the day and 400 at night, etc. And then there's WDR which would be a third.
  • How will people use the numbers? If we say Camera A is an 1100 score and Camera B is a 1000 score, what does that actually mean? How much better will Camera A be or will it be distorted as an absolute metric?
  • These numbers might vary somewhat depending on lens focus, compression level, firmware level, etc. I am a little nervous with a single number. A small range might be more appropriate - like 1000 - 1100 vs 1050.

Overall, trust me, I see the value in a single number because there is real demand for it and it would make us look more 'scientific' (because people wrongly assume a precise number means a precise method). But I am concerned about the downsides.

Is there a way to introduce the effect of motion on your test? You mentioned there was no significant difference using various compression settings, but I would assume that was looking at a static scene. Start moving the chart in front of the camera, and let's see what happens. I assume that is where a 730p 60 IPS cameras will perform much better than a 10 MP camera, etc.

Greg, we have done various tests with motion, like 60fps vs 30fps one and the H.264 I vs P frame.

I am not sure what you are looking for here. Are you interested in seeing if there is going to be compression artifacts or?

For instance, in a test of 720p/60 vs a 10MP camera with motion, I am quite certain that neither will have any blurring or smearing but the 10MP cameras (typically maxed out at ~6fps) will miss very fast motion (like counting money).

Hi

This test is nice, but it's a B/W only testing,so we need additionnal stuff again.

The international EN50132 target is going to be an international test reference

It does combine B/W , Color and details ( Pixel density Inspection / Identification / Recognition / Detection ) and it's very cheap A3 format plastified sold here round 3 for 60$

Marc,

Thanks. Were on the chart is the TV Line count / pixel count / resolution portion? Is it the D / C / B / A portion and if so, what ranges does it cover?

In fact with the European Norm EN 50132-7 includs 76 pages explanations, and also the 9 UK faces ... and how we can test Licence plate reco ..so I include just small parts (in French sorry)

We are working inFrance on the the futur way to measure performance of installations and also maintenance process. Because the technical level is quite low (in general) we try to push the most simple way to test

Bellow kind of classification .

In France 90x60 pixels are required to reco a face (25x15) so +/- 400 PPM

"Compression

In order to test compression's effects on TVL captured, we varied settings from minimum to maximum (0-100, with a default of 30 in the Axis camera seen below). These changes had little effect on TVL. "

It seems you have changed compression with enough light.
Try to change compression on low light or reduce contrast of the test chart image then increase compression. I suppose in these conditions compression will have big influence on resolution.

TVL testing is what led me to IPVM. I see that this is from more than 3 years ago but not seeing any recent. Any plans to actually do TVL testing?

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Lenses

Avigilon H4 Multi-Sensor Adds 32MP, H.265, Analytics on Jun 13, 2018
Avigilon has announced the H4 Multisensor, the successor to their repositionable multi imager line, adding features like H.265, integrated IR,...
Top Features For Easy Camera Installation (Statistics) on Jun 05, 2018
Camera installation is the most fundamental and common task for video security technicians. Because of this, camera manufacturers market their...
Installing Box Cameras Indoors Tutorial on May 22, 2018
This tutorial starts our physical installation for video surveillance series, starting with Box Cameras, one of the oldest and most basic types....
Axis Launches First Stereographic Panoramic Camera on Apr 05, 2018
Fisheye panoramics have not had much innovation in the past few years, with their multi-imager cousins getting most of the advances, e.g.,...
Dahua 4K Micro 4/3 Camera Tested (IPC-HF8835F) on Apr 03, 2018
Dahua is going big. Their new 4K HF8835F Micro Four Thirds sensor is one of the largest now on the market. Dahua is claming: Starlight...
Camera Form Factor Guide on Mar 16, 2018
When selecting surveillance cameras, users may choose from a number of different form factors, each with its own unique strengths and weaknesses,...
Dahua 4K IR PTZ Tested on Aug 21, 2017
4K has made its way to IR PTZs. In this report, we examine the Dahua 6AE830VNI, a 4K PTZ with 30x optical zoom, 200m (~650') integrated IR, and...
IP Camera Specification / RFP Guide 2017 on Aug 14, 2017
RFPs are hard. Do them 'right' and it takes a lot of knowledge and time. Do them 'wrong' and you can be (a) unwittingly locked into a specific...
Technician Personal Protective Equipment (PPE) Guide on May 12, 2017
Technicians encounter multiple hazards when running wires and installing security devices. Wearing personal protective equipment, or PPE, helps...
Varifocal Vs Fixed Lens Statistics 2017 on May 04, 2017
Varifocal camera lenses provide more flexibility at installation time, but come at a higher cost. Fixed lens cameras reduce hardware costs, but can...

Most Recent Industry Reports

Free 100+ Manufacturer-Customized Camera Calculator Released on Jul 19, 2018
Now, any manufacturer has a customized IPVM Camera Calculator, free. The goal is to make it easier for companies to help their customers better...
Improved Security And Surveillance Bidding - 2018 MasterFormat Divisions Examined) on Jul 19, 2018
Navigating the world of system specifications and bidding work can be complex and confusing, but a standard format exists, and understanding it...
Last Chance - Security Sales Course Summer 2018 on Jul 19, 2018
Today is the last day to register. Based on member's interest, IPVM is offering a security sales course this summer. Register Now - IPVM Security...
Directory of Video Surveillance Startups on Jul 18, 2018
This directory provides a list of video surveillance startups to help you see and research what companies are new or not yet broadly known entity...
Ladder Lockdown and Ladder Levelizer Tested on Jul 18, 2018
Ladders are a daily necessity for surveillance and security installers, but working on an unstable surface can be extremely dangerous. In addition...
FST Fails on Jul 17, 2018
FST was one of the hottest startups of the decade, selected as the best new product at ISC West 2011 and backed with tens of millions in...
Axis ~$100 Camera Tested on Jul 17, 2018
Axis has released their lowest cost camera ever, the Companion Eye Mini L, setting their sights on a market dominated by Hikvision and Dahua. Can...
Amazon Ring Alarm System Tested on Jul 16, 2018
Amazon Ring is going to hurt traditional dealers, and especially ADT, new IPVM test results of Ring's Alarm system underscore. IPVM found that...
Hikvision Wins Chinese Government Forced Facial Recognition Project Across 967 Mosques on Jul 16, 2018
Hikvision has won a Chinese government tender which requires that facial recognition cameras be set up at the entrance of every single mosque...
Installing Dome Cameras Indoors Guide on Jul 16, 2018
IPVM is producing the definitive series on installing surveillance cameras. This entry covers one of the most common scenarios - installing dome...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact