We found it to be pretty close to what we determine by eye, except when using 5 and 10 MP cameras, in which case it overestimates resolution by far (listing 2036 when the chart only goes to 2000and it draws the red line seen in the image at 1600). It has worked accurately for 720p, 1.3MP, and 1080p cameras, however.
This short internal video gives a glimpse of what we saw:
What worries us most is that this is freeware, with only a short manual and little to no support if we have questions or issues.
We are also considering using Imatest, which is far more complex, but measures numerous parameters. Imatest has told me that some manufacturers in the surveillance industry are using it. It will test resolution, dynamic range, sharpness, color fidelity, and more and more. Full list here. This video shows just one portion of it:
Does anyone have any experience with either of these packages?
Ethan, the only thing I have found is Acti´s article on using HYRs, http://www.acti.com/getfile/KnowledgeBase_UploadFile/TVL-The_True_Measurement_of_Video_Quality_20101027_002.pdf. So it´s a partial point of view on using the tool
+1 for Imatest. Instead of just one sharpness test it covers pretty much all aspects of image quality, although it is geared more toward still pictures. Kind of easy to use (compared to others) and produces a lot of pretty graphs. Has some problems evaluating images with a large distortion, though. The only downside I can see is that you will have to spend a significant amount of time explaining what each test does.
Other solutions you might consider: Image Engineering has a complete line-up for camera testing, including test charts, software and even complete test stands.
ImageJ is a free image processing software originally designed by the NIH. Comes with its own programming language and is extensively supported by a large community of developers. There is a plugin to analyze resolution using slanted edges according to the ISO12233 here.