H.264 High vs Main vs Baseline Tested

IPVM

Since someone is bound to ask, H.265 is too early even to guess. There's only IP camera manufacturer shipping today (Aventura) and it only works with their own recorders.

I do think we risk similar disappointment with H.265 as we see above with H.264 High profile, depending on how many manufacturers implement support.

Background - H.265 / HEVC Codec Tutorial

Carl Lindgren

I'm kind of disappointed that you didn't also measure the latency difference between profiles.

John Honovich

IPVM

Thank you passive aggressive Carl. Would you like us to measure the latency difference?

Carl Lindgren

In reply to John Honovich

Yes.

Ethan Ace

We didn't set up a stopwatch to test this, but looking at video on these cameras on all three profiles, we saw no real difference in latency when the profile was changed.

Cameras with low latency stayed low latency. Cameras with higher latency stayed higher latency.

Zeb Pena

Very good report! I am curious as to what the results would be from earlier adopters of high profile such as Bosch and Panasonic. Maybe a future report will include them and Axis? HEVC I would expect higher as well especially if SVC functionality is part of the mix and feature set implemented.

Ethan Ace

In reply to Zeb Pena

Hi Zeb, I quickly checked a Bosch stream (because they don't have explicit options to switch profile) and it seems they are using main profile. They do have an option to use B frames, so maybe turning that on switches to high profile, but B frames are also supported in main, so maybe not.

As for Axis, we don't have an ARTPEC-5 camera yet, but when the Q1615 is released we'll test it. And we should have new generation Panasonic cameras coming.

It's simple enough to switch profiles as we're set up for future tests so we can add to this later and check other cameras.

Carl Lindgren

In reply to Ethan Ace

Ethan,

When we tested a Bosch camera, we found it had at least three options but I don't recall it listing profiles, per se. It seems to me the options were:

No 'B' frames (IP)
One 'B' frame (IBP)
Two 'B' frames (IBBP)

I also seem to recall that at least one of the Bosch products we tested had a setting for IBBrBP but I don't recall which product.

On a note related to my original question, our tests found that each increase in 'B' frames caused a corresponding increase in latency. I don't have the exact figures but as I recall, the latency increase between baseline profile (or IP) and higher profiles was fairly substantial, on the order of at least one frame (33ms) for each step.

Luke Maslen

IPVMU Certified

Hi Derek, thank you for a very useful report. Did you also make a note of any changes in image quality when switching each camera model between the 3 profiles? In particular, were there more visible artifacts with any of the 3 profiles? Thank you!

Jim Hall

Basically, as you move 'up' from baseline to main to high, you should get the same quality for less bandwidth....there was no increase in VMS client or VLC media player CPU usage when decoding high profile streams vs. main vs. baseline

Does this mean there is a free lunch to be had after all? Quality is just better with no downsides? What in tarnation were they dilly dallying around all this time for? Is there an ultra-high profile just around the next bend?

Carl Lindgren

In reply to Jim Hall

Jim,

I would venture that, although there may not be any increase in decoder CPU load, there is likely an increase in encoder CPU load.

John Honovich

In reply to John Honovich

IPVM

In reply to Jim Hall

Jim, I think you are misreading that on a few levels. First, we were not saying the quality is 'better'. We said the quality was the 'same'.

Also, this is the theory. As we explain inside the test results, we found that this was not always the case.

Jim Hall

•Jun 29, 2014

John, you learnt me nearly everythng I know 'bout compression, and one of those nuggets was the fact that bandwidth and quality go together. So I just assumed that if you decreased the amount of compression you was applying and give back whatever bandwidth savings you had, you'd get more quality, for free! But I'm ready to hear where I got fouled up....:)

Carl, does that mean that less total stream from a camera is available when high profile is chosen?

John Honovich

•Jun 29, 2014

IPVM

In reply to Jim Hall

Bandwidth and quality go together so long as one is talking about the same CODEC, i.e., the same means of compression.

For instance, MJPEG and H.264 are different means of compressing video. This is why H.264 can have the same quality at lower bit rates than MJPEG.

In the same way, H.264 baseline, main and high are different means of compressing video, meaning that it is possible to lower bitrate while keeping the same quality.

I understand this is confusing, especially since there are compression levels as well, for every CODEC, that determine how much video is compressed, regardless of what CODEC one chooses. Video Quality / Compression Tutorial

Rukmini Wilson

•Jun 29, 2014

In reply to John Honovich

In the same way, H.264 baseline, main and high are different means of compressing video, meaning that it is possible to lower bitrate while keeping the same quality.

It sounds like Jim doesn't mean affect quality directly as a function of changing the profile, like bandwidth, but instead change it indirectly in two steps:

Change from base profile to high profile - bandwidth drops by x - quality constant
Lower the compression amount by whatever means available until bandwidth increases by x - quality increases by y

Net/Net: Quality higher by y, bandwidth the same

Rick Puetter

Hi All,

I have first hand experience with working with the various CODECs in H.264 and H.265 since my company was developing new algorithms for both methods and I lead the compression team. Now the idea that compression increases and performance increases as you move from baseline to the high profiles is simply wrong. Think about it, if the quality goes up and efficiency stays the same, then you are transmitting more information and the bandwidth must go up. Unfortunately, in practice, this is not such a simple trade since the efficiency of the CODEC does go up when you move from baseline to the high profile. The high profile is a very much more complicated algorithm with lots of additional compression tricks. It takes lots more compute power (and I mean a lot) to compress with the high profile. Decoding, however, is about the same. The hard part is figuring out how to do the compression. Once you write down what was done, however, it is easy to follow the decompression steps.

So this is complicated and it is very much scene dependent. That's why you see both gains and losses in the comparisons--there are other reasons, too. For some scenes the additional "compression tricks" in the high profile are not too helpful. Also, in all of the profiles there is a "q" parameter that controls the quality. If the scene is compressed with the high profile and a lower q than the baseline profile with a higher q, then results could be similar, i.e., similar quality and bandwidth. And efficiency of the various profiles depend upon q, i.e., quality. At high quality, the high profile will be more efficient (lower bandwidth), since while it is more complicated to convey all the tricks one is using, in order to compete the less efficient baseline profile has to send so many more of the less complicated sets of instructions that it loses. At low quality, however, since you don't need to be so efficient, the larger sets of instructions in the high profile loses in bandwidth. It is no wonder, then, that the high profile is always used for movie and cinema-quality videos. This is not generally the case, however, with surveillance cameras when we're not so concerned with a few compression artifacts here and there and the arty nature of the video.

So these tests are very important. We just don't know what tradeoffs the camera manufacturers have made and we cannot predict performance without measuring it. There simply is not simple rule-of-thumb.

I hope this is helpful.

Rick

Rukmini Wilson

Good info!

...the idea that compression increases and performance increases as you move from baseline to the high profiles is simply wrong. Think about it, if the quality goes up and efficiency stays the same, then you are transmitting more information and the bandwidth must go up.

I think everyone agrees with that, but since as you say

...the efficiency of the CODEC does go up when you move from baseline to the high profile...

then that implies the bandwidth will go down by changing the profile to high if everything else remains the same, like John indicated, yes? If your answer is yes, then if you were then to bump down the q value one number at a time, that would increase the bandwidth, yes? And if you stopped right when the bandwidth was the same as it was before you switched profiles, then the quality would be higher (from a lower q) and the bandwidth the same, yes?

Using your equation: if the efficiency goes up and bandwidth stays the same, then the quality must go up.

Rick Puetter

Well not exactly. I should have been perhaps a bit more careful in my reply, but you will note that I also said the efficiency of the high profile goes up when the image quality is high and it goes down when the requested image quality is low. That is the tradeoff you always have to make when you have a more complex and descriptive language such as is used in the high profile compared to a much simpler language used in the baseline profile. The issue is when does the greater descriptive power of the high profile with a higher entropy per character exceed the performance of more characters in the baseline profile where each character has less entropy. There is a cross-over point someplace (which is scene dependent). Generally, however, if you demand high quality imagery, then the high profile wins. If the requested quality is low, then the baseline profile wins.

I hope that clarifies things a bit.

Rick

Rukmini Wilson

Yes, it does.

Is it fair to say, to Jim's original point, in those cases where efficiency is increased by switching profiles, can you then trade that bandwidth savings for a more quality?

Rick Puetter

Yes, you can increase quality until the bandwidth gets up to your tolerance limit.

Ethan Ace

Hi Rick, that is helpful and thorough. We did standardized Q levels at about 28 regardless of what profile we were using, so we tried to eliminate that variable.

I do agree though when you say we don't know what tradeoffs manufacturers have made. That much was immediately obvious in our tests!

Sergey Bystrov

•Jun 30, 2014

NetworkOptix

In fact profile is not what most people think it's.
Encoder developer should tend(at least form technical point) to keep his encoder profile as low as possible, while implementing cool algorithms.

Back in the day(not sure about now) it was important to kee your encoder in basic profile in order for all devices to understand your stream.

You encoder can be very pure and do not use even 10% out of features of basic profile, but if you use at least one feature( even a trivial one, which does not save bandwidth ) out of hight profile - you cannot claim you encoder is baseline profile anymore and you cannot count on the fact that it will be decoded by any main profile decoder. You encoder is now hight profile.

Bandwidth much more depends on:
1) was 1/4 pixel accuracy was used for motion prediction
2) CABAC vs CAVLC ?

and such things .. rather than from profile itself.

Vance Kozik

This shows how complicated H.264 is. Two subject matter experts weighed in and the converstation went silent. I agree that it's not simply a matter of which profile but more which specific features and methods are being used to encode the stream.

Rick Puetter

In reply to Vance Kozik

Hi Vance,

Yes, it is complicated. And it is important to remember that a particular H.264 profile is a standard for decompression. It says what capabilities need to be available in the decoder. It says nothing about the encoder. So there can be encoders and then encoders. Some can be very much better than the next one. So who knows what are in those encoders used by the camera manufacturers and what trades were made. The high profile can be very computationally intensive and one doesn't normally think of such a daunting task with the compute power aboard a camera. Still, I haven't kept up with what sort of chip sets there are out there today. Even so, the encoder properties are rather open-ended. That's why these practical tests are needed. One can't really predict performance just by saying you're using a particular H.264 profile.

Rick

Vance Kozik

I agree. The practical tests are definitely required -- they're the only way to see how much or little the bandwidth is affected.

Rukmini Wilson

Has there been any thought to dynamically (or systemically) change profiles on the fly, say based on a calculation done at the camera every 5 minutes or so, where it would do a comparison consisting of several frames using both CODECs and then switch over at some threshold?

From what you're saying, night scenes could respond far better to one than the other, no?

Carl Lindgren

Rukmini,

On the fly? I haven't seen an encoder that can change codecs without rebooting, a process that typically takes a substantial amount of time. I can't imagine any application where the loss of video for many seconds due to an encoder reboot would be a benefit.

Rukmini Wilson

On the fly? I haven't seen an encoder that can change codecs without rebooting...

Obviously you are not a CSI fan; I'm sure their encoders can do it... :)

But even in the real-world many encoders do even a more impossible thing, stream two different codecs (H.264/MJPEG) at the same time!

But maybe you meant "change H.264 profile", not codec per se. I can't test any Indigovision product, but I switched a P3367 from baseline to main with no reboot and maybe a 2 second hiccup. With about the same disruption as a day/night switchover might cause. Which coincidentally might be a good time to change it systematically...

But even if it did take 30 seconds, surely you can imagine what a SOHO customer (non-bank/casino/military/hospital) being presented with the tradeoff of "losing video for 30 seconds a day" for "being able to go back 14 days instead of 10" might choose, no?

Finally, even if it was impossible to do, I was merely blue-skying, which should have been apparent by my timid 'Has there been any thought'.

Carl Lindgren

Profile, bit rate, GOP changes, whatever...

I've seen encoders make changes in a few seconds and I've seen encoders take minutes. You can't predict what changes will be quick vs. what take a long time, at least not from my experience. And I've never seen codec-based changes accomplished as quickly as typical day/night changeovers - which typically take less than a second.

I sure wouldn't want to have to explain to a customer why their camera didn't capture an incident because it was in the process of switching from baseline to high profile.

Rukmini Wilson

Ok, you're right, it just can't be done and that's that.

Profiles changing 'on the fly', ha! See what too much CSI can do to one's brain? :)

Carl Lindgren

I didn't mean it couldn't be done, I just meant there was little upside and substantial downside to the idea.

On another note, encoder CPU capability is one of the largest impediments to better codec performance in my opinion. I see that even with baseline profile h.264. It often shows up in the inability to supply multiple high frame rate streams. Axis cameras, in my tests, could not provide two simultaneous 30fps streams.

This will affect h.265 even more. Although the common claim is its supposed ability to provide equivalent picture quality to h.264 at half the bitrate, I believe that will require many generations of encoder chip improvements before we are likely to see any bitrate gains. At that point, I also have to wonder if the bitrate savings alone would be worth the additional horsepower requirements. Perhaps it will be better to add in-camera features like more complex analytics. Also, when storage and networking capabilities increase to the point that concerns about capacity become less important, bitrate savings become less desirable.

Rukmini Wilson

I just meant there was little upside and substantial downside to the idea.

So what's your price, then? i.e. what constitutes a big enough upside to pursue it? 20% bandwidth savings? 50%?

Also, when storage and networking capabilities increase to the point that concerns about capacity become less important, bitrate savings become less desirable.

Agreed, as we spoke before, one wonders how important h.2.xx etc. will be in 5 or 10 years...

Carl Lindgren

For my purposes, the price will always be too high because even less than a second of video loss could have severe consequences. For other applications, a "risk versus reward" analysis would need to be performed.

Carl Lindgren

As an example: the Integrators of our old system severely miscalculated storage requirements based on a faulty calculator from the manufacturer (or so they said). The net result was that the 96-channel initial installation only retained video for 3-4 days versus the 7 days our RFP called for.

We came really close to cancelling the entire project when the Integrator attempted to call the additional storage needed an extra cost item and wanted to bill us separately for it.

<edit> The point I'm trying to make is that before we reached an agreement on the storage, the Integrator attempted all sorts of tricks trying to get storage requirements down, including using motion to trigger changes to the codec (frame rates, resolutions, etc.). They finally gave up and proposed a compromise because everything they tried caused severe "glitches" in the streams and loss of video.

In the end, we wound up with enough storage to provide at least 7 days to each existing camera plus a few spare inputs but we agreed to pay for additional storage to satisfy future camera additions. They provided encoders and servers for 896 inputs while only giving us storage for approximately 700 inputs.

Rukmini Wilson

We came really close to cancelling the entire project when the Integrator attempted to call the additional storage needed an extra cost item and wanted to bill us separately for it.

Don't tell me, I bet you split the cost down the middle? :)

Carl Lindgren