Surveillance Codec Guide

By IPVM Team, Published Jan 03, 2019, 12:52pm EST

Codecs are core to surveillance, with names like H.264, H.265, and MJPEG commonly cited. How do they work? Why should you use them? What issues may you face? In this tutorial, we examine this in-depth covering:

  • Uncompressed vs. Compressed Video
  • Inter vs. Intra Frame Compression
  • I vs P Frames
  • Other Frame Types
  • H.264 vs H.265 vs MJPEG
  • H.265 Emergence
  • Smart Codec Growth
  • Proprietary Codecs
  • Scalable Codecs - JPEG2000, SVC
  • Quality of Codecs
  • Codec Support in Surveillance
  • Future Codecs
  • What Codecs to Choose?

Uncompressed **. ********** *****

*********** *** ************ ***** is **********, ** ******* and ******* ******* ***** easily ** **** ******* for ************.

**** ***** ** *********, it ** ********* ************. There *** * **** factors ** *** **** of ************ *****:

  • *** ***** ** ****** supported *** **** *****
  • *** ***** ****** ** pixels *** *****
  • *** ***** ****** ** frames *** ******

** **** *** ***** size, *** ******** **** of *****. ***'* **** through **** ** *****.

***** ******

**** ***** ** ***** a ***** *********** ** a ****** ****** * range. *** ***** ********** how ********* *** ***** can ** ******* *** also ******* ******* *** bandwidth/size.

  • **** *********, ***** ********** has * ***** ** 256 ****** (* ****) with * ************ *****, 255 ************ ***** *** the ******* ** ******* representing ****** ** ****.
  • ** ******, ****** *** surveillance ***** ***** ******** color ** *** ***** of ****** ****** ** represent *** *** ****** is *** *******. ** bits ** **,*** ****** is ******.

**********/********* ******

*** ***** *** ******* are *** ****** ** understand ** **** *** the********** ** *** *** camera, *********** ********** ***** vertical ***** ******:

IPVM Image

** **** ******** *** ******, ****** ****, *****, 30fps, ***. *** **** majority ** ************ ******* (>80% ** *** *******) are ******** ** ******* 5 *** ** *** (see******* ***** **** ***** Surveillance ****).

Adding ** **

**** ** ******** **** is *********** *** ******* uncompressed ***** *** ******. Take * ***** ***** camera ** *****. ************ it ** *** ******* of *** *********:

*********** ***** ***** ******* results ** ~* **/* for ************ *****/***** *****. In * ***, ** that ****, *** ***** have ~***** ** ******* from * ****** ******. While **** ****** ******** to *** ******, * 16 ****** ************ ************ system ******* *** ** days, ***** **** ****** 6PBs ** ******* ****** a *** ******* **** drives **** ***** **** hundreds ** ********* ** dollars.

Codecs *** ***

******* *** **** **** of ************ *****, ********** surveillance ******* ** ** cameras ********* *** ** standard *** ** **** small **********. *** **** this ******? ******.

***** ****** *** *********** / ************* *** *** act ** *********** ** the ******* ******* ** reducing ********* / ******* consumption.

****** ***** ***** ** every ***** *** *** unique ***** ** ********** wasteful ** **** ****** are ****** **** * small ****** ** ******* colors. ****** ******** ***** by ******** *** ****** of ****** ******** ***** tracking ***** ****** **** the **** ** ******* values, ******** ** ** transmit *** ***** ******* of ****.

Inter **. ***** ***** ***********

***** *** *** *********** approaches ** ***********: *****-***** and *****-*****. ** ** critical ** ********** *** distinctions ******* *** *** as **** ****** ********* consumption, ********** ***** ************, and ******* *****.

  • *****-***** *********** ** ****** a ****** ***** **** but *** ****** ****** (example - *****)
  • *****-***** *********** ** ****** multiple ****** *** ****** single ****** (******** - H.264, *.***, ****-*)

*** ****** ******* *****-***** compression *** **** **** some ******* **** ***** and ***** ***** ***********.

Intra-frame ***********

*****-***** *********** **** ***** at *** ***** ** a ****, ***** *** best ** ******** **** is ** **** *****. Even ****** ***** ** a ****** ** ******, intra-frame *********** **** **** a ***** ** * time, ******** *** "******." An *****-***** ***** ******'* individual ****** **** **** the ****** ** * movie:

IPVM Image

*** ****** ** **** this ** ****** ** do *************** *** **** significantly ******** *****. *** instance, * *****/***** ***** stream ***** ** *****-***** compression **** ***** *** have *** ******* ******* from ~**** **/* ** ~40Mb/s. *******, *** ******** is **** **** **** savings *** ********* ** you ******* ******* ******.

Inter-frame ***********

***** *****-***** ***********, *** only ** *** ***** within *** ***** *******, the ***** ******** ******** frames ** ******* ******** the *****. **** ** feasible ******* ***** **** little ******* **** *** frame ** *** ****. For *******, ***** *** same ***** ** * person ******, *****-***** *********** would **** **** *** subject's ***.

IPVM Image

**** ** **** ** a ***** ***** ********* static, ******* **** ******* in *** ***** ***** substantial *********/*******. *** ********, the **** *****/***** ****** that ***** **** ****/* with *****, ** *****-***** only *****, *** **** need ***/* **** *.***, a ***** **** **** both.

*******, *** **** ******** to *****-***** *********** ** that ** ** *** more *************** *********, ***** can ******** *********** *** quality ***** (******** *****).

I ** * ******

***** *** *** *** frame ***** ** *****-***** compression **** ** *.***/*.***.

*-******

*** ***** ***** ** a ***** ***** ** pictures ** ****** ** I-frame, ***** *** *****-*****, and ** *********** * full ***** ** *****, as ******* ** **** changes ***** ** * frames. *** ******** ******* two *-****** ** ******** to ** *-***** ********, GOV (***** ** *****), or *** (***** ** pictures).

** ** *******, **** image ***** *** *-***** of ** ******* ***** used ** *** ******* (click *** **** ****):

IPVM Image

*-******

*-****** ********* *** **** image ** *** ******** I-frame ** **** **** the ******* ** *** scene. ******* *** ** small, **** ** ******* noise ** ***** ******* movement, ** *****, **** as * *** ****** moving **** ****** ** preset. *** * ** P-frame ****** *** "**********."

*** ******* **** ** a *-***** **** *** test ***** *** ** seen ** *** ***** below (***** *** **** size). **** *** ***** near *** ****, ***** cars ******, *** ****** foliage ** *** *****, are **** ** *******.

IPVM Image

******* ***** ******** ** the **** *****, *** P-frame ******* *** ** seen "********" **** **** other, ***** *** **** I-frame ** **** (~** seconds).

Other ***** *****

**** **** ***** *** other ***** ***** ** addition ** * *** P, **** ** *, SI, *** **, ****** they *** ********* ****** in ************. **** ** cameras ******* *-***** *******, but *** *** ***** are ******* ** ******** decoding ****, ** **** are ********* *** ****.

Standard ******

** ** ****, **** new *********** ***** *** H.264. ***** *.*** ******* are *** ****** *** VMS *********** ** *********, increased *** **** **** using *.*** *** ******* bandwidth ***** **** **** H.264 ** *** **** common ****** (*** ***** for **** *******).

***** ** ***** **** in **** ***********, *** mainly **** ** ********** applications **** ** *** or ***** *********. *******, manufacturers **** ***** ** limit ******* *** *****, with**** ****** *** ********* an ***** ******.

H.265 ****** *** ***** *******

*** *****,*.*** *** **** *** next *** *****, ********* ** ******* H.264 *** ****** ******** by ******* ~**%. *******, there *** *** ******** preventing *** ********** **************:

  • ******* *****:***** *** ****** **** MJPEG ** *.*** ******** in ******* ******* **********, often **-**% ** ****, our ***** ** *.*** show **** **-**% ******* over *.*** ** **** likely. ***** **** **** and *** ***** ******** in ****/******** ** **** of **** **** ******, the ******** ** *.*** are *** ** ********** as *** ***********.
  • ********* *** ****:** *** *****, ******* H.265 ******* ******** ************* more ********* **** **** equivalent ***/********** *.*** *******, in **** ***** ******, which ***** ******* *** viewing ************ ** **** installations.
  • *********** ********:******, ***** **** ********* H.265 ********, * ***-******* development *******, ****** **** limited ******* *** ** the ***** ****** ** current *.*** ****** *** limited *********/******* *******.
  • ******* ***** ***********:*******, ****** ** *.*** cameras *** ***** ****** up **** ***** *********** until *** ********** ** their ****** *******,******* *, ***** *** ******** in ** **** **** limited ********** ********. ****** manufacturers **** **** ** implement *.*** ***** *****'* 2.4 ****, *** ***** was ** *********** **** until **** ******* *** complete. ******* * ******* have ***** ******* ** 2019/2020/2021, *** *** *********** is ***** ******* ** a ******* ** *********.

** ****** *.*** ** continue ** ******* ** it *******, *** ***** reasons ******** ***** **** it ********.

*** *** ***** ***.*** ** ******* ****** vs *.*********** *.*** ******* **** *******.

Proprietary ******

************ *** *** * handful ** *********** ******, but **** *** ** these *** ** *** in ****. ** ***** analog *******, ***** *** encoded, ****** *** ******* in *** **** ********* (i.e., * ***), ****** it ****** ** *** a *********** ***** ** the ************ ********** *** entire *** ** *** process.

*******, ** ******* ****** video ** ********** *** then **** ** ******** this ******* ***** ** a ********/*** *** ******* and ******** **. *********** codecs ******** *** ********** of *******, ********, *** displaying ***** ** **** must **** ** ********** to *** ***. *** high ***** ** ***** this *** ******** *********** codecs **** ********* **** IP ****** ************ ******.

Mobotix ***** *********

*******'******** *** **** ****** known *********** ***** **** in ***** ************. ********** in ****, ***** ******** inter-frame ***********, ********* **** MJPEG *** ******** ********* consumption ******** ** ** (*** *** *****). *******, ***** ** is ***********, **** *** 3rd ***** ***** ******* it, ******* **** ** either *** *******'* *** VMS ** ** *** Mobotix ******* ** *****. Furthermore, *.***'* ********* *********** is ********* **** ***** than *** ********** ***** stream, ******* ******** *** attractiveness.

Smart ******

** *** **** *** years, *.*** *** *.*** smart ****** **** ****** common, ****** ** ******* reduce ******** ******** ** standard *.***/*. ******* *** these ***** ****** ******** varies, *** ********* **** use *** **********, ******** below. ******* ****** *** our***** ***** ******** **** ******* ** these ************.

Smart ***********

******* ** ******** *** same *********** ***** ** the ****** *****, ***** codecs *********** ****** *********** for ******** ** *** camera's ***. *** *******, looking ** *** ***** below, *********** ***** ** set ** "***" *** the ******** ****** ** keep ******* ****, *** the ***** ********** *** be *** ** "****", since ** ** *** need ******* ** *** white ****.

IPVM Image

Dynamic *-***** ********

******, ***** ****** ********* dynamically ****** *-***** ******** based ** ******** ** the *****. ** ** a ***** *** ****** or ** ******, *** camera ***** *-****** ************ (5, **, ** ******* or ****), *** **** activity ** ******** ** the *****, ** *********** sends ** *-***** *** switches **** ** ****** (typically * ******) *-***** intervals ** **** ** activity *********.

**** ***** **** * stream ******** ***** *** effects ** ******* *-***** interval:

IPVM Image

Smart ***** ********

** *** *****, ***** codecs **** ******* ******** significantly, ~**% ** * minimum, *** ** ** 95%+ ** **** ******. These ****** *** **** effective ** ***** ******, as *-***** ********* ****** longer *** *********** ****** due ** *** **** of ********. *** ******** ***** ******** **** *******.

H.264 **. ***** *******

***** *.*** ** ******* the **** ****** **** codec ** ************, *** many ***** * ****** debate ******* ***** ******* loss ******** ** *****, and **** ***** ** believe *.*** ** ********, even *****. ************ *.*** ** ***** test***** ****, **** ******** configured, *.*** ******** *** same ******* ******* ** MJPEG.

*******, ***** *** **** issues **** *** ********* quality, *********:

  • ******* **** *********** ******:** ************** ********* *** *** ****, the ***** **** **** quality ***********. *** ***** ****** ************ *********** Comparison*** **** ******* *** how ******** ************* ****** compression.
  • **** ********** ******:**** ****** ***** *** visible ** ******* ******, where ***** ** **** of ****** - *************, crowds, ***.
  • ******* *** ******** *** rates:******* **** *** *** bit **** ** *** too ***, *** ***** will **** ******* ***********.

*******, ** **** *****, even **** ***** ******* settings, *.*** *** ********* quality ******* ** ***** in *** *****.

Scalable ******

**** ****** *** **** support * ****** **********. For ********, ******** **** a *** ****** ** a *** *** ***** MJPEG ** *.***, ******** either ******** * ********** new ****** ** *********** (i.e. **-**********) *** *** stream ** **** ** 1MP. *******, ** **** cases *** **** ** change *** ****** ********** without ********* * *** stream, *** ********, ** you *** ******* ** a ****** **** * lower ********* ********** (*.*., mobile) ** **** ** reduce ******* **** ** older *****.

* ******* ***** ** codecs, ****** '********', *** do **** *************, ******* having ** ******* * new ****** ** **-******* an ******** ******. * scalable ***** *** *********** 'pick' *** ****** ** resolution ****** **** **** out ** * ******. This ****** ******* ****** or ********** **** **** as **** ** *********** adjusting ********** / *** for ****** / ****** clients.

***** *** *** **** known ******** ******:

  • *** ** *.*** **** scalability. *** **** **** is **** ** ******** scalability **** *** ********* benefits ** *.***. *************, very *** ************* ******* this. **** ********** **** through *****-********* (******* ******** streams ************** ** ********* resolution/frame *****).
  • ********, ***** ** *********** MJPEG, **** *** ******** of ***********. **** ** the ***** **** ******** traditionally **** *** *** phased *** ** *** cameras. *** **** ******** of ********, **** *****, is ******* *********/******* ******** compared ** *.***, **** partially ****** ** *******.

*** *********** *** **** very ****, **** ****** actual ******** ******* ************ in **** ******* (** least ** *****) *** several *****. **** *** bitrate ********** ** *.*** smart ****** (***** ******* little ** ** *** VMS ***********) *** *.*** taking **********, *** ** unlikely ** ****** * major ****** ** ************ codecs.

Future ******

***** **** *********** ****** are ********* ** *******, it ** ****** ******** that *** *** ************ ones **** **** **** adoption ** ************.

*** ********, **** ********** claim ******* ********* ******* from ***** *********** ******, such ********* ******** *** / EdgeVis. *******, **** ***** require **** ****** ************* and *** ********** ** implement ***** ****** ** their ********. ************, ***** the *********** ******* ********** of ***** ******, ***** are ********** **** *** majority ** ******* *.*** recorders/VMS, *** ********** ** development ** *** ***** proprietary ****** ** **** more ********.

**** **** ********* ******'**** *** ********* ** ************* ** surveillance, ********** ** * royalty-free *********** ** *.*** (which *** ************ *** ******* ****). *******, ***** ****** saw ****** ** ** interest **** *************, **** no ************ ******** ****** to ***** ****, *** have **** ****** ** traction.

What ****** ** ******?

**** ** ****, *** best ***** *********** *** most *** ***** ** H.264 **** ***** ***** support ** ***** ****** significantly ******** ********** ** H.264 **** ******* *********.

*.*** ** *************, *** in **** ***** **** be ***** *** *** to *** ******* ***********. However, ** ** ******* stronger *** **** ***** used ** ***-**-*** ******/*** systems **** * ****** manufacturer (*.*., *****, *********, etc.) ***** ************* ****** are *********.

***** ******* * ***** for ********** ************ *** those *** **** (******* unreasonably) **** **** *****-***** compression.

[****: **** *** ********** published ** ****, *** substantially **-******* ** ******* technology ******** ** *.*** and ***** ******.]

Comments (20)

Very good guide!

10 points!

Agree: 5
Disagree
Informative: 2
Unhelpful
Funny

Great Update!

One question about:

While the change from MJPEG to H.264 resulted in drastic bitrate reductions, often 50-75%, our tests of H.265 show that 15-30% savings over H.264 is more likely.

Was the change usually from MJPEG to h.264 directly, or was there often a baby step of MPEG-4 (Part 2) involved? If so, how big was the change from MPEG-4 to h.264?

Agree
Disagree
Informative
Unhelpful
Funny

Was the change usually from MJPEG to h.264 directly, or was there often a baby step of MPEG-4 (Part 2) involved?

For MP, the move was straight from MJPEG to H.264. See 2008 article - H.264 makes Megapixel go Mainstream.

Agree
Disagree
Informative
Unhelpful
Funny

how big was the change from MPEG-4 to h.264?

How big the change depends on actual codec implementations and scenes. So it's always a challenging question. In the entertainment industry, assuming very good implementation of codecs in both cases ~40%. Practically it always depends...

 

However, H.264 has 2 very sufficient differences vs MPEG4(part2):


1) In H.264 DCT( discrete cosine transform) is done in integer values. 
In MPEG4(part2)(as well as in MPEG1,MPEG2,H.261,H.263) the DCT was done in float values, and standard did not dictate how accurate should it be. So while encoders used one accuracy(due to the hardware of software specifics), decoders doing inverse DCT used another formula (accuracy), which led to the error accumulation with every next P frame. So, by the end of the GOP(with new I frame) it was sort of re-synchronization. Visually it led to image quality jump with every new I frame(typically every second). You might remember this visual effect of MPEG4, that image changes significantly every second. Not the case with H.264. 

This is the reason why GOP size was very limited in MPEG4(image degradation accumulation). It's not the case in H.264. Practically H.264 GOP size could be very huge( practically unlimited). The only limitation comes from editing software( in order to decode frame N inside the GOP, decoder have to decode all previous frames).

So, H.264 made a huge step forward eliminating GOP size issue and image degradation towards the end of the GOP. 

 

2) deblocking filter became part of the standard. So they have to be universally implemented across all H.264 codecs. Wich also made H.264 stream look the same on any device. 

 

So, H.264 is not only bandwidth saver but solves some fundamental issues of previous standards.  

 

Agree: 1
Disagree
Informative: 2
Unhelpful
Funny

Excellent article, the inclusion of all the smart codec data is very useful.

Agree: 2
Disagree: 1
Informative
Unhelpful
Funny

Excellent article. It will add high value add for users. Grate Job.

Agree: 1
Disagree: 1
Informative
Unhelpful
Funny: 1

Sounds like there is always room for improvement on H. 264, but it has proven to be the foundation in which to build on. The H. 265 sounds good at the drawing board, but when put into practice, it has its "ghosts" to be worked out.

Agree
Disagree
Informative
Unhelpful
Funny

I have a better understanding of compression

Agree
Disagree
Informative
Unhelpful
Funny

Thanks for taking the mystery out of compression!

Agree
Disagree
Informative
Unhelpful
Funny

Good article. 

Few points:

P-frames reference the full image of the previous I-frame to send only the changes in the scene. 

Just in case: P frame reference the full image of any I or P previous frames.  Advanced codecs can refer to multiple frames back minimizing the difference(part of the image could be taken from one reference frame, another part form another). Practically P frame most of the time refers previous P frame.  


 

MISC: as a paradox, typically in the surveillance industry(due to static scenes)  H.264 takes less CPU to decode than MJPEG. (since most of the time most of the image parts difference with prev frame is about zero). 

 

H.265 is designed in such a way that besides potential bandwidth savings it's better parallelized for multiple CPU cores to encode/decode. 

Agree
Disagree
Informative: 3
Unhelpful
Funny

Thank you for the excellent summary and outline to explain CODECS. Well done!

Agree
Disagree
Informative
Unhelpful
Funny

With "Inter Framing" if I set my HD to record on motion is there further compression or are they separate?

Agree
Disagree
Informative
Unhelpful
Funny

Motion based recording does not 'further compress' video, it simply determines when to record video or not at all.

Agree
Disagree
Informative
Unhelpful
Funny

I would like to know more about SVC.I am completely ignorant on the topic.Can it be used also to obtain rtsp streams or does it work only with a server/vms that supports the function?

Agree
Disagree
Informative
Unhelpful
Funny

Luca, we know no video surveillance manufacturer that offers a scalable video codec today. It's effectively not used.

Today, it's almost always H.264, H.265 combined with smart codecs, no svc.

Agree
Disagree
Informative
Unhelpful
Funny

Actually UNV allows to enable SVC in the video page.Checking around IPVM i've seen also tests you did with Vivotek

 

Agree
Disagree
Informative
Unhelpful
Funny

My mistake to say none are 'offered'. 

My point is none are used. For example, we have never heard Uniview nor Vivotek recommend using SVC nor any VMSes do so for those cameras.

We are not against SVC, there is just no sign that anyone on the manufacturing side is promoting or recommending using it.

Agree
Disagree
Informative
Unhelpful
Funny

My point is none are used.

Hikvision, Dahua and Vivotek all have a SVC-T setting exposed, that works with at least their own recorders. Of course, this is actually enabled because of the underlying SOC supporting it, as you once mentioned: 

Does Hikvision SVC Really Work?  

On the other hand, neither you nor I consider this a full implementation of SVC, because it is only frame-rate reduction based, temporal -T.  The h.26x SVC we were all waiting for, one with spatial reduction -S, (ala JPEG2000) never arrived: 

SVC - A Better H.264 Coming For Video Surveillance

 

Agree
Disagree
Informative
Unhelpful
Funny

Excellent article. It was good to see and understand how they test to determine what makes one better then the other. Great Job!!

Agree
Disagree
Informative
Unhelpful
Funny

H.264 remains the standard in video surveillance codecs notwithstanding H.265 and other codec introductions such as Google VP8 and VP9. Codecs play a vital function in achieving efficiency when it comes to bandwidth utilization.

Agree
Disagree
Informative
Unhelpful
Funny
Read this IPVM report for free.

This article is part of IPVM's 7,211 reports and 960 tests and is only available to subscribers. To get a one-time preview of our work, enter your work email to access the full article.

Already a subscriber? Login here | Join now
Loading Related Reports