Surveillance Codec Guide

By IPVM Team, Published Jan 03, 2019, 12:52pm EST

Codecs are core to surveillance, with names like H.264, H.265, and MJPEG commonly cited. How do they work? Why should you use them? What issues may you face? In this tutorial, we examine this in-depth covering:

  • Uncompressed vs. Compressed Video
  • Inter vs. Intra Frame Compression
  • I vs P Frames
  • Other Frame Types
  • H.264 vs H.265 vs MJPEG
  • H.265 Emergence
  • Smart Codec Growth
  • Proprietary Codecs
  • Scalable Codecs - JPEG2000, SVC
  • Quality of Codecs
  • Codec Support in Surveillance
  • Future Codecs
  • What Codecs to Choose?

Uncompressed **. ********** *****

*********** *** ************ ***** is **********, ** ******* and ******* ******* ***** easily ** **** ******* for ************.

**** ***** ** *********, it ** ********* ************. There *** * **** factors ** *** **** of ************ *****:

  • *** ***** ** ****** supported *** **** *****
  • *** ***** ****** ** pixels *** *****
  • *** ***** ****** ** frames *** ******

** **** *** ***** size, *** ******** **** of *****. ***'* **** through **** ** *****.

***** ******

**** ***** ** ***** a ***** *********** ** a ****** ****** * range. *** ***** ********** how ********* *** ***** can ** ******* *** also ******* ******* *** bandwidth/size.

  • **** *********, ***** ********** has * ***** ** 256 ****** (* ****) with * ************ *****, 255 ************ ***** *** the ******* ** ******* representing ****** ** ****.
  • ** ******, ****** *** surveillance ***** ***** ******** color ** *** ***** of ****** ****** ** represent *** *** ****** is *** *******. ** bits ** **,*** ****** is ******.

**********/********* ******

*** ***** *** ******* are *** ****** ** understand ** **** *** the********** ** *** *** camera, *********** ********** ***** vertical ***** ******:

IPVM Image

** **** ******** *** ******, ****** ****, *****, 30fps, ***. *** **** majority ** ************ ******* (>80% ** *** *******) are ******** ** ******* 5 *** ** *** (see******* ***** **** ***** Surveillance ****).

Adding ** **

**** ** ******** **** is *********** *** ******* uncompressed ***** *** ******. Take * ***** ***** camera ** *****. ************ it ** *** ******* of *** *********:

*********** ***** ***** ******* results ** ~* **/* for ************ *****/***** *****. In * ***, ** that ****, *** ***** have ~***** ** ******* from * ****** ******. While **** ****** ******** to *** ******, * 16 ****** ************ ************ system ******* *** ** days, ***** **** ****** 6PBs ** ******* ****** a *** ******* **** drives **** ***** **** hundreds ** ********* ** dollars.

Codecs *** ***

******* *** **** **** of ************ *****, ********** surveillance ******* ** ** cameras ********* *** ** standard *** ** **** small **********. *** **** this ******? ******.

***** ****** *** *********** / ************* *** *** act ** *********** ** the ******* ******* ** reducing ********* / ******* consumption.

****** ***** ***** ** every ***** *** *** unique ***** ** ********** wasteful ** **** ****** are ****** **** * small ****** ** ******* colors. ****** ******** ***** by ******** *** ****** of ****** ******** ***** tracking ***** ****** **** the **** ** ******* values, ******** ** ** transmit *** ***** ******* of ****.

Inter **. ***** ***** ***********

***** *** *** *********** approaches ** ***********: *****-***** and *****-*****. ** ** critical ** ********** *** distinctions ******* *** *** as **** ****** ********* consumption, ********** ***** ************, and ******* *****.

  • *****-***** *********** ** ****** a ****** ***** **** but *** ****** ****** (example - *****)
  • *****-***** *********** ** ****** multiple ****** *** ****** single ****** (******** - H.264, *.***, ****-*)

*** ****** ******* *****-***** compression *** **** **** some ******* **** ***** and ***** ***** ***********.

Intra-frame ***********

*****-***** *********** **** ***** at *** ***** ** a ****, ***** *** best ** ******** **** is ** **** *****. Even ****** ***** ** a ****** ** ******, intra-frame *********** **** **** a ***** ** * time, ******** *** "******." An *****-***** ***** ******'* individual ****** **** **** the ****** ** * movie:

IPVM Image

*** ****** ** **** this ** ****** ** do *************** *** **** significantly ******** *****. *** instance, * *****/***** ***** stream ***** ** *****-***** compression **** ***** *** have *** ******* ******* from ~**** **/* ** ~40Mb/s. *******, *** ******** is **** **** **** savings *** ********* ** you ******* ******* ******.

Inter-frame ***********

***** *****-***** ***********, *** only ** *** ***** within *** ***** *******, the ***** ******** ******** frames ** ******* ******** the *****. **** ** feasible ******* ***** **** little ******* **** *** frame ** *** ****. For *******, ***** *** same ***** ** * person ******, *****-***** *********** would **** **** *** subject's ***.

IPVM Image

**** ** **** ** a ***** ***** ********* static, ******* **** ******* in *** ***** ***** substantial *********/*******. *** ********, the **** *****/***** ****** that ***** **** ****/* with *****, ** *****-***** only *****, *** **** need ***/* **** *.***, a ***** **** **** both.

*******, *** **** ******** to *****-***** *********** ** that ** ** *** more *************** *********, ***** can ******** *********** *** quality ***** (******** *****).

I ** * ******

***** *** *** *** frame ***** ** *****-***** compression **** ** *.***/*.***.

*-******

*** ***** ***** ** a ***** ***** ** pictures ** ****** ** I-frame, ***** *** *****-*****, and ** *********** * full ***** ** *****, as ******* ** **** changes ***** ** * frames. *** ******** ******* two *-****** ** ******** to ** *-***** ********, GOV (***** ** *****), or *** (***** ** pictures).

** ** *******, **** image ***** *** *-***** of ** ******* ***** used ** *** ******* (click *** **** ****):

IPVM Image

*-******

*-****** ********* *** **** image ** *** ******** I-frame ** **** **** the ******* ** *** scene. ******* *** ** small, **** ** ******* noise ** ***** ******* movement, ** *****, **** as * *** ****** moving **** ****** ** preset. *** * ** P-frame ****** *** "**********."

*** ******* **** ** a *-***** **** *** test ***** *** ** seen ** *** ***** below (***** *** **** size). **** *** ***** near *** ****, ***** cars ******, *** ****** foliage ** *** *****, are **** ** *******.

IPVM Image

******* ***** ******** ** the **** *****, *** P-frame ******* *** ** seen "********" **** **** other, ***** *** **** I-frame ** **** (~** seconds).

Other ***** *****

**** **** ***** *** other ***** ***** ** addition ** * *** P, **** ** *, SI, *** **, ****** they *** ********* ****** in ************. **** ** cameras ******* *-***** *******, but *** *** ***** are ******* ** ******** decoding ****, ** **** are ********* *** ****.

Standard ******

** ** ****, **** new *********** ***** *** H.264. ***** *.*** ******* are *** ****** *** VMS *********** ** *********, increased *** **** **** using *.*** *** ******* bandwidth ***** **** **** H.264 ** *** **** common ****** (*** ***** for **** *******).

***** ** ***** **** in **** ***********, *** mainly **** ** ********** applications **** ** *** or ***** *********. *******, manufacturers **** ***** ** limit ******* *** *****, with**** ****** *** ********* an ***** ******.

H.265 ****** *** ***** *******

*** *****,*.*** *** **** *** next *** *****, ********* ** ******* H.264 *** ****** ******** by ******* ~**%. *******, there *** *** ******** preventing *** ********** **************:

  • ******* *****:***** *** ****** **** MJPEG ** *.*** ******** in ******* ******* **********, often **-**% ** ****, our ***** ** *.*** show **** **-**% ******* over *.*** ** **** likely. ***** **** **** and *** ***** ******** in ****/******** ** **** of **** **** ******, the ******** ** *.*** are *** ** ********** as *** ***********.
  • ********* *** ****:** *** *****, ******* H.265 ******* ******** ************* more ********* **** **** equivalent ***/********** *.*** *******, in **** ***** ******, which ***** ******* *** viewing ************ ** **** installations.
  • *********** ********:******, ***** **** ********* H.265 ********, * ***-******* development *******, ****** **** limited ******* *** ** the ***** ****** ** current *.*** ****** *** limited *********/******* *******.
  • ******* ***** ***********:*******, ****** ** *.*** cameras *** ***** ****** up **** ***** *********** until *** ********** ** their ****** *******,******* *, ***** *** ******** in ** **** **** limited ********** ********. ****** manufacturers **** **** ** implement *.*** ***** *****'* 2.4 ****, *** ***** was ** *********** **** until **** ******* *** complete. ******* * ******* have ***** ******* ** 2019/2020/2021, *** *** *********** is ***** ******* ** a ******* ** *********.

** ****** *.*** ** continue ** ******* ** it *******, *** ***** reasons ******** ***** **** it ********.

*** *** ***** ***.*** ** ******* ****** vs *.*********** *.*** ******* **** *******.

Proprietary ******

************ *** *** * handful ** *********** ******, but **** *** ** these *** ** *** in ****. ** ***** analog *******, ***** *** encoded, ****** *** ******* in *** **** ********* (i.e., * ***), ****** it ****** ** *** a *********** ***** ** the ************ ********** *** entire *** ** *** process.

*******, ** ******* ****** video ** ********** *** then **** ** ******** this ******* ***** ** a ********/*** *** ******* and ******** **. *********** codecs ******** *** ********** of *******, ********, *** displaying ***** ** **** must **** ** ********** to *** ***. *** high ***** ** ***** this *** ******** *********** codecs **** ********* **** IP ****** ************ ******.

Mobotix ***** *********

*******'******** *** **** ****** known *********** ***** **** in ***** ************. ********** in ****, ***** ******** inter-frame ***********, ********* **** MJPEG *** ******** ********* consumption ******** ** ** (*** *** *****). *******, ***** ** is ***********, **** *** 3rd ***** ***** ******* it, ******* **** ** either *** *******'* *** VMS ** ** *** Mobotix ******* ** *****. Furthermore, *.***'* ********* *********** is ********* **** ***** than *** ********** ***** stream, ******* ******** *** attractiveness.

Smart ******

** *** **** *** years, *.*** *** *.*** smart ****** **** ****** common, ****** ** ******* reduce ******** ******** ** standard *.***/*. ******* *** these ***** ****** ******** varies, *** ********* **** use *** **********, ******** below. ******* ****** *** our***** ***** ******** **** ******* ** these ************.

Smart ***********

******* ** ******** *** same *********** ***** ** the ****** *****, ***** codecs *********** ****** *********** for ******** ** *** camera's ***. *** *******, looking ** *** ***** below, *********** ***** ** set ** "***" *** the ******** ****** ** keep ******* ****, *** the ***** ********** *** be *** ** "****", since ** ** *** need ******* ** *** white ****.

IPVM Image

Dynamic *-***** ********

******, ***** ****** ********* dynamically ****** *-***** ******** based ** ******** ** the *****. ** ** a ***** *** ****** or ** ******, *** camera ***** *-****** ************ (5, **, ** ******* or ****), *** **** activity ** ******** ** the *****, ** *********** sends ** *-***** *** switches **** ** ****** (typically * ******) *-***** intervals ** **** ** activity *********.

**** ***** **** * stream ******** ***** *** effects ** ******* *-***** interval:

IPVM Image

Smart ***** ********

** *** *****, ***** codecs **** ******* ******** significantly, ~**% ** * minimum, *** ** ** 95%+ ** **** ******. These ****** *** **** effective ** ***** ******, as *-***** ********* ****** longer *** *********** ****** due ** *** **** of ********. *** ******** ***** ******** **** *******.

H.264 **. ***** *******

***** *.*** ** ******* the **** ****** **** codec ** ************, *** many ***** * ****** debate ******* ***** ******* loss ******** ** *****, and **** ***** ** believe *.*** ** ********, even *****. ************ *.*** ** ***** test***** ****, **** ******** configured, *.*** ******** *** same ******* ******* ** MJPEG.

*******, ***** *** **** issues **** *** ********* quality, *********:

  • ******* **** *********** ******:** ************** ********* *** *** ****, the ***** **** **** quality ***********. *** ***** ****** ************ *********** Comparison*** **** ******* *** how ******** ************* ****** compression.
  • **** ********** ******:**** ****** ***** *** visible ** ******* ******, where ***** ** **** of ****** - *************, crowds, ***.
  • ******* *** ******** *** rates:******* **** *** *** bit **** ** *** too ***, *** ***** will **** ******* ***********.

*******, ** **** *****, even **** ***** ******* settings, *.*** *** ********* quality ******* ** ***** in *** *****.

Scalable ******

**** ****** *** **** support * ****** **********. For ********, ******** **** a *** ****** ** a *** *** ***** MJPEG ** *.***, ******** either ******** * ********** new ****** ** *********** (i.e. **-**********) *** *** stream ** **** ** 1MP. *******, ** **** cases *** **** ** change *** ****** ********** without ********* * *** stream, *** ********, ** you *** ******* ** a ****** **** * lower ********* ********** (*.*., mobile) ** **** ** reduce ******* **** ** older *****.

* ******* ***** ** codecs, ****** '********', *** do **** *************, ******* having ** ******* * new ****** ** **-******* an ******** ******. * scalable ***** *** *********** 'pick' *** ****** ** resolution ****** **** **** out ** * ******. This ****** ******* ****** or ********** **** **** as **** ** *********** adjusting ********** / *** for ****** / ****** clients.

***** *** *** **** known ******** ******:

  • *** ** *.*** **** scalability. *** **** **** is **** ** ******** scalability **** *** ********* benefits ** *.***. *************, very *** ************* ******* this. **** ********** **** through *****-********* (******* ******** streams ************** ** ********* resolution/frame *****).
  • ********, ***** ** *********** MJPEG, **** *** ******** of ***********. **** ** the ***** **** ******** traditionally **** *** *** phased *** ** *** cameras. *** **** ******** of ********, **** *****, is ******* *********/******* ******** compared ** *.***, **** partially ****** ** *******.

*** *********** *** **** very ****, **** ****** actual ******** ******* ************ in **** ******* (** least ** *****) *** several *****. **** *** bitrate ********** ** *.*** smart ****** (***** ******* little ** ** *** VMS ***********) *** *.*** taking **********, *** ** unlikely ** ****** * major ****** ** ************ codecs.

Future ******

***** **** *********** ****** are ********* ** *******, it ** ****** ******** that *** *** ************ ones **** **** **** adoption ** ************.

*** ********, **** ********** claim ******* ********* ******* from ***** *********** ******, such ********* ******** *** / EdgeVis. *******, **** ***** require **** ****** ************* and *** ********** ** implement ***** ****** ** their ********. ************, ***** the *********** ******* ********** of ***** ******, ***** are ********** **** *** majority ** ******* *.*** recorders/VMS, *** ********** ** development ** *** ***** proprietary ****** ** **** more ********.

**** **** ********* ******'**** *** ********* ** ************* ** surveillance, ********** ** * royalty-free *********** ** *.*** (which *** ************ *** ******* ****). *******, ***** ****** saw ****** ** ** interest **** *************, **** no ************ ******** ****** to ***** ****, *** have **** ****** ** traction.

What ****** ** ******?

**** ** ****, *** best ***** *********** *** most *** ***** ** H.264 **** ***** ***** support ** ***** ****** significantly ******** ********** ** H.264 **** ******* *********.

*.*** ** *************, *** in **** ***** **** be ***** *** *** to *** ******* ***********. However, ** ** ******* stronger *** **** ***** used ** ***-**-*** ******/*** systems **** * ****** manufacturer (*.*., *****, *********, etc.) ***** ************* ****** are *********.

***** ******* * ***** for ********** ************ *** those *** **** (******* unreasonably) **** **** *****-***** compression.

[****: **** *** ********** published ** ****, *** substantially **-******* ** ******* technology ******** ** *.*** and ***** ******.]

Comments (20)

Very good guide!

10 points!

Great Update!

One question about:

While the change from MJPEG to H.264 resulted in drastic bitrate reductions, often 50-75%, our tests of H.265 show that 15-30% savings over H.264 is more likely.

Was the change usually from MJPEG to h.264 directly, or was there often a baby step of MPEG-4 (Part 2) involved? If so, how big was the change from MPEG-4 to h.264?

Was the change usually from MJPEG to h.264 directly, or was there often a baby step of MPEG-4 (Part 2) involved?

For MP, the move was straight from MJPEG to H.264. See 2008 article - H.264 makes Megapixel go Mainstream.

how big was the change from MPEG-4 to h.264?

How big the change depends on actual codec implementations and scenes. So it's always a challenging question. In the entertainment industry, assuming very good implementation of codecs in both cases ~40%. Practically it always depends...

 

However, H.264 has 2 very sufficient differences vs MPEG4(part2):


1) In H.264 DCT( discrete cosine transform) is done in integer values. 
In MPEG4(part2)(as well as in MPEG1,MPEG2,H.261,H.263) the DCT was done in float values, and standard did not dictate how accurate should it be. So while encoders used one accuracy(due to the hardware of software specifics), decoders doing inverse DCT used another formula (accuracy), which led to the error accumulation with every next P frame. So, by the end of the GOP(with new I frame) it was sort of re-synchronization. Visually it led to image quality jump with every new I frame(typically every second). You might remember this visual effect of MPEG4, that image changes significantly every second. Not the case with H.264. 

This is the reason why GOP size was very limited in MPEG4(image degradation accumulation). It's not the case in H.264. Practically H.264 GOP size could be very huge( practically unlimited). The only limitation comes from editing software( in order to decode frame N inside the GOP, decoder have to decode all previous frames).

So, H.264 made a huge step forward eliminating GOP size issue and image degradation towards the end of the GOP. 

 

2) deblocking filter became part of the standard. So they have to be universally implemented across all H.264 codecs. Wich also made H.264 stream look the same on any device. 

 

So, H.264 is not only bandwidth saver but solves some fundamental issues of previous standards.  

 

Excellent article, the inclusion of all the smart codec data is very useful.

Excellent article. It will add high value add for users. Grate Job.

Sounds like there is always room for improvement on H. 264, but it has proven to be the foundation in which to build on. The H. 265 sounds good at the drawing board, but when put into practice, it has its "ghosts" to be worked out.

I have a better understanding of compression

Thanks for taking the mystery out of compression!

Good article. 

Few points:

P-frames reference the full image of the previous I-frame to send only the changes in the scene. 

Just in case: P frame reference the full image of any I or P previous frames.  Advanced codecs can refer to multiple frames back minimizing the difference(part of the image could be taken from one reference frame, another part form another). Practically P frame most of the time refers previous P frame.  


 

MISC: as a paradox, typically in the surveillance industry(due to static scenes)  H.264 takes less CPU to decode than MJPEG. (since most of the time most of the image parts difference with prev frame is about zero). 

 

H.265 is designed in such a way that besides potential bandwidth savings it's better parallelized for multiple CPU cores to encode/decode. 

Thank you for the excellent summary and outline to explain CODECS. Well done!

With "Inter Framing" if I set my HD to record on motion is there further compression or are they separate?

Motion based recording does not 'further compress' video, it simply determines when to record video or not at all.

I would like to know more about SVC.I am completely ignorant on the topic.Can it be used also to obtain rtsp streams or does it work only with a server/vms that supports the function?

Luca, we know no video surveillance manufacturer that offers a scalable video codec today. It's effectively not used.

Today, it's almost always H.264, H.265 combined with smart codecs, no svc.

Actually UNV allows to enable SVC in the video page.Checking around IPVM i've seen also tests you did with Vivotek

 

My mistake to say none are 'offered'. 

My point is none are used. For example, we have never heard Uniview nor Vivotek recommend using SVC nor any VMSes do so for those cameras.

We are not against SVC, there is just no sign that anyone on the manufacturing side is promoting or recommending using it.

My point is none are used.

Hikvision, Dahua and Vivotek all have a SVC-T setting exposed, that works with at least their own recorders. Of course, this is actually enabled because of the underlying SOC supporting it, as you once mentioned: 

Does Hikvision SVC Really Work?  

On the other hand, neither you nor I consider this a full implementation of SVC, because it is only frame-rate reduction based, temporal -T.  The h.26x SVC we were all waiting for, one with spatial reduction -S, (ala JPEG2000) never arrived: 

SVC - A Better H.264 Coming For Video Surveillance

 

Excellent article. It was good to see and understand how they test to determine what makes one better then the other. Great Job!!

H.264 remains the standard in video surveillance codecs notwithstanding H.265 and other codec introductions such as Google VP8 and VP9. Codecs play a vital function in achieving efficiency when it comes to bandwidth utilization.

Read this IPVM report for free.

This article is part of IPVM's 6,746 reports, 909 tests and is only available to members. To get a one-time preview of our work, enter your work email to access the full article.

Already a member? Login here | Join now
Loading Related Reports