ONVIF Conformant Products Failed To Comply With Mandatory Test Cases?

When you say "ONVIF tool", are you using the official ONVIF test tool? 

Theoretically, I believe manufacturers are supposed to re-test conformance on firmware update. For example, if you look at this listing from Axis, they're on the fifth firmware version since that camera was first conformant (8.30.1, previously 7.40.1, previously 7.20.1 and so on).

However, not all manufacturers do. For example, you mentioned Uniview, which shows the most recent conformant firmware as IPC_G6102-B5013P10D1611, while we just updated cameras to a newer version for our tests (IPC_G6102-B5016P30D1705).

Ethan, Yes. It is official ONVIF tool. I believe most of the vendors don't follow retest procedure. I have tested few models of HIKVISON and UNIVIEW. They failed to pass the mandatory test cases

ONVIF has confirmed that technically a manufacturer must submit a new declaration of conformance / test result for new firmware. However, ONVIF has not yet been able to provide any proof of how they enforce this or if they even try to enforce this.

How many manufacturers claim ONVIF compatibility but don't actually have it.  Or let me ask a question that maybe they can answer, what kind of penalty would a company receive if they falsely claim that they are ONVIF compliant and are not?  To me, the lack of enforcing has ruined the credibility of ONVIF.

what kind of penalty would a company receive if they falsely claim that they are ONVIF compliant and are not?

Related: List Of Manufacturers Faking ONVIF Conformance

The answer is generally nothing. ONVIF does minimal enforcement.

Are there any penalties for manufacturers that claim onvif compliance and fail? Whats the point of it otherwise? Could you have a class action law suit for false advertisement? 

 Another flaw from ONVIF is, there is no clear guidelines about how the products should be tested by their test tool and what is the environment expected. For example I was running ONVIF tool with a cameras which is listed in ONVIF site, It failed in IPV4 test cases. I have assigned IPV4 manually to the camera.  When I approached the OEM about this issue, they recommended to have a DHCP assigned IPV4 to the camera, which in turns passes the test case. From ONVIF side there is no such guideline stating that user should have DHCP server in the environment while performing test 

Onvif is, and probably always will be, a best practices type of tagline. It does not carry any certification weight like UL listings do, but it's more like an "organic food" label.