Verkada Notification Outage

By Sean Patton, Published Dec 12, 2019, 10:52am EST

Verkada is suffering an event notification outage and analytic search failures.

Inside, we examine what the issues are, what Verkada told IPVM and the risks of these outages for VSaaS users.

Update **/** - ***** *** *** ************* ********* / ** ******

** ** *** ******* of **/** ***** *** SMS ************* *** ****** have **** ******** **** our *******. ****** *** notifications *** ***** *** working. ** ******* *** to ******* *** ********** feedback *** **** ****** the ****** **** ** is ********.

************, ****** ****** ** also ******* ********, ******* for *** ***(*) **** it *** *** ***********, people ********* ****** *** still *** *****.

Outage ********

*************, *** *****, *** and ****** ***, *** all *** *******, ********** users **** ******* ****** about ********** *** ******** risks.

**** ********* *** ****** has ****** ** ***** 1 **** *** ** the **** ** **** publication. ******* ******* *** still ********* ** *******'* servers *** ********* ** functioning ********, ** ********* notifications ** *** ******** were **** ** *******.

*******, **** ** ********* Verkada ******* ** **/** at * ** **, they ********* **** **** aware ** *** ***** and **** ********** * fix:

******* *** ****** ** provide ** *** *** the ***, *** ***** it *** *** *** priority *** ***********:

*************, *** ******* ***** noted **** ****** *** issue ******* ******* **** are ***** ******* ******* in ***** *****:

** ******** * ************* at ~* ** **, by *** *** *****, but **** *** ******** any *****. ******* ****** App ************* *** **** not *******.

****** **** ***** ** the ***** *** *** trigger *** ******:

No ******* ************

******* *** *** **** email ************* ** ********/*** users ** ****** * blog ***** ***** *** outage ** ** *** morning ** **/**, ~** hours ***** **** ****** aware ** **. ******* this *******.*** ******* ** ***** to ******** *.* ***** after ***** ****** *******(* **** ***** ** was ********).

*** ********** ***** *****.*** experienced * **** ****** outage, ***** ******* ** suffering * ******* ****** of ***** ************* *** analytic ********.

** ***** ******* **** the ***** ** *** issue *** **** **** not ******** * ***** response. ** **** ****** with ***************** **** ******* ***, which ** ****** ** the ** ********.

Cloud - ***** ************ *****

*** ************ ****** ***** cause ***** ** **** critical ****** (****** ** vehicles ** ***** ********), especially ** ****** ***** or ***** ****** ******** hours **** ** *** may ** ******** ********** their *******. **** ** especially ********* *** ***** users *** **** ***** ** *******'* *** 5 ********, "***** ******** *** Alerts", **:

***** ***** ** ******** situational ********* ******* ****** to *********** ******* ************ around *** *****. **** notifications **** *********** ***** stakeholders ** ******* **********, such ** * ****** detected ** *** ******** of * ****** ***** hours, *** ********** ** on-site ********* ** ******* or ********** **********.

**** ** *** *** same ***** ** *** loss ** *** ******** at * ********, ***** could ********* ****** **** traditional ***/*** *******. ******** events *** ****** *** very ***** *** ******** that ********* *** ************* have ********* *** *******, and *** ********* ******** are *** *********** ** complicated.

Person ********* ****** ******

** ******** ** *** notification *******, ****** ****** is **** *** *********** properly. ***** ** *** basic ******** ******* ******, people *** ******* *** highlighted ** *** ******, switching ** ****** ****** returns "** ****** ******** within **** **** *****":

******* **** **** ***** was ******* ** *** camera *** ******* * fix ********, ******* ** of *** **** ** publishing, *** ***** ***** persists.

Cloud ** **-**** *******

******* ********* ******* ***** the ******** ** ******** outages *** ****** **** come **** ************ *************. And **** *** ********* right **** ***** *** such *****.

** *** ***** ****, as **** ******* ******* notification ****** *****, ***** are **** ***** **** one's ***** ******** *** problems **** ***** **** to ****** **** *** hope **** ** ******** as ******* ** ********.

Comments (37)

Undisclosed #1

12/12/19 04:04pm

** ******** ** *** notification *******, ****** ****** is **** *** *********** properly.

** *** ****** ** anything ***** **** **** notifications ** ******? **** what ** ******* ** the ******, **** ****** as ** ** *** be ********* **** **** just "*************", *** *** be ********* **** **** of *** ********* ******* in *** ***** ** well.

** **** ** *********** to *** ** *** search ******** ***** ***** things *** ********, ************ on ***** ******** ****** the **** ** *** outage.

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Sean Patton

IPVM | In reply to Undisclosed #1 | 12/12/19 04:17pm

****** ************* ******* ********* restored *** ******** ****** today, *** *** *** some ********** ******** *****.

Agree
Disagree
Informative: 1
Unhelpful
Funny
Avatar

Sean Patton

IPVM | In reply to Undisclosed #1 | 12/12/19 04:20pm

*****, *** *** ***** question, *** ****** *** email/SMS/app ************* ** ******** events (******, ********) ***** on * **** ******* schedule (******, ******** *****, after *****, ***). *** analytics **** ** ** working ********, ** *** people *** *********** **** bounding ***** ** *** timeline, *** *** ****** results **** *** **********.

Agree
Disagree
Informative
Unhelpful
Funny

Undisclosed End User #2

12/12/19 04:06pm

***** *** *** ********** to ****** ** *** cons ** *** ******* technologies. ****** **** **** didn't *****.

Agree: 7
Disagree
Informative: 1
Unhelpful
Funny

Undisclosed Manufacturer #3

12/12/19 04:13pm

** ***** ** ***** beneficial ** **** **** this ******* ******.

Agree: 12
Disagree: 1
Informative
Unhelpful: 1
Funny: 1

Undisclosed Manufacturer #4

12/12/19 04:21pm

** ** ************ **** any ********** ***** *** a ***** ********** ******** for ********* *** ****** or * ******** ***-***** interface ****** ********. **** is *** ** **** ways ******** ****** *** go *********.

* ****** **** ****** on *** ********** *** it ******* ***** **** useless ** **** *** using ******* *** *** like. **********, ******** ** foreign, **** **** * field *** **** ******* who ****** ***** *******, just **** **** ****** anything **** ******** ***************.

*** **** ***** ****** will *** **** ******** of ****** *** ****** vulnerable.

Agree: 9
Disagree: 6
Informative
Unhelpful: 1
Funny: 2

Undisclosed #1

12/12/19 05:55pm

** ** ************ **** any ********** ***** *** a ***** ********** ******** for ********* *** ****** or * ******** ***-***** interface ****** ********.

"************", ** *** ** all. ******** **** ***** payroll, ***, ***, *****, and ***** ******** ********* are *** ****** ******** outsourced ** *****-****** *********. Why ***** ** ** "unbelievable" **** ***** ******** cameras ***** ** *** different?

** ***** **************** **** ** *** cloud-based *******, ** ***** short-sighted ****, *** ** you ***** ***** ** is "************" **** *** as * ************ *** in *** * **** rough **** *****.

Agree: 6
Disagree: 1
Informative: 3
Unhelpful
Funny: 1

Undisclosed Integrator #5

In reply to Undisclosed #1 | 12/12/19 06:04pm

* *** ******** ** watching ****** ** ******* for ***** **** "****** 365/Google ***** ** ****, guess *'** ** ****!" It ******* **** ***** than ***'* *****, *** a *** ** ************* seem ** ** **** with **.

Agree: 2
Disagree: 1
Informative
Unhelpful
Funny

Undisclosed Manufacturer #4

In reply to Undisclosed #1 | 12/12/19 07:05pm

** ** "***********" *** you ** ***** **** losing *****, *******, ***, etc ***** ****** ** loss ** **** **** losing * ************ ****** using ********* *** ******** areas *** *** **** you ****** *** ** would ** *** *********. Or ***** "************", ********* on **** **** **** in *** ******** **.

***'* **** ********* *******, fire ***** **** ********, and ***** ********* **** to *** ******* *** cloud, ***. ***'* **** access ******* ******* **** to ** ** *** cloud *****, ****** ******* a **** *** ********* egress. **'* *** *** same ** ***** *** CRM.

******** ************ ***** *** (or **** *** ***) for ******** *** * very ***** **** *****.

Agree: 2
Disagree: 3
Informative: 2
Unhelpful: 5
Funny

Undisclosed #1

In reply to Undisclosed Manufacturer #4 | 12/12/19 07:08pm

** ** "***********" *** you ** ***** **** losing *****, *******, ***, etc ***** ****** ** loss ** **** **** losing * ************ ****** using ********* *** ******** areas *** *** **** you ****** *** ** would ** *** *********.

* ***** *** ****** my *****. * ***** that *** *** ***** of ********* ******** ****** not ** ******** ** the **** ***, *** I *** ******* *** how *** *** ***-***** are ***** ******* ****. Calling ** "************" ******* that *** *** *** understanding *** ****** ****** of *** ******, *** are ******** ** ***** accordingly, ***.

Agree: 5
Disagree: 1
Informative
Unhelpful: 1
Funny

Undisclosed Manufacturer #6

In reply to Undisclosed #1 | 12/12/19 08:08pm

** *** **** ** minimize ** ******* ********** demands *** *** ********* costs ** *** ****** that *** ****** ** manage *** ********* ** networked *******, **** ******* cloud *** ****** ************ operating *****.

** *** **** ** own *** ******* ** your ** *** *******, data *********, ********, **********, fault *********, **-****, ***, to ******** ****, *** have ** *** *** price *** ** ** in-house.

****'* ** ****-************** ** two ********* ** ******, but **** ** *** essense ** **. *** lesser ** *** ***** or *** ******* ** two *****.

***** *********** **** **** it's *** *** ** risks ** ******** *** the *********** ********* ** mitigate ***** *****.

** *** ********* ***** as * **** ****** technology, ** ** ** house ** ** ******** to ****** *** ************ of ********** ** ****** (the *****). ** *** does *** ******** ***** as * **** ****** technology, ***** *** **** sense.

** ****** *****'* **** more ** **** ** each ************ ***, *****, I ***** ****'* *** essence ** **.

Agree: 4
Disagree
Informative
Unhelpful
Funny

Undisclosed Manufacturer #4

In reply to Undisclosed #1 | 12/12/19 09:56pm

***, **** ***** *** ill-communicated. * ***** **** that ****** *** *** a ****** ** **** industry ***** **** ********** and *********** ** * huge ****** ****** ** IT. * ****'* ***** I *** ** ***** the *******. **** ** unbelievable ** *** **** that **** ****** ****** has ******* ******** **** reliability ** * ****** used ** ******* ***** and ******. **** ** ERM ***** ** ****, "unadvisable" ** ** ********** adjective ** ******** *** ignorant *** ********* **** is, ***.

*** ******* ** **** executives\board ******* ***'* ********** technology **** * **** perspective ** ******** ** says, *******. ** *** cloud ******** ***** ** that ***** ***** **** be ****** ******* *********** logical ********, ** **** for ** ******* ******** about *** ********** ** physical ********.

Agree
Disagree: 1
Informative
Unhelpful: 2
Funny

Undisclosed Manufacturer #4

In reply to Undisclosed Manufacturer #4 | 12/12/19 09:58pm

***, * ***** ***** based ******** *** *** time, **** *********** ** the **** *****.

Agree
Disagree: 1
Informative
Unhelpful
Funny

Undisclosed Manufacturer #6

In reply to Undisclosed Manufacturer #4 | 12/12/19 10:17pm

**** * **** **** I *** *** ******** in *** ************ ** undesirable *********, **** ****** to **** * **** to ******** *** ************ and ************* ** *** given ******** ** *** risks *** ********** ** those ***** ** **** particular ***********.

Agree: 1
Disagree
Informative: 1
Unhelpful
Funny

Undisclosed #1

In reply to Undisclosed Manufacturer #4 | 12/12/19 10:52pm

***, **** ***** *** ill-communicated.

*** **** ******** **** was * ***** ** vague *********** ******** ** with ************ * *****-***** botnet **** ***** **** corporations *** ***** *****-********* surveillance *******.

**** ** *** *********** I **** **** **** their *** **** **** multiple ********* *** *** not *********** ** * "simple" **** ******. ****, Verkada ****** **** ***** viewing, ***** **** *** have * ***** **********.

**** ** ************ ** the **** **** **** buying ****** *** ******* priority **** *********** ** a ****** **** ** protect ***** *** ******.

******** ******. **** "*****" are *** *********** **** show ****** ****** ***************** *** *********** ** the ******?

Agree: 1
Disagree: 2
Informative
Unhelpful: 1
Funny
Avatar

Sean Patton

IPVM | 12/13/19 04:00pm

Update **/** - ***** *** *** ************* ********* / ** ******

** ** *** ******* of **/** ***** *** SMS ************* *** ****** have **** ******** **** our *******. ****** *** notifications *** ***** *** working. ** ******* *** to ******* *** ********** feedback *** **** ****** the ****** **** ** is ********.

************, ****** ****** ** also ******* ********, ******* for *** ***(*) **** it *** *** ***********, people ********* ****** *** still *** *****.

Agree
Disagree
Informative: 1
Unhelpful
Funny

Undisclosed #1

In reply to Sean Patton | 12/13/19 04:07pm

* ** **** ******* to *** *** ******* is ***** ** ******* the ****-****** ** ****.

********* **** ******** *** events ** ********* ****: Person ******** --> ****(*) matched --> ***** **** logged ** ******** --> Event **** ********** ** notification *******.

* **** ** ********** that *** ***** ************ stack ***** ** ** somehow ********** **** *** rules *****, *** ***** separate **** *** ****** detection **********.

**** **** **** *** been ******, ** ***** more **** * ******* in *** ***** ****** that *** *** *********** events *** ******* *** event *** ** *** notification **********, *** ** course * ***** *** say *** ****.

** ** **** **** common ** *** * third ***** ******* *** the ************ ******** (**:******), ** ***** ** interesting ** **** ** Verkada ** ******** ** have ****** (** ******) their *** **** ************ stack.

Agree
Disagree
Informative: 3
Unhelpful
Funny

John Honovich

IPVM | 12/13/19 11:54pm

******* *** * *** funny, ********* ******* *** not **** * ** team ***** ********, **** a**** ********* ******* *********:

** ******* ***** ** a ***-****** ***** **** before ****** * ** team.

Agree
Disagree
Informative: 2
Unhelpful
Funny: 7
Avatar

Sergey Bystrov

NetworkOptix
In reply to John Honovich | 12/18/19 11:58pm

***** * *** ** people **** ** *****, sometimes **'* ** *** to **** * **.
***** *** * **** driven ********** **********
**** ****** *********** - Google ******

******* ********, **** ** 100% ******* **** *** autotests, ***** ** **** more ******** *** ********* than ****** **. ** should **** ** **** if ********* ***** *** be *********.

Agree
Disagree
Informative
Unhelpful
Funny

John Honovich

IPVM | In reply to Sergey Bystrov | 12/19/19 12:16am

******, **** ******* ***** have * ** ****? Why ** *** ***?

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Sergey Bystrov

NetworkOptix
In reply to John Honovich | 12/19/19 12:44am

** **. *** * few ******* *** * cannot *** ***** *** is *** ***** *** more ********* ***, * just **** **** ***. And **'* *** **** binary...

*) ** *******. ***** TDD ** **** **** efficient, ********** ** * startup ** ****** ** ship *** **** **** and **** **** ** manually ****** ** ******** with ***********. ****** ***** term ****, *** **** term **** ****. ** we ***** ** ** again, ** ***** **** a **** ******* **.

*) ** *** ****** client *** **** ** the ****, ** *** much ****** ** ********.

*) *****, ***** *** things ***** ****** ** is **** *********(********** ***** term).

*) ** *** ************, QA **** ***** *** role ** ********* *******. They **** *** ********* to ****** ********** ***** and ****** ********** ***** to **** ****** **** simple *** *****.

***** **** * *** of *** ******** ** members ******** **** ****** QA ** **** *******.

*** ****** * **** some *********/******** ** *** use ****** ** ** all. * ******* ***** would ** ** ******* form *** ********. **'* totally ****, **** *********.

Agree
Disagree
Informative
Unhelpful
Funny

John Honovich

IPVM | In reply to Sergey Bystrov | 12/19/19 01:37am

**** ******** ** ********* reasonable. *** "********* **'* ** *** to **** * **" is ***.

** *** ******** ** the ******-**, ***** ********** test ******** ** *********, in ********, *** ****** any ***** ** ******* an ************'* ********* ** all ***** ** **** and ********.

*'* *** ** **** list, * ***** *****, that *** ****** ********** and *** * **** for ***** ********* ** interaction, ****** ******* *** can't ***** ******* *** the ******** ******** ******* that ** *** ***** will ******** ***.

********* * **** *** are **** ******* *****'* advocate, * ***'* ******* you ****** ***** **'* sensible ** **** *** salespeople *** ** *** in **...

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Sergey Bystrov

NetworkOptix
In reply to John Honovich | 12/19/19 01:47am

**** *** ***** ********* interview ** **** **** they **** * **** of************(*** **** *****). ** means **** ****** ***** have ****** *** ** role ** ************ *******(*'* sure **** ***).

** **'* *** ***** "no *** ** **.." or "*** ****** *** human **".

Agree
Disagree
Informative
Unhelpful
Funny

John Honovich

IPVM | In reply to Sergey Bystrov | 12/19/19 01:54am

***, * ***** **** someone ****** ** ******* even ******** ***** **** out ** ******* ****** deploying ** **********. ** still ** *****, ********* to **** ** **, that ** *********** **** them **** **** ** formalize **.

**** ****, *** ********** advocated *** **********:

******* ********, **** ** 100% ******* **** *** autotests

*****, *** ********** ** detracting **** *** **** of **********, **** ****** that *** ********** ********, whether ******* ***** ** now *******, ** ***** to **** * ********** QA *******, ****** ***.

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Sergey Bystrov

NetworkOptix
In reply to Sergey Bystrov | 12/19/19 05:41pm

* ******** **'* ***** to **** * ***** good ****** ***** ***** product *******.

Agree
Disagree
Informative
Unhelpful
Funny

Undisclosed Manufacturer #7

12/16/19 08:37pm

**** ******* ** ***'* that *** **** ***** oil??

Agree
Disagree
Informative
Unhelpful
Funny: 2
Avatar

Sean Patton

IPVM | 12/20/19 04:58pm

** **** ******** ******* of ********* ************ *******, with ** *** *** resolution ******** **** *******. A ******* ******* **** us **** *** ***** having ****** *** **** not ******** * ******** in * ****.

** **** ********* ** receive *****/*** *************, **** still ** ****** *** notifications.

******* *** *** ******** a ****** ** ********** for *** ******, *** we **** ****** *** report **** ** ******* one, ** ******* ********* a ****-******.

Agree
Disagree
Informative
Unhelpful
Funny

Undisclosed #1

In reply to Sean Patton | 12/20/19 05:37pm

**** ****** ** ***** like **** ****** * software ****** **** **** massively ******* * ******** schema, ** ******* **** other *********** ************* ** the ************ **** ** very **** **** ** roll ****.

** **** *** *** fault ** * *** party ******* ********, ** would **** **** ****** been **** **** ** a **** *****, ******** along **** ******* ********* to * ********* *** party. *** ** *******, we ***** ** ******* of ***** ********* ****** notification ******** *** ***** products.

** **** *** *** result ** * ***** bug, *****, ***., **** would **** ****** **** back, ** ***** *** issue ** ***.

** **** **** ******* out ** ****** *********, they ***** **** **** a ***** ** **** tumblers ** **** ** fund **** **** *** cycles :)

**** ****** ** ** believe **** ** **** very ******* ** **** architecture, *** ******* **** Verkada **** **** *** very *********, ** **********, to *******.

* ****** ** ******* customers **** **** ** given *** ****** ***** how *** **** *** changes, ************, ***. *** rolled *** ** ***** personal ********* *** ********. This ** *** ** the ***** ********* ** SaaS-style *******, **** ******** experience ** **** ** good ** *** ******* link ** **** ********* resources.

Agree
Disagree
Informative
Unhelpful
Funny

John Honovich

IPVM | 12/24/19 12:25pm

*********** *******'* ********* ***** them ** ***** '****** reliable, ****** **':

****** *** ******* ***** of **** *********, ********* rarely ***** ** ** 'always' ******** ** '**' as *** ******* (******* its ******* ** ***** or ***, ***.) ** any ******* *** **** issues ********.

Agree: 1
Disagree
Informative
Unhelpful
Funny: 1

Undisclosed #1

In reply to John Honovich | 12/24/19 12:38pm

*********** *******'* ********* ***** them ** ***** '****** reliable, ****** **'

****, ** **** ***** the ******* *** **** impacted **** ****** **** you ***************** ** ***'* *** fixed *******.

Agree: 1
Disagree
Informative
Unhelpful
Funny

Undisclosed Manufacturer #8

In reply to John Honovich | 12/24/19 05:04pm

*'* ******** **** *** rethinking ******** **** ** sign ** ** * customer ***** ***.

Agree
Disagree
Informative
Unhelpful
Funny: 1
Avatar

Sean Patton

IPVM | In reply to Undisclosed Manufacturer #8 | 12/24/19 07:10pm

** ********* *** ******* through * ***** ******* dealer *** ********** *** system ******* ******** **********.

***** * ******* ***** they ***** ***** ******** created **** *** ***** address/domain, * ** *** know ** **** ***. They **** ** **** channel, ****** *** ******** through ***/************.

Agree
Disagree
Informative
Unhelpful
Funny: 1

Undisclosed Integrator #5

In reply to Sean Patton | 12/24/19 07:39pm

***** *** **** ******** partnering **** * ***** integrator ** ** **** covert *******, ** * manufacturer ***** ** **** their ******** **** **** you?

*'* **** ****** *** of *********; ** ******* doesn't ******* ** ************.

Agree
Disagree
Informative
Unhelpful
Funny: 1

Undisclosed Integrator #5

In reply to John Honovich | 12/24/19 07:34pm

*** **** ** ** uptime ** **** ********* in ***** ***? ** they *** *** ******* like **********.** ** ****** outages?

** **** ** *** advertising ******* **** ** SLA ** ********** ** back ** **, ****'*... not ********** ** *** from **** *******.

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Sean Patton

IPVM | In reply to Undisclosed Integrator #5 | 12/27/19 07:59pm

* *'* ** ******, we **** ***** **** are ******** ******* ******* for ***** **** **** not **** ********* *************.

**** ******* *** *********'* *** ******* ****.

Agree
Disagree
Informative: 1
Unhelpful
Funny

Undisclosed Integrator #9

12/30/19 05:16am

* ** * ******* partner. ***.

******: **** ******* *** been ***** ** *** own **********:* ** * ******* Partner. ***.

Agree
Disagree
Informative
Unhelpful
Funny: 1

John Honovich

IPVM | 04/12/20 02:05pm

******:******* *** *** * public ****** ****, *********:

Agree
Disagree
Informative: 2
Unhelpful
Funny
Read this IPVM report for free.

This article is part of IPVM's 6,896 reports, 921 tests and is only available to members. To get a one-time preview of our work, enter your work email to access the full article.

Already a member? Login here | Join now
Loading Related Reports