Axis Suffers Outage, Provides Postmortem

By: Ethan Ace and John Honovich, Published on Aug 15, 2019

This week, Axis suffered an outage impacting their website and cloud services.

axis outage_2

Inside this note, we examined what happened, what was impacted and what response Axis has provided, contrasting it to recent outages for Alarm.com and Hikvision.

**** ****, **** ******** an ****** ********* ***** website *** ***** ********.

axis outage_2

****** **** ****, ** examined **** ********, **** was ******** *** **** response **** *** ********, contrasting ** ** ************* *** *****.***************.

[***************]

** *******, ****** ****, Axis ******* ******* ** show ** ***** *******:

************, ********** **** ** ****** media, *.*.:

***** ***** ******** **** impacted ** **** ******* they *** ** **** to *****. ***** ********** loaded **** *****, *** this **** **** ** seconds *** **. ******* video **** * **** time ** ****, *** eventually ********, ** ************ below:

*** ****** ****** ******* 8 *****.

**********

**** ******** *** ********* postmortem ** ****:

******* ******* ** *** a ******** ******* ******* in *** ** *** sites. *** ******** ********* and **** ********** **** switches ****** ******* *** stopped ******* ** * way **** ****** ***** be ********. *** ****** is ******** *** ******* uptime, ******* *********** *** complete **********. ************, ********** went **** ******* ******* at ************* **:** ***.******** **** *** ******* staff *** *** **** switch/network ******** ** ******* working ** ** ******** and ***. ***** *** size ** *** **** and **********, **** **** until ************* **:** *** Tuesday *******, ********* ******* within **** *** ****** experts **** *** ********, both ** ******** *** remotely **** ***.*** **** ** ******* everything ** ******* ********** started ** **** ** we *** ** ************* of *** *****. ** approximately **:** ***, *** core ******* *** ******** and ** ***** ***** bringing ******* ** *****. As ** **** ******* around **:** ***, ** were **** ** **** back ** **** ******** with * *** ********* issues. ** ********* (*****) 12:00 ***, *** ***** issues *** **** ******** and ********** **** ** normal.*** ******** **** ***** analysis **** ** ********* in *** ********’* ****; the ******** **** **** some **** *** ** are ********** **** *******. All ******* **** **** designed *** ******** **********. Despite ****, ** *** problems, ***** ** ****** is * ******** **********. We **** ******* *** design, ******** **** *** supplier, *** **** ********* changes.

** ***** ***** ********** and **** **********:

*** ***** ****** ** yes, ** **** ***** site ********** *****. *** slightly ****** ****** ** that *** *****-**** ***** of *** *** ***** uses * ****** ******* and **** *** ***** caches **** ** ****** to * ******* ******* somehow. ****** ** ******* a ***** ***** ****** or ** ******** *** cache *** ******** *** content. ** *** ********* roadmap *** *** ***, all ***** ****** **** be **** ** ** more **********, **** **** some ******* *******. ** is * ******* ******* ensuring **** ******* ** available *** *******. ******** to ***, ** **** look **** **** ******* again, *** ** *** constantly ****** ** ******* even ******* ********** **** as *** **** ****** experience.

*********** (*** **** ****), ** **** ** prompt ****** *************, *** the **** ** **, though ** ***** ************ and ******** **** *** less ******.

Alarm.com *** ********* *******

**** ** *** *** outage (** ***** **** we **** ********** ** the **** *** ******). Alarm.com *** *** **** serious ***, ********* ******** of ********'* ******* **** 5 *****. ********* *** the ******* *** ********* one ** ********* *****'* website *** **** ** least * ****, *** Hikvision *** *** **** sporadically, *** ***** ** a ****, **** * month.

** *** *, **** was *** **** *** who ******** ****** ****** while ** *** ********* and ******* *** **********.

** *** *** *** online ******** ****** ************ critical *** **********, *** ability ** ************ ** keep ***** ******** ****** and ******** ****** ** issues *** *********** **** be ** ********** **************.

Comments (4)

**** ** **** ******* hour? * **** ****, but **** **** ********* for **** **** ******* without *********, ****** ****** (and ********), *********, **********, and ********** **** ** mind *** ******* ** Memory **** ***** **** are ******** ** ** fault ******** (*** **** to **** **** ***** systems **** ****, ********* etc), ******, **** ******* consistency, *** *** *** be ******* ** **** if "*** ** *** expensive"

* *** ****** ** download *** ****** **** Axis *** ********** ** took **** ***** ********* on ********* ******** ** get ********** * ****** and **** ** ** min ** ** ** time

****** *****. **** ******** none ** *** ********* want ***** ****** *** demand ** *** **** "reliable" *******.

*** * **** ********* or *** **** **** say...

"*** **** **** ****, the ******** **** ******, our ******* ********** *** a ****** ***** ** failure, *** ** *** throwing *** ****** ****** under *** *** **** though *** ***-*********** ******* apparently *** ** *********** complex *************."

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports

HTTPS / SSL Video Surveillance Usage Statistics on Apr 01, 2019
HTTPS / SSL / TLS usage has become commonplace for websites to improve security and, in particular, to help mitigate attackers reading or modifying...
Verint Victimized By Ransomware on Apr 18, 2019
Verint, which is best known in the physical security industry for video surveillance but has built a sizeable cybersecurity business as well, was...
Manufacturer Favorability Guide 2019 on Jun 12, 2019
The 259 page PDF guide may be downloaded inside by all IPVM members. It includes our manufacturer favorability rankings and individual...
Security Fail: ASISNYC Auto Emails Passwords In Plain Text on May 14, 2019
ASIS NYC automatically emails a user with the password the user just entered, in plain text, when one registers for the site / event, as the...
'CCTV' Is the Past, Cloud Video Surveillance Is the Future on Jul 08, 2019
A fundamental shift is happening. For decades, video surveillance was overwhelmingly 'closed' and off the Internet. This is changing. More and more...
Alarm.com Suffers Outage on Jul 26, 2019
Alarm.com suffered a major outage this week, impacting its 5+ million customers. Inside, we examine what happened, what Alarm.com told IPVM and...
Warning: Windows 7 Update Crashing NVRs on Aug 26, 2019
Windows 7 updates are causing VMS servers to fail to boot. After running the update, impacted systems do not boot as normal, instead display this...
3 Weeks Later, Honeywell Still Cannot Say Whether They Are Vulnerable To Dahua Wiretapping [Now Admits] on Aug 27, 2019
The Dahua wiretapping vulnerability and Dahua's decision to delay disclosing it until IPVM inquired underscored problems with cybersecurity and...
Verkada Notification Outage on Dec 12, 2019
Verkada is suffering an event notification outage and analytic search failures. Inside, we examine what the issues are, what Verkada told IPVM...
Wyze Massive Data Leak on Dec 26, 2019
Wyze has exposed millions of user's data, as reported by Twelve Security, and confirmed by IPVM, who has spoken with Twelve Security and reviewed...

Most Recent Industry Reports

Motorola / Avigilon Drops ISC West on Feb 26, 2020
Motorola Solutions has pulled out of ISC West 2020 effective immediately, because of coronavirus concerns, IPVM has learned. This is done amidst...
Cancel or Not? Industry Split Over ISC West on Feb 26, 2020
The industry is split, polarized, over whether ISC West 2020 should run or be canceled. New IPVM survey results of 400+ respondents show heated...
Coronavirus Hits Sony, Bosch Says Switch on Feb 26, 2020
Sony's fall in video surveillance has been severe over the past decade. Now, they may be done. In this note, we examine Bosch's new...
Video Surveillance Cameras 101 on Feb 25, 2020
Cameras come in many shapes, sizes and specifications. This 101 examines the basics of cameras and features used in 2020. In this report, we...
Favorite Video Analytic Manufacturers 2020 on Feb 25, 2020
Video analytics is now as hot as ever, driven by the excitement of advancing deep learning offers. But what are actually integrator's...
Latest London Police Facial Recognition Suffers Serious Issues on Feb 24, 2020
On February 20, IPVM visited another live face rec deployment by London police, but this time the system was thwarted by technical problems and...
Masks Cause Major Facial Recognition Problems on Feb 24, 2020
Coronavirus is spurring an increase in the use of medical masks, which new IPVM test results show cause major problems for facial recognition...
Every VMS Will Become a VSaaS on Feb 21, 2020
VMS is ending. Soon every VMS will be a VSaaS. Competitive dynamics will be redrawn. What does this mean? VMS Historically...
Video Surveillance 101 Course - Last Chance on Feb 20, 2020
This is the last chance to join IPVM's first Video Surveillance 101 course, designed to help those new to the industry to quickly understand the...
Vulnerability Directory For Access Credentials on Feb 20, 2020
Knowing which access credentials are insecure can be difficult to see, especially because most look and feel the same. Even insecure 125 kHz...