Subscriber Discussion

Simple Solution To Default Password Conundrum...

U
Undisclosed #1
Mar 06, 2016
IPVMU Certified

Short and easy default passwords are so nice, until they're not.

Here's is an obvious answer to the problem, so obvious in fact I'm sure it's either stupid or somebody is already doing it:

Only let default (and/or weak) passwords work on the local LAN subnet. Installers can still use then, VMSes can still use them, just not Shodan.

?

U
Undisclosed #2
Mar 06, 2016

It assumes the local LAN and every user on it should be considered trustworthy. That's not a great idea. I've seen things like smaller schools which didn't have an IT staff who could set up a VLAN for the cameras. I'm not sure I'd put all students in the trustworthy category. Also a lot of smaller installs in SMB aren't going to have isolated cameras. And if they are watching for employee theft then it may be reasonable to assume they may not be trusted with the camera system.

Less likely but still possible is the idea that the LAN can be compromised from outside attack. While it's extremely unlikely that a hacker would be working a group looking to break in, there can still be danger. Axis had a very old issue where you could upload things via FTP and run them in the linux environment. There wasn't a lot of memory there but enough for small port scanner or other tool. Future issues along those lines are still possible.

So you can't just have default passwords that are left that way. The passwords need to be changed. And documented.

(1)
U
Undisclosed #1
Mar 06, 2016
IPVMU Certified

It assumes the local LAN and every user on it should be considered trustworthy.

If the users on the local LAN are not trustworthy by design then such a network would typically need active administration to be secure in any sense. The staff would likely have their own policies and password creation scheme for all devices. Without that such a LAN would be a free-for-all regardless.

In any event the overwhelming majority of attacks on default passwords come from the WAN.

Do you agree that if this was implemented 5 years ago by all the majors that there would be no Shodan for cameras today?

U
Undisclosed #2
Mar 06, 2016

Except in a lot of spaces you're gonna find people who do not have the resources for full time administrators. Particularly in the SMB space. If that space could afford administrators then the Managed Services industry wouldn't exist. So just waving a hand and saying well an admin should administer it doesn't work.

Yes, it may protect against some outside attacks and things like Shodan. But so does forcing a password change, and the password change also protects against potential internal attacks. So your solution is a partial solution verses a solution that is much more complete.

Why do you think this is a better and more secure option then changing the password?

(1)
U
Undisclosed #1
Mar 07, 2016
IPVMU Certified

Forcing a password change to a unique, difficult password is certainly more secure, if implemented.

But let's back up a bit. First off, I fully expected to hear from someone why it wouldn't even work for the WAN side, for some obvious technical or financial reason. I still expect there may be such a reason and that reason is not "we didn't do it because the LAN would still be unsecured, so we figured why do anything."

The BIG difference between the force password change, (besides increased development and support costs) and the block WAN default password, is that it doesn't make the integrators job more tedious and time consuming by having to comply.

Integrators that DO secure their network properly don't feel like they should be penalized by additional work due to the ones who DON'T.

To the degree that the integrator is the customer, manufacturers are going to loathe alienating them by increasing work loads.

That and increased support costs are why most passwords are still not forced to be changed.

But we are making progress, right? Two of the biggest that do force password changes are Axis and Hikvision.

Yet in the case of Axis, who talks a good game, the camera is STILL accessible by root pass UNTIL the password is changed, as I show here. Bug? Or trying to have it both ways?

Hikvision has implemented an even tougher password policy, yet after receiving a brand new model, I found that a new default of 123456789abc was now valid, as I show here.

I'm not sure that this is really HIK's doing, it may be a reseller in between or a location based modification. But the effect is the same, if distribution has decided that it's worth their while to add a default in transit to a camera, it's going to be hard to stop them.

But, why let that default be accessible from the Internet?

U
Undisclosed #2
Mar 07, 2016

As for the technical feasibility the answer is no at the enterprise level. The reality is that larger deployments can have quirks that would cause LAN only detection to fail. For instance, Ford bought a staggeringly large number of IP addresses a long time ago. They have enough addresses that they can use them internally instead of the traditional class A,B, and C networks. Same with the DoD a number of other companies.

Then you can run into all kinds of fun edge case stuff where you have networks from companies that have merged or aquired multiple locations and have a routing table that is nightmare fuel. Also you have stuff like companies doing remote management of multiple locations via VPN. So if an IT person is logging in offsite via VPN, the connection is likely secure but won't show up as a LAN IP address.

Would it be impossible to try to make something that handled all of this? No. But it would get expensive. Writing an engine for this would be a multi-month project for at least one coder, and probably another month of combined time for a couple more. From a QA perspective, it would be extremely time intensive to test. Some aspects could be automated but there would be a lot of network set up and configuration to work through. So it's not a trivial amount of expensive labor to do. QA time isn't super expensive, but developer time would be. By comparison, forcing a default password change is pretty trivial. The easier option might simply be setting up an ip address white list but that would take you more time to set up then a default password change, and would be prone to breaking if the VMS/NVR IP address changes for any reason.

And the customers it's gonna bite in the ass are the Enterprise customers and dealers. If the engine ends up buggy, and I expect there would be some teething issues, then the integrator on the sharp end is looking at potentially major issues. There is so much with this kind of engine that can end up broken. And from a business case perspective, if I have to spend a ton of money and potentially screw the hell out of guys doing big projects to save another integrator a few minutes per install than that's a bad idea for everyone. Even if in a perfect world it doesn't create a drain on support, or sales staff attention and support, it does mean that features that could help the integrator sell more might not be implemented in a timely fashion.

Particularly when the functionality to do bulk password changes exists in a lot of camera finding tools. And if you use a camera manufacturer that doesn't have that functionality, then that's a good feature request to ask for. That benefits their entire customer base. And it reduces the time needed to change the default password to a minute or two.

So let's review:

1. Technically it would be extremely challenging to do and get work right. It has the potential to become a support nightmare when it goes wrong.

2. From a security perspective, it's less secure then a default password change.

3. From a business perspective, it's a expensive with little potential revenue to be booked. This isn't an end user related request so it's not gonna drive unit sales. I'd be deeply skeptical of it being a big enough differentiation that could be used by sales staff to drive sales. The labor savings on a massive project might be the difference although usually on most bids the difference will be greater then a few hours of labor. And honestly the functionality exists to do it quickly in a lot of camera finders.

So you're proposing a solution that is worse then a default password change at it's primary goal of being secure. It's going to be more prone to creating problems and angry customer phone calls. And it's going to be dramatically more expensive to implement then forcing a default password change.

U
Undisclosed #1
Mar 09, 2016
IPVMU Certified

1. Technically it would be extremely challenging to do and get work right. It has the potential to become a support nightmare when it goes wrong.

Strongly disagree. It would only check for two things in one place:

  1. Is the default password being used?
  2. Does the request come from outside the local IP subnet?

If both are yes, then it would deny access and give the error.

Determining #2 is trivial. On LANs that are comprised of multiple routed IP subnets requests would be treated as WAN if they originate on different segments.

So IF they are using multiple routed networks AND they are accessing the CAMERAS directly from other subnets AND they are using the default credentials then it would deny them.

So when you upgrade the firmware in this specific use case, they would be denied.

Contrast that to rolling out forced change of password where when they upgrade EVERY camera without a new password would fail. Oh sure, if you happen to be coming in on the manufacturer web page you can change it right then. But if not, you will just get an error as the client won't know what to do with it.

The cameras that would get blocked in my proposal are always a small subset of the ones you would block, so your disruption is always greater.

As for which is easier to roll out? Again, take Axis as I showed, who still hasn't got it right. What do you say about this?

2. Agree. IF implemented, changing the password is more secure.

3. Strongly disagree. The business risk in your case is alienating integrators by making their life much more difficult as well as increasing the number of support calls recieved.

You have yet to acknowledge this point, and hand waving will not be an acceptable answer. :)

U
Undisclosed #2
Mar 09, 2016

Determining #2 is trivial. On LANs that are comprised of multiple routed IP subnets requests would be treated as WAN if they originate on different segments.

Why do you consider this a good default behavior? It's going to immediately cause the potential for problems for multi-location customers and other customers that I pointed out. You can't just wave them away because it's inconvenient. You're talking about customers who can represent large orders via their integrator. Any solution you demand has to address that situation or it is absolutely a non-starter.

And what you're demanding isn't a trivial thing. With TCP connections you can cheat by checking the first time the connection is established. But for any incoming UDP packets, you're going to have not check on a per connection basis but a per packet one. And generally the reason you use UDP is speed. So you're going to be adding further latency to things that tend to be depending on speed.

You keep proposing this solution as it's trivial and easy. If you're gonna hold up an Axis goof as a reason to never do it, why do you think a significantly more complicated solution is going to be less likely to be buggy?

You also handwave away a lot of use cases in the non-enterprise space. You're starting to handwave a way a lot of customers.

3. Strongly disagree. The business risk in your case is alienating integrators by making their life much more difficult as well as increasing the number of support calls recieved.

The problem is that I'm having trouble buying in on the idea that the burden to intergrators is so dramatically large here. I've pointed out that camera manufacturers have tools to make bulk camera password changes. At least a couple of VMSes have that option even if they don't make it obvious or easy to find. But even if they don't, if they are using a database then making a SQL script to do it isn't rocket science. Hell a couple of VMS/NVR manufacturers will pre-program the NVRs with all of the IP addresses and passwords already set to what ever you want. I bet if you'd ask John, Brian or Ethan they would be happy to do a survey and tell exactly which ones will do that. Same with bulk camera and password changes.

So what I'm seeing is you demanding expensive programming time and risking alienating intergrators who bring in large projects to save you a minute of using the camera manufacturer tool to do bulk changes, and three minutes to use the VMS equivalent of the same tool.

So show me the massive labor savings. Tell me why the bulk password change tools offered don't solve your solution. If you can do that, I might be able to start seeing where implementing your solution is a good idea. When ever an intergrator or end users proposes or demands a feature implementation it's not just added to the list. Manufacturers look at the idea and weigh it based on three factors:

1. Can it increase sales?

2. What will it cost? These can be development costs. They can be lost sales.

3. Is there an intangible benefit?

And you bring up the threat of alienating intergrators. The blunt truth is that as a manufacturer, everything you do will alienate some intergrators. You can't avoid it. If you could then sales people wouldn't need to maintain relationships. So the question is never "Will this alienate intergrators?" because the answer to that is always yes. Trust me, I've had to sit on phone calls listening to an intergrator bitch about a UI having a slightly different shade white as a background color. So the question is how many and which ones. And can the sales people off set it. Right now you haven't convinced me that this is more the a four minute loss to labor. So I have to say yes they probably can.

U
Undisclosed #1
Mar 09, 2016
IPVMU Certified

With TCP connections you can cheat by checking the first time the connection is established. But for any incoming UDP packets, you're going to have not check on a per connection basis but a per packet one. And generally the reason you use UDP is speed. So you're going to be adding further latency to things that tend to be depending on speed.

Come again? Look, no offense but I'm not sure you understand what I'm talking about on a technical level, which may be the reason you think this is so difficult, but..

No, you only have to check one packet per login attempt. If the packet that brings the password does not have a local source IP and the password presented is the default, then you deny the login. End of story, like any other failed login, no checking UDP packets, no latency, no "staggering large number" of IP addresses to sort.

Why do I think this is easier, than forcing a new password?

Here are the two main reasons:

  1. No UI change. You will need to create a new screen flow, that handles redirecting to and from your force change screen, as well as new help for those screens.
  2. Not dependent on state. You need to check some global value in some database which tells you if the password has been changed. You will also have to write (possibly concurrently) the same state after password is actually created. BTW, make sure and update your auto password tool to maintain your global flag correctly as well. My way is stateless, requiring only the incoming packet to function.

As for the automated password change mechanisms, you maybe right, I'm not aware of what percent of cameras have them. That certainly would help.

I've certainly heard plenty of complaints over and over, haven't you?

Not sure what you background is, but every manufacturer just wants people to plug their cameras into the LAN and have them automatically show up on every VMS, NVR, LAN client, etc. Not even go into a mass update tool if it's avoidable. Integrators like simple, easy and universal discovery. Why wouldn't they, when every minute spent resolving connections issues is money out of their pocket.

So regardless of the actual burden, if the integrators believe it is a burden, that's bad enough. And make no mistake, for a lot of these mfrs, the integrator IS the customer.

So, no matter how much you wave away and say it's nothing to be concerned about, it is in fact the VERY reason that your solution has taken so long to get to market. You think they couldn't have done it 5 years ago? It took enough Hack attacks and fallout from to motivate SOME mfrs into action.

Cut to the chase, I asked why don't they implement this WAN check, you say because they should do something better.

But the reality is that the majority have not yet, so what do you say, are you in favor of the WAN block for those who are NOT planning a force original password scheme?

Yes/no.

Avatar
Brian Rhodes
Mar 07, 2016
IPVMU Certified

Then all those routers with default password on Shodan let you reconfigure things so you can see all the cameras with default passwords?

(1)
U
Undisclosed #1
Mar 07, 2016
IPVMU Certified

Ha! That sounds like something I would say...

Sure they could do that. Pretty much if your router still has the default password AND allows remote management you're pretty much toast in all respects...

But home routers, from what I've seen, don't have remote management enabled over the WAN by default.

Take a look at the cameras and see if the management of the router is compromised as well; I think the majority will be just the cameras.

U
Undisclosed #1
Mar 07, 2016
IPVMU Certified

Can anyone say why manufacturers who have not yet implemented systems to force a complex original password should not take the simple step of denying the default password from the WAN? At least in the interim while developing a comprehensive new scheme and mitigating the business issues surrounding that.

This is code-wise as simple of a change that there is, far less than a sophisticated new password scheme which would likely touch many areas and require extensive testing and support changes.

With virtually no negative ramifications to the user base and the elimination of 99% of the hacking, what is the reason for not taking this easy step?

Avatar
Jon Dillabaugh
Mar 08, 2016
Pro Focus LLC

Why not just give a fake default gateway? Instant local only filter!

/sarcasm

(1)
New discussion

Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.

Newest discussions