Integrators Know Admin Passwords For Nearly All End-User Systems (Statistics)
By IPVM Team, Published Nov 01, 2017, 09:03am EDTWith cybersecurity concerns rising, more scrutiny is being applied to various elements of security implementation.
One of those is who knows the admin passwords of systems.
140+ integrators answered the following questions:
What percentage of your customer's video surveillance systems do you have admin access / know the admin password? Why?
Almost All Have Customer Admin Passwords
Over 90% of respondents answered that they had admin access to most or all of their customer systems, with only ~9% saying they did not.
The main reasons cited were:
- Keeping customer passwords enables/speeds service
- Customers forgetting/losing passwords
- Customers trust integrators/do not care
While those opposed to the practice cited security concerns as the main reason for not keeping passwords.
Passwords Necessary For Service
Service access, both local and remote, was the most common reason for maintaining access to users' admin passwords. Integrators pointed out that many users have limited desire or skills necessary to maintain their own systems and expect integrators to be able to do so.
- "Currently we have admin access at every site that we are the provider for, as well as getting admin access on any system that we install. It is critical for our ability to perform service in the case that the IT/security director is not there to provide us admin access. So far our clients have been very receptive to our requests for admin access."
- "We have admin access to about 90% of our customer's systems. Most of our customer's aren't interested in maintaining their own system or the end users have a lack of basic computer skills so we take care of them most of the time. We also have admin access for most customers so we can offer remote support without having to bother the end user with logging us in or having to remember passwords."
- "All of them. We maintain them for them. We also provide them with an admin user name and password should they decide to go to another integrator. It is our responsibility to keep the customer happy with our service, but we don't lock them to us."
- "Customers want us to manage if there is a problem. Most of our customers are consumers/residential homeowners"
- "100% due to maintenance contracts or maintenance needs when customer calls for assistance."
- "Probably in excess of 90%; they rely on us for higher level support and we can provide that quicker with remote access and admin rights."
- "90% Remote management and support are important for the vast majority of our customers."
- "80% we give them the option of setting up their own PW but most want us to have access so we can log in to troubleshoot"
- "70%. We can't keep all of the passwords because some of our customers will not allow us, however, we try to keep records of Admin passwords so we can assist the customer when they let go employees who had the responsibility and now 'no one knows'"
Customers Forget Passwords
Some integrators simply pointed out that users often forget or lose admin passwords, so maintaining credentials themselves allows them to reset passwords or create users in the event the original password is lost.
- "100% if we need to service we have rights, when and if they lose their password we can get in to resolve problem"
- "90% we are typically maintaining the equipment and find customers will typically forget the password if left to them"
- "80% We typically store an integrator password in the system so when/if they forget the password we can get into the system and update/add users and passwords."
Customers Trust Integrators
Finally, some replied that they had access to users' systems simply because the customer trusted them to have access, or did not care otherwise.
- "Probably all of them, small customer base and they are all small companies that don't care if we have access"
- "90%. Customer trusts. I have to admit most customers, especially if they are from IT, do not take access security on a security system that seriously. It ends at switch and server level. Information sharing and organization are also a key problem within our company. There is no protocol in setting up credentials. IT directors do not take it that seriously. Physical security professionals actually demand to take over credential on each hardware and software."
- "100. Faster customer service. P.s. we ask for agreement before. if customer denies then we don't but as of today all my customers agreed"
Integrators Against: Security Concerns
The main concern given by those who were against keeping admin passwords was cybersecurity, feeling it is irresponsible for integrators to maintain access to users' systems after installation is complete, and potentially leading to unauthorized remote logins without users' knowledge.
- "At last IPVM have listened to real and tangible issue that shadows all cybersecurity issue. Engineers should never have the details AFTER a system had been commissioned and handed over. But most irresponsible installers love to have remote logins."
- "Nearly all of our large client systems retain their own Admin rights, which we do not have access to without end user intervention."
- "We have an installer admin account if the client approves it and it is done by a person. We will not support a general login for multiple people nor will we have the main or only admin account. If your client does not agree with this then we feel we have not done our job as security professionals by providing value."
- "Typically we can but don't keep access. We prefer to transfer that over. There are a small percentage of our clients who insist that we maintain rights."
Exceptions: Large Organizations / Strong IT Departments
The main exception to end users allowing integrators to have ongoing admin access tended to be larger organizations with stronger IT departments, as these comments explain:
- "Most of our customers/installs are larger operations, and those customers tend to have their own I.T. or operations staff."
- "No more thant 10%. Because of security issues when the systems are commercial (like gov.)"
- "Nearly all of our large client systems retain their own Admin rights, which we do not have access to without end user intervention."
- "Between 30 and 50%. some clients IT provider have taken full control."
- "Only when there is heavy IT support by Customer, is there time to not need to have access to what we are to support."
- "I can only think of a couple of larger customers that don't want us to have access, and they will typically give us admin access when we're on site. Military sites of course are always locked down for good reason."
2 reports cite this report:
Comments (34)
Hi,
how you keep clients user/pass database?do you use some special software/app that can be acces buy the tech guys who have rights??
thx
Curiously, integrators may be less sure about whether customers should have admin access to their own systems.
You need a policy. I don't care if it's a one-page PDF that says in effect "we take care of passwords". I get it that in the real world you need to handle this stuff for your customers. Please please document your policy. (Hint: it makes the IT audit conversation into a 30 second handshake, instead of the Spanish Inquisition.)
You should use vaults. You should use strong auth when possible (smartcards? SAML?) There should be a password management policy that applies. You should be able to show password hygiene. Your inner locksmith should guide you through showing a customer password some respect, kind of like a Master Key.
Do you have a policy pdf draft/model? It be helpfull for us....
thank you!
Fascinating. None of my integrators have any of our passwords, nor are they allowed to access any of our systems once they're installed. They install, I have network visibility, then they leave without having any idea what programming we've done, or what we're using these devices for. Seems like the easiest way to maintain OPSEC to me.
Michael, depends on who you are and how much ownership of the system you want, I think. Some places don't want to have to think about the system. Some are super high security.
If you don't mind me asking, do you buy your systems with any type of onsite warranty or service plan, or is it strictly time and material after install?
And I'm not saying a company can't provide a full service plan after install without having remote access and admin privileges to the system, but that all is going to be factored into the cost.
We have a very large system, thousands of devices connected to it. I design these systems, then bid them out to a list of trusted integrators which we run annual background checks on. All of this stuff is time and materials, the only warranty we have is with the factory on the equipment. It's more expensive, but far more secure.
I feel like any end user that's paying for an IPVM subscription is going to want their admin passwords. The other 90% are going to throw a hissy fit when they inevitably lose their admin passwords and get billed for the reset. I had a bar owner years ago that I took over and immediately made the owner change her Hik default passwords, stressing they were not secure. She entered her new password, and it was 3 months before I found out she had changed it right back to default "because she doesn't have time to remember all these passwords". Most smaller EUs need to be protected from themselves.
I feel like any end user that's paying for an IPVM subscription is going to want their admin passwords. The other 90% are going to throw a hissy fit when they inevitably lose their admin passwords and get billed for the reset.
Joseph, while I don't think that is literally true :), I agree with your general assessment. Related, IPVM end users definitely tend to skew towards larger organizations with dedicated physical security specialists. While we have some smaller end users, typically IPVM end-user members are schools, government, military, airports, Fortune 2000 corporations, etc.
Is this a joke?
I am an end user system admin with 1160 cameras spread over 53 facilities(read=servers). I would never trust password administration to an installer or integrator. You need a password for the time you're onsite at one of my facilities, I will issue it, and when you're done, so is that account.
What could possibly be a situation where I need your installer or integration tech to have full root access to my infrastructure forever? I trust you? I trust any tech you send to my site? You mean the one who smelled like weed when he hit the jobsite this morning? Oh, you fired him? Did you remove his accesss immediately upon doing so? On all 53 servers? Can i trust you on that?
You get the access you need until you dont need it and i'll be happy to manage that, thanks. I dont have a million dollars of infrastucture that youre going to put at risk because of cost cuts, short cuts, or your pissed off employee who wants to get back at you. Any security administrator who doesnt have the time to manage the security of their infrastrucure, is in the wrong job. It takes 3 minutes to create a user account in vms software. Maybe 5 minutes on a server. And i will manage that, thanks. It takes even less to disable them. I'll manage that too and know its done.
The conversation would go something like this:
How did this data breach occur? Well it seems that the individual had root access because he was an employee of our integrator.
What project are they working on? They aren't, they did the "A" building last year/month.
How did they get into the system? Well they manage the passwords.
To what? The entire system.
How in the hell do we allow outside people to manage access for critical infrastructure? Thats the way weve always done it.(There is no answer here that will suffice)
Isn't that your job? What in the hell am i paying you for? uhhhhhh...end of conversation, and job.
I know who has access to my systems. I audit monthly. I can see every user account change since the system was started 9 years ago. I audit the servers every 8 weeks. I have the time. Its my job. You don't care about my job more than do. The cost of damage control, remediation, and damage to my organization's reputation...you dont want that responsibility. Trust? Really?
Is this a joke?
I am an end user system admin with 1160 cameras spread over 53 facilities(read=servers).
Questions?
Amen.
Sad thing is, sometimes the software/hardware doesn't work like it should, even if you did everything right yourself. With big and wide systems unexpected problems take a lot of time, and routine management & maintenance is still there to be done. Some level of trust is needed in practice - even if you didn't give out even temporary credentials, someone you don't know may have a physical key to access the networking equipment etc. out of convenience.
Maybe it is just me but you sound a little arrogant.
Not everyone has 1000+ camera counts. For every 1000+ camera install, there are hundreds of smaller installs with 10-20 cameras.
I have been involved in a few enterprise installs and we could not even use our own laptops let alone get admin rights. And that makes sense.
We have admin rights on every system we have installed. We let clients know they are free to change it, but it will make it that much harder for us to maintain. Nobody has denied us access.
But these are smaller installs, less than 100 cameras and on a separate network.
As a Systems Integrator I couldn't agree with you more. I would rather have my role in the relationship clearly defined than up in the air. If you want complete control of your system then I am more than happy to hand over the keys.
I don't understand the mentality of wanting to keep admin control out of the customers hands. They have PAYED for the system and they OWN it. If they want my assistance managing their system then they know I am avaible to help them.
Or, they PAYED and still OWN it :)
Ring ring ring
Me: "Thanks for calling Greenwire, how can I help?"
Client: "Hey it's Bob over at Acme. I just hired a new manager and I need him to have access to the cameras on the widget production line"
Me: "Sure thing Bob! You have the admin account, so I can either walk you through the process or initiate a remote session so I can do it for you after you login. That's the way I'd advise, because you can watch me do it and maybe do it yourself next time."
Client: "No just do it for me I'll never remember. I don't remember my password anyway"
Me: "You mean the password I just helped you set a month ago and repeatedly told you it was critical not to lose? The one you wanted so you could do things like setup new users without being billed for it? The one we can't add users without? "
Client: "Can't you just reset it? I don't mind paying, I don't want to mess with it"
Me: Headdesk headdesk headdesk
I fully agree. I'm an end user and had a mid-size regional integrator install a video system for me. He REFUSED to give me the admin credentials citing the potential for the customer to create issues in the system.
I see this as a cheap tactic to keep the customer captive and dependent on the integrator for all service calls, as well as preventing us from seeing just how poor a job the integrator did on the set-up.
I bought the system; the credentials belong to me. Period. After some back and forth, he gave me the credentials.
Ironically, the integrator did a really poor job of system set up and one of my team members reconfigured all of it. It worked fine after that.
I guess what this basically boils down to is:
- If customer asks for credentials, just comply and give them the responsibility for admining the system
- If they never ask, admin it for them for the time being (while they knowingly pay for such service)
There are people who can and want to handle their systems and those who simply could not, so it's good to have someone take care of it, or at least make a decent effort.
In the case of a fairly big installation, the effort placed by an average integrator on the software configuration is likely in the ballpark of 1/100 of the effort the actual sysadmin there would put in the following two months, anyway. Personally I'm fine with someone else physically mounting the cameras, but that's where their privilege ends.
He REFUSED to give me the admin credentials citing the potential for the customer to create issues in the system. I see this as a cheap tactic to keep the customer captive and dependent...
I agree that could be a reason and unfortunately not an uncommon one. Where there'd be an exception is if the system is actually leased (so technically you don't own it), and there is a flat rate service contract for the installer to maintain the system, and every service call comes out of their pocket.
We've had clients insist on admin credentials and usually we relent. 50% of the time though they did tinker with something we had to fix. But even though we warned them if they messed with something and we had to fix it we'd make it billable, we never really did.
REFUSED to give me the admin credentials
Related: ADI Declares: "Don’t Give The End-User The 'Admin' Or 'Root' Level Password"
Well I think the fact that you had to reconfigure it explains everything. The system is yours, you get full access to it. I as an integrator may also have Admin access for maintenance, support or configuration as well but in the end you get the keys and no one should argue that. If they do IMO that is a solid sign they aren’t a good integrator.
Michael, #5, good feedback!
We did find a pattern where larger end users with stronger IT departments did not allow integrators to keep admin passwords ongoing. A section emphasizing that has been added to the report above, with the following integrator quotes:
- "Most of our customers/installs are larger operations, and those customers tend to have their own I.T. or operations staff."
- "No more thant 10%. Because of security issues when the systems are commercial (like gov.)"
- "Nearly all of our large client systems retain their own Admin rights, which we do not have access to without end user intervention."
- "Between 30 and 50%. some clients IT provider have taken full control."
- "Only when there is heavy IT support by Customer, is there time to not need to have access to what we are to support."
- "I can only think of a couple of larger customers that don't want us to have access, and they will typically give us admin access when we're on site. Military sites of course are always locked down for good reason."
As an Integrator/Service Provider are you willing to put you're business on line if there was a breach? Read the terms of the Contract/MSA you have with the end user carefully, I am sure you will find some clause in there that puts you on the hook.
In today's day and age this is the biggest Cyber NO-NO. In my field Regulatory Compliance mandates Access Management, we will not even source products that have Privileged Accounts that cannot be modified or deleted. Only one person (a FTE) gets Admin/Root and that is not their primary log in account.
This is a scary read. As a big organization we change all default passwords and lockout the system so that only specific accounts have access. Where possible these accounts are managed from a directory like LDAP. Our policy is not to allow integrators to admin account access. If needed they will be shadowed for the particular work or given a temporary account on the specific device.
There was a comment higher up talking about policy. This is a must. It is not an issue for a service provider to have passwords, as long as you have a mutual agreement on the policy and ensure audits are performed.
I believe that too often manufacturers first and then integrators second, believe that the end user has to do security. Yes the end user needs to be aware and conscious and have policies in place, but the manufacturer and integrator need to know today what they probably did not know 10 or 20 years ago. Security starts at the beginning and not the end. My favorite quote "no point in closing the barn door, once the horse has bolted".
