Subscriber Discussion

No Native Linux VMS Found, All Windows-Based

UI
Undisclosed Integrator #1
Oct 27, 2016

I am tired of my customer complaining to me every few days about instability on his Windows 7-Based VMS Software. The NVR has to be turned off completely and cold-booted after 5 minutes each time until the next time it freezes again. I am considering migrating entirely to a LINUX-based CCTV platform, but it seems there are no native Linux VMSes out there and the 16 channels max. standalone LINUX NVRs won't be enough for my customer (he has 128 channels of IP cams recording).

Will I gain some stability by virtualizing Windows inside Linux ? Ideas, suggestions, best practices ?

(2)
Avatar
Hans Kahler
Oct 27, 2016
Eagle Eye Networks

Full Disclosure - I work for Eagle Eye Networks

The Eagle Eye Networks Security Camera VMS is 100% Linux. It may or may not be a fit for this specific scenario, but we support more than 16 cameras.

Avatar
Tyler Blake
Oct 27, 2016
BCI Integrated Solutions

I'm super confused by this post since most VMS servers are Linux. The clients are usually the part that is normally only written for Windows/MAC/Android/IOS, although the bigger manufacturers normally have linux clients.

In this instance I would just run things on an Exacq linux server. You can use a virtual deployment on his servers or an Exacq NVR. Milestone is out since their NVRs are Windows10. Hell, you could even do a 128 cam HikNVR that runs off Linux and it would be fine. Sounds like the issue is your NVR/What product you are using. Maybe if you tell us what is installed we could help you trouble shoot as well as a restart every 5 minutes is not normal.

(1)
(2)
U
Undisclosed #2
Oct 27, 2016
IPVMU Certified

I'm super confused by this post since most VMS servers are Linux.

Only if you are counting embedded NVR's as VMS servers. Generally VMS server indicates software running on COTS PC hardware. Exacq is a valid example of one, but otherwise most are Windows.

Related: Which VMSes Run On Microsoft SQL Server?

(4)
Avatar
Tyler Blake
Oct 27, 2016
BCI Integrated Solutions

Well the OP was referring to restarting an NVR so I thought he was implying the catchall of VMS to include what is running on embeded devices. IMO just use Exacq running linux. That's probably going to be the best 'open' vms in an enterprise environment on linux.

(2)
BS
Brian Sharbowicz
Oct 27, 2016
IPVMU Certified

Try considering the Digital Watchguard Spectrum NVS (Linux x64).

Build 2 boxes (Dell server running Ubuntu) and then you can install their Linux debian onto both of them.

The reason why I'm suggesting 2 boxes is because you can load-balance, have failover, or just have lighter loads on each box. Even though you can put 128 cameras on one box, why not have a better design with 2?

Running a single Dell server "could" be enough depending on your frame-rate, resolution, and storage requirements. Be sure to have 2 NIC cards on each server.

Please note that I do NOT hook up a monitor to either server. By eliminiting the high graphics card output, that allows their "RaaS" algorithm to really work well and keep resources down.

Plus you can simply access the server via a workstation client (Spectrum Client), smartphone (IOS/ANDROID App), or SSH/RDP to perform maintenance (Any PC for management).

(1)
MM
Michael Miller
Oct 27, 2016

What hardware and VMS are you using? When setup properly you should have zero issues.

(4)
UI
Undisclosed Integrator #3
Oct 27, 2016

I agree with Michael's assessment. However, there is definitely some buggy VMS out there, though certainly less than before.

In addition to Michael's question - what hardware are you using? A home-brew PC/server, NVR, or a full-fledged Dell/HP/server? Model numbers would be beneficial.

You are unlikely to gain any stability virtualizing Windows inside of Linux, particularly if it is on the same hardware. Whatever VMS / Windows problem you are running into is likely to manifest even in a virtual machine, just the Linux machine may continue to work.

(1)
UI
Undisclosed Integrator #1
Oct 28, 2016

We are using an assembled machine the size of 4U rackmount units (it is NOT a typical PC tower, but a rackmount case with lots of internal fans) with a top ASUS Motherboard (not any cheap ones) that has a lot of bells & whistles, i.e.: Additional on-circuit ESD transient-voltage-suppression (TVS) diodes, ESD TVS diode protection on the LAN circuit, 5K-hr solid-state capacitors that last 2.5X longer, etc.

Specs are > CPU: Intel Core i7 Processor, RAM: 16 GB Dual Channels and OS: 64-bit Windows Embedded Standard 7.

I've also used (not for purposes of NVR for IP cameras, but for general office use) HP and DELL branded PCs and Laptops, and the only-added value that I see are: partition with recovery and DVDs media creation for burning a set of discs for restoring to factory-defaults when disaster strikes and the online registration which gives you a dashboard with all the warranty time and information.

I've yet to use branded NVRs because my reasoning is that if said brands also assemble theirs with ASUS, DFI, FOXCONN, Intel motherboards plus genuine INTEL chips on the inside, why pay premium if I don't need said added features for my customers CCTV platform ?? Opinions ?

(1)
UI
Undisclosed Integrator #4
Oct 28, 2016

Wouldn't using a branded NVR in this case save you having to chase ghosts to resolve this? I mean people pay a premium for them to get the manufacturer warranty and usually overnight replacement on hardware and software should something happen.

(1)
MM
Michael Miller
Oct 28, 2016

We have hundreds of Avigilon (dell) or Seneca workstations and servers in the field with no "ghost" issues. We tried the whole build your own server route and it just doesn't make sense for us in the long run. There is no way we could service the number of customers we have if we were building our own boxes and dealing with those issues in-house. Having both Avigilon and Dell behind us if there is an issue saves everyone a lot of time, money and headaches.

(2)
MM
Michael Miller
Oct 28, 2016

Also what VMS are you using?

UI
Undisclosed Integrator #1
Oct 28, 2016
MM
Michael Miller
Oct 28, 2016

So you are using GeoVision. I would try using a major VMS manufacturer on commercial hardware and I bet all your problems will go way.

(4)
UI
Undisclosed Integrator #3
Oct 28, 2016

Not saying your issue necessarily pertains to hardware, but here would be my concerns:

1. Great motherboard for home use. Consumer grade motherboards are not designed for 24/7 365 reliability. As Michael and others have mentioned we have all tried the DIY server route because anyone can build their own PC. The extra features that will never be used in a commercial environment are fantastic for home use, but are just additional complexity/points of failure. The service and support are where you will suffer.

2. Dell, HP, and others are far better at understanding all of the relevant specs necessary inside the PC for proper cooling,

3. Are you using SATA drives or SAS? SATA drives have less error reporting features and are more likely to shut down due to the lack of categories to slap errors into.

4. Xeon processors equivalent to the specs of an i7 are not much more. They are almost the exact same processor with some minor additional features enabled. The real value is they usually have a lower TDP and are overall a better/more reliable part due to binning.

5. The Windows error log might be your best friend on your current issue. That will help to identify if the issue is hardware, software, or OS related.

6. Windows embedded 7 is a good choice, particularly if you have your own build with unnecessary features turned off.

All lessons I've learned the hard way. The customer threshold for system instability is far lower in surveillance than in home/gaming PCs. I used to build DVRs/NVRs for use with Geovision to save a few hundred dollars per machine. We more than ate it up in support costs and I wasted many nights troubleshooting off hours to save the company money on techs resolving it. Now I just call Dell (through Avigilon) or HP (through BCD), get their tech onsite within 24 hours or less, and have a replacement part installed.

If you still want to pursue DIY NVRs, my recommendations would be to look for server grade boards from Intel, Supermicro, or similiar. At 128 cameras you are solidly out of the consumer grade component realm. The i7 is generally fine, but since there is such limited cost differential on an Xeon E3, may as well pay slightly more for top binning. Don't get the cheapest ram on the market as that will cause many issues that are difficult to identify. Verify all components are on the supported hardware list for the mobo.

I haven't had quite as good of luck with Seneca as Michael appears to have, but if you want reasonable pricing on homebrew PCs you don't have to support yourself, they are good option.

(1)
(1)
Avatar
Luis Carmona
Oct 29, 2016
Geutebruck USA • IPVMU Certified

"I've yet to use branded NVRs because my reasoning is that if said brands also assemble theirs with ASUS, DFI, FOXCONN, Intel motherboards plus genuine INTEL chips on the inside, why pay premium if I don't need said added features for my customers CCTV platform ?? Opinions ??

Because they have spent the time and money making sure they work together, and they have optimized their libraries and system settings for the nuances and known factors for that hardware.

You may be using "best in class" hardware, but that doesn't mean any combination will always work well together. The generic "IBM clone" specifications that allow so many different manufacturers to make the same components introduces nuances that can conflict with each other. That's why Mac's tend to be more reliable than other systems- the hardware and software is built to one company's specification, Apple's, so has greater probability of optimal performance and less probability of conflict.

Now I say that more with an eye on the top quality brand names. Your cheap discount brands you can get off the shelf at box stores or the Internet don't mean they are going to be great parts inside.

PK
Patrick Kelly
Oct 27, 2016

Full Disclosure: I work for DW

UI#1,

You can find the DW Spectrum MediaServer in both X32 and and X64 Linux version HERE .

In addition, you will find Linux, Windows and Mac Clients. This download includes a 4 channel 30 day evaluation licence.

If you would like more, have any questions or a quick demo. I can be reached via patrick@digital-watchdog.com or 813-545-9549

(1)
(1)
Avatar
Josh Hendricks
Oct 27, 2016
Milestone Systems

While Linux is usually considered more stable and secure (when setup by a knowledgeable Linux admin), it is not normal to have to constantly cold boot a Windows machine. You are probably at your wits end and there are a few good recommendations for Linux based VMS here.

If you're up for it though, you might consider testing the current software on different hardware if possible. Requiring a cold boot makes me think something is overheating or a component is failing in some way - there's nothing within Windows that would require power to be removed from the system to my knowledge. If you put Linux on the same hardware, you might still be stuck doing the same thing.

(1)
Avatar
Sarunas Pavilionis
Oct 28, 2016

Exacq is Linux based.

(1)
Avatar
John Bazyk
Oct 28, 2016
Command Corporation • IPVMU Certified

We use spectrum and install it on Ubuntu. I hate windows. Lol

(1)
(1)
U
Undisclosed #5
Oct 28, 2016

You could use Dahua or Hik 128/256 channel NVR.

(2)
SD
Sinisa Dumanic
Oct 28, 2016

You have Linux based VMS's here https://www.exacq.com/index.php and here http://www.networkoptix.com/. In my opinion both of them are small, fast and
reliable.

NB
Nicolas Bellego
Oct 28, 2016

Hello

If you want to consider a real VMS running Under linux. You should then look at a french solution named Camtrace ?

http://camtrace.com/?lang=en

Stable, reliable. Used by the french army and police during the COP21.

Running 128 cameras is not a big deal. Calculation of course should be done to fin the best hardware architecture.

Best Regards

(2)
UI
Undisclosed Integrator #4
Oct 28, 2016

Seems like a solid idea to go with a product with practically 0 US support. I think the OP would be best to use a western line as others have mentioned with an active US support team.

(1)
UI
Undisclosed Integrator #1
Oct 28, 2016

Well, to be honest, it does have a good Live Chat U.S. native English support based in California that speak perfect U.S. English as well as a well-documented F.A.Q. Forum... I have no complains with the support, but with the instability of the system itself when operating continuosly for several days and then collapse ... and I have yet to pinpoint the exact cause (maybe it is Windows 7, maybe it is the internal hard drives, maybe overheating I don't know) and it is frustrating enough : ( so that's why I am asking you guys on this forum for help ...

UM
Undisclosed Manufacturer #6
Oct 29, 2016

UND 1, I have seen comments by Geovision users on IT forums and even if they have nice support, it doesn't make up for issues in the software itself, which some have commented on. When we get RFP's involving an existing Geovision system, it is usually to replace it. If it is not replacing it, the comments we get from the security staff are usually," We hate the system, but the IT department makes the decision to keep it." IT departments probably like it because they can buy it direct and "tinker" with it themselves (which let's them get themselves into trouble).

Like others here, we have seen more problems with more problems with VMS systems running on home built or Dell and HP machines than when coming with a certified machined from the VMS, and Windows or Linux doesn't seem to make a difference. We have Windows VMS based systems running for years that rarely required a reboot.

Be careful not to jump onto IT populist bandwagons that obfuscate real issues with perceived ones.

(2)
NB
Nicolas Bellego
Oct 28, 2016

Hi

The only limitation is the time difference. But the support in English is not a big deal.

And as mentioned in this article, there is NO (few) need of support :-)

http://camtrace.com/?p=3091&lang=en

Best Regards

Avatar
Christopher Freeman
Oct 28, 2016

Had same problem with HV Software on Server.

Changed to windows 7 pro , Exacqvision , have not had problem since ( 2 years )

also could be hardware problem in server. not always vms as the culprit

Avatar
Christopher Freeman
Oct 28, 2016

I have linux servers out there and I dont like them , takes too much to troubleshoot the software and the clients , customers cannot do anything over the phone.

Like the window s platforms or factory set up units best.

No problems when I have the set up in advance. Only Dell or Hp or custom servers have glitches , but that s usually problems with set up , or software. usually something not set or set wrong.

lol

(1)
AW
Alex Wasilesku
Oct 28, 2016
IPVMU Certified

Milestones HuskyM10 is 100% Linux if you are looking for a small platform

TS
Tariq Saleh
Oct 28, 2016

Netavis.net has VMS which is based on Linux (Red hat and CentOS). They have multiple options from small scale to large scale, from single server to multiple servers. Also they have option such as NVR or you can install the software on your chosen hardware.

Avatar
Christopher Uiterwyk
Oct 28, 2016
IPConfigure

Orchid VMS and its free for Raspberry Pi or $69/camera for Linux or Windows. Rich browser based UI makes it simple and accessible.

http://www.ipconfigure.com/products/orchid

(1)
MI
Matt Ion
Oct 29, 2016

The NVR has to be turned off completely and cold-booted after 5 minutes each time until the next time it freezes again.

If you actually have to power it off for five minutes before it will start back up properly... that's hardware. You may have parts bad out-of-the-box (bad drive, bad RAM, bad mobo), assembly mistakes (ever had a screw wedged between the motherboard and the case?), mismatched components (wrong speed RAM for the board, etc.)... lots of possibilities.

(5)
AR
Austin Rich
Oct 30, 2016
This. I would start to look at utilities to verify CPU temperature under load. Did anyone do anything silly like overclock the CPU or GPU? Sounding more and more like an unstable system.
UM
Undisclosed Manufacturer #7
Oct 29, 2016

The NVR has to be turned off completely and cold-booted after 5 minutes each time until the next time it freezes again

I did have a problem like that once, with an older ASUS Workstation board. The computer would freeze every day or so, (usually in the afternoon when it was hotter), and I finally traced it to over heated RAM (you can check the RAM temp. in the BIOS). When I installed a memory cooler on the motherboard the problem went away and the computer ran fine for several years, until one day the MB burst into flames. I have a lot of confidence in ASUS MBs, but in this case I believe it was a faulty design.

I do not believe that Windows 7 or the VMS is your problem here. Because a VMS is almost certain to run on the OS in what is called "user mode" it is very unlikely it could crash the OS. Also, I find Windows 7 to be very stable, but it can only do so much to protect itself from bad drivers and bad hardware. If you have an unstable Windows (or Linux) installation, it is almost always caused by 1 of 2 things;

1) Buggy drivers. In this case run Windows update first, and if the problem is still there try updating all the drivers manually. I recently had another unstable installation of Windows 7 that would get slower and slower and eventually require a reboot by early afternoon everyday. I fixed the problem simply by running Windows update.

2) Hardware problem, as in the example I gave above.

(1)
(1)
Avatar
Campbell Chang
Oct 30, 2016

Power cycling the NVR sounds like a hardware issue rather than a VMS issue.

There are heaps of Linux based VMS's out there. Bear in mind though that with 128 cameras, you're looking at 128 licenses plus labor to migrate everything.

SL
Sandro Lassmann
Nov 24, 2016

Have a look to www.netavis.net, we have a LINUX based software for big and huge installations. And uptimes of 1 year and more are usual.

(1)
AW
Alex Wasilesku
Nov 28, 2016
IPVMU Certified

Access Control Engineer / Software Developer / Sr. Networking Specialist here.

Ill start by saying there are a lot of Linux Based VMS applications out there. However can be limited when speaking of Industry Leaders.

There always seems to be some confussion with going along and running completely unkown VMS Applications and Suppliers vs Running a commonly run, idustry leading, VMS partner that has a huge dedicated support team and quality control team doing their jobs in the background.

Is there a reason you are not going with Hikvision, Milestone, Exacq, Avigilon, etc? I believe from earlier readings I saw that the choice not to go with one of these systems is paying for unnecessary premiums? I can tell you as a software engineer myself those premiums you pay for are Quality Control (which sounds like your real issue), a huge tech support team with extensive testing on every product they manufacture, over night turn arounds when experiencing issues and it turns out to be hardware, constant updates and patch fixes, remote access troubleshooting where they drive and resolve the issue while you watch, the list can go on and on but ill stop it there.

There is a commonly used slogan here in the networking field and that is

"you will never be fired for buying cisco"

Which basically means if you buy unknown products expect problems and expect little to no support when trying to resolve the issues which sounds exactly like what your experiencing as well.

(1)
U
Undisclosed #2
Nov 28, 2016
IPVMU Certified

"you will never be fired for buying cisco"

"though you'll have to be laid off to pay for it." is a common retort.

(1)
(2)
AW
Alex Wasilesku
Nov 28, 2016
IPVMU Certified

Haha as much as I wish I could disagree with you that is the best comeback possible to the statement. You pay for quality, you pay for support, you pay for what you get. I know you understand what I meant when I originally said it.

Video is a whole different Monster than building a pc with extremely high specs, major gaming video cards, I7 quad or octo processors, etc. I remember reading the original poster is building their own machines. Well I can easily say my personal 3000$ with 900$ video card homemade computer will not perform the same as our 1000$ build sooley meant for vms applications. But with better parts shouldn't it perform better? Not necessarily because certain cards and processors have features that specialize for optimizing video management.

Best Tech Tip I can provide to everyone here. Call into Milestone and find out what they put in their machines and why they put those specific products in them. As described earlier the 150$ video card can out perform the 400$ card depending on features for this specific application.

(1)
Avatar
Luis Carmona
Nov 28, 2016
Geutebruck USA • IPVMU Certified

I like that comeback, too. And to keep things in context, that phrase is for networking equipment. But I know of people who were probably fired after procuring Cisco VMS.

(1)
Avatar
Danny Vallejo
Nov 28, 2016

IndigoVision has Linux NVRs as well as Windows NVRs.

The first thing I would try to do in this situation is verify the hardware of this NVR. Many times people attempt to use old hardware past it's prime. Since most NVRs are now handling 100s of cameras I would guess that this NVR is very old.

I would not blame Windows for this since there are many functional Windows servers in the field. The Linux vs. Windows debate will continue but without any knowledge of this situation I will only say that many vendors including IndigoVision have Linux NVRs that handle 100s of cameras.

(2)
Avatar
Jon Dillabaugh
Nov 28, 2016
Pro Focus LLC

What type of PC are you running this VMS on? I have quite a few VMS servers running on Windows 7 or 10 and don't need rebooted. In fact, I haven't ran across any PC of late that needed frequent reboots.

I have a feeling you are running on defective, poor quality, or underperforming for the task hardware. Either that or your VMS is unstable.

EDIT: I should have read the replies before trying to help.

Your problems are:

1) Any PC you build will likely be inferior to a Dell/HP/Lenovo built server. I have built tens of thousands of PCs since 1995 and the one thing I can assure you is that buying the "top of the line" ASUS board will usually cause the exact issue you are reporting. Stop buying top of the line, cutting edge products. Buy stable, well known and tested versions instead. I generally prefer Gigabyte motherboards, but I'm not exclusive.

2) Geovision, in my past experiences, is FAR from a polished product. This is going back to the days of buying their analog camera cards. While I admit I haven't used their IP products, I knew enough about their software developing that I didn't need to. If people here think Dahua or Hikvision level software is craptastic (me included), you should try out Geovision and some others to get a feel for rock bottom.

3) While Linux may or may not be more secure or stable, you should be well versed with a given OS before selling it to a client you are supporting. I am reluctant to change to Linux, due to the fact I know Windows much better and it is far easier to find support for Windows than Linux.

4) If you are using your server as a workstation as well, you should really consider using it as a server alone. I have clients that wanted to use their VMS server as a daily use computer, in addition to it being the VMS server. This is a very bad idea. Stability comes from simplicity. Don't over complicate things. K.I.S.S.

(1)
(1)
New discussion

Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.

Newest discussions