From VMS Manufacturer's View, 4kn Or 512e Drives?

Any special considerations using either/or? Or not worth considering?


Luis -

This is a good question, I haven't seen any VMS manufacturers specifically call out application support for 4Kn drives yet.

For those unfamiliar, this post gives a good brief overview of 4Kn vs 512e drive layout options.

Given that 4Kn drives were very new and rare just over a year ago, and seem to only be available in 6TB and 8TB formats it will probably still be a while before VMS manufacturers add 4Kn support at the application level.

As many VMS's define their own storage "block" on the hard drive, I probably wouldn't want to risk going 4Kn without explicit support from the manufacturer, or at the very least a solid test that included overwriting old data and adjusting the VMS storage size a few times.

Sticking with 512e for now, where you get most of the benefits of the 4K block size, but the drive emulates the older 512B sectors is the safer bet.

Thanks, Brian. I only just recently started reading on this and admit I wasn't even aware of the new specs in the hard drive world. But it appeared it would be something that might (or might not?) affect video storage so wondered if VMS companies had any input or insight on it.

These 4kn drives sound really good for survellience storage!

Like jumbo frames for hard drives.

With the possible exception of MJPEG recording, I think video, because of its large file sizes stands to benefit substantially.

Thanks for the tip Luis!

These 4kn drives sound really good for survellience storage!

Be careful. The risk in going to 4Kn vs. 512e is if an application tries to do any sort of direct writing to the drive and is using the assumption that the drive has a 512B layout instead of a 4KB layout.

512e drives will give you most of the benefits of 4K blocks, but still emulate (thus the e) a 512B block. Going to 4Kn requires an OS that supports it (Windows 8 or later for example, not that much of a restriction), but also requires apps that deal directly with the disk also support it.

This may not be an issue with some VMS's, but I'd advise doing a thorough test if you care about losing data at random times.

Yes, only when supported/approved by the VMS.

But I think if it's supported, 4kn is the way to go, since 512e is just a stopgap and there are performance penalties for continuing to process in 512 byte sector sizes, as well as drive cost/complexity in the emulation.

Are there any VMSes that you know of that write directly to raw space, i.e. without file system or DBMS support? There must be, maybe embedded systems? Of course many use SQL Server, but SQL Server has support for 4kn.

Also, SQL Server has had performance/integrity problems with 512e drives and so recommends using drives with native sector sizes instead.

But I don't know of any VMS's that store video files in SQL. I believe they use SQL only for indexing and configurations, not video data storage.

...I don't know of any VMS's that store video files in SQL...

I wasn't implying any do.

However, both video data and/or indexes are critical to the system, no?

Would the VMS even care, I would be under the impression that the disk format would be an issue for the operating system itself, the OS just provides the storage to the VMS as a "disk" and the VMS does not really care what format etc.

If the VMS is using a file system abstraction provided by the OS, e.g, NTFS, then it should be fine.

Applications can also choose to manage raw storage space using low-level drivers directly. They may do this to implement their own caching/buffering scheme or for custom logging or sharing semaphores, etc.

In this case, knowledge of the logical sector size is necessary to make the system calls. In theory, if a vendor has written their code with a variable sector size or with run-time binding, they might not have to change much.

However, it sounds like a lot of these apps are written hard-coded with literals, i.e. 512, or (slightly better) compiler directives, e.g. #SECTOR_SIZE, so new versions need to be in place.

That said, I'm not sure which VMSes use raw devices directly, the ones I'm familar with use NTFS.

I'm not sure which VMSes use raw devices directly,

it'd be interesting to know if any of them do, surely this would bring pain when you are running behind a raid controller or something like that?

It'd also be interesting to see if any VMS have issues with the different formatted NAS's, but again are they not just provided as a Disk via the OS?

We received a comment from Milestone on 4Kn drives:

Milestone writes to the disks through Windows, so our software will work on all the Windows OS that our software supports and that supports the 4Kn drives (we support windows 7 and newer – and I think the same OS’s are the ones supporting the 4Kn drives)

If it gives you a performance benefit using these drives over the 512 or 512e drives or not we don’t know as we haven’t specifically tested it.

We suspect that it will give a better performance as we know the formatting of the drives influences the performance.

In our manual, we recommend to format the disks using an allocation unit size of 64KB as we know this improves the performances a lot.

Below is from page 78 in the manual:

Important: Milestone recommends that you use a dedicated hard disk drive for the recording server database to prevent low disk performance. When you format the hard disk, it is important to change its Allocation unit size setting from 4 to 64 kilobytes. This is to significantly improve recording performance of the hard disk. You can read XProtect Advanced VMS 2016 - Administrator Manual Management Client elements 79 more about allocating unit sizes and find help on the Microsoft website http://support.microsoft.com/kb/140365/en-us.

the only place I think you should beware is embedded hardware solutions where the VMS is actually built into a piece of hardware.( which wouldn't make a piece of software but whatever)

it would be like a NVR, they say they only except xTB size drives because the OS ( linux, unix, or windows) is built around a certain kernel with certain packages that allow only certain hardware to be swapped out or upgraded to.

Software apps that run "on" an OS shouldnt have a problem as the OS handles all the heavy lifting where the hardware is involved.

VMS that is built "into or as a part of" the OS may have severe issues with the new drives as updating a package just for the drives could brick the machine, or may not be possible to do without rewriting the OS/VMS hybrid.

but those hybrids have started dying out with the emergence of VMS's like milestone, and the like so it would be more of an issue for NVR's and proprietary storage options I would think.

True, I agree, manufacturers of the "storage" portion of video surveillance would probably have the biggest say- but I've just been used to the VMS developer and storage provider being the same.

Much of the comments so far are filling in the gaps I think and I appreciate it.

but those hybrids have started dying out with the emergence of VMS's like milestone...

How are they dying out exactly? It seems more and more companies are creating them, including Milestone.

I may have worded it poorly but I meant VMS's that are built into the unit as part of the unit , not storage built for VMS's like milestone is doing with their husky NVR's

I wonder if these Hybrids are just throwing their software on an OS anyway (just that the OS in question might be Linux so you would not know it's there)

wonder if these Hybrids are just throwing their software on an OS anyway (just that the OS in question might be Linux so you would not know it's there)

Yes, for sure, and almost as sure it's Linux.

And there is, IMHO, even less of a chance of the recording software in such devices utilizing raw partitions than in a classic VMS.

On the other hand, the exact OS architecture, i.e. CPU/kernel/libraries/drivers, varies enormously, and if a particular run-time implementation does not support 4kn, good luck making it so!

you can get some Indian/Pakistani programmers to cut you a custom UNIX/Linux kernel on the cheap I hear. maybe an NVR OS for a couple hundred US dollars.

most of the time its flashed onto a chip and locked out physically on the really cheap units.

the ones with up-gradable firmware though dont run full featured OS's like REHL or Debian styles. the have just enough to run certain scripts and programs loaded into them.

...maybe an NVR OS for a couple hundred US dollars.

buildroot and busybox.

I am video Product Line Manager at Genetec.

Genetec never produced any specific tests or qualification for video recording to 4KN drives.

By design, the Hardware Disk technology shall be transparent to us at the application level considering our VMS leverages Microsoft Windows file system, and therefore Security Center Omnicast doesn’t perform raw disk access nor does it define block size.

A solution built with 4KN drives would be supported like other storage technologies, while performance may vary based on the solution and vendor.