And just so no one else has the same problem, it was an issue with the edge storage sd card format of ext4. As soon as this was enabled with or without the same stream profile, we lost anywhere from 7-10 fps on the VMS.
Ryan, thanks for elaborating on the comment. When you say you 'lost anywhere from 7-10fps on the VMS', do you mean the frame rate dropped (like from 20 to 10-13fps) or? I am assuming you hit this problem and it took a while to figure out what was causing this?
Yes, that's exactly what happened. On a casino, no less. We caught it before the system went live, but talk about liability...
Specifically happened on a P3354, with a Kingston Class 10, running OnSSI
Did they ever confirm what caused the frame rate drop? For instance, did somehow Milestone re-set the frame rate lower or did the Axis camera automatically drop the frame rate when sd card formatted with ext4 was used?
I will say, speaking as an organization that tests lots of new features, that crazy bugs happen a lot across all product lines for new features. Unfortunately, edge storage / support still is in that category. I am not excusing this but anytime we test something new, we figure that something weird / bad is going to happen. It does not always but it's not uncommon at all.
Well edge wasn't the primary storage, it was backup. So that at least was good. But it impacted the performance of the main stream, which is what the VMS records off of. Basically, as soon as the sd card was enabled, my fps dropped in system status of Ocularis (Milestone). This has nothing to do with the VMS other than that was how I was conducting the fps investigation.
What was the ultimate solution? Did new firmware allow using edge storage without any fps drop or did you turn off edge storage as a backup?
No, in fact on the P3346 units rolling the firmware back was what narrowed the problem down. ext4 wasn't available on 5.20. It is only available in the newest versions. Basically just using vfat fixed the problem, until Axis figures out why the fps drops. My guess is that the ext4 format takes more processing power to condense, and the performance drop is seen in the fps.
Ok, thanks. Now it sounds fairly clear to me. Milestone/OnSSI weren't part of the problem but someone at Axis along the way suggested it might be them.
Returning to your opening question, "I've never heard of a vendor cutting labor checks to snoop out their problems.... Anyone have any luck with this?"
We actually had just this discussion in the old LinkedIn group where I asked, "If a product is found to have a bug, should manufacturers compensate integrators for wasted labor?" It got over 100 comments and I swear a handful of manufacturer death threats :)
If you are an Axis partner, I'd ask them for credit for future purchases. Explain the amount of labor involved and how it involved an Axis bug, especially if this was not listed as a bug in the firmware release notes when you were deploying this.
Cutting checks is not standard practice but the better manufacturers, when they realize that a mistake is on their end, will do something to compensate integrators impacted.
And what about Axis not having a backup and restore feature? Am I the only one that thinks this is just insane?
Ryan, for backup and restore, do you mean camera settings? As in, I'll dump the camera's current settings to a file and load then back later if need be?
Yeah, as far as backup and restore goes, there isn't any with Axis. Backing up of camera settings has been gone for a little more than a year, and no eta for release. Axis literally gave us a website in the phillapines that an engineer that works for Axis created for a temporary solution. Straight from my IT Engineer, 'That's not enterprise. It's not even small business!'
Ryan, I asked Axis for feedback on this. They are quite responsive so I am sure they will review and have feedback.
I have too. I actually opened a case, and their level 1 tech support tried to close it out by saying, 'it's not available, we don't know when it will be'. I said, escalate it and keep the case open until the feature is released because my problem isn't solved until it is.
One other thing you might ask them is why they only test their cameras with their own VMS. A tech specifically told me on the phone today that Axis only tests their cameras with their own VMS. I'm sorry, but are you too big to test integrations? Milestone, Verint, OnSSI, DvTel all test their software with multiple camera manufacturers- why does it not work the other way around? Especially when you are the largest! I'm looking for the philapines website now...
It was actually Singapore, and the site is down now. So all those backups we've done in the past with this third-world site are useless I guess. Notice the date is July 2012, and still no backup options 2 quarter of 2013...
Any word from Axis? My guess is they will give you their 'template excuse'. You can create a template.... Not the same!!
Ryan, what configuration settings are you missing from the camera that's not templateable?
I find this interesting, as their cameras run Linux. Everything is a file in Linux, so everything would be capable of going through a backup routine. I wonder even if you can pull everything via SSH? I'll have to test...
Maybe a list of files to backup in the camera would suffice? I would love to store camera configuration files like Cisco switch configs...
There's a ton. From SNMP and alerts- such as email alerts, ASCII, etc (which can be very time consuming on larger projects), camera names, ptz titles and presets (on ptz cams and encoders), theres more than just this small list, it's just late. I had to spend nearly 7 hours last week manually reconfiguring a bunch of the above which couldn't be done via templates.
I've wondered about the whole lack of configuration backup before aswell and mailed Axis about it.
I had a Q1755 at the time and since they're quite critical in the installation they're in I was wondering if I could easily get the configuration out of it and install it on a new one in case one died.
They send me a simple Java program which could export the configuration and import it as well.
It hasn't been working perfectly though, but it something. Still find it weird though that it isnt just standard in the programming.
I contacted Axis in Sweden via telephone last week and asked to speak with a Team Leader in their tech support department. I explained the necessity if having a backup and restore, and the team leader fully agrees with me. I gave her my international telephone number and she said that she would personally inquire within R&D as to when it is expected to be out back into their devices. This am I got the return call, only to be told that backup and restore is not something they will be developing or releasing anytime soon- nor do they ever have plans to have it again. I guess enterprise products just don't need enterprise features. Cisco not only has a backup and restore, but also has configuration wizard that is essentially the same thing as having templates. There are uses for both, however backup and restore is extremely critical compared to templates. John, did your poking around get us anywhere different on this?
(Disclosure: I am a Bosch employee) We do support saving/backup of camera configurations in our IP cameras and encoders. This includes PTZ cameras settings (ie. presets, tours, etc...). The only parameters not included in backup config files are: IP address; time server address; and multicast.
Additionally, you may find our free Configuration Manager software to be a useful tool for these backup processes - as well as collecting and documenting necessary camera/encoders settings by exporting into a .csv or Excel friendly format. Especially helpful with large systems.
Has anyone gotten anywhere on this? I appreciate the reach out by the Bosch employee, but the last I checked your ip cameras and encoders are stupid expensive. There are other companies out there that have the features needed, it's just troublesome that the worlds largest ip surveillance company has these kinds of shortcomings.
Ryan, Ethan sent you an email last week. Did you respond to him?
As for Bosch, their cameras are generally in a similar price range to Axis. They have the Advantage Line which is like the Axis M series and then their professional stuff which is like the P series (for both fixed and PTZ offerings). I am not saying you should switch, just that their products are not stupid expensive (e.g., from just last month, Bosch's new 16 channel encoder).
We priced out a casino with a Bosch solution and got killed by their $1800 ip dome. And their Autodome ptz series is way over priced, in my opinion. Ill have purchasing take another look I suppose. I never got an email, no. Did you get any response from Axis? Looking back 11 days ago you had said you were going to look into it on your end. All in all, we found more bugs than what was listed earlier. Ill update all of them tomorrow on this thread. Found a few issues with the new M3006 and M3007.
I just reforwarded you the email. Email email@example.com and firstname.lastname@example.org to discuss further.
I should clarify, we used the flexiDome and Dinion analogs quite a bit. We find their image quality for analog cameras to be above most others. Their IP line is just above most people's price range.