Ex-Arecont Leaders Launch Super Resolution Startup Entropix
Sooo are they calling it a on demand FaaS (Forensics as a service) ? All these dumb cameras around and they get infused with Entropix technology become little minions that serve the Grid. They just take all the information in the Fast and Furious 7 kind of moment for common people to visualize. The Grid with all its computing power does all the Analytics, people counting, LPR/LPC etc. on demand as fast as google fiber can deliver.
It searches all your metadata and will predicts what you are going to do next. If it cant predict it will tell you what you are going to do.
Sigh if it only supported open protocols at the moment :)
Vincent, you may want to do marketing for them, lol
From what I understand, they are not working on video analytics right now. Their focus is on the super resolution element.
But, in general, I do think cloud analytics have potential. Related: Genetec Cloud LPR Examined
I'm confused, are they going to be using 'new camera hardware' or 'standard commodity camera hardware'?
I'm surprised you're not more skeptical on this one. For instance would you agree that:
"Resolution of video surveillance systems, which has been rising year after year, has finally hit a hard physical barrier of light diffraction limit beyond which it cannot increase” said Dr. Michael Korkin
Even if the pixel size is at the diffraction limit, there's nothing limiting sensors from getting larger, except for cost, which is not a 'hard physical barrier', but rather one that becomes easier to overcome as more demand creates greater economies of scale. Like in the recent larger sensors of Sony and Axis among others.
Not to mention multi-imagers solutions, which Dr. Korkin is obviously familiar.
We often bemoan companies who tacitly play off end-users CSI misconceptions, did you notice Entropix does this explicitly:
Hollywood creates expectations of extreme forensic zoom and video quality enhancement techniques... At ENTROPIX we’re excited about bringing those capabilities to surveillance customers at commodity pricing.
Do you think that 'one million dollars' in this day and age are likely to develop, produce and ship technology that will improve resolution 9x?
I'm not saying it's impossible, but it seems a long shot, no?
"I'm confused, are they going to be using 'new camera hardware' or 'standard commodity camera hardware'?"
New camera hardware means you can't load an app on Axis Q series (e.g.) and make this happen. Standard commodity camera hardware is their claim that whatever secret sauce they add can be done on 'standard commodity camera hardware'.
"I'm surprised you're not more skeptical on this one."
They have a track record of delivering on video surveillance products. I'd be more skeptical if it was a kid out of school or my neighbor down the block.
"Even if the pixel size is at the diffraction limit, there's nothing limiting sensors from getting larger"
Agreed. This is a prime risk for them. That is point #2 in my key questions section.
"Do you think that 'one million dollars' in this day and age are likely to develop, produce and ship technology that will improve resolution 9x?"
First, it's not a real-time continuous improvement. I'd be more skeptical about that, just given the computational load of doing this 24/7/365.
Secondly, if they make progress, they are going to get more money. Listen, IC Realtime just got $15 million for basically scotch taping 2 fisheye cameras together.
There's definitely risk here, but it is clearly people who have credible backgrounds and expertise to try it.
[NOTE: Poster is a Founder of Entropix]
Undisclosed A... Your skepticism is healthy and appreciated however you're making several assumptions about our technology that I'm afraid are inaccurate. John's post mentions that we are using super-resolution technologies, powered by massively parrallel GPU-based super computing. However that is only one layer of our multi-layered approach. (a key layer to be sure)
You also state that nothing is preventing sensor size from getting larger except cost and that economy of scale can solve that problem. That is not technically true, as those involved with making the 'wafers' of CMOS sensors are quite aware. The die for such wafers involves mathematics of wafer 'yield' that rise exponentially with sensor size and yield per wafer. The cost curve is much steeper than you might imagine. See graphic here:
Not to mention that if you go higher sensor size you have also exponentially raised the cost of lensing for such a camera. See Avigilon 7k camera for reference.
You also stated:
"Do you think that 'one million dollars' in this day and age are likely to develop, produce and ship technology that will improve resolution 9x?"
Again, you're making large assumptions that are incorrect. Where did you come up with the idea that we plan to "develop, produce and ship" our technology from this article? We raised a million dollars in seed round of funding to continue our work and expand our team. We are most certainly going to raise additional funding as it would be impossible to go into production of sophisticated hardware on mass scale without doing so.
Again, your skepticism seems healthy and we look forward to you getting to analyze and scrutinize our technology in person some time next year.
Hello Nathan.
...however you're making several assumptions about our technology that I'm afraid are inaccurate. John's post mentions that we are using super-resolution technologies, powered by massively parrallel GPU-based super computing. However that is only one layer of our multi-layered approach.
I'm not assuming super-resolution/super-computing is all you do, but it is all you say you do on your website. Where are the other layers mentioned?
Indeed the first sentence of your press release is
Entropix, Inc. secures Series Seed funding to develop GPU-powered super-resolution video surveillance technology.
As for your comnent
You also state that nothing is preventing sensor size from getting larger except cost and that economy of scale can solve that problem. That is not technically true...
You then explain yields and give further reasons why you believe that the cost of creating such higher resolution sensors will limit their adoption. Even if this is the case, (which is certainly debatable), this is no "hard physical barrier" that we've finally "hit". Remember the Entropix statement under question here is:
"Resolution of video surveillance systems, which has been rising year after year, has finally hit a hard physical barrier of light diffraction limit beyond which it cannot increase”
Please clarify this statement as it stands and then I will be glad to discuss wafer yields further.
In addition your "fisking" skips right by my comment about multi-imagers, and their ability to create high resolution cameras without larger sensors. What is the response to this objection?
Where did you come up with the idea that we plan to "develop, produce and ship" our technology from this article?
from the part that said
Entropix is aiming to start shipping products by the end of 2016
I assume you have to develop and produce before shipping?
We are most certainly going to raise additional funding as it would be impossible to go into production of sophisticated hardware on mass scale without doing so.
Here we agree, my comment was only pointing out the difficulty of meeting a ship date 20 months away with only this round. More funding is necessary, as you say.
Having said that, I am actually optimistic about the benefits of SR and don't doubt that Entropix will deliver value here. Most of the skepticism relates to your overly rhetorical and bombastic copy, not to the underlying technologies.
You're right: especially in IR, it's not that rare to find cameras who's effective resolution is less than their focal plane array pixel count, because their lenses can't focus a point of light down to a single pixel. Focal plane arrays have evolved to the point that, often, the lens is the factor that limits the total system resolution. Manufacturers could easily choose a lens that can focus light to the pixel size of the selected focal plane array; or alternatively they could easily choose a focal plane array that has larger pixels. But either option would increase price, which is counter to current market forces.
Enter super resolution. Video provides a stream of images, and some information within each frame is a little different than information in any of the other frames. Super resolution algorithms "boil down" the information collected across multiple independent images into a single image that contains some of that extra goodness. Collection over time (or over space if using multiple imagers) can provide higher resolution imagery beyond the Rayleigh diffraction limit of any single imager.
Speculating as to why it might need custom hardware, I'm guessing that compression discards detail that is mostly below the human perceptual threshold but that is important to the super resolution algorithms. If this is the case, then the words "forensic video quality" might be an implicit acknowledgement that streaming raw uncompressed video might exceed available bandwidth, but that it's no problem to export select clips whenever desired.
It sounds like an exciting new capability that could be very useful. I recall a relevant experience from the dark ages when I had a Commodore Amiga. For its time, it had unprecedented video processing capabilities. I was showing it off to a group that happened to include a prosecuting attorney. He asked me to assist him on a case which in part depended upon video evidence of marginal quality. We were able to digitize the clip and use the Amiga's image processing capabilities to improve the clip's clarity. He was so pleased that he came back for more assistance whenever his video evidence could benefit from improved clarity. This personal experience, plus the many cases in which IPVM members have asked for help with video of marginal quality, suggests that a rich market probably exists for this sort of capability. It's exciting to see a highly qualified and proven team tackle this sort of industry advance, and it'll be interesting to see what they deliver in terms of limitations, quality, and pricing.
"We were able to digitize the clip and use the Amiga's image processing capabilities to improve the clip's clarity. He was so pleased that he came back for more assistance whenever his video evidence could benefit from improved clarity. This personal experience, plus the many cases in which IPVM members have asked for help with video of marginal quality, suggests that a rich market probably exists for this sort of capability."
Horace, good feedback.
My main concern for this type of application is that on a per camera basis, this need is typically rare. In other words, if you have a 100 camera system, how many times per year do you really need to apply super resolution? Maybe a few at most?
To that end, I think it is pretty critical that Entropix makes it inexpensive and easy enough that people can justify using it or else you get in the Briefcam situation where most users say "Wow, that's neat but I can't just spending X more for every camera when I may only use it every so often."
You're right, of course. The special camera issue is huge. If compression is the issue, often camera bandwidth requirements are quite limited, and with today's network topologies, most of today's cameras could probably get a software update to stream raw to the server, but then what? Storage is getting less expensive, but when would it be cost effective to store two weeks or a month of raw video for every camera? I think it'll be quite a long time yet.
BTW, by now the technique of super resolution is well understood and the basics are pretty much a commodity in the research community. However, moving it into a commercial product can still offer many challenges. I wonder if someone, somewhere who owns an early patent is quietly biding their time until someone becomes so successful that they're worth suing? I hope these guys have sewed up that risk with ownership in some early intellectual property.
BTW, by now the technique of super resolution is well understood and the basics are pretty much a commodity in the research community.
Horace, do you know of any peer-reviewed papers involving actual optics where 2MP image was forensically increased to effectively 18MP? Or anything even close?
I'm skeptical, but would love to see them succeed.
1) I've not seen any super-resolution research papers claiming 9x resolution improvement in real world scenarios. Love to read such a paper if I missed it.
2) My impression, along with others in these comments, is that this cannot be done effectively after compression. This means it must be done in the camera which implies a large compute engine with the associated power and cost in the camera. A tough sell. Even an FPGA-based approach, which might beat a standard processor, won't be cool or cheap.
3) If it all works, then you still have the problem of commpressing 18Mpix at 30fps to send the super-resolution video to the NVR. I don't know of 18Mpix 30fps H.264 compresssion engines (they are coming, but aren't here yet). Plus storing 18Mpix at 30fps, without compressing it so much that you loose the benefits, won't be cheap.
I hope I'm wrong on point 1 and 2. SuperResolution gets very cool if it can enhance a 2Mp/30fps stream AFTER compression. That would be great.
[Note: Poster is from Entropix]
Craig, thanks for your comment!
1) I've not seen any super-resolution research papers claiming 9x resolution improvement in real world scenarios. Love to read such a paper if I missed it.
Regarding the super-resolution research, there is “classic” super-resolution which required multiple frames of video to work and relied on motion between frames, and then there is a more recent approach which does not require any of that. It does work in real-world scenarios producing very impressive results. We have developed it further, as well as the related camera architecture using the same image sensors and same processors as in today's commodity cameras. It relies on an off-camera computational power in the cloud that is orders of magnitude higher. To give you an idea of how much computation is involved, it would take minutes per frame to produce the same result using the computational power available on-camera.
2) My impression, along with others in these comments, is that this cannot be done effectively after compression.
Compression is always an issue, however our technology works under a typical range of compression settings, short of extreme levels of compression.
3) If it all works, then you still have the problem of commpressing 18Mpix at 30fps to send the super-resolution video to the NVR.
We do super-resolution on demand in the cloud from our camera's stream. The stream itself is fully compliant with standard decoders, however it includes an additional data component produced on-camera which provides for an extra resolution increase and is utlized in the off-camera computation.
Michael, I assume that even though you plan to use similar hardware in your cameras as others, that you are not able to use an off-the-shelf camera's API, to obtain the 'secret sauce' sensor data that you require.
VAPIX, for instance, allows access to image data before compression, but its sounds like the camera could easily be overwhelmed if you tried to do to much with it in real-time. Do you agree or is there another reason why you cannot have an embedded agent running on a camera's native OS obtain the needed SR data?
I'm asking only because looking superficially at the image enhancement cloud play, one would imagine the ease of adoption would be far greater without a required hardware component.
Lastly, without revealing any secrets, are you doing anything in the near-field as part of your innovation?
Thanks.
I'm asking only because looking superficially at the image enhancement cloud play, one would imagine the ease of adoption would be far greater without a required hardware component.
This technology is not "image enhancement cloud play", and not even in the category of "image enhancement". It is in the category of "hardware-enabled software". The new camera architecture is essential for computational super-resolution reconstruction to work, and vice versa. The value to be delivered to, and experienced by the user is a product of both.
"The new camera architecture is essential for computational super-resolution reconstruction to work, and vice versa."
To A's point, that is a business / market limitation, since your camera market share is 0%.
So either you now build up your own camera offerings or you find a mega-manufacturer to license or acquire your technology to embed in theirs. Both are possible, neither are simple.
John, we don't see this as a limitation -- to the contrary, this is a major market opportunity in a commodity market. A comparable value today is 30-50 times more expensive, not to mention the slow frame rate. If the frame rate were any higher, it would require 10x bandwidth / storage, and power / heat. All major selling points, like wide area coverage with a single camera, forensic image quality, improved low light performance. Plus, the effective resolution of the upper range megapixel cameras today is quite diffferent from their nominal megapixel count. In fact, under anything but a bright sunny outdoor conditions, a camera with 1.4 um pixels has a lower effective resolution than a camera with 2.8 um pixels, see this paper (especially Fig 1). Our offering addresses all of these points.
Not buying this. And you completely ignored my concerns.
Whether or not your technology is good or bad, since it cannot be added to existing cameras easily, it will require A or B above. If you think either are easy from a sales / business perspective, you are deluding yourself.
"A comparable value today is 30-50 times more expensive, not to mention the slow frame rate."
This is a classic Arecont style scam claim. It's wrong and misleading on multiple levels.
(1) You are not claiming to provide super high-resolution continuously. Your conventional super high res competitors (like Avigilon) are. You force users to request processing on-demand, connecting to a cloud service with unclear VMS integration. This reduces the value and increases the complexity versus a 'regular' super high-resolution camera.
(2) The claim about comparable value being 30-50 more expensive does not make sense. Let's say the comparable is an Arecont Pro camera at ~$6,000 to $10,000. This implies you are selling your cameras at $200 each with no recurring cost. I don't believe you can or will price your products at this level.
(3) More importantly, the 30-50 times claim ignores the point that almost no one chooses to pay $6,000 to $10,000 per camera (the market share of those cameras are something like 0.01%). Those cameras are not overall very attractive in the marketplace.
From the claims so far, it appears you will have a 'regular' HD camera with a 'resolution burst' mode.
It's neat but very few people are going to see it as delivering 30-50 times more value than the 4K / 12MP cameras that are already shipping and are not going to require a cloud service and requesting for processing each time they want increased resolution.
I respect what you are doing on the technology side but you seem to be greatly underestimating the business challenges of taking this to market.
If you think either are easy from a sales / business perspective, you are deluding yourself.
John, I recall that when I made a decision to join my last employer back in 2005 it was a five-person startup with zero market share. That was a few hundred million dollars in sales ago... The first multi-sensor panoramic cameras I designed were introduced at ISC West 2006. Everyone finally caught up by ISC West 2015, and this market segment I predict will grow to quarter billion annually within five years. Like they say, "There is no substitute for experience" :)
(1) You are not claiming to provide super high-resolution continuously. Your conventional super high res competitors (like Avigilon) are. You force users to request processing on-demand, connecting to a cloud service with unclear VMS integration. This reduces the value and increases the complexity versus a 'regular' super high-resolution camera.
Nobody in video surveillance ever needed the entire field of view captured and streamed at the highest resolution. This is done out of necessity because there are no other options. You are paying for it in camera cost, bandwidth and storage cost, power and heat, reduced frame rate, poor low light performance... and all of that is entirely wasted except for a small region of interest! Resolution on demand is a much better value proposition. The Moore's Law does not apply to image sensors and lenses, they don't scale in cost over time -- don’t expect an 8K camera and beyond to become available for $200. That is essentially what we are setting out to do.
(2) The claim about comparable value being 30-50 more expensive does not make sense. Let's say the comparable is an Arecont Pro camera at ~$6,000 to $10,000. This implies you are selling your cameras at $200 each with no recurring cost. I don't believe you can or will price your products at this level.
Entropix cameras will be priced well within the regular commodity camera price range. Nothing in the new camera architecture suggests high cost.
(3) More importantly, the 30-50 times claim ignores the point that almost no one chooses to pay $6,000 to $10,000 per camera (the market share of those cameras are something like 0.01%). Those cameras are not overall very attractive in the marketplace.
Exactly my point. Their market share is tiny, and they are not attractive at that price. Imagine if you could get comparable image quality and image detail at the price of a 1080 camera, and have that at 30 fps with great low light performance and WDR...
It's neat but very few people are going to see it as delivering 30-50 times more value than the 4K / 12MP cameras that are already shipping and are not going to require a cloud service and requesting for processing each time they want increased resolution.
The 30-50x value comparison was not in relation to 4K- 12 MP cameras, but to 7K/8K cameras and beyond. If you expect that a low-cost small optical format 4K - 12 MP camera gives you 4K - 12 MP effective resolution, then make sure it is a bright sunny day at 100,000 Lux. Otherwise you may be getting the same or worse than a 1080 camera resolution due to small-pixel low SNR and poor dynamic range, please refer to Fig 1 in the paper I linked to in my earlier post. Even some of IPVM's own tests seem to point in the same direction.
I respect what you are doing on the technology side but you seem to be greatly underestimating the business challenges of taking this to market.
Thanks, John! All three of us, the founders of Entropix, have been cautioned about this very thing on a few earlier occasions in previous companies we launched or helped build. If only we listened.. :)
What an arrogant answer that misses the point.
Your multi-sensor example is not appropriate to your new technology. Multi-sensor cameras are basically multiple cameras in a same box and are therefore easy to integrate. What you are proposing now is far harder to integrate. It's more like adding in Briefcam than multi-sensor. Good luck getting the big VMSes to spend money integrating your cameras with 0% market share.
"The 30-50x value comparison was not in relation to 4K- 12 MP cameras, but to 7K/8K cameras and beyond"
So now you are claiming 30MP super resolution by the end of next year?
You really should keep quiet rather than set expectations you can't deliver.
Nothing in the new camera architecture suggests high cost.
So no reset button?
This technology is not "image enhancement cloud play", and not even in the category of "image enhancement".
This is the wrap-up statement from your press release:
"Hollywood creates expectations of extreme forensic zoom and video quality enhancement techniques... we’re excited about bringing those capabilities to surveillance customers at commodity pricing”. Entropix.com
In addition, this is the genuinely impressive before/after image provided by Nathan, showing what you "are able to do".
If you want to call this technology "hardware-enabled software", that's up to you, but I think its a mistake to deride someone for using the term "image enhancement", since it is a natural and functional description that most anyone would use here. 'Hardware-enabled software' might play well with the VC crowd, but for most security pro's its a non-starter.
The snake-oil detector is shaking on the lab bench but has not started blinking. A million dollars sounds low for startup doing anything. If there were mathematical magic that made it so you could get 9x by shipping your pixels to the cloud I would think someone would know about it. Patented==bad, get over it, we know this from many many many repeated bad experiences. Yes I know the investors love it. I work for the customers, not the investors. Never underestimate the amount of magic you can pull out of that whole "cloud" thing. It's not weird to hear engineers calmly talk about summoning thousands of processors with zettabytes of storage. Processor per pixel, anyone?
Net conclusion: money sounds low, having trouble beliving the hype but no reason to challenge it. Yet. Not thrilled with the lineage.
I work for the customers, not the investors...
Well, they are looking for a Director of Engineering:
This is an excellent ground floor opportunity to become a key early player in a recently funded technology startup, currently based in Los Angeles area and running in full stealth mode.*We are looking for a Lead Developer / Director of Engineering with both creative vision and technical know-how.
*First rule of 'full stealth mode' is...
Not thrilled with the lineage.
Hi Rodney. I'm curious...
Dr. Korkin was the VP of Engineering, author of numerous camera patents and lead inventor of the first ever full line of H.264 surveillance cameras ranging in resolution from 1.3mp on up to 40mp and including several largely unique multi-sensor cameras. Before that he was the founder and CTO of a company that was listed in the Guinness Book of World Records for having created an FPGA-based massively parallel AI super-computer for Japan's Key Technology Center.
I left that same camera company to found a software company that launched and grew an enterprise VMS product into a profitable and respected international company doing business in nearly one hundred countries.
So with those two resumes coming together to co-found a new cloud enabled camera hardware and enterprise software massively parallel super computational imaging startup... What I'm curious about is what "lineage would thrill you?" :)
"Not thrilled with the lineage."
I don't want to speak for Rodney, but he could be alluding to the two of you having worked for Arecont. As you might have heard, Arecont is not the most beloved manufacturer in the industry....
So by High resolution, they mean actually doing this properly and using correctly sized sensors and Lens? that'd be amazing, remake the Bosch for $600..
or we just continue to wait for hik to do it?
Right, Undisclosed A. Thanks for prodding me from vague pontifications to real-world particulars. LOL. I got the sense that you already knew what to expect, so I did a bit of digging and I've not found credible reports of resolution improvements approaching 9x.
Using Google, I'm not seeing recent work on the limitations of superresolution, which tends to suggest that the matter is pretty well settled.
A 2002 paper by Baker and Kanade of Carnegie Mellon suggest that significant improvements can be made by modelling recognized characteristics of the image. For example, if a text is recognized as Times New Roman 12 point and the letters are recognized, one could argue that it is possible to reconstruct the original to an infintessimal resolution. However, their paper only achieved about a 2x improvement for text, which is comparatively easy to recognize. They concluded with a fact that analytics users might already know -- that recognition has been impractical for complex scenes such as surveillance video.
A 2004 IEEE paper coauthored by Lin and Shoum of Microsoft examined the question, "Do fundamental limits exist for superresolution?" For multi-image superres that is restricted to linear transformations, they concluded that real-world improvements would likely not exceed 2x, and even synthetic improvements (eg improving images that have been degraded with known techniques) would likely barely exceed 5x. They suggested that other methods such as rotations, nonlinear transforms, and frequency based analysis, might marginally improve these expected outcomes.
Mr. Korkin, are you able to shed any light on "a more recent approach which does not require any of that?" Thanks!
Horace -- if you interested in the theory of it, I would like to refer you to this seminal 2012 paper: http://statweb.stanford.edu/~candes/papers/super-res.pdf. If you are prepared to commit to its 48 pages of math, there is a hint there suggesting that a higher super-resolution factor may be feasible based on a more recent insight that was unavailable to the authors you cited, just a few years earlier. National Science Foundation awarded the author with its highest prize calling his research “nothing short of revolutionary”.
Having said that, please note that we are not exclusively relying on computational super-resolution alone to produce the final result under real world scenarios, the overall super-resolution factor is partially achieved via the new camera architecture itself. Also, note that the factors in these and other papers are linear, while we are using square values: in terms of these papers our factor is in the range 3x-3.3x.
...in terms of these papers our factor is in the range 3x-3.3x.
A picture (or two) is worth a thousand words.
Can you share a before and after image?
Certainly that would be interesting.
Undisclosed A... Here is a picture demonstrating what we are able to do. The original full scene image is 1mp (720p). The top forensic zoom window is the native digital zoom from the original resolution. The bottom zoom window shows what the same image looks like after run through our post processing engine and achieving a 9x increase in the effective pixel count and useful resolution for that area.
This is an example of what we are looking forward to being able to achieve in near real-time for surveillance end users and essentially why we refer to it as a sort of "CSI" button on-demand. It is also scalable up to any common resolution available on the market right now including 4k native. In order to show you these native video clips and post processed full clips, I'd need to get you under a signed NDA.
The stream itself is fully compliant with standard decoders, however it includes an additional data component produced on-camera which provides for an extra resolution increase and is utlized in the off-camera computation.
Curious, for this 1MP image, what is the size in bytes of the 'additional data component'?
More super-resolution technology, this one slated for Fujitsu phones later this month.
Came across this video on Nvidia’s website from back in March 2018.
Nothing really new on their website though.
3 years is a decent amount of time to get a startup going. Anybody know what’s up with them?
optics and pixels are not all that a camera can process. looking forward to more 2019 technology.
Nathan still posts regularly on LinkedIn regarding Entropix. I have seen some demo images but haven't seen anything saying they are ready for market.
3 years is a decent amount of time to get a startup going. Anybody know what’s up with them?
Certainly enough time to get a startup going but that's not all we were doing. We were also inventing an artificially intelligent solution for bypassing the laws of physics. Specifically light diffraction limits. Such endeavors can take a bit of time. Especially when you're not venture funded and bootstrapping your way to success.
The science was proven 18 months ago. The software was operational 6 months ago. The product will be available for purchase in six months. Assuming of course we get funding early this year to go to mass production and launch it. If we don't, we will continue licensing it for non-surveillance customers such as robotic vision applications like we currently have, and will release a security product of our own once sufficiently funded with licensing revenue.
Happy to provide anyone who reaches out to me directly with more information. We're only really working with strategic partners or specific regional resellers interested in being first to market in their region or channel with it. If that's you... I'd be happy to have a conversation about it.
Best,
Nate
Especially when you're not venture funded and bootstrapping your way to success.
I thought you had two rounds already?
The science was proven 18 months ago.
Science has a pesky way of getting around. Have you been able to secure your IP while preparing your commercial entre?
I thought you had two rounds already?
We've raised less than $2M in seed funding only. First was an angel round in early 2015, followed by a seed note round which is an ongoing raise for us and less than a mil. We are actively pursuing a Series A round of funding of several million now.
Have you been able to secure your IP while preparing your commercial entre?
We have two granted patents, 3 years of internal R&D, and a whole lot of neural network secret sauce in our software. Our hardware is all made with commodity components so there's nothing secret about it. We gladly share our camera spec with anyone and actively encourage manufacturer's to build to it because the 'magic' lives in the GPU software and our vast library of trained and proprietary neural networks.
We gladly share our camera spec with anyone and actively encourage manufacturer's to build to it because the 'magic' lives in the GPU software and our vast library of trained and proprietary neural networks.
In the time since you last updated us, the prospect of NN residing at the edge is becoming viable. Has this changed your model of a heavy hardware, pay per clip, cloud based SR to possibly a self-contained, always enhancing, SR camera?
Entropix presented at Nvidia's GTC recently, below is the embedded video of the presentation:
@Nathan,
Interesting video. One question though, for several minutes in the opening you bemoan the use of Bayer color masking (and the resultant signal attenuation), and implying that the mask exists solely because of the way visual perception occurs in humans, and that what’s good for human vision isn’t necessarily good for computer vision.
To overcome this degrading human accommodation, your design adds back a monochrome sensor, which when combined with the Bayered sensor allows you to increase resolution substantially thru your pipeline.
question though, if a system didn’t have to be concerned with human perception, for instance a vehicle tracking system that was used only to generate textual records about vehicles that pass thru a point, would the bespoke camera use just one monochrome sensor, or would you use two mono, or?
regardless, how would such a mono-only system be able to determine the color of the car?
I agree it's an interesting video. I would take issue with any suggestion of "degrading human accommodation". It's actually a case of, this is the best we can do with a sensor that isn't absurdly expensive.
The human eye has about 24 stops of dynamic range, and estimates of 480 MP to 576 MP resolution and higher. This article gives a decent summary of the human eye/brain combo and what it is capable of.
https://www.forbes.com/sites/quora/2016/10/06/what-is-the-resolution-of-the-human-eye-in-megapixels/#393f9b545912
Anybody mistake a cat for a human lately?
The human eye has about 24 stops of dynamic range, and estimates of 480 MP to 576 MP resolution and higher. This article gives a decent summary of the human eye/brain combo and what it is capable of.
disagree.
first off I have to object to the estimate’s misleading false precision, “480 MP to 576 MP”, not 577?
secondly, assuming we’re speaking of the resolution of a fixed gaze (allowing for the eye’s micro-tremor movement but not major FOV changes, like a PTZ), I think 500 MP is way high.
a simple experiment for you to try. right now as you read this, center your gaze right here at the word FOCUS. Then without moving your eyes or changing their focal length, try to read the sentence above this one, then the one above that, etc.
you will quickly find it impossible, even though you can “see” the peripheral sentence. on the other hand a 10 MP image of your screen at a natural angle of 50 degrees easily captures all the words on the screen in sufficient detail, in one instant.
one might argue, sure the detailed FOV is smaller than a camera, but in that area the resolution is higher. but consider that you are unlikely to count individual pixels on a smart phone with your eye, and that is nowhere near 500 MP.
I think 500 MP is way high.
Yeah, about that eye chart test. What if you do it with someone with severe nearsightedness? How about someone like a baseball player with super 20/10 vision? You're going to get very different results. (Not that the test isn't useful in terms of choosing security cameras.)
Also, this test also ignores what I just commented on, the human eye pair's field of view, and what the brain does with it. Here is the better link, referencing multiple scientific studies:
http://clarkvision.com/imagedetail/eye-resolution.html
In looking at a high resolution print, which would come the closest to your eye chart test, the estimate is 74 MP. (I believe there is a typo in the parenthetical where he refers to people being able to sort photos by ppi. The last value should be 1200 ppi.)
The 576 MP full FOV explanation is just below that.
If you're still not liking this definition of human eye/brain resolution, try looking at some pictures of a large natural wonder of some sort, such as the Grand Canyon. Then go look at it with your own eyes and see if the pictures accurately conveyed the scene in full detail.
What if you do it with someone with severe nearsightedness? How about someone like a baseball player with super 20/10 vision? You're going to get very different results.
Or you could ask what it was for someone with 20/400 vision. The point is for 'normal'/ideal 20/20 vision.
There are 2 studies in the link you provided, both studies return 404 / not found.
A better way to put it would have been to say you're testing in a way that I don't believe is used for cameras, lenses alone, or for camera sensors alone. The line pairs method Roger Clark mentions is very common. This link works. It's for lens testing, but of course there will be a fixed measuring device such that the discussion applies for a complete lens/sensor system:
https://www.cambridgeincolour.com/tutorials/lens-quality-mtf-resolution.htm
Roger Clark refers to studies/measurements where he does not provide links, and for at least a couple he mentioned in the text he did not bother to put them down in the references section. I see that the 2 links he does provide are dead.
In a quick search, I did not find a free version of the cited piece by Blackwell, but you can buy it here for $35. Member cost is only $15:
https://www.osapublishing.org/josa/viewmedia.cfm?uri=josa-36-11-624&seq=0
Apparently it has been used heavily in the years since, based on the pdf available at this link:
https://www.researchgate.net/publication/325416263_Visibility_of_targets_Beyond_Blackwell
Whether you like the MP number, or the number is way off, you're confusing what the brain is currently focused on versus the entire scene with your sentence reading exercise. If you're going to do that, try looking at a Waymo vehicle and all the stuff on it that is there to try and equal a human driver. (Granted, a human driver that is actually looking at the road instead of a cell phone, for example.) There is a LIDAR, multiple radar units, and multiple cameras. Why is all that needed versus one set of eyes and one brain? Because the brain can very quickly ignore things that the "AI" must continue to track. Again, only refuting your point on focus.
Why is all that needed versus one set of eyes and one brain? Because the brain can very quickly ignore things that the "AI" must continue to track.
we’re not talking AI. we’re talking optics/sensors corneas/cones.
you human eye statement was quite specific:
The human eye has about 24 stops of dynamic range, and estimates of 480 MP to 576 MP resolution and higher.
I don’t know what the “resolution of the brain” is. the brain will fill in for all sorts of missing data, like for instance the blind spot right in the middle of your retina caused by the optic nerve. but it’s just filling in.
Whether you like the MP number, or the number is way off, you're confusing what the brain is currently focused on versus the entire scene with your sentence reading exercise.
In the exercise, my brain IS focused on the peripheral sentence, it’s only my eyes that are focused on the word FOCUS because of their positioning and focal length. Despite my brain’s focus I am unable to read the peripheral sentence, because the optics and the retina are not providing sufficient information to do so. if I move my eyes, then I can see it fine.
there are less than 10 million cones (the receptors used in color and sharp focus) in any case, so even with Nathan’s 9x super resolution, it’s still off by an order of magnitude.
the problem is the eye and the retina provide a varying degree of resolutions from center of focus outward. more like the way an 180 degree panoramic resolution quickly diminishes, but even quicker.
Your comment about FOCUS does indeed refer to AI, not total human sensory resolution. Your position suggests that one could look at a series of pictures of a scene broken up into segments, one at a time, and it will have the same visual effect as seeing the whole scene at once. Try an experiment with a few dozen viewers and see if they agree.
And you can't separate the brain from the eye, either. As noted in the presentation, and as you note above, indeed the brain can do all sorts of processing that factors into human final resolution. The presentation included human brain processing because it is relevant to the final evaluation. Read the analytics reviews here and watch the videos/look at the pictures, and see how many times you make a mistake versus where the various systems made mistakes, in spite of the fact that a camera focuses on the entire scene at one single time with equal resolution.
if you don't like the estimate by Roger Clark, or the visual acuity and other studies he cites as part of that estimate, you probably need some sort of study of your own to counter them.
And you can't separate the brain from the eye, either.
that is exactly what the question “what is the resolution of the eye” is trying to do.
and although you can hem and haw about the brain doing this and that, it’s still reliant on raw data coming via the optical nerve from the eyes.
and whether it’s a nerve or an Ethernet cable, it’s still just an encoded sequence of electrical impulses conveying information.
essentially, we’re comparing the amount of information generated between a retina on the one hand, and an imager on the other.
here’s another way to think of it:
if and when an eye could be hooked up to a computer, what would it look like?
As noted in the presentation, and as you note above, indeed the brain can do all sorts of processing that factors into human final resolution. The presentation included human brain processing because it is relevant to the final evaluation. Read the analytics reviews here and watch the videos/look at the pictures, and see how many times you make a mistake versus where the various systems made mistakes, in spite of the fact that a camera focuses on the entire scene at one single time with equal resolution.
don’t confuse concept with clarity.
the brain does an incredible job at interpreting the input that it receives via the visual cortex. but that doesn’t necessarily mean it has incredible resolution.
an experienced installer can spot a shoddy install with just one look. that’s not resolution, that’s conception.
finally, what is the resolution of your mind’s eye?
I will be sure to look for citations of "Undisclosed #1" the next time I do any research on human vision.
Meanwhile, since your vision is limited to one sentence at a time, please do not drive a vehicle. I ride a road bike, and my safety often depends on all those other human drivers that can see the road, the other vehicles on the road, the instrument panel in their car, road signs, signal lights, etc., and me on my bike, all within a very short span of time.
Meanwhile, since your vision is limited to one sentence at a time, please do not drive a vehicle. I ride a road bike, and my safety often depends on all those other human drivers that can see the road...
next time you ride your road bike try not moving your eyes...
A more subtle point about Entropix tech is that humans and machines see the world very differently when it comes to deciding what is a high quality image. What humans perceive as a high quality image may not be perceived as such by computer vision, especially when using the latest technology of deep convolutional networks. The reverse is also true: images that produce higher analytics accuracy are not necessarily perceived as higher quality by the human eye. This may sound counter-intuitive, but this is exactly what we have found, and some of the recently published computer vision research confirms that. Some go as far as saying that human and machine perceptions may be anti-icorrelated. So, when it comes to improving video analytics accuracy, it is best to tune the imaging system to computer vision itself as the primary "consumer" instead of relying on eyeballing the video as always.
"So, when it comes to improving video analytics accuracy, it is best to tune the imaging system to computer vision itself as the primary "consumer" instead of relying on eyeballing the video as always."
what does this mean?
while I think I agree with your premise (maybe), I'm not sure that I can trust a machine to make an innately 'human' decision when perceptual nuance is involved.
am I wrong?
I'm not sure that I can trust a machine to make an innately 'human' decision when perceptual nuance is involved.
Then I guess you will never agree to ride in a self-driving car :) Machines are quickly becoming the primary consumers of pixels, and in the coming years they will become the only consumers of pixels. The "innately human" decisions will be made by the analytics of all kinds, whether it is just alerting humans or making the decisions.
"Then I guess you will never agree to ride in a self-driving car :) "
wrong.
I am a loud and proud advocate of removing the human equation in most day to day things - like self-driving cars... after all, it is the human that will always be the least common denominator when it comes to passionate (meaning perception-based) choices dictating any reaction.
Machines are far better at such things - primarily because the 'human perception' (or lack there of) can't be expected to be able to actually analyze the amount of data that machines can - at the pace that machines can perform this task.
This doesn't mean that humans - and their ability to fathom nuance - is not a factor in the ability to analyze data - at least at this time.
what do you think?
I can't speak for Michael Korkin, but I would suggest talking to Waymo about machine choices and what is needed for self driving cars. Uber, maybe not so much:
https://www.reuters.com/article/us-autos-selfdriving-uber-idUSKBN1GV296
I agree that the machine needs different input than the human. I'm not so convinced that the machine will become the decider in the near future.
This "Moral Machine" exercise, from MIT no less, is so simplistic and contrived that I find it ridiculous.
https://www.wonderoftech.com/mit-moral-machine/
It makes you wonder about some of the people designing autonomous vehicles.
A more subtle point about Entropix tech is that humans and machines see the world very differently when it comes to deciding what is a high quality image.
which humans are we talking about exactly? Photogs? Teenagers taking selfies?
but in the sense that we are talking, forensic identification/face rec for example, done by an trained FBI agent, is the difference as large?
I’m sure there are many times when computers and humans each have their optimal results using different pictures.
but to suggest that they are anti-correlated is to imply that in general things like high dynamic range/low noise are not helpful to both, or motion blur is not generally harmful.
again I’m sure there are many outlier examples that would be quite striking, but do you think it’s right to say there anti-correlated in the general case?
An example of how humans see things differently: we see the school bus on the left and on the right with no perceivable difference. Same with the dog example. A deep neural network sees an ostrich on the right in both cases, with high confidence. Object classification may change dramatically with very small pixel modifications.
Photogs? Teenagers taking selfies?
but in the sense that we are talking, forensic identification/face rec for example, done by an trained FBI agent, is the difference as large?
A trained FBI agent would not see an ostrich here, I think.
The anti-correlation between visual and machine perception is a separate but related issue: when using traditional image quality metrics, such as PSNR and SSIM that are widely used in assessing image quality, images with higher PSNR and SSIM have lower visual quality than images with a lower PSNR and SSIM synthesized or processed by deep networks.
when using traditional image quality metrics, such as PSNR and SSIM that are widely used in assessing image quality, images with higher PSNR and SSIM have lower visual quality than images with a lower PSNR and SSIM...
this is great news for crappy camera manufacturers everywhere!
;)
Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.