$59 Smart Camera Module (Pixy)

Beset by lawsuits, the video analytic industry has seen little innovation recently. Now, a new offering from Carnegie Mellon researchers offers a super low cost smart camera module, raising over $270,000 on Kickstarter. We talked to Richard LaGrand, founder of Charmed and CMU partner, about the technology and its possible applications. 

Background

LeGrand said Pixy was mainly developed to give engineers an affordable way to use computer vision. 

“When I was in school I remember I did some robotics and had to beg my advisor for $25,000 to get a computer vision system. That was considered the going rate for something that would give you the computation power of Pixy. What Pixy can do is pretty basic. If you want to detect a specific object like a charging station or chase after a ball, it can do that kind of thing."

The idea behind Pixy was to provide and affordable computer vision technology for hobbyists, researchers and engineers doing robotics. The project started in 2009 and is on its fifth version. It is 10 times more powerful than its first version and about a quarter of the price (currently $59 on Kickstarter, but the first backers got it for $39).

Watch their video demo below (warning: it starts slow but finishes strong):

Possible Applications

Pixy was built to interface with other devices:

“There are a lot of other ideas for things like bar code readers or QR code readers. It has a usb port so it could be used as a pointing device, but people are mostly asking if Pixy will work with specific algorithms so it's hard to tell what the application is,” he said. “One out of 10 emails asks if we can do face detection ... so we’re committed to providing that down the road (using the Viola-Jones algorithm [link no longer available]).” 

Could it be used for security applications? Possibly. But Charm's founder did not speculate on those applications. He said the main focus was to develop the technology and make it open source to the people couple apply them to all sorts of applications.  

“If you want to engineer a low cost camera like this, and be able to get frames out of the sensor, and do it cheaply, that’s been a hard problem. I think that people are responding to the fact that we have solved this hard problem and this is something they can use to leverage technology that's already been figured out.”

How it works

Pixy can detect hues and saturation, using an algorithm to assign unique color signatures to different hues. It stores those signatures in flash memory and uses them to process future images. 

It does not detect objects, but can detect colors and report its findings based on assigned values given to those colors or color combinations. This image show it detecting various colors:

However, the labelling is misleading because it does not know that an object is a ball or a box or a car, etc. Pixy cannot detect people or objects. User defined labels are added, matching colors to objects (e.g., red = ball, blue = box).

“You just press down on the button and hold down the object in front of it and it ‘learns’ the object you’re interested in,” LeGrand said. Pixy can store signatures for seven different colors and can detect hundred of objects within those signatures simultaneously.

Capability and Limitations

Detection Range

 “There’s not really a set distance. It has more to do with how much of the field of view it occupies,” said LeGrand. “Depending on the focal length of the lens, it could be a really small object.” The version of Pixy on the Kickstarter has a 2.75 mm lens and has a 75 degree field of view. 

Pixy processes information coming in from a 640X400 frame. As long an object takes up at least 4x4 pixels, Pixy can detect its color information.

Low Light

Charmed admitted that “You need plenty of light to get a good image. The sensor is a quarter inch sensor. So it’s not tiny, but not big either, but it’s pretty sensitive. What we rely on though is that you can train it. If you went from outside to a fluorescent lit environment, it might effect Pixy’s ability to identify that object, but since it’s easy to re-teach, it helps with that issue.”

Dark Colors

The technology does best with colors that have strong color signatures so browns and black are more difficult to detect, LeGrand said. “It’s more possible with brown because there is some hue in there, but it’s best to use an object that has a decent color signature.”

Potential Surveillance Applications 

While Pixy makes a neat demo and offers quite low pricing, given its limited intelligence (only making out certain colors and in well lit conditions), it is difficult to imagine mainstream surveillance use cases. 

Tech Specs:

• Processor: NXP  LPC4330, 204 MHz, dual core
• Image sensor: Omnivision OV9715, 1/4", 1280x800
• Lens field-of-view: 75 degrees horizontal, 47 degrees vertical
• Lens type: standard M12 (several different types available)
• Power consumption: 140 mA typical
• Power input: USB input (5V) or unregulated input (6V to 10V)
• RAM: 264K bytes
• Flash: 1M bytes
• Available data outputs: UART serial, SPI, I2C, USB, digital, analog
• Dimensions: 2.1" x 1.75" x 1.4"