Key Moments
PhotoTechEDU Day 25: Open-source-based high-resolution cameras
Google TalksWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
Open-source modular cameras offer high-resolution video and customizable software for a fraction of the cost of commercial alternatives, but require significant engineering effort to utilize fully.
Key Insights
The founder of Elphel, Andrey Filippov, was inspired to create open-source hardware after witnessing the limitations and difficulties of proprietary Windows systems for multi-camera synchronization in 2001.
Elphel's cameras integrate processors, FPGAs (Field-Programmable Gate Arrays), and dedicated memory to handle tasks like JPEG compression and, in newer models, Ogg Theora codec, achieving resolutions like 1280x1024 at 30fps.
The company's commitment to open-source extends to licensing, with all software distributed under the GPL (GNU General Public License), refusing commercial licenses to maintain the open nature of their designs.
Achieving higher frame rates with high resolutions is a multi-faceted challenge, involving limitations in the sensor, FPGA processing bandwidth, network transmission, and the direct system memory transfer speed.
Elphel is developing an Ubuntu-based distribution to simplify the user experience, embedding development tools like Eclipse and offering pre-configured IDEs to lower the barrier to entry for customization.
Synchronization of multiple cameras can be achieved via network or wired connections, with wired connections offering negligible latency (microseconds) but requiring additional hardware, while network synchronization incurs latency due to the non-real-time OS aspects.
From Windows limitations to Linux innovation
Andrey Filippov, founder of Elphel, Inc., began his journey in hardware development in 2001, moving away from proprietary Windows systems due to their limitations in managing multiple firewire video cameras simultaneously. He observed colleagues struggle for weeks to set up a system that he later saw could be achieved much faster with Debian Linux, albeit with initial challenges. This experience led him to believe in the power of open-source and Linux for hardware integration, prompting him to quit his job and establish Elphel, focusing on building hardware around Linux. His initial foray involved using Atmel's (now Microchip) XScale processors, chosen for their ability to run standard Linux, a network bootloader for easy firmware updates, and readily available SDKs and documentation. Within three months, with minimal prior Linux experience, he developed a camera capable of serving JPEG images, a feat that previously took half a year with embedded assembly language. This early success validated his vision of leveraging existing software to accelerate hardware development.
Early modular designs and FPGA integration
Elphel's initial projects focused on creating networked cameras. The first iteration, a gated intensified camera module, was developed with a former employer's needs in mind and was not a general-use product. Following interest sparked by articles on LinuxDevices.com and Slashdot, Filippov shifted to creating a more general-purpose network camera. This second project incorporated a configurable Xilinx FPGA, not just for controlling internal camera functions but also for processing, such as JPEG compression. The decision for Xilinx over Altera was partly due to their software licensing policies at the time, which offered more long-term flexibility for developers. This modular approach, separating the sensor board from the computer board (housing processor, memory, FPGA, and network interface), allowed for easier upgrades and product lifespan extension. The computer board design was kept consistent across different camera models, with variations primarily on the sensor board, enabling reuse of significant FPGA and software code for new camera projects.
Scaling up resolution and adopting Ogg Theora
As demand grew for higher resolutions, Elphel developed subsequent camera generations. A key project addressed a client's need for cameras with 10-plus megapixels, a significant leap from the earlier models' capabilities. This camera utilized the same processor board but required a more complex sensor board due to the CCD sensor's needs. To overcome performance limitations of the processor's lack of hardware checksum calculation, two processor boards were sometimes used, with each handling half the image, simplifying development by reusing code. The third generation featured increased memory and FPGA capacity, enabling video compression at 1280x1024 resolution and 30 frames per second. Filippov chose to implement the Ogg Theora codec instead of MPEG due to MPEG's complex and prohibitive licensing fees, which at the time were charged not just per device but per megabyte-second of stream. Ogg Theora, being royalty-free, aligned better with Elphel's open-source philosophy (GPL2, moving to GPL3). This implementation focused on static scenes as it lacked motion compensation, a feature he hoped to enable with future, more powerful FPGAs.
Hardware bottlenecks for high-speed, high-resolution imaging
Achieving higher frame rates with high-resolution cameras presents several limitations. Filippov identified the sensor's pixel clock, the FPGA's processing capabilities for compression, and the network bandwidth as primary bottlenecks. Additionally, the data transfer rate between the FPGA and the system's main memory (DMA transfer) is critical. Current implementations require multiple memory cycles per data word, limiting throughput. Newer processors support bus acquisition, which could theoretically increase this transfer rate fivefold by decoupling memory from the processor, but this requires careful clock synchronization and controller implementation. The sensor technology has also advanced, with newer 5-megapixel sensors capable of higher frame rates (up to 15fps) than the current FPGA compression code can handle (around 11fps), although FPGA code can be optimized. The network bandwidth, especially for uncompressed or lightly compressed formats like JPEG, also becomes a limiting factor, dependent on image detail and quality rather than just pixel count.
Open-source philosophy and licensing
A cornerstone of Elphel's operation is its unwavering commitment to open-source principles and licensing. All software developed by the company is distributed under the GNU General Public License (GPL). This means that while the designs and code are freely available, they cannot be incorporated into proprietary products without also making those products open-source. Elphel explicitly states they do not offer alternative commercial licenses, even when approached by other IP camera companies interested in their designs. This principled stand ensures that their technological advancements remain accessible to the wider community for modification and further development, aligning with the spirit of collaborative innovation that drove Filippov to start the company.
Simplifying development and user accessibility
To make their powerful and customizable cameras more accessible to a broader audience, Elphel is actively working on simplifying the development environment. Traditionally, users had to compile C/C++ programs and FTP them to the camera, or delve into FPGA design using tools like Xilinx's. Elphel is creating an Ubuntu-based distribution that can run as a live DVD/CD, providing a working environment out-of-the-box. This distribution will include pre-configured development tools, potentially within an Eclipse framework, allowing users to recompile or modify code with a lower barrier to entry. They are also adapting software to run PHP in fast CGI mode for web interfaces and aim to expose more low-level functions accessible via PHP scripts, making the camera's capabilities more manageable for web developers and hobbyists. This initiative is crucial for fulfilling the goal of truly enabling users to shape the cameras to their specific needs.
Multi-camera synchronization strategies
Synchronizing multiple cameras is essential for various applications, such as 3D reconstruction or high-speed event capture. Elphel offers two primary methods: network synchronization and hardware-based synchronization. Network synchronization relies on packet timing over the network, which is simpler as it requires no additional physical wiring. However, due to the non-real-time nature of standard operating systems, there can be inherent latency and jitter. For applications demanding the utmost precision, Elphel provides hardware synchronization through opto-isolated inputs and outputs that connect directly to the FPGA. This wired approach bypasses network latency, achieving negligible delay (microseconds), but necessitates extra cabling and connectors, introducing potential grounding and noise issues that opto-isolation helps mitigate.
Mentioned in This Episode
●Products
●Software & Apps
●Companies
●Books
●Concepts
●People Referenced
Common Questions
Andre Philippov founded Alphal after encountering limitations with Windows systems for multi-camera synchronization tasks. Witnessing the power and flexibility of Linux and free software, he decided to build hardware around these technologies, leading to the development of open-source cameras.
Topics
Mentioned in this video
An operating system that Andre Philippov decided to build his hardware around, finding it powerful after initial challenges.
An architecture of processors that Andre chose for his early cameras due to their ability to run normal Linux and having a hard-wired network bootloader.
Synchronous Dynamic Random-Access Memory, used as additional memory for frame storage and buffering in Alphal's FPGA-based cameras.
A hardware description language used for designing FPGAs. Alphal's FPGA code is written in Verilog.
An image compression standard implemented in Alphal's cameras, achieving significant frame rates at 1.3 megapixels.
An application that inspired Andre's approach to the camera's web interface, aiming for a richer, more application-like experience within a browser.
A simulator used for FPGA development, with a limited free version and a commercial option.
An integrated development environment (IDE) framework that Andre envisions for Alphal cameras, to simplify the development process for users without prior FPGA experience.
A popular Linux distribution that Alphal is now switching to for its distribution to allow for live DVD mode and easier installation of software.
Motion JPEG, a video compression format used in Alphal cameras that generates a lot of data and can be limited by network bandwidth.
Dedicated memory used for buffering and image processing in the newer generation cameras, not sharing bandwidth with the system bus.
A server-side scripting language that Andre is porting to Alphal's new cameras to make low-level functions accessible to a wider range of developers.
Software Development Kit provided by XScale, which was comprehensive and allowed downloading documentation from one place.
A tech news website where articles about Andre's cameras gained significant attention and interest from the public.
The GNU Debugger, a tool used for software development that Andre aims to integrate into a pre-configured IDE for easier development on Alphal cameras.
A Linux distribution that Andre's colleagues installed, taking several weeks to get running but proving powerful for multi-camera setups.
A royalty-free video codec that Alphal implemented in their cameras, chosen over MPEG due to licensing complexities.
A free and open-source simulator for the Verilog hardware description language, which Andre is using to convert his test fixtures.
The operating system Andre Philippov used initially, which presented limitations for connecting multiple firewire cameras.
A set of video compression standards that Andre considered implementing but ultimately avoided due to complicated and expensive licensing.
Small computer boards used by Andre's former colleagues that struggled with connecting multiple video cameras.
Charge-Coupled Device, a type of sensor used in an earlier high-resolution camera project that required more support circuitry.
A future FPGA release that Andre hoped would allow for full implementation of motion compensation in their cameras.
A type of processor that Andre mentioned as being too slow for viewing high-resolution video from live DVDs.
A current Xilinx product line that uses 65nm technology, which Andre noted for its potential.
Field-Programmable Gate Array. Used in Alphal's cameras for control, image processing, and compression. Andre experimented with one-time programmable and reconfigurable FPGAs.
The company founded by Andre Philippov that develops open-source cameras.
A manufacturer of sensors. Andre mentions a new 5-megapixel sensor from Micron that runs faster than their current compressor code.
A competitor to Xilinx, also manufacturing FPGAs. Altera's licensing model required a choice between free trials or potentially unusable code later.
The company behind Ubuntu, which Alphal is working with to create a distribution that supports their cameras and includes hardware development tools.
A company manufacturing configurable FPGAs. Andre chose their products over Altera due to licensing policies.
A large company in the camera manufacturing industry that Andre initially felt he couldn't compete with for high-resolution camera projects.
A major camera manufacturer whose products were being evaluated by a potential client, leading Andre to develop higher-resolution cameras.
More from GoogleTalksArchive
View all 48 summaries
58 minEverything is Miscellaneous
54 minStatistical Aspects of Data Mining (Stats 202) Day 7
45 minKey Phrase Indexing With Controlled Vocabularies
63 minMysteries of the Human Genome
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free