# Announcing libobscura

Let me formally introduce you to my most recent overly-ambitious experiment: *libobscura*.

**Libobscura is a friendly library to use cameras on Linux.**

At least that's the goal.

## What does "friendly" mean?

• *It's hard to use it wrong.* No segfaults. Errors guide you to the right track.
• *Point-and-shoot.* If that's all you need, you get a RGB buffer in ten lines of code.
• *It's easy to add support for new devices.* Great documentation and a good internal API are the goals.
• *It's easy to contribute to.* Send patches using the web interface, not a mailing list.

TL;DR: with the simple buffer API you can get a frame in 6 calls, and map it to CPU in another 2:

```
let cameras_list = vidi::actors::camera_list::spawn()?;
let cameras = cameras_list.cameras();
let camera = cameras_list.create(&cameras[0].info.id)
    .expect("No such camera")
    .expect("Failed to create camera");

let mut camera = camera.acquire();
if let Ok(ref mut camera) = camera {
    let mut stream = camera.start(
        Config{fourcc: FourCC::new(b"YUYV")},
        4
    ).unwrap();
    loop {
        let (buf, meta, _next) = stream.next().unwrap();
        let mmap = buf.memory_map_ro().unwrap();
        let data = mmap.as_slice();
    }
}
```

Go to project examples for more.

Figure 1: Libobscura will never be friendly enough for every audience.

## What cameras?

Any webcams, industrial cameras, image sensors working exposed by the V4L2 interface are currently in scope.

## What does "experiment" mean?

There are already other libraries for camera support on Linux. You can use the V4L2 APIs directly, or use libcamera, or libmegapixels.

They all strike various middle points on the power vs user-friendliness scale. Having worked with all of them while developing camera support for the Librem 5, I never got the impression that any of them are particularly easy to use.

Libobscura is an experiment because it tries to find an API that fulfills the needs of most people and remains hard to use wrong.

Figure 2: My "Perfect tool" conjecture. Imagine you have a perfectly well useable tool covering some of your needs. If your needs grow, the perfect tool covering those tools cannot be easier to use than your old tool. And humans are imperfect. This applies to APIs, as well. I put my rough idea of where I think a couple of examples fall. Libcamera tries to be as general as possible, so it's on the far right. OpenCV has Python bindings, so it's up top. Libobscura starts out a bit to the left and top, and I hope to push it to the right, to explore how many use cases can be added while staying easy.

Perhaps it's impossible to improve on what current libraries do. But now libobscura is a space where it's possible to try out radical changes without bothering the maintainers of existing libraries – for example, to try out an entirely new approach.

## Radical approaches

There are a couple of radical approaches in libobscura that earn it the "experiment" status.

### Rust

Rust is a memory-safe systems language. Libobscura uses it to ensure that the APIs are hard to use wrong. A Rust API built with a little care does not let you crash with a segfault – the compiler will alert you before you can create problems.

The Linux kernel started its own Rust experiment in 2020. Linux is a low-level project. Camera support is a low-level topic (many people working on libcamera also work on the kernel). If Rust is interesting for Linux, then it's interesting for libobscura.

### Get RGB data

A camera library can't be described as easy to use if the pictures it gives you need to be processed before displaying.

The typical format to display data is RGB while cameras return either Bayer or YUV data. With libobscura, the goal is to provide those conversions transparently, without any extra effort on behalf of the user or camera backend.

### GPU acceleration

The conversions might not be implemented in hardware, but don't worry, libobscura has your back! It comes with a GPU-accelerated image processing library called crispy-img.

Why GPU? The idea started with the Librem 5, which will often record video while running on battery. Offloading this work to the GPU is a clear win, and I expect most Linux video-capable devices to have some form of GPU. And we're open to a CPU decoder if you want to contribute!

Figure 3: An artist's impression of a pixel format conversion shader.

## Status

Because libobscura is only two months old as a funded project, the current status is "proof of concept". There's a safe and limited user API, another more tricky but zero-copy API, the GPU accelerated library can handle some conversions, and camera support is relatively simple. USB camera demo works.

But there are still goals to achieve:

• change controls like brightness or focus,
• transparently integrate GPU processing,
• have a way to choose your preferred output format,
• implement auto-* algorithms when the camera doesn't have auto mode,
• replace LGPL-licensed pieces from libvidi and crispycam (not great for link-time-optimized code)
• add Pipewire integration, so your browser can just pick up your camera feed for teleconferencing,
• add Librem 5 support as a realistic, useful verification of concept.

Those will be the next steps for the project. First implement all the functionality and then extend support for more devices. This way we can catch corner cases in the API that are bound to appear with unusual setups.

## The future is YOU

I'm making libobscura in the hope that it will be useful to people. As a base to build software, or to ship devices, or to learn what software architectures fit this problem.

When I sent my funding request to Prototype Fund, I didn't expect to be taken seriously. After all, what motivated me most was that it was a cool challenge. Apparently they believed in me more than I did, because I got funding until March 2025.

What happens until then and what happens next depends on how useful this work actually is to YOU. The ultimate goal of any software is to be useful, otherwise what's the point?

So I invite YOU to analyze it, try it out, give feedback, experiment with it, copy the concepts, contribute to docs, illustrations, and code, fork it entirely! Regardless if you come from the Mobile Linux community, or Raspberry Pi, or you have a laptop with this IPU thing, or if you're from libcamera. When you improve, libobscura fulfills its goals.

So see you on the project page!

## Thanks

Thank you to Purism, for inviting me to the Linux camera work and funding the base which I'm using now.

Thank you to all the libcamera people! I stol... was inspired by libcamera's architecture, and you also answered my countless questions both during Librem 5 development and recently.

Thank you Martijn Braam for showing how to create simple config files for many mobile phones.

Thank you BMBF for providing the funding money, and thanks Prototype Fund for connecting me to it :)