Wayland input method project post-mortem

Input methods on Wayland have a bit of a chicken-and-egg problem.

That topic has been my main occupation for the past year, thanks to NLNet, and thanks to the fact that I was a cause of some of the problems in the first place.

But the time has come to move on, and take a less active role. So here's a summary of what I did, what I didn't do, and what I think about the whole topic.

Chicken and egg

Long story short, the chicken is users and the egg is implementations.

An input method is a very user-facing thing. It's already in the name: it's used for input. It's for the user to interact with the computer. Let the smartphone user type, the Japanese speaker write Kanji. Propose autocompletion from a dictionary. Use the right layout or language. Stay out of the way when unneeded.

For such a user-focused feature, the work I did was exceptionally user-unfocused. Tech demos instead of user testing. There was no one to use what I did every day, or even show me an example of a daily task. What I created isn't enough even for myself. I couldn't empirically evaluate if what I did was an improvement.

Why?

It comes down to manpower and demand.

Manpower

Most application software people use daily are built using the GTK or Qt toolkits. They most likely communicate with the input method through Mutter or Kwin as the compositor. The third relevant component to me is an Input Method Engine (IME), like fcitx or ibus.

To test my work on someone's daily workflow, I'd have to modify all 3 components and find a user who uses that specific combination. Then take them through the trouble of setting up the patched versions on their system.

It's not impossible, but for a single person, it's infeasible. Right from the start of this project, taught by the lessons implementing text-input in GTK and wlroots, I knew that I'm too stupid to write C/C++ code, and I can't afford the time debugging what I would have written. So I selected a Rust-based stack of smithay, floem, and, uh, my own input method because I couldn't find any (now I know about kime – a bit late, though). Sadly, pretty much no one is using that combo, but at least it made proofs of concept possible.

Demand

But that's not the only way to get a working stack for users to test. What if someone else did the work? What if the relevant projects decided to implement my experiments themselves, to let their users test? I submitted all the protocols as official experiments in wayland-protocols, after all!

But that hasn't happened.

What gives? My suspicion is that most of the time, there isn't an expert in the area who could do the work. The rest of the time, the improvements aren't seen as worth the extra effort. Users aren't burning developers' effigies, demanding better Kanji. Linux mobile users aren't flooding bug trackers with complaints that the keyboard shoves their apps around for no good reason (it totally does). Linux Mobile is Android, after all, right? (No, it's not.) Android doesn't use Wayland.

A bit more puzzling is the relative silence of the Chinese users. But only a bit. Think about it: they are on the other side of The Great Firewall. The other side of the English-Chinese language barrier. The other side of the chasm between the Western and the Eastern cultures. How can we not expect to see a disconnect?

Good enough

I think the most important reason driving this lack of demand is that the existing solution is good enough. Input methods on Wayland work 90% of the time. They fall apart only 10% of the time, and everyone knows the last 10% is 90% of the work.

Unfortunately, that last 10% also contains the difference between "kinda sorta works" and "a pleasure to use". The latter has been my motivation for the past year. And common programmer's hubris. Are you saying, I can't do that???

So let's take a look at the 90% we already have, and then move on to guess at what the remaining 10% could look like.

The 90%

After reading this, you'll have a pretty good idea what each feature is necessary for.

I'll mark features relevant to the mobile use case with an M, and those relevant for input methods with an I.

• IM: Send text
• IM: Select a language
• IM: Announce text purpose
• IM: Display pre-edit
• IM: Less ambiguity in corner cases
• IM: Compatibility
• M: Display a panel
• M: Cursor navigation
• I: Send key events
• I: Grab keyboard
• I: Style the pre-edit
• I: Display a popup

Common features

Purpose and language selection inform the input method what's expected inside the text field. This is such a generic thing, that honestly, it could even be useful with a keyboard and nothing else. Automatic layout switching between windows is already a thing, with the input method protocols it could be per text input field, and determined by the application.

Pre-edit is a piece of text that you just typed, but which may change and is not yet considered final.

Chinese, Japanese, Korean (CJK) input methods use them to help composing the characters.

On my Maemo, it's used to display the most likely autocompletion.

Features mostly useful for CJK input method

CJK IMEs often show a popup window next to the typed text, with several candidate outcomes. This is called a candidate window, which needs to be allowed in the protocol.

When the IME is paired with a keyboard, it needs to prevent the application from getting some key events. It wouldn't be helpful if, when you typed "[kou](https://en.wikipedia.org/wiki/W%C4%81puro_r%C5%8Dmaji", the text editor received "kou". You want to stop those events, and give them to the IME, which can turn them into "こう". This is what they keyboard grab is for. At the same time, you still want to forward events like "Ctrl+C" to the application, so that's why we have sending key events.

Features mostly useful for mobile text input

No matter if you're using a QWERTY input method or MessagEase, on your mobile, you need to display a special panel to be able to press the buttons. (If you thought about T9, you can feel smug now. You still need an input method, though.)

Hitting an exact position in the text with your finger that's 3x the size of a letter on the screen is still hard, though, and application toolkits are failing to make that useable, so cursor navigation, which makes it possible for an input method author to come up with something better, remains relevant.

Complication

That would be about it. Except those features are a bit spread across different protocols, some of which aren't even sanctioned officially. So you can't have both popups and a panel on your system at the same time.

And what about the other 10%?

The other 90%

If the results of my work as they are today were adopted and displaced all the existing protocols, we'd get the following improvements:

CJK IME

Control over the position of the popup window. Maybe you are typing right-to-left. Maybe up-to-down. I don't know, but the experimental popup uses the same positioner mechanism as the popups you see after right clicking. In this protocol, you can even have multiple popups!

Actually not an improvement because a dedicated panel window is missing from experimental. That's an intentional oversight coming from lack of manpower. Sorry. Layer-shell works OK for me.

Another missing feature is pre-edit styling. That one is missing because no user ever came to me to have a chat how they use it and why it is important. And less features is less complexity = more better. Lack of demand in action.

Then, there's no language information sent from the application to the IME. That's an oversight that should be fixed, although in such a way that doesn't favor the officially recognized languages, but instead lets the speakers define the language they use, no matter how obscure or disorganized, without having to beg an authority for recognition.

No more keyboard grab. No more generating keyboard events

This is another missing feature, but it's actually an improvement!

You see, behind keyboard event handling hides a realm where things – mostly key maps – wait for any misstep you made to turn your day into a hell of edge cases.

It's a source of real life bugs that had been reported to me unofficially, as "the Wayland input method authority".

Consider the happy path: the user presses Ctrl+B. The IME forwads the key press, to the application, and the application makes text bold.

Let's make things more complicated. The user presses Ctrl+Б (same key as Ctrl+B, just on a Russian Phonetic keyboard). The input method forwards the Б key sym, the application is confused and can't figure it out because it actually listens to key code 56.

Maybe we should send key codes then? Okay: the user presses Ctrl+Б (Russian Phonetic). The input method engine forwards the key code for Ctrl+Б, but - oops - in the regular Russian key map, which is key code 59. Not 56. The application is confused.

Those examples represent bugs. The one app could translate the keysym back to keycode (but what if there are 2 valid solutions?). The other input method engine could use the correct key map. But why open the opportunities for all those bugs where all you really want is forward the exact events the user already pressed?

So the experimental input method protocol provides a filtering mechanism instead. The IME can choose to intercept button presses that help typing your Kanji. Those will then not reach the application, and things work as expected.

As a bonus, this makes it easier to support improvements to the keyboard protocol like server-side key repeat, which solves more problems.

No more generating keyboard events, again

I hear you scream: "foul! Stop! Generating keys is still useful! Think of automating the desktop! And mobile keyboards need to send the »enter« key when submitting text!" To this, I reply: don't worry, there's a better way out.

I wasn't actually done complaining about keyboard events handling. In this Pandora's box, you can't know the consequences of your key presses in advance. There is no standard mandating that pressing Ctrl+X results in removing text. There isn't even a standard saying what to do when you press the humble "a"! Applications could do anything they want based on the current key map, locale, and the phase of the Moon, if their authors so please.

Trying to predict any of that in an input method engine is folly. You'd need to know everything about every application. Or cover 90% and hope for the best, but we're trying to be better than that.

So for the mobile use case, there are actions. Currently, there's only one: finish editing. Works like pressing "Enter" would: submit the field, or go to the next one.

There are more ideas outside of experimental, depending on the use case: adding more generic actions, defining a shortcuts protocol, and fixing virtual-keyboard. Keep reading, they are described later in this post.

Across the board

Consolidation. While you still kinda have to choose between popups and panel, the experimental protocols are designed to be exactly compatible with text-input-v3 and its future developments, where the (precious little) social momentum seems to be.

Atomic updates group requests into units, applied all at once. Going from "blog pose|" to "blog post|" takes two logical steps: remove the last letter, add "t". If the IME submits them as two separate steps, the application doesn't know the intended output. As it receives the first step, it will draw and display "blog pos|". On receiving the second step, it discards that, and draws "blog post|". The first step ends up a waste of time and battery life. In extreme cases, it could lead to flickering as, for example, the candidates popup changes position to follow the cursor.

With atomic updates, the application can safely wait until the end of the command to start doing its work.

---

The protocols so far make a dangerous assumption: that requests get received immediately. What makes it dangerous is byte indexing.

Whenever there is a need to change cursor position, the new position is specified as a byte offset. This works well if the part of the system choosing the offset knows exactly and without delay what text is being indexed.

But Wayland is a distributed system. After the IME receives the contents of the text field, and before it chooses a byte offset, the text can change. With a bit of bad luck, the new context may fall inside a UTF-8 code point. The result could be trying to delete half a character like "я". Complete nonsense.

Existing protocols don't say what to do, but the experimental ones remove such ambiguities.

Compatibility

This is a bigger problem, so it deserves its own section again.

With the protocols evolving, the need to stay compatible between the IME and the application arises.

In this model of input method, the compositor acts as a little more than a forwarder of events between the application and the input method. All is fine if they use matching protocol versions – ones that were literally made for each other, like text-input-v3.1 and input-method-v2.1. But what if they don't? Wayland allows clients to negotiate any protocol version they like, up to the highest one supported by the compositor, so there's no guarantee they match.

Imagine that (1.) the application lands on a higher protocol version and the IME on a lower one. In the best case, the user won't be able to take advantage of the app's features. Worst case, the protocol semantics differ slightly (which Wayland allows) – for example strings are in UTF-8 in version .4, but version .5 switches to UTF-16 – and both sides can't communicate, the user can't type and throws their useless laptop at the developer.

What about the opposite situation? 2. The application uses a lower protocol version, and the IME uses a higher one, that the application never heard of. Even if the above doesn't happen, the user may try to use some feature in the IME that the app just doesn't understand. Not a good look – it would be good to at least hide those. But that needs another communication channel.

So that's my solution: to have the compositor – which knows who uses which protocol version – communicate a compatibility level to the IME. In addition, when the protocol contains optional features (like different actions), it directly communicates what it can handle, so the IME can not present the unsupported options to the user.

But this is not a perfect solution. It only addresses the case where the application is less capable or older than the input method. After a little brainstorming with other Wayland developers about two-directional capability negotiation, it was suggested that it's too much trouble to implement: it would mean that every application library needs to implement an unusual, complicated version negotiation mechanism.

Instead, we cut the Gordian knot by requiring that the IME is always up to date.

This is a trade-off, and like all trade-offs, it comes with downsides. The users can't keep using unsupported IMEs any longer. Also, when shipping a distribution, all shipped IMEs must be updated or removed before any compositor starts supporting newer protocol versions.

Addendum: key forwarding.

Forwarding keys through the IME actually shares many of the above considerations, regardless if we're talking about event filtering or grabbing all and re-generating some key events. The IME needs to understand the events to make a forwarding decision, and the application needs to understand what the IME sends, so they must agree on a common set of functionality.

Thankfully, keyboards are a lot better understood than input methods. The protocols change a lot less, and it's clear what the base functionality is. Still, the protocols sometimes change, like the addition of server-side key repeat.

With grab and re-send, the common features were defined in the input method protocol directly. That's simple, but you need to redefine everything keyboard-related.

The filtering protocol lets the IME and the application use the keyboard protocol. The only alteration is that key events may be withheld. That should cause only few situations where sides can't just communicate. As a downside, the compositor must figure out how to make it work when they can't.

While making the protocol, I was aware of this tradeoff, so this protocol is separate from input method and could be rejected if the consensus is that the tradeoff sucks.

Input method version 3.2

What do we gain if we don't go experimental yet, but do take in the v3.2 improvements?

With it, the application takes part in setting the visibility of the on-screen keyboard panel, if there is one. I think it's something Android applications like to do.

There's a small improvement in synchronizing the displayed windows that fixes edge cases.

It allows the IME to take over displaying pre-edit for applications that really don't want to do it.

Also, it gives the application more say in what kind of text the input method engine should provide it. While it's useful to send information like numbers-only, I expect this attempt to be primarily used in a way that makes things worse for the user. But hey, we don't have any input method user testing, so there's literally zero empirical information to help make the right call.

It also completely neglects the IME counterpart protocol, where things like popups live.

Future

Of course, there's more. There are a few users who made their wishes known.

What to do when the text field loses focus while pre-edit is ongoing? Korean writers expect the pre-edit to be turned into permanent text. Chinese writers don't.

Some Chinese writers even suggested that the IME could come back to the same state when focusing the same field again.

That leads to wider persistence considerations. If I always chat with Дима in Russian, I'd like to always have my preferred Russian layout and the Russian language dictionary activated. The application could supply stable text input field identifiers for the IME, so the IME could save my settings permanently.

The keyboard Pandora's box could be closed back again with more work around shortcuts and actions. Think copy, paste, undo. There's a bunch of actions already defined in XKB in the form of keysyms (XKB_KEY_Undo, XKB_KEY_Redo, XKB_KEY_Menu, and so on), but it would be nice to have a protocol that's separate from keyboards entirely so that we don't try to extend it back into the problem we're just trying to avoid.

Further along that way, using global shortcuts or something similar, we could eventually build a system that allows generating arbitrary actions, while also preventing shortcut clashes (because all shortcuts would be registered centrally).

I saw type-to-search functionality being requested: where the application doesn't focus a text field, but the field will get focussed as soon as something is typed in. Thinking about it deeper, text-input-v3.2's ability for the application to hide the panel seems like it could solve it: for the purpose of the IME, the field is focussed, but no panel is shown.

Guesses

Things that no one asked for, but I think input method is the right place for them, are:

• speech to text. I think all is in place for that already.
• handwriting recognition. On a train, I saw a lady write on a tablet with a stylus. The text turned into typed text automatically. I think we'd need to define another popup type, which covers the whole text field, and which the IME can make transparent to receive stylus events. This definitely needs more thought.
• gamepad input, and other unusual input devices. I want my computer to be a proper game console! So does Valve, and they already do it. But Valve uses their own input method protocol. Hey Valve, why not join the standards track?

Non-goals

The broad umbrella of user input is broad. Some things look like they share concern with input methods, but the commonality might be not so big.

One important out-of-scope case is keyboard emulation. If you look at an on-screen-keyboard from an input method and one used to emulate an array of switches, you may not immediately see the difference. Looking deeper, one may have buttons like "copy", the other will have "Alt" and "Control".

Apart from needing a panel, though, those are entirely different things. My input method effort completely ignores keyboard emulation. It would be just too much to handle.

But I can offer you a hint if that's what you need: fix the virtual-keyboard protocol. Rip out key map switching, and make it always follow the system key map. Switching key maps is a headache and the reason why the v1 of the protocol was not accepted in wayland-protocols. Go ahead and try to implement it and you'll see why.

Typing in terminal windows with an IME is also out of scope. We're back to the problem where key presses have no standardized consequences. When you type ":q!", will you get this text or will you lose all your data? Nobody knows, because there's no protocol to inform the input method whether it's in text edit mode.

If you wanted to correct this, you'd basically need to take all I did in Wayland and redesign it for VT100. Or whatever protocol terminals use these days.

Sorry, fellow hackers, I'm not motivated enough to do that. I code in Kate. Implement dumb keyboard emulation instead.

Upstreaming

This effort will live or die depending on adoption, and adoption survives on upstreaming.

All along, I kept in touch with KDE through their input goal. They aren't yet implementing the experiments in their on-screen keyboard or in Kwin, so I hope this writeup helps clarify the benefits.

My own experimental IME is, sadly, useless. It's just too slow for practical use. If you're familiar with integrating egui and you want to fix reinitializing the GPU on every redraw, you can be my friend.

When upstreaming things to toolkits, I had generally good experience. Thanks, winit maintainers! Thanks, floem folks! But I only managed to add some things missing from text-input-v3.1 before running out of steam, so experimental improvements haven't been tried yet.

I made the least progress on the compositor side, where even MRs bringing it up to date with base Wayland protocols have been languishing for months.

Perhaps the lukewarm response to my work means that it comes before its time. Maybe what we need is a breakaway success of Linux Mobile, which would push input methods onto the main stage. Maybe we need a large business cough like Valve cough to do their own thing while contributing to the upstream.

Or maybe we need to wait until X11 stops being supported and the previously working ways to input CJK text stop working, causing riots that are too hard to ignore by the developer community.

What next?

Myself, I'm taking a back seat now. I plan to push this forward only when I feel particularly inspired – after all, I have other things I'm burning to do now.

So, will input methods become a pleasure to use on Wayland? Now it's up to you – the community.

This is a translated version of a talk I gave at P.I.W.O in June, with cleanups and adjustments for the blog form.

Free Software hasn't won

…that doesn't sound right. I made the slides in Inkscape, on a computer running KDE and Linux, I use Firefox regularly. But maybe that's just me. What about you, are you using Free Software? Hands up! [hands go up in the audience] Of course! What nonsense, "Free Software hasn't won". Someone replaced my slides, hey conference staff!

Staff: The other folder.

[Browsing to a directory named "other folder", opening file called "your slides dimwit.pdf"]

Now, those are finally my slides.

Hello audience, my name is Dorota, and I'm going to talk about how

Open Source has won.

And that's not recent, the news has been out in 2008, and has been regularly repeated since by reputable press: ZDNET, Linux Journal, Wired, and so on.

Those press articles list a multitude of examples to prove it.

Linux, Ruby, Red Hat, uh, GitHub? Does that mean I can download GitHub and run it on my own server? Microsoft? Come on, that's some kind of a joke. Those slides are manipulated! So what else do they contain? Oh, this quote is all right:

Open source won. It’s not that an enemy has been vanquished or that proprietary software is dead, there’s not much regarding adopting open source to argue about anymore. After more than a decade of the low-cost, lean startup culture successfully developing on open source tools, it’s clearly a legitimate, mainstream option for technology tools and innovation.

Oh, the name of the quoted person is wrong. Looks like an attack on my reputation! Anyways.

The point is, if we want to build something new, using Free Software is not a hindrance. And thats super important, because software is eating the world. What does it mean? It means software keeps appearing in areas where there used to be no software before. That, in turn, means that we're slowly giving up control over more and more areas of life to those who made the software. After all, their software controls those areas of life from now on.

That's why it's great that there's always an alternative available, that we can select software that is free which grants control to us, and not just its manufacturer. So we have alternate operating systems made with Linux. There are very many programming languages to choose from. We can choose one of the open games, or graphics or audio creation software without resorting to closed software.

Similarly, we don't need closed software to print in 3D, or to build a mobile computer (also known as smartphone) or a smart watch. There are graphics cards which run completely free of closed firmware (Upon asking nouveau devs, they confirmed they wrote some firmware. Nvidia Kepler from 2012 is the last model where free firmware is allowed). There are such bicycles! Pretty much everyone owns one that rides without closed software. There are also sewing machines:

There are comms systems:

There are cars, and have been for a long time:

There are hard drives:

[the slides go blank]

There are wireless headphones, TVs... [slides remain blank] wait, something's wrong. There are phones! [Slides stay mockingly blank, noise of frantic clicking.]

[…]

[…]

[…]

Crap. This isn't right.

Oh, now I get it! The only kind of a phone that grants us openness is an analog phone. That reminds me of the time we were building the aforementioned Librem 5. There was a problem finding the modem for it. The reason is, one company controls the necessary patents necessary to connect to cellular networks. That company can and does impose arbitrary conditions on anyone using their integrated circuits. That made it very difficult to find modems that match our needs, and at the same time any reseller is willing to sell us. The resellers worried that by passing them on to us, they could break some distribution rule and not be able to get any modems in the future.

So that's a no. But I know what will be open for sure.

Richard Stallman started an important project called GNU in 1983. In one of his interviews, as he describes how he started the project, he mentions a certain device that his university bought, but which didn't work very well. He wanted to improve it, but no one wanted to share the device's sources with him. That was an offense! Why wouldn't anyone share the code?

What was that device?

That was a printer. Considering that the GNU project started in 1983 and that the story's from 1981, it works out to over 40 years of fighting for printer freedom. So let's reveal our open alternative.

Oh come on. This cannot be. Have you ever used a printer? If you even manage to find a driver, if you even manage to connect to the printer, then it's still going to print single-sided black-and-white when you asked double-sided color. And despite having to put up with that, Free Software people still haven't gotten frustrated enough to solve the problem once and for all? Unbelievable.

I have a theory. People who say "Open Source has won" are only taking into account a small part of what software is out there. Take a look at this list: it's a map showing which kinds of software force you into running something closed (bold) and which have open options available (italics).

• Applications: Blender, Firefox, KiCAD – Twitter, YouTube
• Operating System: GCC, Apache, OpenSSL
• Kernel: Linux, Zephyr, FreeRTOS
• Firmware: Coreboot – modem, GPU
• Appliances: Prusa 3D, Airgradient – washing machine, TV

What picture does this paint? Things programmers care about directly, like the OS and the kernel, are quite well covered. Whatever we need, there's an open version. Applications are also more or less fine. There's a Web browser, there's creativity software. The problem appears when you try to participate in social media. Sure, there are alternatives. But Mastodon, or PeerTube are separate networks from the closed ones, so they won't help much when trying to reach people who aren't yet using them.

Looking at the lower layers, like appliances or firmware, there seem to be options. But those options are limited to a couple niches, and with most things we buy, like a TV or a PC component – sorry, pal, there's simply no choice at all.

All the firmwares in the average laptop

How many processors are there in a typical laptop? By "processor" I mean something that needs its own software. For example, a GPU has its own processor that needs software, or a hard drive, or a keyboard. Here's a diagram of my personal estimate of what separate components need software:

I estimate there are 10 to 15 separate processors on a typical laptop. Just the graphics card may host five of them.

What does that mean for free software? Normally, all that's open – Linux, drivers, applications – all of this is confined to the main CPU. Now imagine you want to use this operating system through some human-friendly interface, like the touch screen or the keyboard. Those are running closed software, so if you want to enter any sort of data on your average laptop, it's game over: you can't make a move without dependence on closed software.

Same story with the graphics card. You won't display anything without closed software. What a fail. Okay, let's ditch keyboards and displays because this is a server. But that's a fail, too: to communicate over a network card, you still need software that it's running and that hasn't been opened. Suppose that we managed to somehow solve this problem. We hit the wall anyway when we try to store data: SSDs as well as HDDs are running their own closed software. I haven't heard of a single case of open software ever running on a storage device!

But that's not even the worst. The peak of lameness is the processor inside the processor. Have you already heard of Secure Boot? It's a piece of BIOS that is loaded onto the processor inside the main processor before the main operating system. Secure Boot allows the manufacturer choose which software the user can run. A similar system exists on Android phones to lock them to a particular system. Manufacturers of Android-based phones are not shy about restricting what the user can run on their devices.

That runs against user's freedom!

User freedom exists only when the Four Freedoms of Software are upheld:

• 0: freedom to run the program for any purpose,
• 1: to study and change it,
• 2: to share copies of it,
• 3: to improve it and share the improvements.

…but those are just words. Who cares about that? This theory is only ever going to be relevant to us computer experts, right?

Except… could it be that you're the family's tech support expert? Does your uncle/mum/grandma come to you carrying their malfunctioning Android phone, hoping for you to make everything right again?

And have you ever disappointed them? Has it already happened that their phone was simply too old and unsupported to be useful any more? Have you already told someone they need to pay up to replace a phone that seems perfectly functional?

Sadly, that's what the Android manufacturer support timelines say: typically after 4, exceptionally after 8 years, they will no longer release security updates. That makes devices too insecure to use, and turns them into e-waste.

What does Free Software have to do with it? I don't know, but my Lenovo laptop, 13 years since release of the hardware, the operating system is still receiving regular security updates. I suspect this has something to do with the lack of a boot loader lock and the openness of all the drivers. That's unlike Android. Even if there's no explicit lock, the drivers are so rarely open that the community rarely has the manpower to create a custom ROM for a given device.

Rug pulls

The couple hundred bucks that your aunt might need to pony up to get a new phone pale in comparison to how much you need to pay for some cloud-only devices. Some cloud-enabled gadgets don't let the user choose an alternative provider for services the device requires. What happens when the company shuts down the online service? Of course, the device becomes an expensive brick. Imagine someone setting your 2300 bucks on fire just like that.

That's still nothing compared to what some other people have to deal with. Imagine you're a farmer, and your harvest is on the field, ready to get cut and brought in. T here's a storm brewing, so you jump into the combine harvester, and start the work. Oh no! The machine broke down! Not to worry, you're a resourceful farmer and you have the necessary spare part. You install it and start the machine, except… it tells you: "Unauthorized component. Please contact customer service". Now you're in real trouble because it could take 9 months for the customer service to solve the problem. You can't harvest the food worth tens of thousands of dollars, you're that much in the red. Game over, your farm is bankrupt. But that's not the end of the world, is it?

This is what a pacemaker looks like. Why would I mention those in a talk about software freedom? You see, a pacemaker is a complex device which must examine and diagnose the patient continuously, in real time, in order to perform its function. Its task is to detect a dangerous condition and perform a medical procedure in response to it. It needs software to do this complicated task. But if the device isn't perfect at diagnosing, that's a big problem. I'm not a medical expert, but getting your heart shocked when it's not necessary sounds dangerous in its own right. When it runs closed software that does not grant us the freedom to modify it, we have to resort to begging the manufacturer to fix it. And when we get no freedom to study it, we can't even avoid the circumstances that make it misfire!

But don't take my word for it. I only know of this problem because of Karen Sandler, whose involvement with Free Software is intertwined with this problem since the beginning.

The bottom line is, if we have people who have no other choice but to trust their own body to a piece of closed software and a single manufacturer, how could we possibly say that Open Source had won?

Appliances and copyleft

Are you responsible for building an appliance? I bet you're using Open Source software in it, aren't you? Then licenses like the MIT require you to include a notice about the authors of the source code together with the software you distribute. There's a whole gallery of those on the curl website, ranging from cars to food processors. Are you feeling proud for releasing a device with Free Software in it? Not so quick! Can the user of your device study and modify the software you gave them? Have you actually granted them the Four Freedoms?

Permissive licenses like the MIT license are Free Software, so they let you do all that the Four Freedoms promise. But they also allow you to do another thing: to close the software again by never granting those freedoms regarding your own modifications. If that's what happened, then freedom for me, not for thee. You, the manufacturer, reaped the benefits, the user can't, sucks to be the user.

The responsibility to prevent this falls on us, computer experts. When we create software, we have the choice of license we want to release it under. And we should be using what's called "copyleft": it's a term that applies to licenses which prevent code once released under that license from being closed again. The most widespread copyleft license is the Gnu General Public License (GPL), and I recommend that you all use that one.

Licenses and more

Licenses are not the only thing relevant for Free Software. There are other things to fight:

• patents, like in the case of cellular modems,
• hardware locks, like Android's,
• project management.

As for the last point, recently Google gave an amazing example by restricting access to sources in development to select manufacturers. Everyone else will not get continuous updates, but only once per major release. This illustrates how much influence over the practical usability of a project management decisions have. This is not a change in licensing, and it's also not a technical change, so it's not immediately visible under those lenses.

Instead, it's a consequence of who's in charge. In this case, it's not a community who controls the Android project, but a for-profit corporation. At the same time, it's regular people who are on the user side of the project. Is it any wonder that the goals of a corporation and those of regular people differ? Is it any wonder that the corporation is making changes that suit it even when they don't suit the community of users? When those are the conditions under which a project is developed, it can have deep consequences, even on an architectural level.

Take Debian as a point of comparison. The first statement on the web page already says "Debian is a Community of People!". The software is being developed and used by the same people. They won't make it harder to use. They provide a complete operating system, publish all the sources, and purge anything that isn't open enough. On the other hand, Android has long been replacing open components by closed ones, making AOSP (the open part of android) all but unuseable on its own.

Why?

I suspect this situation has something to do with how computers and appliances have been developed historically. Computers have their roots in academia. When they were sold, they were always advertised as blank slates, general purpose devices, as opportunities to do what you choose to do with them. Not so much for appliances. Those have always had a single purpose. Except they kept getting complicated, until they entered a level of complexity where they needed to incorporate computers in order to perform their function. But they kept being manufactured as appliances, with only a handful of people being expected or allowed to exercise control over them. Incorporating computers didn't change the culture around them.

This is just a guess and I don't know how correct it is. For example Apple was always a computer manufacturer, but they are making computers now as if those were appliances.

What now?

The responsibility is ours – computer nerds' – to make Free Software win. When we build a hardware device, we must publish the firmware sources. We must publish technical documentation – it often so happens that the device documentation needed to make open firmware is missing or incomplete (another war story from the Librem 5, camera sensors this time).

As users, or institutional customers, we should demand that the manufacturer provides open sources for any firmware they are shipping with their devices.

But there's one more way: political pressure. I expect this to be a more effective method than individual action. After all, EU managed to convince phone manufacturers to standardize on USB-C ports for charging, as well as to extend the warranty period. Perhaps they could also force computer manufacturers to not install boot loader locks. It would fit nicely into the Information Society Directive. It says things like:

Member States must provide legal protection against any person knowingly performing without authority any of the following acts:

• the removal or alteration of any electronic rights-management information;

…oh. So instead of jailing people who put locks on devices they no longer own, it enforces the jailing of those who remove them from their own devices. Great.

Dear European Comission, please always have someone with a clue in the room who can explain the consequences of your ideas in a way you can understand. You can do it, already did a couple times, like above. But work on consistency, okay? Pinky promise?

Here are some people with a clue: Free Software Foundation Europe with their Public Money Public Code open letter, the Right to Repair movement, as well as the European Pirate Party.

I recommend anyone who cares to join forces with them. But if you don't want to engage politically, there are also financial way of support. And I don't even mean (although I do encourage) donating. I mean supporting Free Software friendly manufacturers! Buy the Librem 5 from Purism, or a 3D printer from Prusa, or a smartwatch running Espruino. You see, it's expensive to manufacture any sort of hardware. It doesn't help that the markets are already saturated with closed products. Even if open source, hackable products are superior, it will take people at large a long time to realize that this is a superpower. Free Software thrived in culture of repair and modification. But this culture has been suffocated in the wider society with closed, throwaway items, so few people recognize its benefits. That unsustainable crowding out makes another obstacle for Open Source friendly products in the current markets.

There's a noble exception here. What makes it even more unusual is that it comes from Google. It's Chromebooks. Google has a set of requirements that all Chromebook manufacturers must fulfill, and one of them is having a completely open BIOS, together with the Embedded Controller firmware. All Chromebooks I'm aware of run Coreboot. They still contain some closed software, notably the RAM startup software, which, I believe, is present in all laptops, but! ARM-base Chromebooks are able to run with a completely open BIOS apart from that. So if anyone wants to take care of this together with me, I have this NLNet project to make it as easy as possible to run regular, mainline Linux on them. So please, contact me, if you're that person.

The world

A short quiz: how many devices can you count around you which contain processors?

Some hints: TV, camera, toothbrush, oscilloscope, e-book reader, radio receiver, dishwasher, router, washing machine, vacuum cleaner, bathroom scales.

Now think wider. When I went to the supermarket, the vegetable section had a scales that printed labels with barcodes. They were equipped with touch screens. You bet there's a processor and a load of firmware in those. But shops are chock-full of processors in my part of the world. There are thousands of price labels in each of those stores, and they are all e-paper screens. I'm fairly sure you need software to drive those and receive wireless updates.

Keep going and you might realize that the software running in your car allows remote control. Or in your train. That snafu wouldn't have occured if the railways had access to sources of the train software.

What about other business uses? Car diagnostic stations? Medical equipment? Accounting software?

Software is really eating the world, and it's closed software which is everywhere around us, without free options. What's the regular person's role in this? They give up control over entire areas of their lives to others, others who often can't be supervised or replaced.

You know, we messed up. There's no other way to put it. We even let closed software sneak into our own home field: computers. Sure, the interfaces are open. There's SATA, there's PCI. We can swap parts if we want to, we can run Linux there, all is fine. Except it's not, because peripherals are as important as the core, and we, software people, lost control of the peripherals of our darlings already.

Wasted potential

In theory, it's possible that someone opens a piece of software regardless of the wishes of the original authors. The whole game modding scene is about that. Here's an example of someone running Tetris on a pocket camera:

But going against the manufacturer is just wasted work. Imagine the difference between hacking it in and modifying the official sources. The potential, things we could achieve if we didn't have to break doors that are open! So here's a silly example: I have an action camera. Due to some stupid law, the camera breaks off every recording as soon as it reaches the 30 minutes mark. Now I have 20 years of coding experience. Having source code, I could have fixed the problem and went on with my life. Another example, another camera: I am making a time lapse from my window. Every day at 10:00, I take a picture from a camera that just sits there. But this camera has no time lapse feature, so I must go there in person every time. Why can't I fix this? Of course, no source code.

Epilogue

There's now a new printer project that advertises itself as open source. But if you look at the details, it's actually not. Instead, it uses a source-available license which does not grant you Freedom 0 – you must not use the sources for commercial purposes. Better than nothing, I guess.

Notes on DMABUF and video

While working with the Linux video subsystem, especially while working on the Librem 5 even before libobscura, the term "DMABUF" appeared a lot. Yes, it's a buffer. No, you can't always use it.

It's always been some kind of a magically limited but also powerful thing that you must master if you want fast video, but you're likely to hit a corner case if you're not careful.

And the knowledge is spread all over.

So this is my latest understanding of what it is and how to use it. I intend to update this in follow-up posts, so please correct me if you see something wrong.

Background

Video devices operate on pictures. Those pictures are large amounts of data, and they need to travel somewhere to be useful: your screen and your eyes, or your hard drive, or the internet. Typically, your screen, though.

Moving around big pictures at 30 frames per second means moving around a lot of data across different components: all the way from the camera device to the GPU and the display. Moving data from one place to another is copying, and every copy along the way takes up some computing resources. So in order to have a smooth experience, unneeded copies should be eliminated.

DMABUF

One mechanism to avoid copies is called DMABUF: DMA (direct memory access) buffers. Those buffers can be used directly by devices like the camera controller, or the GPU. In the case of video coming from the camera, the idea is that once the camera data arrives from the hardware processing units into the main memory, the GPU can access it directly (with its DMA engine), without creating another "GPU area" and copying image data there from a "camera area".

Unless it can't. DMABUF buffers are always associated with a device. This allows for the buffer to live in a special-purpose area of memory, like the GPU VRAM, where other devices might not be able to access it.

Location is one reason why DMABUF buffers are not exchangeable, but different devices can do different things. If your camera's DMA engine can access only the first 4GiB of memory, the buffer you allocated for your GPU might be out of reach. Or take the shape of the data: if your camera produces buffers with rows being a multiple of 4 bytes, but your GPU can read rows being multiples of 32 bytes, then the GPU may misinterpret the images.

DMABUF life cycle

I'm working on a camera application, so this is the case I focus on: buffers get filled, I read them out later.

At a high level, every buffer needs to be allocated, then fed to the camera capture device, then received once it's filled and read out - by the GPU or otherwise. Then, finally, the buffer is de-allocated.

In V4L2, this is achieved by the following sequence of IOCTLs:

1. VIDIOC_REQBUFS(N, MMAP) allocates multiple (N) DMABUF buffers, without giving any access to the user yet
2. fd = VIDIOC_EXPBUF returns a handle in the form of a file descriptor (fd)
3. VIDIOC_REQBUFS(N, DMABUF) - not sure why this is needed
4. VIDIOC_QBUF(fd, DMABUF) passes the buffer to the device for writing
5. VIDIOC_DQBUF waits until a previous buffer is ready and returns it
6. VIDIOC_REQBUFS(0, DMABUF) not sure what it does

TODO: which call assigns a size to the buffers? They are magically the right size... and what if the format changes? TODO: Which call releases the memory?

In kernel documentation, there are some mentions of "importing" a buffer into the device. I find this unhelpful for understanding how this works. The "import" operation is not the opposite of the "export" operation, which was very confusing when I was just trying to figure it all out. It just seems to be the act of enqueueing. It would mesh with my mind better if all mentions of "import" were unwrapped and instead the focus was on what the operation actually does.

Reading the data

The handles to the DMA buffers are file descriptors. To read the data out, mmap them. In Rust, it's as simple as:

let outf = unsafe { File::from_raw_fd(fd) };
let outfmap = unsafe { memmap2::Mmap::map(&outf) }?;

println!("  length    : {}", outfmap.len());

// Prevent File from dropping and closing the fd implicitly.
let _ = File::into_raw_fd(outf);

Or is it?

The buffers may reside in some other part of memory, like maybe on the GPU.

And can you expose a GPU buffer directly to the CPU? Maaaybe – there are unified memory architectures out there, like some systems with integrated graphics. But unless you know exactly what hardware you're running on, you may find out that the buffer cannot be mapped.

Even if it can be mapped, I'm guessing that access might be slow, for example if using a bounce buffer (can anyone fact check me on this?). That kind of defeats the speed point.

But that's not all the problems coming from different memory placement yet. You have to pay attention to caching. See, the buffer receives data by DMA, meaning that the memory is updated behind the back of the CPU. Which is exactly the point: the CPU should not be busy watching the transfer. BUT! If the CPU is caching some data from the buffer, any update will make the cache out of date and the CPU will never know.

So you have to make sure the CPU throws away or updates any old cached memories of the updated area.

There's an IOCTL for that: DMA_BUF_IOCTL_SYNC. I won't get into details, unless someone asks me to explain the docs.

Complete reading example

For your convenience and my own, I took an existing dma-buf crate and adjusted it to be a little nicer to use in libobscura. Here's how you map a buffer without fretting about safety in dma-boom:

use dma_boom::DmaBuf;
use dma_boom::test;

// It's up to you to find a working buffer.
let buf: &DmaBuf = test::get_dma_buf();

{
    // Request sync and create an access guard.
    // Multiple read-only accesses can co-exist
    let mmap = buf.memory_map_ro().unwrap();
    // The actual slice
    let data = mmap.as_slice();
    if data.len() >= 4 {
        println!("Data buffer: {:?}...", &data[..4]);
    }
} // `mmap` goes out of scope and unmaps the buffer

Announcing libobscura

Let me formally introduce you to my most recent overly-ambitious experiment: libobscura.

Libobscura is a friendly library to use cameras on Linux.

At least that's the goal.

What does "friendly" mean?

• It's hard to use it wrong. No segfaults. Errors guide you to the right track.
• Point-and-shoot. If that's all you need, you get a RGB buffer in ten lines of code.
• It's easy to add support for new devices. Great documentation and a good internal API are the goals.
• It's easy to contribute to. Send patches using the web interface, not a mailing list.

TL;DR: with the simple buffer API you can get a frame in 6 calls, and map it to CPU in another 2:

let cameras_list = vidi::actors::camera_list::spawn()?;
let cameras = cameras_list.cameras();
let camera = cameras_list.create(&cameras[0].info.id)
    .expect("No such camera")
    .expect("Failed to create camera");

let mut camera = camera.acquire();
if let Ok(ref mut camera) = camera {
    let mut stream = camera.start(
        Config{fourcc: FourCC::new(b"YUYV")},
        4
    ).unwrap();
    loop {
        let (buf, meta, _next) = stream.next().unwrap();
        let mmap = buf.memory_map_ro().unwrap();
        let data = mmap.as_slice();
    }
}

Go to project examples for more.

Figure 1: Libobscura will never be friendly enough for every audience.

What cameras?

Any webcams, industrial cameras, image sensors working exposed by the V4L2 interface are currently in scope.

What does "experiment" mean?

There are already other libraries for camera support on Linux. You can use the V4L2 APIs directly, or use libcamera, or libmegapixels.

They all strike various middle points on the power vs user-friendliness scale. Having worked with all of them while developing camera support for the Librem 5, I never got the impression that any of them are particularly easy to use.

Libobscura is an experiment because it tries to find an API that fulfills the needs of most people and remains hard to use wrong.

Figure 2: My "Perfect tool" conjecture. Imagine you have a perfectly well useable tool covering some of your needs. If your needs grow, the perfect tool covering those tools cannot be easier to use than your old tool. And humans are imperfect. This applies to APIs, as well. I put my rough idea of where I think a couple of examples fall. Libcamera tries to be as general as possible, so it's on the far right. OpenCV has Python bindings, so it's up top. Libobscura starts out a bit to the left and top, and I hope to push it to the right, to explore how many use cases can be added while staying easy.

Perhaps it's impossible to improve on what current libraries do. But now libobscura is a space where it's possible to try out radical changes without bothering the maintainers of existing libraries – for example, to try out an entirely new approach.

Radical approaches

There are a couple of radical approaches in libobscura that earn it the "experiment" status.

Rust

Rust is a memory-safe systems language. Libobscura uses it to ensure that the APIs are hard to use wrong. A Rust API built with a little care does not let you crash with a segfault – the compiler will alert you before you can create problems.

The Linux kernel started its own Rust experiment in 2020. Linux is a low-level project. Camera support is a low-level topic (many people working on libcamera also work on the kernel). If Rust is interesting for Linux, then it's interesting for libobscura.

Get RGB data

A camera library can't be described as easy to use if the pictures it gives you need to be processed before displaying.

The typical format to display data is RGB while cameras return either Bayer or YUV data. With libobscura, the goal is to provide those conversions transparently, without any extra effort on behalf of the user or camera backend.

GPU acceleration

The conversions might not be implemented in hardware, but don't worry, libobscura has your back! It comes with a GPU-accelerated image processing library called crispy-img.

Why GPU? The idea started with the Librem 5, which will often record video while running on battery. Offloading this work to the GPU is a clear win, and I expect most Linux video-capable devices to have some form of GPU. And we're open to a CPU decoder if you want to contribute!

Figure 3: An artist's impression of a pixel format conversion shader.

Status

Because libobscura is only two months old as a funded project, the current status is "proof of concept". There's a safe and limited user API, another more tricky but zero-copy API, the GPU accelerated library can handle some conversions, and camera support is relatively simple. USB camera demo works.

But there are still goals to achieve:

• change controls like brightness or focus,
• transparently integrate GPU processing,
• have a way to choose your preferred output format,
• implement auto-* algorithms when the camera doesn't have auto mode,
• replace LGPL-licensed pieces from libvidi and crispycam (not great for link-time-optimized code)
• add Pipewire integration, so your browser can just pick up your camera feed for teleconferencing,
• add Librem 5 support as a realistic, useful verification of concept.

Those will be the next steps for the project. First implement all the functionality and then extend support for more devices. This way we can catch corner cases in the API that are bound to appear with unusual setups.

The future is YOU

I'm making libobscura in the hope that it will be useful to people. As a base to build software, or to ship devices, or to learn what software architectures fit this problem.

When I sent my funding request to Prototype Fund, I didn't expect to be taken seriously. After all, what motivated me most was that it was a cool challenge. Apparently they believed in me more than I did, because I got funding until March 2025.

What happens until then and what happens next depends on how useful this work actually is to YOU. The ultimate goal of any software is to be useful, otherwise what's the point?

So I invite YOU to analyze it, try it out, give feedback, experiment with it, copy the concepts, contribute to docs, illustrations, and code, fork it entirely! Regardless if you come from the Mobile Linux community, or Raspberry Pi, or you have a laptop with this IPU thing, or if you're from libcamera. When you improve, libobscura fulfills its goals.

So see you on the project page!

Thanks

Thank you to Purism, for inviting me to the Linux camera work and funding the base which I'm using now.

Thank you to all the libcamera people! I stol... was inspired by libcamera's architecture, and you also answered my countless questions both during Librem 5 development and recently.

Thank you Martijn Braam for showing how to create simple config files for many mobile phones.

Thank you BMBF for providing the funding money, and thanks Prototype Fund for connecting me to it :)

IPv6 in your home

IPv4, IPv6, who cares? As long as it works, right?

The death of IPv4 is greatly exaggerated

We ran out of IPv4 addresses, and so what? I can still watch Youtube videos, have calls over Zoom, buy things on Amazon. The Internet works due to the magic of NAT! Channeling Oprah, you get an address, you also get an address, and you too get the same address!

Except this all works because of lots of money and effort.

Effort because you can't just make a call to your friend over IPv4. You're both likely behind a one-to-many NAT, meaning you can't receive connections. So who will initiate one? STUN offers some painful, costly, partial, and annoying solutions involving having a dedicated address anyway.

An IPv4 address costs money, too. A single IPv4 address costs 26 USD in bulk as of 2024-11-05, and I'm paying for a 3EUR per month for a cheap VPS just for the sake of having one address for myself.

Why bother? Because if you want a little place for yourself on the internet to serve stuff from, you need an address. Manage your own game server? Expose a quirky service? Have an actual, direct call? Seed torrents of your favorite classic art? Connect to your home network while away? I want to do all those things, so I bought myself an address.

Of course, taking an address from the limited pool makes me part of the problem.

But I want to connect to the plant watering system that I left at home! But I want to download stuff from the off-site backup that's at my friend's place, behind NAT! Yes, I could buy a server in a data center and install Wireguard there. I did it, it sucks. High pings, slow transfers. Help! Let me have direct connections.

IPv6

IPv6 comes to the rescue! The address pool is about 420^π bazillions addresses, so every address is really cheap. If you want to apply for a subnet, a random 500 EUR package gives you 2¹²⁸⁻⁴⁸=2⁸⁰ addresses, which is 2417851639229258137600 addresses per EUR yearly.

Global IPv6 addresses are effectively free.

If you have a decent Internet provider, they will assign you a /60 subnet and let all your LAN devices grab a /64. Now anyone can connect to you, so enjoy! And put up your firewall.

That /64 thing

What's that about a /64 subnet? A subnet for every device?

Well, that's how IPv6's DHCP equivalent works. You can't normally get anything smaller than that through autoconfiguration. Which is not a problem if you have a decent Internet provider.

But if you've got only a half-decent provider, they might only offer you a /64. That typically happens on mobile connections, but not only. (Also, this can happen if you like subdividing networks like Russian dolls. IPv4 you could just chain NATs, IPv6 has no NAT.)

So what do you do?

OpenWRT

I don't believe I have to introduce OpenWRT to any of my readers. Newcomers, this is the Open Source operating system for routers.

It can extend IPv6 connections to connected computers even if there's only a /64 available! And it has this nifty Web interface called LuCi.

While there are multiple guides for the command-line, there are no guides for configuring IPv6 forwarding for the Web interface. So here's mine.

IPv6 relay mode in Luci

First, set up an IPv4 WAN network (if you care). I'll call it wwan. Remember to set up a LAN interface if you don't have one. Ready? Then set up a WAN network for IPv6. I'l call it relay6, set it to DHCP client and select "Alias Interface: @wwan" as the device.

Once you have it, navigate to "DHCP server" and set one up.

Yes, I know, it's weird. We don't want to provide addresses on this interface. That's why the next step is checking the "Ignore interface" box.

Once that's done, go to DHCP IPv6 settings.

Make this interface a designated master and change 3 dropdowns (RA-Service, DHCPv6-Service, NDP-Proxy) to "relay mode".

We're done with the WAN interface, but the LAN needs to be adjusted. Here, it's just a static address on a bridge device.

Go to DHCP Server → IPv6 Settings and change all the dropdowns to "relay mode".

Save all the changes and apply them. For me, the router immediately received an address.

My laptop also got an address immediately, but I had to reconnect to get the default route populated (otherwise you can't connect to the Internet).

Check a device connected to that router, it should get the address, too.

Default route

One snag, though.

The default route was not set for my laptop. I had to modify the connection manually and add one. The result looks like this:

[me@foobar ~]$ ip -6 r
default via fe80::abcd:ef12:fecd:6573 dev wlp2s0 proto ra metric 600 pref high

Bonus: ULA

ULA is something that doesn't exist in IPv4.

ULA is the closest counterpart to an IPv4 local address. I use it to have stable addresses within my network. Even if the upstream changes their prefix (effectively your network address) and all previous addresses become invalid at 6:00 every morning, breaking all connections (thanks Telekom), and even if the upstream goes down at all, ULA will keep your network internally connected.

I put the names of all hosts on my network in /etc/hosts, like a troglodyte on the early Internet before DNS.

It sounds complicated if you come from IPv4. There, every computer knows its own address. 192.168.1.77. Great. That's it. What's my public address? No idea.

In IPv6, every computer can easily have multiple addresses. A global one, some ULA addresses. It doesn't become a mess because those addresses are completely independent (link-local ones are a bit special, though). ULA networks are dropped on the edge of the public Internet, so they can't even be used for Internet access.

OpenWRT supports using ULA in Network → Global network options.

For me, it doesn't always get picked up, but it's there on some "Static address" interfaces. I don't really know what controls it, but I have one on the same interface as my LAN:

The end

Now I have IPv6 in my local network! I can seed the world with torrents! I can host a quirky server for 10 minutes (remember to open firewall)! I can connect to the servers I manage at a friend's place!

Oh wait, that server is behind its own firewall and I never enabled IPv6 on that one -_-. I guess I'm not done blogging about IPv6 yet.

Bike dynamo

I was recently planning a longer bike tour, where I would enjoy the wonders of sleeping under a tent and a wide open sky.

Retreating to nature is fun, but as a society, we're addicted to electricity. As a data hoarder, the very least I need to do is to keep my GPS tracker charged. The camera with a full battery is nicer than a dead one, too. This poses some logistical challenges if you don't want to regularly check into hotels, or libraries, or restaurants. And when cycling, I like having the freedom of not having to do many things.

This means I should produce my electricity myself to rely less on civilization.

I'll mention solar cells just to dismiss them: they don't work when I cycle, they don't work when I sleep. Next!

Dynamo

If the rumor is to be trusted, a bike dynamo is not a dynamo, but rather an alternator. It produces alternating current. But how exactly? What are the pitfalls of using a dynamo to power circuits?

AC

Alternating current is not directly useful for charging batteries. The AC needs to be rectified first, and a rectifier bridge made out of 4 diodes takes care of that.

Voltage

After the rectifier stage, we have direct current. But what voltage? Most bike dynamos are rated for 6V. But if you connect a multimeter (and nothing else) to the dynamo and spin the wheel, you'll get more like 30V to 100V AC peaks. Those rectify to way more than 6V. You're about to fry your USB device if you plug it in.

On the other hand, attach a small resistance and you don't even get the rated 6V. What the heck?

So I decided to run an experiment. I attached various resistive loads to a wheel with a hub dynamo, caught the axle in a vise, whipped out a hand drill, and held it against the tire to spin the wheel at an equivalent of about 20km/h.

This is a table of voltages I got:

resistance [Ω]	AC voltage[V]
3200	28
1000	23
1000	22.4
470	26
470	23.2
68	22
22	13.2
6	8.6
4	4
2	1.8
0.8	0.6
0.4	0.4

Some values repeat because I couldn't believe my eyes (j/k I was just sloppy at taking notes).

Clearly, a dynamo is not a voltage source. You'll find out as much after five minutes of research.

Current

But is it a current source? If so, then the current should be equal across all measurements. And current = voltage / resistance, so we can quickly calculate it and add it to the table:

resistance [Ω]	AC voltage[V]	AC current[A]
3200	28	0,0087
1000	23	0,023
1000	22.4	0,022
470	26	0,055
470	23.2	0,049
68	22	0,32
22	13.2	0,6
6	8.6	1,4
4	4	1
2	1.8	0,9
0.8	0.6	0,75
0.4	0.4	1

The current isn't always equal, as it goes from almost nothing with a high resistance to over 1A. So no current source, either?

Fitting a dynamo model into a current hole

There's no clarity from looking at raw data in a table, so instead I decided to guess a model and see whether it matches the data.

First I had to decide whether I wanted to model a current source, or a voltage source. Fortunately, Thévenin's theorem says that one can be converted to another. So I simplified my life by modelling my circuit as a voltage source with internal resistance:

The current(resistance) function to fit to match the data looks like this:

i(r)=v_s/(r_s+r)

where r_s is the internal source resistance.

After letting gnuplot find the best fit, it finds v_s = 23.9184 and r_s = 14.492. The plot looks like this:

Given that the data already contains 28V, I don't think it's a great fit. And I suspect there's more power hiding near the 1.4A, which should form a peak, but there isn't one in the best fit. Given that, the model seems wrong.

But I don't know what other model I could use. If you know, I'm eager to hear from you!

But this model is enough to estimate that any circuit dealing with the dynamo should withstand about 30V at about 20km/h. Investigation at higher speeds will continue after I find a faster hand drill.

Spikes

Except when the voltage spikes. If you're like me, your connectors will not be attached with micrometer precision and steam engine forces. It may get torn by accident or just get loose from all the experimenting. As the spinning dynamo on a bike in motion disconnects from the load, its resistance jumps to infinity, and voltages on the leads spike to 100V or maybe even above that.

And when the connection catches again, your circuits are in for a shock. Literally. Look at this vertical line at the beginning of the graph. That's the moment I connected the spinning dynamo.

The shock isn't long, but apparently it's enough. This matter has been brought to my attention because someone had had multiple lights fried for no clear reason.

Thankfully, it's possible to mitigate. Experiments showed me that sticking a 30µF capacitor rated for 50V in parallel to the load is enough to absorb the shocks. A TVS diode (or two) might also be sufficient.

Overvoltage

Okay, but what if 30V is still too much? You'd like to cut it off, even at the cost of losing some of the energy, because you just can't find a reasonably sized or priced component that can handle it?

Notice that current goes WAY down above about 23V and stick two Zener diodes in series, parallel to the load. A Zener diode cannot handle much current, but there isn't much to divert, either.

If that's still too much for you, add a resistor in parallel to the load. Choose one based on your preferred voltage from the table above, just be aware that it caps the voltage by always diverting some of the power.

Power

One more thing we can estimate is power. The spice must flow into those batteries! How much can we count on? Power = voltage · current. Skipping directly to the plot:

Again, the fitted curve is kind of crazy, but at least we already know that we can cross 10W. I strongly suspect this is not the last word of the dynamo, given online tests, but more measurements are needed.

Sources

The data for the plots and the code can be downloaded: data, plot. Run with gnuplot -p ./dynamo.gnuplot

State of input method

This is an edited version of the "Input method on Wayland is broken and it's my fault" talk which I presented at FOSDEM 2024.

My encounter with input methods started with the GNU/Linux-based Librem 5 phone, when I got hired to make sure people can type text on it.

As you can see, the phone has no keyboard. That would make entering text difficult on a typical Linux computer. Thankfully, we're not the first who needed to enter text without depending on a physical keyboard. The solutions to that are collectively called input methods. They can cover handwriting recognition, or entering scripts like Chinese where the characters on the keyboard are by themselves insufficient, or – and this is what we're looking for – selecting characters on on-screen "keyboards".

But how do we place an input method within the context of a GNU/Linux operating system?

Consider Wayland, which has been chosen as the centerpiece of the user interface on the Librem 5.

Wayland is a set of protocols connecting applications to the user, by the means of outputs, like the graphical display, and inputs: the mouse and keyboard. (It doesn't cover anything relating to sound, though, nor joysticks, for some reason). My task was to find an alternative to typing, so it would cover some of the tasks that a keyboard typically does. That suggests that Wayland is indeed the right place to put an input method – in place of the keyboard.

It turns out that I wasn't the only one having that thought. Wayland has seen attempts to create protocols for this: zwp_input_method_unstable_v1 and zwp_text_input_unstable_v2.

I took those protocols and tried to implement them, but I used them wrong, resulting in breakage which I'm about to warn you against.

Haha, just kidding. My transgressions against Wayland were much worse.

The two existing protocols were very basic, and didn't really cover the actual needs of nontrivial input methods. I decided to update them and create improved versions. This is where it gets much worse: I embedded mistakes within the protocols, so no matter how hard you try, you can't use them correctly.

Mistake 1: being too conservative

A user of a modern, keyboard-less mobile phone expects the on-screen keyboard to do more than just input text. Actions like moving the cursor within a text field, moving to the next one, or submitting the form are not related to typing – they don't produce text. Yet, because of the expectation, the tech stack on the Librem 5 had to support this.

I decided to not introduce too many changes at the same time, and re-use an experimental keyboard emulation protocol that we supported for other reasons. That protocol would be used to submit just the non-text actions.

It looked great, until it became clear that it's a dead end.

To understand why, you have to take a look at how keyboard protocols work. What the user wants is to submit text. With a text-input protocol, it's easy. Entering the text "Błąd" would look something like this:

1. Use text "B"
2. Use text "Bł"
3. Use text "Błą"
4. Use text "Błąd"
5. Finish

But a keyboard is not at its core a device to enter text, but to press keys. It's concerned with buttons and whether they were pressed and released. Our simple example looks more like:

1. KeyDown SHIFT
2. KeyDown B
3. KeyUp B
4. KeyUp SHIFT
5. KeyDown AltGr
6. KeyDown L

and so on.

In reality, the buttons don't even have names, but numbers that need to get resolved to names. It looks closer to this:

1. KeyMap *0xffffbced
2. ModifierDown 0x1
3. KeyDown 0x102
4. KeyUp 0x102
5. ModifierUp 0x1
6. ModifierDown 0x4
7. KeyDown 0x56

and so on.

The tables responsible for resolving button numbers to actual text are called keymaps, and they are the problem here.

There are keyboards that support multiple scripts. A button can be labelled with "Q" and "Й" at the same time, even though its number can't be changed. The user is responsible for telling the software to use the currently intended keymap.

Then, there are custom keyboards. Those can send arbitrary key numbers as well, but there's no guarantee that a button labeled "P" on one sends the same number as the button labeled "P" on another. Again, the user must indicate the intended keymap.

An emulated keyboard is a kind of a custom keyboard, needing a custom keymap, independent of any physical keyboards that might be connected.

But Wayland combines all keyboard events into a single stream before giving them to an application.

That means the application must always have the correct key map for every key press. This works until you consider that a user might want to smash and hold keys on multiple distinct keyboards at the same time. Long story short, Wayland maintainers told me that it's basically impossible to make this work due to corner cases, and they won't accept the keyboard emulation protocol.

This is bad. A protocol without Wayland buy-in will not get widespread adoption. And we must support non-text events if we want to meet users' expectations about how a phone on-screen keyboard works.

This makes emulating keyboards a dead end, and this mistake results in having no viable solution for the phone use case.

Actions

Not all is bad. There has been some talk about an "actions" protocol to cover those needs. It's still unclear, but it could allow the user to do things like copy and paste, select all, submit the form, etc.

Mistake 2: bad synchronization

What should happen if you're entering two words into a text field but the application lags for a moment?

If you're answer is "the application should show both words", then I agree with you.

Except this is forbidden in my protocol. The text-input protocol I came up with will drop events a lot for that reason, and you'll end up with missing letters. That's bad.

I did it for a reason. An input method must be aware of the text inside the input field, if only to present the user with autocomplete suggestions. I came up with a protocol which carries the state identifier with every request.

Every event of an input method pertains to some state, so that the application can reject events if they apply to text which is no longer there.

So when the input method sends two events at once, and both apply to the original state, then after the first event is applied, the original state is already gone and the second event is automatically invalid.

"That's wrong", I hear you say. "Why not just let the input method override state changes? It would only be a problem if the user is typing into the input field from another source, and this is such a contrived case that it's not worth caring about."

To this, I say: I don't want to mandate which use cases are contrived or not. Initially, that was my only reasoning behind this aspect, but before FOSDEM, I came up with an actually realistic use case: two people editing the same text field. We don't even have to get into Wayland seats. It's already a thing in collaboratively edited Web documents: the text may change here while you're typing there. Should your input method automatically override the other person's edits?

This is a hard problem, and I have no solution to it.

Progress

Sadly, since I was moved to other work, there has been very little progress on those fronts. The last change to the new text-input protocol is 2 years old, and it adds my 4 years old commit.

If you want to take part in the effort to change this, feel free to contact me.

After RustLab

Last week I attended the RustLab conference for the first time. This report was going to be rather boring, but one of the presentations triggered me, so I'm going to put the (hopefully interesting) hot take right here in the front.

Last-minute talk

The talk about "confidential computing" was not on the schedule. It took the slot of a cancelled one, after some confusion regarding lightning talks in the same spot. Please excuse me that I don't know the exact title or the author (I will update the post once I find the recording).

Same, but different

The speaker asked the audience for a show of hands: who heard the term "confidential computing"? Very few hands went up.

Then he explained that "confidential computing" prevents anyone from accessing confidential data, and even the staff at the data center cannot access your computations. This is due to hardware features like TrustZone, hypervisors, Intel SGX, and similar.

Sounds familiar?

How many people would have put their hands up if he asked about "trusted computing"? Or "DRM"? Was the selection of the term intentional? I can only wonder.

Trusted computing can be used to empower. At Purism, my co-workers used trusted computing to give the user control over who can tamper with their OS. My own team added a smart card reader to the Librem 5 for similar purposes. But this is an exception, rather than a rule. Hardware manufacturers use trusted computing technologies to prevent people from loading their software of choice on the devices they own. Some don't give up, and eventually break into their own devices to gain full(-er) access.

The speaker completely ignored that topic, instead describing the rather tame use case where we pay someone else to do our computations – the classic cloud computing arrangement. Trusted computing then ensures that our contractor doesn't see or mess with our inputs or outputs. That's a quite reasonable take on things.

Except it's like presenting the scientific benefits of the ballistic rocket and staying silent on its applications in war.

Root of trust

For a talk about controlling access to data, there was disappointingly little said about who's wielding that control. Even when someone from the audience (moi) asked about it, the answer didn't really include the core concept of processing secrets: the root of trust.

In very simple terms, the root of trust is the part of the system that the data cannot be hidden from.

As the presenter said, CPU doing the actual processing is one. Translating to more useful terms: you must trust the CPU maker to create a CPU free of bugs and not to intentionally exfiltrate your data. This is the CPU has direct access to your data, and verifying that the CPU is OK after buying it is damn near impossible.

But there's an even more important party here. It's the one who actually wants to process the data. The one who loaded the application. If the application could not access data it wants to confidentially process, it wouldn't be useful for much. So the application loader also holds the keys to the kingdom. Quite literally, because the CPU interface is designed to grant access to the confidential computing facilities to anyone who holds one of the cryptographic keys burned (in)directly into the CPU.

If you load the application, but haven't thoroughly verified its source code, then the authors of the source also become roots of trust – but that's a topic for another day.

It's mine! No, it's mine!

So, who's the key holder in practice?

For Secure Boot on x86, Microsoft is one by default. Linux distributions like Fedora need to beg Microsoft to give them access, or otherwise installation would stop with scary warnings by default.

For Android boot, the phone manufacturer typically holds the keys. The firmware running in TrustZone typically cannot be replaced by anyone else.

But if you're the user, the situation is not hopeless. Many CPU manufacturers on the ARM side keep TrustZone accessible. On the Librem 5, we deliberately left all 4 key slots open, for situations where the owner wants to create their own trusted computing environment.

Some people say that not allowing the owner to access the keys is a good thing. After all, the keys protect the owner's confidential data from attackers, and if the users got the freedom to protect themselves, they would certainly hurt themselves, therefore they should not have the option. I will mercifully not comment on this kind of argument.

Instead, I will encourage you, dear hardware and software maker, to stop for a moment, and think. Are you using confidential computing to empower people who own their hardware, or to split them from the computing power they paid for and own? Are you letting them make decisions, or making it for them?

This talk forgot to place humans in context of confidential computing, even though all computing is one for humans. Please don't make that mistake and don't forget who your work serves.

Two-factor login

Do you still type a password to get into your laptop? What about in the office? Or when you're on the train? What if there are cameras above your head? Have you ever worried that a snooper might get your password and do bad things with it?

I'm sure you haven't. Like a properly security-aware hacker, you disabled password-based SSH login already. You also don't reuse your laptop's password for anything else, especially not for your backups. Even if someone snoops the password, all they get is access to the laptop itself. Right?

Haha, I betcha. Let's face it: opsec is hard. Upgrading your opsec skill takes effort. Good opsec reduces the likelihood that secrets will be spilled.

That's why I decided to augment my laptop with two-factor login.

Now, I can whip out my laptop on the train, and log in to it without worrying too much whether the password gets intercepted. It's unique, so the attacker won't hack any of my other properties (and I'm not leaving my laptop sitting around). I see you grumble: "another unique password is hard to remember! What's the point if you could just change your login password?" The hardware token makes short and memorable passwords possible, because it bricks itself after 3 failed guesses! So I can get away with a password that's really easy to remember, without losing resistance to brute forcing.

Now I can also leave the laptop in a public space for 5 minutes. Even the security guard who used the ceiling camera to record me typing the password won't get into my laptop undetected. After all, I took the hardware token with half of my login credentials with me!

WARNING: I only cover the user login procedure in this guide. Not unlocking the disk encryption. The evil security guard can still compromise the laptop by watching me enter the decryption password and then rebooting the system. Stealing the laptop is not in my threat model for this blog post.

Second factor

Two-factor authentication splits your secret into pieces of two different types (factors), which have different security properties. This is to reduce the likelihood of the entire secret getting intercepted. Each factor may get intercepted easily on its own, but both at the same time is going to be much more difficult because of how they differ.

There are two most common factors: "something you know", like a password, and "something you have", like a hardware token. Passwords can be intercepted by watching someone type them, or by key logging, or by retrieving it from badly secured storage. A hardware token cannot be duplicated, and its secret can only be intercepted by physically stealing the token.

While hardware tokens contain some data constituting the secret, it's not possible to get that data out. If you can duplicate the secret, then the token not doing its basic job as a "something you have", and stops being a second factor to a password. It basically becomes a password with extra steps.

Login setup

This guide will show you how to enable 2-factor authentication with a password and a smart card (Nitrokey Pro) on Fedora 37. It does not disable the original password, so that locking yourself out is harder.

First, install some packages:

sudo dnf install sssd opensc p11-kit gnutls-utils nitrokey-app easy-rsa

Let's play around with the smart card itself. Plug it into the USB port and give it a go. First, run nitrokey-app and set up the user and administrator passwords (if you have trouble finding the options, check out the menu bar).

Now, play around with the key.

$ p11tool --list-tokens
[...]

Token 2:
        URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=ZeitControl;serial=xxxxxx;token=OpenPGP%20card%20%28User%20PIN%29
        Label: OpenPGP card (User PIN)
        Type: Hardware token
        Flags: RNG, Requires login
        Manufacturer: ZeitControl
        Model: PKCS#15 emulated
        Serial: 000500005c61
        Module: opensc-pkcs11.so


Token 3:
        URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=ZeitControl;serial=xxxxxx;token=OpenPGP%20card%20%28User%20PIN%20%28sig%29%29
        Label: OpenPGP card (User PIN (sig))
        Type: Hardware token
        Flags: RNG, Requires login
        Manufacturer: ZeitControl
        Model: PKCS#15 emulated
        Serial: 000500005c61
        Module: opensc-pkcs11.so

Remember this command, cause the URL field will be needed later.

Certificates

The login procedure involves checking PKCS12 certificates, so let's set up one using easy-rsa. First, create a file called "vars" in some directory, and put this inside:

set_var EASYRSA_KEY_SIZE 4096

Now, let's create the entire certificate chain for the user "thatsme". It will ask you for different passwords: one for the certificate authority (you only need to use it when adding another certificate), and one for the private key (you'll need it once, when uploading the private key to the smart card), so pay attention to them.

ln -s /usr/share/easy-rsa/3.0.8/ easyrsa
EASYRSA_VARS_FILE=`pwd`/vars ./easyrsa/easyrsa init-pki
EASYRSA_VARS_FILE=`pwd`/vars ./easyrsa/easyrsa build-ca
EASYRSA_VARS_FILE=`pwd`/vars ./easyrsa/easyrsa build-client-full thatsme

Whew, this wasn't as difficult as the last time I tried to set up a new certificate authority. Good job, easy-rsa folks!

Now, store the generated private key and the corresponding certificate (approved by the newly minted certificate authority) into the smart card.

WARNING: we generated a private key on the host computer. It can be intercepted by a rogue application on this computer, especially if you don't delete it fully (and deleting is harder than it seems). The interceptor will be able to follow the next steps to duplicate your smart card. If you want to be super careful, you should generate the key directly on the smart card, and use Certificate Signing Requests (CSRs) with easy-rsa to approve the key. This is not relevant to me, because I not will use the smart card to log in to any other computer. If someone snooped the key from the laptop, I'm already hosed. So I chose the easy way to generate the key.

pkcs15-init --delete-objects privkey,pubkey --id 3 --store-private-key ./pki/private/thatsme.key --format pem --auth-id 3 --verify-pin
pkcs15-init --id 3 --store-certificate ./pki/issued/thatsme.crt

See how snugly the key and the certificate both sit on your smart card now? Use the URL you noted earlier to show them.

$ p11tool --provider /usr/lib64/opensc-pkcs11.so --list-all pkcs11:model=PKCS%2315%20emulated;manufacturer=ZeitControl;serial=xxxx;token=OpenPGP%20card%20%28User%20PIN%29
Object 0:
        URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=ZeitControl;serial=xxxx;token=OpenPGP%20card%20%28User%20PIN%29;id=%03;object=Authentication%20key;type=public
        Type: Public key (RSA-4096)
        Label: Authentication key
        Flags: CKA_WRAP/UNWRAP; CKA_EXTRACTABLE; 
        ID: 03

Object 1:
        URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=ZeitControl;serial=xxxx;token=OpenPGP%20card%20%28User%20PIN%29;id=%03;object=Cardholder%20certificate;type=cert
        Type: X.509 Certificate (RSA-4096)
        Expires: Sun Sep 28 09:07:36 2025
        Label: Cardholder certificate
        ID: 03

I tried listing the private keys, but for some reason, p11tool won't display any.

Login

Now that the card is prepared, let's set up the login. First, copy the certificate authority to where sssd can find it. Any key signed with this certificate will be candidate for login.

sudo cp pki/ca.crt /etc/sssd/pki/sssd_auth_ca_db.pem
sudo chmod ugo-w /etc/sssd/pki/sssd_auth_ca_db.pem

Then, set up sssd itself by creating the file /etc/sssd/conf.d/base.conf with the following contents:

[sssd]
services = nss, pam
domains = files
disable_netlink = true

[domain/files]
id_provider = files

[pam]
pam_cert_auth = True
pam_p11_allowed_services = +kde, +kscreensaver, +sddm

[certmap/files/thatsme]
matchrule = <SUBJECT>^CN=thatsme$

NOTE: The common name (file name) given to easy-rsa must match the username on this system. If anyone knows how to break that relationship, please tell me! I know it has to do with sss-certmap, but the docs are incomprehensible to me.

Finally, enable sssd. Do this as root:

systemctl enable sssd
authselect select sssd with-smartcard --force
systemctl restart sssd

And that's it. When the smart card is plugged in, you should be able to see this:

$ sudo echo "ok"
PIN for OpenPGP card (User PIN):

rather than the normal

$ sudo echo "ok"
Password: 

Similarly, the KDE screen locker will accept the smart card password rather than the user password. It doesn't indicate that this is happening though, other than being really slow with the card in.

Debugging

Enable logging by adding the debug_level entry:

[pam]
pam_cert_auth = True
debug_level = 9

and restarting sssd.

The two relevant log files are here:

/var/log/sssd/sssd_pam.log

and

/var/log/sssd/p11_child.log

but they are not that helpful, because sssd documentation doesn't really explain a lot.

Good luck upgrading your opsec!

Actually, no login

I did all of that, I rebooted my computer, I plugged in the Nitrokey, and I entered my password at the SDDM login screen. The prompt takes some long seconds to resolve – good, it's working with the card.

Login refused

Wait, what? I try again, same result. I try my regular password, and that goes through.

The logs in /var/log/sssd/sssd_pam.log show:

[pam_reply] (0x4000): [CID#3] pam_reply initially called with result [9]: Authentication service cannot retrieve authentication info. this result might be changed during processing
[pam_reply] (0x0200): [CID#3] blen: 22
[pam_reply] (0x0200): [CID#3] Returning [9]: Authentication service cannot retrieve authentication info to the client
[client_recv] (0x0200): [CID#3] Client disconnected!

Meanwhile, a successful use in sudo:

[pam_reply] (0x4000): [CID#9] pam_reply initially called with result [9]: Authentication service cannot retrieve authentication info. this result might be changed during processing
[pam_reply] (0x0040): [CID#9] Assuming offline authentication setting status for pam call 249 to PAM_SUCCESS.
[pam_eval_prompting_config] (0x4000): [CID#9] No prompting configuration found.
[may_do_passkey_auth] (0x0400): [CID#9] Passkey auth not possible, SSSD built without passkey support!
[pam_reply] (0x0200): [CID#9] blen: 22
[pam_reply] (0x0200): [CID#9] Returning [0]: Success to the client

I don't know if this difference is relevant. My search engine skills failed me, and I eventually gave up.

I guess I'll just have to use the main password for login. Thankfully, when traveling, I resume the laptop from sleep, and the screen locker accepts the smart card password, so it's not that bad.

Either way, if you know how to solve this problem (any sddm devs around?), please let me know! I'll update this post with your info.

---

Thanks to one of my coworkers at Purism, for showing me that not every password must be guarded jealously.

Non-English speech synthesis

If you're like me, you've been on the English-language Internet for so long that English is how you think of computer things.

Sadly, the English Internet has blind spots. The most obvious one of those is that the Eglish Internet doesn't care about other languages. So if you go and ask the search engine about "Linux speech synthesizers", hoping to get a robot voice to say "jazda", you'll realize that the results have nothing to offer you.

Instead, you stop for a moment, and decide to go to the smaller, a bit backwater, non-English Internet, and you ask "syntezator mowy linux", knowing that search results are likely to be infested with non-free solutions which you don't want to see, and you might not even find a free one.

The results are just as bad as in the English search, but somehow still worse. There's the unmaintained, one-person effort, a tutorial for Festival which asks you to download the voice bank from a long-dead URL, and a bunch of links referring to espeak which sounds awful even for a robot voice when asked to go outside of its English comfort zone. The guides to installing an Android software aren't even worth mentioning.

But there's something sparkling in the mud. A university website links to RHVoice, which is even packaged for Debian – although I don't know why it's in the non-free repo, the license on the github page is listed as GPL. If it sounds like an easy solution, it almost is. Almost.

RHVoice

I don't run Debian, so I install the software in a container.

podman run -ai debian:bookworm

Once inside, I install vim, and I install the package and some voices:

echo 'deb http://deb.debian.org/debian bookworm non-free" >> /etc/apt/sources.list
apt update
apt install rhvoice rhvoice-polish

Then, I tell it to say something:

echo "jazda" | RHVoice-test -p natan -o - > test.wav

This will place the recording in a file. On the host side, I retrieve it using the container ID:

podman cp 1ea7197c68f6:test.wav Documents/jazda.wav

That's it! The recording sounds okay, and I can put it alongside the IPA pronunciation on the web site.

A biased guide to tech conferences

This year's Camp was my second so far. Despite knowing it already, and knowing how much is happening there, I left it feeling that I missed out on a lot. The previous camp was so much more exciting! Was this year's Camp different, or was it me?

The steamy weather affected everyone badly, putting all of us campers into a sleepy mood. But, once I thought about it, I noticed that I made some unprovoked mistakes, myself. Information wants to be free, so here's an easily digested guide for all of you, so that you don't have to make the same mistakes.

My biases

Out of the lack of experience otherwise, I'll describe what works for me when I come alone by train, and with an amount of luggage that does not impede my mobility much. The advice may need some adjustments for other cases.

Early

To get the most out of a conference, becoming a speaker might be useful. Forget about the long-term popularity boost. It's worth presenting just because of the immediate benefits. You'll meet the most fun people: the organizers before your talk, and afterwards, the listeners who are interested in the same topics as you are. Don't leave the stage immediately after you finished presenting. Oh, and did I mention that a speaker's badge sometimes grants access to the backstage area?

To snatch a slot, you have to prepare early. The deadline for a typical "Call for Papers" is 6-2 months before the event. Look for announcements. If you miss it, then, depending on the event, you might still get another chance.

It may be worth to get the ticket early. In some places, tickets go up in price as the date of departure approaches. If you have a choice, snatch a ticket with a fast connection while you can. Sure, you can travel to the GPN in Karlsruhe from Berlin on the flat-rate Deutschlandticket, but you're going to face over 12h of travel. Maybe it's worth it to get there in half the time by paying a couple EUR for an IC ticket well in advance.

Now that you know you have to choose your tickets early, when to go? There are conferences where participants are the main attraction, and where they come with all their own toys. Find out if your conference is one of those, and if it is, come a day early, to help with unloading and the set-up. People who come early are the organizers, and the most proactive members of the community, so this is whose friendship you can win by carrying boxes and connecting cables. You can fall back on it over the conference, and you'll be remembered online and at future events.

Staying after the party is over will not earn you friends for the conference that just ended. But it may give you eternal gratitude by the few people who stayed. As I'm writing this, there are still people on the Camp grounds dismantling the event. It's also going to pay off if the people you help with the teardown are the same who you meet at your local hackerspace.

CCC events

Events like the Camp and the Congress are self-organized to the extreme. They explicitly want visitors to come in groups, and organize activities, rather than to leave everything to the central organizers. Groups provide stages to have talks on, toys to play with, electricity to use, places to hang out, and, at the Camp, food to eat. While some of those benefits are available to anyone, you'll be seen a lot more welcome if you earn your participation by helping the group, and it's best to start early.

Go to your local hackerspace, and check who's going. Offer your help unpacking, and people will simply like you. Help organizing the talks on your stage, and you'll get another shot at presenting a talk (see above why talks are important). Come to the meeting before trip, and you might find out that someone has free space in the car to carry you or your cool hardware projects to the destination.

Hotels

There's not much to be said about them. It's best to choose the same hotel where others from the conference are already staying. This way, you'll be able to take the conference chat all the way back, and maybe even start nerd-talking at breakfast.

Gadgets

A tech conference cannot exist without blinkenlights. If you have the space, bring your favorite toys. But only if --. don't carry your huge 3d printer on the train, thank you. Take your projects, posters, stickers, -junk- rare stuff you don't need --. anything that draws attention --. and give them away or show them off later. Just don't get so lost in the project that you hack on it alone. You can always do that at home.

Oh, and some events run a phone network with DECT phones, so if you have one, take it with you.

Travel

The day of the trip is now close. You've prepared well. A printed ticket to prevent troubles with a dead phone battery. A bottle of water, because train trips can be exhausting. Some money, to buy food and a replacement bottle when you inevitably lose the one you came with.

But don't forget that the conference starts already on the train. Decorate yourself with nerd jewelry: armbands from previous hacker conferences, helper T-shirts, sticker-adorned laptops, and IKEA sharks are dead giveaways of a nerd coming to a social event. When you start seeing those signs, the conference is officially open. You've met conference people. Talk to them as if you already arrived! After hours on the train, they are probably just as bored as you are. (Be careful not to talk to them so intensely that you miss your next train – true story.) Maybe you'll discover that you have common interests. Some prompts, depending on the exact situation: "Are you going to FrOSCon too?", "Is this a Framework laptop? Can you remove the HDMI port?", "Hey, I love Rust, too!", "Your cat ears are falling off."

If you don't know how to continue, the train trip is a good place to practice. A lot of fun from the conference comes from finding people with cool interests. The more people you quiz, the most likely you'll find the outstanding ones, so talk to many people! In case of a stall in the conversation, it's best to ask about their projects and reasons for coming.

Arrival, helpers

After the arrival you might want to set up your base. This is obligatory if you're camping, but it's still useful if you just want somewhere to place your toys. Choose it based on the people around. Your hackerspace? Perfect. Sleeping spots? This way the party never pauses. A cool installation? More opportunities to catch passersby for a chat.

There is something more important, though. Register yourself as a helper (angel, troll, whatever it's called this week), and have your schedule filled for you with hanging out with people who care (orga team, bar shifts), leaving your mark on things things (construction), and going behind closed doors (video team), among other things. And all of that while being respected for helping make the event run! Maybe you'll even get a T-shirt for your trouble. Being a helper is a win-win-win.

Schedule

Just don't overdo it. You won't be the only helper, and if the only open shift remaining is babysitting the children playground at 7:00, maybe that's not the best use of your time.

Treat the shift schedule as another conference track. Welcome desk shift at 11? Great, but the talk at 11:15 about cyber security in Bhutan is presented by this guy I want to meet.

Walk, watch, talk, play

Every conference is different, even if it's just a re-edition of the same event. Go around, discover the hidden corners of it. Pay attention to what you see. Maybe there's a poster with the DECT number to the model railway operator group. Check it out. Maybe there's a puzzle to solve. Maybe there's an easter hunt going on, and the corner hides a prize. Or maybe there's a secret cabal of uber-nerds discussing plans of cyberspace domination. Join them! Don't be afraid to talk to people. Or if you're still afraid, find another curious hacker, and go explore together (it helps). Go sweeping, and visit each stall, asking people there what they are up to (they are usually up to something).

Stay curious, and not just because it's a hacker's virtue, but also because curiosity will make your day exciting.

Style me! — announcement

Regular readers of this blog (meaning: me) could notice a change last month. The change is a new section of the web page called "Styles", linked from the navigation section. What is it?

It's a return to the roots. It's a revolution that comes back to the beginning. It's the return of control of the Web experience back to the user.

It's a dark style for dorotac.eu.

Hoops to jump

Of course, I'm not doing this like everyone else, oh boy no I don't. There is no JS on this web site which activates when you click the style link. The website does not set any cookie to select the style in the future. There is no button which you can click on an unmodified browser to make the style work – sadly.

The style is a (User)CSS file, containing the CSS to add to the page, and pretty much nothing else.

Unfortunately, no browser I'm aware of is able to use user styles without modifications. I recommend the Stylus extension: install it first, then choose the style.

User agent

Not many people realize, but browsers were once called "user agents". To me, it sends a clear message: this browser is your agent. It serves you, respectable user. It's there to further your interests on the Web.

Sadly, this was not meant to last. Browsers have been co-opted by interests other than the user. The most popular browser is made by an advertising company, which have strong incentives to respect the rights of individuals on the web – oh wait, never mind.

Those interests, naturally, try to push their own interests, even if it wrests choice and control of their Web experience away from the user. First, DRM support was standardized, encouraging companies to give the user a choice putting them in control: either we control your video player, or you can't see our stuff. A little further on, Google Chrome switched to Manifest v3, which removed support for a large swath of ad blockers (connect the dots yourself). Most recently, Google pushed for standardizing a version of DRM which would allow web sites to put the user in a situation where they either use a pre-approved browser, or go home via the WEI proposal. Control is going away from the user.

User styles

User styles are one of those things that bring a bit of control back to the user. They allow the user to see the Web as they please, even if the original designers imagined it otherwise. And the designers of the original Web page can't do squat about this!

This is the reason that dorotac.eu is so sparsely styled. I don't want to take your control away. Do you have a fancy green-on-blue default? Sure, I'll play with your rules. Do you want to add your own CSS via user styles? I keep custom stuff to the minimum, so that you can reorganize it easily.

The dark style on here is actually just a tweaked copy of the style I apply to all Web pages. Take it, and modify it! Oh, and if you create something really wild, send it to me. I'll publish it alongside!

Debugging Rust in QtCreator

Want to get straight to the juice? Here's the repository with the helpers.py file for debugging Rust with QtCreator.

They say debugging is twice as hard as coding in the first place, so if you write the most clever code you can, you can't debug it by definition. Instead, you should write code at most half as clever as you are, so that you are able to actually debug it.

In reality, a lot of your skill depends on your tools, so if you have a bad/good debugger, you have to be more/less careful.

I'm only a human, and I make mistakes: sometimes I write clever code. It's embarrassing to say, but sometimes I even enjoy it. When the inevitable bug appears, I desperately search for any ways to make the chore of debugging easier. For me, that means avoiding a naked gdb for something more visual. I like to see an overview of variables all at the same time, know their types without asking explicitly, and to be able to see which variables have changed.

Because of those preferences, I use QtCreator as my debugger.

C? C++? Rust?

QtCreator has been built with C and C++ in mind. That's all fine, but when I use Rust, it doesn't really know what to do with the variables I want to see. It doesn't understand enums, inserting weirdly named fields into them, and it gets completely lost when it comes to showing the contents of a slice or a string. Take a look at what a String looks like:

And now at this Option::None value:

That's not great. Slices, strings, and enums like Option or Result are Rust's bread-and-butter. It's challenging to write any code without those. Does that mean that I have to go back to gdb? Gdb understands Rust, after all:

test::printn (n=...) at test.rs:57
57          println!("{:?}", n);
(gdb) p n
$2 = test::Newtype (5)

...more or less:

test::printS (t=...) at test.rs:33
33          println!("{}", t);
(gdb) p t
$1 = alloc::string::String {vec: alloc::vec::Vec<u8, alloc::alloc::Global> {buf: alloc::raw_vec::RawVec<u8, alloc::alloc::Global> {ptr: core::ptr::unique::Unique<u8> {pointer: core::ptr::non_null::NonNull<u8> {pointer: 0x5555555abaa0}, _marker: core::marker::PhantomData<u8>}, cap: 3, alloc: alloc::alloc::Global}, len: 3}}
(gdb) c
test::printSl (s=...) at test.rs:73
73          println!("{:?}", s);
(gdb) p s
$5 = &[u8] {data_ptr: 0x555555598061, length: 2}

Helpers

Worry not. QtCreator's authors were smart enough to predict the need for such functionality. They provided a hook for something called "debugging helpers". Those are small programs which interact with QtCreator's gdb's session in order to turn the data into something the debugger panel can understand.

Long story short, I spent a bunch of time hacking a helpers.py file which you can put into QtCreator, and which decodes the most common Rust types. Get it in my Framagit repository and read the README.

Limitations

The project for which I needed this didn't make use of many types, so many things from the standard library are missing and may not work. Things like Path[Buf], OsStr[ing], Cells, Mutex, Arc are completely untouched. They shouldn't be hard to add, though. Patches accepted!

Beware! Rust doesn't have a stable ABI for the debug information, so those helpers are tuned for binaries produced with a specific version of Rust. I'll try to keep them up to date, but it's not a guarantee.

Finally, because I try to keep compatibility with plain C and with C++, I keep building on the basic helpers provided by QtCreator. Honestly, they are not thought out that great, and they need a rewrite, but instead I monkey-patch what was needed. If your answer it "well, contribute it upstream", then I'll just make it clear that I refuse to sign over my rights to the code by signing the Qt Company's Contributor License Agreement. Qt Company: if you're reading this, you're allowed to take the code under the terms of the GPLv3, but you do not get the permission to relicense it, just like I didn't.

SSH keys everywhere

This blog post is an edited and expanded version of the lightning talk I gave last week at GPN21.

As a reader of my blog, you probably know what SSH is. If not — the relevant part for this session is that it's a system allowing for secure, authenticated connections between two computers. It allows using passwords to log in, but the interesting part is when the user is logged in based on the provided key. Nearly every Linux hacker has a key pair which is used to log in to remote computers [citation needed].

Normies also enjoy secure systems, but those are based on a different technology. Connecting to web sites is secure and authenticated thanks to the layer called TLS. However, unlike with SSH, pretty much no one uses TLS keys to log in to web sites. Instead, the typical login process proceeds by making a TLS connection anonymously, and then a login mechanism built on HTTP or some other protocol (IMAP, XMPP, SMTP, IRC all have one) on top of TLS takes over.

Needless to say, using TLS to authenticate the server, and then a custom protocol to authenticate the client is more complicated than using TLS to authenticate both. SSH wins here by using the same mechanism (a public/private keypair) in both directions.

Another advantage of SSH is the one I mentioned before: many hackers already have a set of keys, and most understand how to use them.

However, there are not many libraries for creating custom services using the SSH protocol, compared to the ubiquitous TLS. What if I wanted to make a custom, secure, authenticated application, but avoid hacking around SSH's niche protocol?

Would it be possible to use SSH keys in a TLS connection?

You'll be pleased to hear that the answer is YES.

I put some effort into building a proof-of-concept pair of programs (server and client) which communicate using a TLS-based connection secured and authenticated using existing SSH keys.

Why is that possible?

Both TLS and SSH protocols are based on a similar set of cryptographic primitives. TLS 1.3 certificates support three signature schemes: RSA keys, ECDSA NIST P-256, P-384, and P-512 elliptic curves, and the pair of elliptic curves called ED25519 and ED448 (which seem to be unsupported by most Web browsers). SSH public keys come in different flavors: dsa | ecdsa | ecdsa-sk | ed25519 | ed25519-sk | rsa. The ecdsa key, once generated, expands to ecdsa-sha2-nistp256.

As you can see, there is some overlap: rsa, ecdsa, ed25519.

If we manage to extract the key data from the SSH public/private key files, and turn them into TLS-compatible public/private key files, then we should be pretty much set!

Except that there is no "public key file" in TLS. The closest equivalent would be the certificate file. The certificate adds some extra data about the certifying authority on top of the key file. But that's fine. We can skip all that, and create a self-signed certificate. (While certificates also exist in SSH, most setups don't use them, so there's no need to bother with them in a proof of concept.)

Now we have a clear path to SSH key support in TLS connections!

SSH security model

Remember known_hosts and authorized_keys? Those are not things used in most TLS-based applications. TLS applications are usually based on the Web PKI concept, where some entities are trusted by default by everyone to prove the identity of the servers. SSH is different: nothing is trusted by default. When you connect to a server, you get a choice to trust it to be who it promises to be or not. If you don't, the client will prevent the connection. Your choice gets stored in known_hosts. Similarly, the server administrator registers your key as allowed to log in by placing it in the authorized_key file. If you're not registered, the server will reject your connections.

By reusing existing SSH keys, we apply the SSH model to TLS, and throw away centralized certificate authorities.

Implementation

The proof-of-concept application is simple: the client tries to connect, checks the server public key, presents its own, and if everything goes right, the connection gets accepted by both parties, and the server sends the client a single "hello".

$ cargo run --bin server -- -k foo 
    Finished dev [unoptimized + debuginfo] target(s) in 1.27s
     Running `target/debug/server -k foo`
listening on [::1]:4433
connection incoming
connection remote [::1]:43131, proto "<none>"
established
accepted

$ cargo run --bin client -- -i foo2 --known_hosts known
    Finished dev [unoptimized + debuginfo] target(s) in 0.22s
     Running `target/debug/client -i foo2 --known_hosts known`
connecting from [::]:43131
opening
hello

The only supported key type on both sides is ED25519. Make sure to create new keys for the client and the server: this is a proof of concept, full of cut corners. It's possible that some of the corners became a security hole!

ssh-keygen -t ed25519 -f my_special_key

This is what happens if a client with an unknown key tries to connect:

     Running `target/debug/server -k foo`
listening on [::1]:4433
connection incoming
Unknown pubkey with fingerprint: [F0, 34, FB, EA, 59, B, 2D, 40, BD, 37, C1, 24, B1, 7D, 4C, D7, E4, 49, E3, 86, 7, A8, A9, 23, 1B, 1A, 9B, 6A, 5B, C5, 74, D6], rejecting connection.
IF THIS KEY SHOULD GAIN ACCESS, add this entry to your authorized_keys file:
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOD833SxhcyNayIdwoZfNJ12PK7heXfl5iHHKJWU/okB
connection failed: the cryptographic handshake failed: error 40: invalid peer certificate contents: Public key unknown

ssh-agent

The best way to ensure security of keys is to not have direct access to them. This is the job of ssh-agent: it loads a key, and signs everything you throw at it. The application never sees the key, which means it cannot leak it. This is perfect for applications written by someone so unskilled as me, so I implemented ssh-agent support in the client. I skipped it for the server, but it should be easy to do as well.

ssh-keygen and openssl

Without those tools, I would not have gotten anywhere. The first one can make a PEM public key from a SSH private key, and the other can inspect created keys and certificates.

ssh-keygen -e -m pem -f my_private_key > my_pub.pem
openssl asn1parse -inform pem -in my_pub.pem
    0:d=0  hl=2 l=  89 cons: SEQUENCE          
    2:d=1  hl=2 l=  19 cons: SEQUENCE          
    4:d=2  hl=2 l=   7 prim: OBJECT            :id-ecPublicKey
   13:d=2  hl=2 l=   8 prim: OBJECT            :prime256v1
   23:d=1  hl=2 l=  66 prim: BIT STRING        

Those help verify that the keys converted inside the application are indeed those which openssh would convert itself.

My eyes are bleeding

Have you seen the source code? Yes, I'm using 3 different ssh-related libraries. Each has parts which the two others don't, and which I needed to pull through. Also, the project came up because I was trying to learn QUIC, so it's made more convoluted because of that. As you can see, it's possible to write garbage, duplicates, unstructured code in Rust as well. I swear I will fix some of those things once I come back to the project.

Contributions welcome ;)

Lost SSH goodies

The SSH network protocol is not just security and key exchange. There are also things like channels and some awareness of X11/port forwarding specified in the RFC. Most standalone applications would not feel their lack after switching to TLS, but what about those which would? Those who open multiple channels and forward data around?

TLS can run over TCP which is single-stream-at-a-time, but I took another approach. QUIC incorporates TLS, and also [supports streams[(https://www.rfc-editor.org/rfc/rfc9000.html#name-streams) natively. Best of both worlds!

That being said, there exists an expired draft for SSH over QUIC already. According to my reading, it does not reuse SSH keys to establish the TLS layer, so in the end, it encrypts everything twice: once on the TLS, and once on the SSH layer (please correct me if I'm wrong). Not great for my server which is already limited by CPU, rather than network throughput.

There's also an implementation of SSH over QUIC, but it looks more like a simple proxy, than anything more involved. Twice encrypted again.

Future

Quishka, the project I wrote, is intended to become a library supporting arbitrary applications. If you decide that it's worth exploring the idea with me, get in touch!

With QUIC, there's few remaining reasons to stick to the vanilla SSH protocol, and I would definitely want to see the official server support it. Multiple nonblocking streams would make multiple file transfers in sshfs so much smoother, for example. I would be totally stoked if this toy project revived the idea!

Shell recipes

The last time I organized a workshop at the local hackerspace, I made copious use of the shell. The audience ranged from ambitious youngsters to greybeards who ate their teeth on decades of Linux. And yet, I managed to surprise them with a tiny piece of bash.

Here's the piece:

mv /home/dcz/file.md{,.bak}

Notice the brackets and the comma? This syntax turns the path into two paths: /home/dcz/file.md and /home/dcz/file.md.bak. How could a shell-hater like me know of it, while true hackers didn't? I think it's a testament to shell's discoverability of features. (Hint: it's awful.)

The tricks file

It turns out that other people I know have a similar problem with the shell: they remember that they did something, but they can't easily find the way after some time. They create a file containing useful commands or incantations. I do as well, and I was asked to share it.

Most of those tricks will not directly apply to you. But they can serve as a base to whatever use case you have in mind, after slight adjustments. And of course I'm not responsible for any loss of data resulting from using my recipes.

So here goes!

ARM emulation wth binfmt

This one is useful before you want to run a binary compiled for the ARM architecture. It requires QEMU to be installed, and a chroot containing an ARM file system. I usually use podman to pull an arm64 container, so I don't have to create the file system myself. I use it to build binaries for the Librem 5 without having to set up the cross-compiler.

echo ':aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7:\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:/usr/bin/qemu-aarch64-static:F' > /proc/sys/fs/binfmt_misc/register

echo ':arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm-static:F' > /proc/sys/fs/binfmt_misc/register

The podman command for armhf is:

podman run -ti -v /containers/build:/mnt/build:z --name=debian_armhf multiarch/debian-debootstrap:armhf-buster /bin/bash

Copy a hard drive with bad sectors (untested)

When a hard drive has bad sectors, the reading process will eventually hit an area that cannot be read immediately. Most copying programs will retry several times, or abort outright. This is wasting lots of time if you have thousands of sectors. Because I am diligent with backups, when my hard drive failed, I wanted to read it back to save the 2 hours needed to recreate some workspaces. A couple of missing files would not have hurt. So I came up with this command to read back everything, but without trying to hard.

Note that I specify disks by ID which contains the serial number. I never use /dev/sdx because it's too easy to make a mistake and overwrite something important.

CAUTION: I think I actually wrote this and never tested it.

dd iflag=fullblock conv=sync,noerror bs=1M if=/dev/disk/by-id/ata-Hitachi_xxx of=/dev/disk/by-id/ata-Hitachi_yyy

Trace file access

Applications break in thousand ways. Sometimes they crash outright, or don't pick up the correct configuration, but they won't tell you what they were actually expecting. For those cases, I take them on the strace dissection table and observe their calls in vivo.

Here's the command that will tell you everything about how an application uses files. Most useful for figuring out where it attempts to look for configuration files, or dynamic libraries (the LD_LIBRARY_PATH env var), or pkgconfig instructions (PKG_CONFIG_DIR). One of my most used recipes.

strace -fe trace=access,creat,open,openat,unlink,unlinkat --decode-fds=all

Socat

Socat is a great tool to translate streams from one form to another. It can listen or connect, convert between Unix pipes or TCP sockets. Here's a basic forwarder from localhost to another:

socat TCP-LISTEN:8000,fork TCP:192.168.4.4:8000

Git change authors

There are reasons you might forget to set your git identity. A new computer, or a computer where you want to work under multiple identities. You might realize that you had used the wrong email for this repository. Here's a recipe which alters the history of any repository by replacing your name on all commits:

git filter-repo --name-callback 'return b"dcz"' --email-callback 'return b"dcz@example.com"'

Git change identity

Here's a related one. If you have different identities, you might have different ssh keypairs for each. Here's how you set a custom keypair for the current repository. Don't forget to ssh-add.

WARNING: Opsec is hard. I think SSH will still disclose your other identities to the server when connecting.

git config --local core.sshcommand "ssh -i ~/.ssh/id_rsa.dcz -F /dev/null"

Git serve

When you want to share your local git repository to the wide world, you need git-daemon installed, and the following entry in the .git/config file:

[alias]
    serve = !git daemon --reuseaddr --verbose --base-path=. --export-all ./.git

GPT partitioning tool

This is the simplest recipe I have, but not the least useful.

sgdisk

Search for text files

The grep program will by default search for the given string in all files, including binary files. This is almost never what I want: when I look for text, I want to find text, and including binaries slows the search for nothing. Even worse, even when my string exists in the binary file, all grep says is "binary file matched", dropping any context. Completely useless. I use this to search only in text files:

grep -r -I text dir

Warm-plug of SATA drives

You can plug in SATA hard drives when the computer is running. But they will not always get detected. This is how you poke the bus to try to find newcomers (try host0 through host3):

echo '- - -'  > /sys/class/scsi_host/host0/scan

Then you can kick out the drive again by doing:

udisks --detach /dev/disk/by-id/ata-xxxx

and unplug it.

Show most recent messages in Gajim

sqlite3 ./.local/share/gajim/logs.db
select * from logs where message != '' order by time desc limit 10;

Show network connections in real time

It works also on OpenWRT. Needs tcpdump on the remote host and etherape on the host which inspects the traffic.

ssh root@192.168.5.1 tcpdump -n -i br-lan -w - not port 22 | etherape -m ip -r -

Nmap ping scan

Check who's up on the local network. A regular scan with nmap takes too much time.

nmap -sn -PE 192.168.5.0/24

Graphical podman

If you want to sandbox games in your podman, it's possible with some tricks:

Wayland:

podman run -e XDG_RUNTIME_DIR=/tmp            -e WAYLAND_DISPLAY=wayland-0     --security-opt label=disable       -v $XDG_RUNTIME_DIR/wayland-0:/tmp/wayland-0:rw -ti debian:buster /bin/bash

X11:

podman run -ti -v /tmp/.X11-unix:/tmp/.X11-unix:rw --security-opt label=disable game_container

Ripping a CD

I never actually tried this one. It's supposed to read all the data, and produce a more complete file than just ISO. ISO does not contain audio tracks, for example:

cdrdao read-cd --read-raw --read-subchan

Touchpad

Ever played a shooter game on the touchpad? Well, I was once bored enough on the train… But I needed to to make it possible to aim and move first. I did it by not disabling the touchpad while the keyboard is in use. First, check the device number in xinput list, and then:

xinput set-prop 11 298 0

Unrar

RAR files have an Libre decompressor, but it doesn't work on all files. I refuse to pollute my system with such tools, so I keep them contained in a container.

podman run --rm -v $PWD/unrar:/files:z maxcnunes/unrar:latest unrar e -or -r myfile.rar

Listening to WOL packets

Nothing tricky in this one.

tcpdump -i enp9s0 'ether proto 0x0842'

Widen

I don't use fixed width fonts in editors. Proportional fonts are mostly superior, but they have a downside. Sometimes "ASCII" diagrams are useful to quickly sketch an idea visually, and they are typically drawn assuming a fixed font on the reader end, so they appear as misaligned garbage when I see them.

Instead of assuming a fixed font, the authors could ensure fixed font. This is what this Python function does: it converts ASCII to fixed width, on the character level.

WIDE_MAP = {i: i + 0xFEE0 for i in range(0x21, 0x7F)}
WIDE_MAP[0x20] = 0x3000

def widen(s):
    """
    Convert all ASCII characters to their full-width counterpart.

    >>> print widen('test, Foo!')
    ｔｅｓｔ，　Ｆｏｏ！
    >>>
    """
    return s.translate(WIDE_MAP)

Video tricks

Ffmpeg is so versatile that it warrants an entire section.

Keyframes

Dropping all frames from the video except for keyframes results in a movie at normal speed but a very low FPS and lower size.

For video streams encoded in h264:

ffmpeg -i test.mkv -c copy -map v -bsf:v "filter_units=remove_types=1" key.mkv

This one is h265, I think:

ffmpeg -i IN.MOV -c:v copy -c:a copy -bsf:v noise=drop='not(key)' out.mkv

Metadata

To write all metadata (global, video, audio) to a file, use:

ffmpeg -i in.mp4 -c copy -map_metadata 0 -map_metadata:s:v 0:s:v -map_metadata:s:a 0:s:a -f ffmetadata in.txt

To add all metadata from a file, use:

ffmpeg -i in.mp4 -f ffmetadata -i in.txt -c copy -map_metadata 1 out.mp4

Reencode as h265

Those settings worked for me to get good quality:

ffmpeg -i ./XXX.MP4 -acodec copy -vcodec libx265 -bufsize 2M -maxrate 1M ./20.mkv

Extract the music

Losslessly. This commands works for music encoded using the AAC codec, but other codecs mean you have to use different output formats.

ffmpeg -i ../Music/movie -vn -acodec copy -bsf:a aac_adtstoasc out.m4a

Replace the sound track

When your camera can't record the sound in a good enough quality and you have to use a separate sound recorder, you will end up with a video file and a music file. This is how to synchronize the times and merge them:

ffmpeg -itsoffset -00:00:59.8 -i an.wav -i XXX.MOV -ss 00:00:00 -c:a flac -c:v copy -map 0 -map 1 -disposition:a:1 none -map_metadata 1 out.mkv

User-oriented desktop, part 1

Are you unhappy about current computer user interfaces? Do you mourn the unrealized potential of the desktop metaphor? Are tears of nostalgia coming to you when you see Windows 95?

I those feelings are real. You might be seeing something more than just personal taste. There's something actual that we have lost over time – possibly something few people ever saw – that is rubbing against the loudest complainers about user interfaces.

I don't know exactly what it is, so I'm writing this series of essays to narrow it down. Let's start with

Windows 95

What is the first thing you notice when you launch a couple programs on this ancient version of Windows? No, I don't mean the skeuomorphic look-and-feel. Tell me the second thing! That's right: most windows look similar to every other window. the control panel, the start menu, Word, Paint Shop Pro, ACDSee, Creatures 2, WinRAR. Apart from the occasional program from Windows 3.11, and the notable exception of Winamp, they are following the same theme, and they will all obey if you change your scroll bar sizes or button colors.

Image credit: Nathan

Image credit: WinWorld

Now let's come back to today and take a look at Windows. From reviews, I gathered that this is no longer the case. Steam, Discord, iTunes all look different, and they also look different than builtin software like the Control Panel, which looks different from another builtin, the Microsoft Store.

Image credit: Pureinfotech

Image credit: http://windows10quick.com/

Linux does not fare a lot better, with 3 different look-and-feels on my computer. GTK2 applications differ from GTK3, which look different from Qt, and all of those have a different setup for the user to apply themes (which carry over only imperfectly). On top of that, I have the occasional totally-out-of-place application like Blender, Cura or RawTherapee, which come with their own look-and-feel, and might or might not support a bespoke theme setting system.

I'm not entirely pleased by this. Spending time making every other app not burn my eyes out is not my idea of a good time. Why do I bother? Because I have needs which are fulfilled by having consistency. Let's put down some factors promoting → consistency in a diagram.

See, consistency promotes ease of use when I know what to look for and what to expect. It promotes my self-expression when tied to my custom themes. That boosts my feeling of ownership, which is also a need of mine.

Then why is the diagram not showing any reason to be inconsistent, you ask? It's because it's missing another relationship: factors inhibiting ⊣ consistency.

One thing is certain: making software consistent with your competition's software makes your brand blend together with the competitor's. Your software becomes less distinct and memorable, which is a grave sin in an economy as focused on attention and loyalty as ours. So let's redraw the diagram from the perspective of a developer who has needs and who chooses what to influence o⊣ o→ to fulfil those needs.

There are two main forces which influence consistency: the need to attract users mostly promotes consistency (and works in favor of user's needs), and the need to stand out as a brand, which is entirely against consistency and the user. At the end of the chain, increases in brand recognizability provide a feedback signal ⤏ that the need to have the brand distinct is getting fulfilled. Similarly with ease of use and the need to satisfy users.

This makes our model contain two opposing feedback loops! They balance each other to some extent. Branding cannot take over to abolish consistency entirely because it damages ease of use to an extent that nobody wants to use the software. The need to satisfy users cannot achieve peaks of consistency if the software has to compete with others, because it makes the software forgettable, and at some point it's more efficient to get more users by investing in branding rather than ease of use.

Those feedback loops can be viewed as working for two masters: one for the user, and one for the developer. And, watching the progress of user interfaces, I think we shifted to the developer more. The branding feedback loop is especially strong now, and keeps consistency levels low.

As a user, I'm asking myself the question: how can the balance be shifted back towards the user?

What would ultimate consistency look like?

If the need for branding comes from competition, could more cooperation break the feedback loop's power? And what other factors influence this network of relationships?

In part 2, we'll expand this simplified picture with additional factors.

Keypress entropy

A good password is hard to type. Is it a passphrase? My hand is strained from just looking at diceware's suggested passphrase NinetiethStunnedPurityStumbleDorsalReps (39 chars, 45 Keypress). pass is not much better with ''~E;I|pWZ)K:\?~=U&q-|8^V. That's 25 chars, but 17 shift presses on a US layout. Together it's 42 presses, almost as many as the passphrase. It's not a problem for pass. It's a manager, and it will remember the passwords for me so that I don't have to type it.

But I still need to type the master password. And it needs to be good, with 128 bits of entropy (entropy measures how many choices were made to generate the password) and a glittery wrapper. So I should just take the one pass gives me, right?

No, not right. There's no reason I should be mashing shift like crazy. It only doubles the available characters - 1 extra bit of entropy for each. For the 25-characters password, the shift key is responsible for 25 bits of entropy (about 6 characters), but for 17 extra key presses. That's not a good deal at all.

The wrong choice of a password generator hurts multiple times every day, when I type in the various passwords: my computer unlock password, the one to unlock the password manager, another computer, and whenever I need to install some new package or just use sudo. That's too much typing to rely on badly generated passwords.

Generating key presses

But I know how to generate passwords specifically for typing them in. It's to switch focus from characters to key presses. Just like the choice of a character from a given set represents entropy, so does choosing a key from the available ones. So instead building the password by choosing among characters:

a b c ... ABC ... 1 2 3 ... ! ? , ...

let's choose among keys:

` 1 2 ... q w e ... shift ...

Let's forget for a moment about how many thousands of symbols Unicode contains. No password manager generates ⸘, because they do typically make a concession to typists by limiting themselves to some version of the ASCII set of characters.

The actual difference between generating characters and keys is that there are about 100 characters the typical US keyboard layout can produce, using about 50 keys. See that 100/50 = 2? That's the extra bit of entropy we're giving up per character. We have to make up for it if we don't want to trade security for less button mashing.

A password generator

So how do we generate a password that still has our 128 bits of entropy, but isn't as buttony as the naively generated one?

We write our own generator, of course.

Hasword is a short Rust program I wrote for this purpose. (It's listed at the end of this post.) It generates a password, and tells you how many bits of entropy were used to generate it, the length in characters, and the number of key presses.

Here is an example of 128-bits-of-entropy passwords generated naively, from all characters:

# cargo run 128
g`{m>|.4->Q1.^(&n7<J
Entropy: 131.39711
Chars: 20
Presses: 30
Modifiers: 10

And here's the same, except generated from key presses:

# cargo run 128
t i8ep51\:,.9y0me 94;z
Entropy: 129.07819
Chars: 22
Presses: 23
Modifiers: 1

Oh no, the lower per-character entropy in the typist's password makes it 2 characters longer than the naive password. But it's a total win in the button mashing department: it's 10% longer but it needs only 23/30≈3/4 the key presses without being any less secure!

Now, the only problem left is to commit it to memory…

---

Here's the source code for the password generator. I use rdrand to block as many ways as possible for the attacker to guess the random numbers, but I do not guarantee that I did anything right here. You use it on your own responsibility!

/* COPYING: allowed under GNU GPL v3 or later. 
To build, do `cargo init` and add this to Cargo.toml:
[dependencies]
rand = "0.8"
rdrand = "*"
*/
use rand;
use rand::seq::SliceRandom;
use std::env;
use std::iter::repeat;
use std::str::FromStr;

enum C {
    Char(char),
    Shift,
}

use C::*;

const PLAIN: &[C] = &[
    Char('`'), Char('1'), Char('2'), Char('3'), Char('4'), Char('5'),
    Char('6'), Char('7'), Char('8'), Char('9'), Char('0'), Char('-'),
    Char('='), Char('\\'), Char(']'), Char('['), Char('p'), Char('o'),
    Char('i'), Char('u'), Char('y'), Char('t'), Char('r'), Char('e'),
    Char('w'), Char('q'), Char('a'), Char('s'), Char('d'), Char('f'),
    Char('g'), Char('h'), Char('j'), Char('k'), Char('l'), Char(';'),
    Char('\''), Char('/'), Char('.'), Char(','), Char('m'), Char('n'),
    Char('b'), Char('v'), Char('c'), Char('x'), Char('z'), Char(' '),
    Shift,
];

const SHIFTED: &[char] = &[
    '~', '!', '@', '#', '$', '%', '^', '&', '*', '(', ')', '_', '+', '|', '}', '{', 'P',
    'O', 'I', 'U', 'Y', 'T', 'R', 'E', 'W', 'Q', 'A', 'S', 'D', 'F', 'G', 'H', 'J', 'K',
    'L', ':', '"', '?', '>', '<', 'M', 'N', 'B', 'V', 'C', 'X', 'Z',
];

fn all_chars() -> Vec<char> {
    PLAIN[0..(PLAIN.len() - 1)].iter()
        .map(|c| match c {
            Char(c) => *c,
            _ => panic!(),
        })
        .chain(SHIFTED.iter().map(|c| *c))
        .collect()
}

/// Generates random characters. Considers entropy from each character.
fn chars_generator<R: rand::RngCore>(mut r: &mut R)
    // entropy, keypresses, character
    -> (f32, u8, char)
{
    let chars = all_chars();
    let bits_entropy_char: f32 = (chars.len() as f32).log2();
    
    let c = chars.choose(&mut r).unwrap();
    let presses = 1 + SHIFTED.iter()
        .find(|p| *p == c)
        .is_some() as u8;
    
    (bits_entropy_char, presses, *c)
}

/// Same, but considers entropy from each keypress.
fn typing_generator<R: rand::RngCore>(mut r: &mut R)
    -> (f32, u8, char)
{
    let bits_entropy_plain: f32 = (PLAIN.len() as f32).log2();
    let bits_entropy_shifted: f32 = (SHIFTED.len() as f32).log2();

    match PLAIN.choose(&mut r).unwrap() {
        Char(c) => (bits_entropy_plain, 1u8, *c),
        Shift => (
            bits_entropy_plain + bits_entropy_shifted,
            2,
            *SHIFTED.choose(&mut r).unwrap()
        ),
    }
}

fn main() {
    let needed_entropy = env::args().skip(1).next()
        .and_then(|s| f32::from_str(&s).ok())
        .unwrap_or(128.0);
    
    let mut r = rdrand::RdRand::new().unwrap();

    let gen = typing_generator;
    // uncomment the following line to use naive mode
    //let gen = chars_generator;

    let seq = repeat(()).map(|()| gen(&mut r))
    
    let mut e = 0.0;
    let mut chars = 0;
    let mut presses = 0;
    for (en, p, c) in seq {
        e += en;
        presses += p;
        chars += 1;
        print!("{}", c);
        if e > needed_entropy {
            break;
        }
    }
    println!();
    println!("Entropy: {}", e);
    println!("Chars: {}", chars);
    println!("Presses: {}", presses);
    println!("Modifiers: {}", presses - chars);
}

Maps à la carte

Those who follow my Mastodon account will know that I have a thing for bicycle trips. The bicycle is a technology that gave humans immense freedom back when it appears, and it remains one of my most favourite ways to meet the world on my terms.

Sometimes unrestricted freedom is the best, and sometimes we need to settle on a direction. And the best thing balancing freedom and directions on a bike is a map.

Maps are typically made for a group of people. Thanks to OpenStreetMap you can take a look at a map drawn for everyone, for the cyclist, or even for a biker and hiker. There's a service for generating maps ready for printing, too. However close they come, there are no easy tools to make a map for me, according to my preferences, showing things I need to plan my cycling.

Yes, I'd like a map made, please. For cycling. Make the asphalt local roads thickest. Motorways? Skip them. Actually, draw them the same way as fences. And canals. Mark footpaths too, please. And information posts, and tourist information points. While we're at it, mark monuments in a different color. Oh, and don't forget about bike repair shops, public water sources and grocery stores. Make them red! Mountain springs? Well, only if there aren't any wells nearby...

– me, ordering a cycling map in my imagination

Thankfully, I'm a problem solver, and I know a solution must exist.

Playing with crayons

If only I could draw the map on the computer like I drew fictional maps with crayons as a kid. A stroke there, a pictogram there, each takes two seconds total.

The maps I mentioned before use some sort of styles to generate the differently colored pictures out of the same data. The services are made for serving maps, rather than editing them, though. Tools like Mapnik require lots of set up, and even after it's done, editing the styles presumably isn't so comfortable.

What tools do the serious style designers use? Such a tool would put the ease of changing the looks at the forefront, even sacrificing speed…

A while later

Well, I found it. The project at the core of it is understaffed, and written in JavaScript, but works amazingly well. It's called TileMill, it's open source, and the maintainers are looking for fresh contributors!

Here's how to set up everything and render a map of your region.

TileMill

As a first step, I recommend creating a separate user on your machine. The project uses NPM, and that wil leave lots of cruft in your home directory. In addition, we use the dreaded curl-to-shell pattern, which better be contained away from valuable data.

I'm using Fedora 37 as the base system. Fedora doesn't ship with nvm, and the required version of npm doesn't work, so let's fix this:

curl https://raw.githubusercontent.com/creationix/nvm/master/install.sh | bash
source ~/.bashrc

Now, install some needed packages:

sudo dnf install git wget osm2pgsql postgis postgresql-server postgresql-contrib vim

Then follow the installation instructions with some changes:

git clone https://github.com/tilemill-project/tilemill
cd tilemill
nvm install lts/carbon
nvm use v8.17.0
npm install # this will take a while
npm start

Ignore the rest of the installation instructions. Now we have an empty instance of TileMill running on http://localhost:20009. Sadly, have no idea how to change the ports :( Anyway, navigate there with your browser and start a new project. Uncheck "default data", we'll use our own.

You'll see an empty project. Now that we have the renderer running, let's change tracks and prepare some data.

OpenStreetMap Data

You won't draw a map if you don't know anything about your area. Thankfully, collecting geographical data is what the OpenStreetMap project is best at. Don't be fooled, the map is only a side thing. It should really be called OpenStreetData.

Let's get the data of region that interests us from Geofabrik. I'll choose Münster for demonstration purposes: Germany is dense with all kinds of detailed data added by volunteers, and Münster in particular is famous for its bike-friendliness. Now download the .osm.pbf file:

https://download.geofabrik.de/europe/germany/nordrhein-westfalen/muenster-regbez-latest.osm.pbf

TileMill does not support loading it directly, but it supports something better:connecting directly to a geographic database. The cost of that is that we need to set up the database ourselves.

A while back, we installed the necessary packages: Postgresql with Postgis. Now it's time to configure them.

sudo postgresql-setup --initdb --unit postgresql

CAUTION: it's possible that I messed up the access controls here. An attacker on the same network might be able to get in your (data)base and kill your landmarks.

Edit the file /var/lib/pgsql/data/pg_hba.conf to widen permissions. Afterwards, the relevant part should look more like this:

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
# IPv6 local connections:
host    all             all             ::1/128                 trust

Create the database "osm", to keep our geographical data.

systemctl restart postgresql
psql -U postgres -c "create database osm;"
psql -U postgres -d osm -c 'CREATE EXTENSION postgis;'
osm2pgsql -c -G -U postgres -d osm ./muenster-regbez-latest.osm.pbf # you can list additional files here!

This will take a moment, but if it succeeds, your data is now safely stored. Test your network access:

psql -h 127.0.0.1 -p 5432 -U postgres -d osm

If this succeeds, then TileMill will be able to access the database, too.

My own map

Go back to the browser, and find the "layers" icon. It's shown on the picture:

Add a new layer, select PostGIS, and give it "dbname=osm host=localhost port=5432 user=postgres" in the "Connection" field. Write "highway" in "Class". Finally, enter the query in "Table or subquery":

(select * from planet_osm_line where highway!='') as lines

We're almost there. Now try this style:

Map {
  background-color: #b8dee6;
}

#highway {
  line-color: #808080;
  line-width: 1.0;
}

Enter it in map.mss, and press "Save" (or Ctrl+S). Suddenly…

What is that? Zoom in, please.

Now, this looks like the communication network in the region!

Come on, paint my world

I'm not going to give you a hand-holding here, but I'll leave you with a couple useful tips.

1. TileMill's documentation is quite decent for educating you how to style things.

2. Remember that your database holds 3 types of objects: points, lines, and polygons. Those don't have to be the same as in OSM, and, in fact, I don't know how relations are represented, if at all.

3. You can filter objects using CSS styles, like this:

#highways[highway='path'][bicycle='permit'] { foo; }

4. But dedicated layers are faster at filtering data:

(select * from planet_osm_line where highway='path' and bicycle='permit') as bikes

5. Take a good look at your database with psql -h 127.0.0.1 -p 5432 -U postgres -d osm.

Tables:

osm=# select * from # I pressed tab here
geography_columns    pg_toast.            planet_osm_roads
geometry_columns     planet_osm_line      public.
information_schema.  planet_osm_point     spatial_ref_sys
pg_catalog.          planet_osm_polygon  

Tag values:

osm=# select distinct highway from planet_osm_line where highway!='';
    highway     
----------------
 trunk
 road
 disused
 footway
 cycleway
 services
 secondary
 traffic_island
 tertiary
 abandoned

Example data (caution, long lines):

select * from planet_osm_roads limit 5;

6. Use more layers!

Printing

This application is not perfect for printing. It doesn't have a built-in rose of winds, nor a scale, and you're stuck with the Web Mercator projection. QGIS is way better in that respect, but also creating styles in QGIS is a lot more painful.

In a pinch, however, it's sufficient, and exporting is easy, too. The export menu is on the top-right. Make sure to aim at 600pixels per 2.54cm, and to select a zoom level that makes things readable.

That's it! Have fun with your new set of crayons!

Example

The picture in the first section is from a live demo I performed at the local hackspace. Here's the style for it:

Map {
  background-color: #fff;
}
 
#lines {
  line-color: #808080;
  line-width: 0.0;
  
  [waterway='stream'] {
    line-color: #aaf;
    line-width: 1.0;
  }
  
  [waterway='river'] {
    line-color: #aaf;
    line-width: 2.0;
  }

  [highway='path'] {
    [bicycle='permit'],
    [bicycle='yes'],
    [bicycle='designated'],
    [bicycle='official'],
    [bicycle='permissive'],
    [bicycle='use_sidepath'] {
      line-color: #444;
      line-width: 1.0;
    }
  }
  //[bicycle='permit'],
  //[bicycle='yes'],
  [bicycle='designated'],
  [bicycle='official'],
  //[bicycle='permissive'],
  [bicycle='use_sidepath'],
  [highway='cycleway']{
    line-color: #444;
    line-width: 2.0;
  }
  
  [highway='track'][surface='asphalt'][bicycle!='no'],
  [highway='service'],
  [highway='unclassified'],
  [highway='residential'],
  [highway='tertiary'] {
    line-width: 1.0;
  }
}

#lines {
  line-color: #808080;
  line-width: 0.0;

[surface='asphalt'] {
    //[bicycle='permit'],
    //[bicycle='yes'],
    [bicycle='designated'],
    [bicycle='official'],
    //[bicycle='permissive'],
    [bicycle='use_sidepath'],
    [highway='cycleway']{
      line-width: 2.0;
    }
  }
}
#roads {
  line-color: #808080;
  line-width: 0.0;/*
  [highway='secondary'] {
    line-width: 2.0;
  }
  */
  [highway='primary'],
  [highway='trunk'],
  [highway='motorway'] {
    line-color: #d8c1b1;
    line-width: 2.0;
  }
}

#toiletten {
  [amenity='toilets'] {
    marker-width: 6;
    marker-fill: #070;
    marker-line-width: 0;
  }
  [amenity='drinking_water'] {
    marker-width: 6;
    marker-fill: #700;
    marker-line-width: 0;
  }
}

#rathaus {
  [amenity='townhall'] {
    polygon-fill: #f00
  }
}

#towns {
  [place='village'] {
    text-name: [name];
    text-fill: #f00;
    text-face-name: 'Droid Sans Regular';
  }
  [place='town'],
  [place='city']{
    text-name: [name];
    text-fill: #f00;
    text-face-name: 'Droid Sans Regular';
    text-size: 20;
  }
}

Spots on the Sun and worn-out clothes

Have you seen the latest XKCD? It imagines a world where the Sun gets completely covered with dark spots, as opposed to only partially:

Can you see the surprising observation? The more black you add, the more spots you have. Until a certain point, where adding more black causes spots to merge. If this problem sounds remote, imagine a shirt. It starts out with 4 holes, but as it accumulates tears, it gains extra holes. Keep wearing it, and the holes will grow in size, eventually merging, until your shirt will be as good as new! At least when it comes to the number – but not the size – of holes.

But xkcd's plots didn't look quite right. Why is the sunspot number sinusoidal? And will it really get seriously dark while there's more than one spot?

I can't answer the first question definitively, as I don't know what model the author chose for darkness emerging. But I can check my own guess: if spots appear randomly and uniformly, as if a child ripped holes in your shirt deliberately, then I expect a sharp rise at first (unlike xkcd), then a slow incline until peak spots, and finally a slower decline… I guess. Like this:

But xkcd is explicit about the number of spots everywhere, so the second question is possible to answer definitively. Why do I think Sun wouldn't get so dark unless there's only one spot? It's because there's plenty of space for bright areas even with a couple big spots. When you add extra dark areas, it's going to be hard to keep them separated.

Okay, but how do I intend to test it? I don't have a miniature Sun, after all. If you're guessing that I'm going to use up my shirt supply for science, I'm also going to disappoint you. Of course, I'm using a simulation (code at the bottom).

The simulation makes some assumptions. First, we're poking white holes in a black canvas. The canvas is square and flat. Each poke leaves a white plus-shaped mark containing 5 pixels.

Experimentation

So I ran the simulation on a couple canvas sizes, gathered the number of pokes to reach perfect black, and watched the brightness as the peak was reached and crossed.

That's what I gathered:

Looks different, doesn't it? The most striking is that the number of spots falls to 1 and remains there for the majority of the cycle. It's not sinusoidal at all. That's easy to explain: it's hard to hit a solid piece when there are only holes remaining, and my model doesn't try to be smart about it. It usually takes 9000 pokes to cover the entire picture, whereas a single hole emerges before the 3000 mark.

But that doesn't mean that it's always completely dark, either. As I predicted, reaching a single sunspot is not enough to put out the Sun. The grayness on the lower plot doesn't succumb to complete darkness immediately. But it's hard to see, so I prepared another plot:

On this plot, we still have about 1/3 brightness when a single superspot takes over. That's not what xkcd expected! Another interesting thing is that we hit peak spots before dropping to 50% brightness.

Here's my methodology for the charts:

I don't have a firm opinion on how to model the disappearance of spots, so I just mirrored their appearance in the plots. I also assumed that spot sizes are uniform, and fudged the average spot size to be 1/10_000 of the whole (after looking at figure 4 in On the size distribution of sunspot groups in the Greenwich sunspot record 1874–1976), giving a canvas 70px in size for a 5px spot.

Late nights bring weird ideas.

Simulation code

To run the simulation on a 40x40 pixels canvas, use python3 holes.py 40. This is holes.py:

#!/usr/bin/env python3
# how many holes?

import cv2
import numpy as np
import itertools
import random
import sys

rand = random.randrange

w= int(sys.argv[1])
h=w

image=np.zeros((h,w,1),np.uint8)

try:
    for i in itertools.count():
        cv2.circle(image, (rand(0, w), rand(0, h)), radius=1, color=(255), thickness=-1)
        contours, hierarchy = cv2.findContours(image, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)

        if i % 10 == 0:
            cv2.imwrite('out_{}.png'.format(i), (255 - image))

        spots = [0 for c in contours if cv2.contourArea(c,True) < 0]
        whites = cv2.countNonZero(image)
        print(', '.join(map(str, [i, len(spots), whites])))
        if i > 100 and len(contours) == 1:
            break

except KeyboardInterrupt:
    cv2.imwrite('out.png', image)

Graphical debugging on the Librem 5 using QtCreator

This blog post was paid for by Purism, as part of the work I do on the Librem 5 mobile phone.

I've always had a weak spot for debugging with graphical, user-friendly tools. Whenever I run bare GDB, I'm limited to what I can see at once, and learning about the choices available to me is a chore. Navigating the code is not integrated either. Clearly, I need some extra tooling to make GDB worthwhile.

QtCreator is such a tool. It checks the above boxes, and also some more. One feature in particular was taunting me for a while.

Remote debugging

Wouldn't it be great to compile stuff on my powerful desktop and debug it from my laptop on the sofa? Or, for that matter, to debug programs that access the camera on the Librem 5?

It isn't as easy as it could be. Sure, you could always attach to a GDB server ad-hoc, but then you have to deploy manually, start the GDB server, and you don't get to see the code you're debugging. Kinda useless.

There's also the mode where the debugger integrates with your project, taking care of deployment and startup automatically using SSH. Just like local debugging! Unfortunately, it quickly became very clear that it's meant for embedded devices. There's a strong focus on setting up a cross-compiling toolchain, compiling locally, and only then doing the remote debugging. Sorry, ain't nobody got time for that.

But QtCreator is flexible enough to allow for custom deployment, and I finally cracked it! This is how you get

Remote debugging on the Librem 5

Instructions for QtCreator 4.13.2.

Devices and Kits

Let's make QtCreator know that you have a device you want to take care of. Go to Go to Tools → Options → Devices → Devices tab → Add. You'll be presented with a choice, select "Generic Linux device". Fill in choices in "Connection". I chose the name "evergreen". The next screen will let you choose the SSH key to use. If you're confused here, you should read up about SSH key-based authentication. QtCreator will open SSH connections to the Librem 5, so you don't want to get password prompts all the time (once you have the key, make sure it's in your keychain. ssh-add ~/.ssh/my_key.pub works for me).

QtCreator has a notion of "kits", which define the set of libraries used for building a project. We'll need to define one on the new device for the Librem 5. Go to Tools → Options → Kits, and add a new one. Duplicating the default one also works, I think most fields don't matter. Choose the name for your kit (I chose "evergreen" again), change device type to "Generic Linux Device", and select the device you just added. Make sure to select some compiler, otherwise you'll get complaints from QtCreator later.

Building and deployment

You need to import a project if you want to compile and then debug things. I'm going to skip this – you can easily look up some tutorials online – and go straight to the details regarding remote deployment.

Deployment is configured in the "projects" section on the left panel. I select the project I'm interested in (simplecam), and notice that, in addition to the default "Desktop" kit, there's also an "evergreen".

There are two positions here: "Build" and "Run".

Build

Let's rework "Build" first. Make sure that there are no build or clean steps. (If you intend to build the project manually, you can now skip to the "Run" subsection.)

We're going to use this step to synchronize git sources between the local checkout and the remote one. That means we need to create a checkout on the remote host. Adjust the following command to match your preference, and issue it:

ssh purism@10.42.0.185 git init ~/simple-cam

Now, add the new repository as your remote in the local git repository (adjust to match the remote host):

git remote add evergreen purism@10.42.0.185:~/simple-cam
git push evergreen master:foo

Now come back to the remote host and initialize the build directory.

# this is on purism@10.42.0.185
mkdir simbuild
cd simbuild
meson ~/simple-cam

Now, save the following script somewhere on your local computer. It'll be responsible for deploying each version. Make sure to adjust the paths if yours are different!

#!/bin/sh
set -e

REMOTE=purism@10.42.0.185
DEST="~/simple-cam"

ssh $REMOTE "git -C $DEST checkout -f foo"
git push evergreen -f HEAD:master
ssh $REMOTE "git -C $DEST checkout master"
git push evergreen -f HEAD:foo
ssh $REMOTE "cd ~/simbuild && meson $DEST && ninja && ninja install"

Finally, go back to QtCreator's "Project" section, and choose the "Build" configuration for Evergreen. Add a process step, and point it to the file you just created. Make sure it executes in the git checkout by setting "working directory" to "%{sourceDir}".

I used "sh" as "Command", and the path to the script (redacted) as Arguments.

Run

Here's the meat of the operation. You're given one method: "Deploy to Remote Linux Host". We don't actually deploy anything here, because the executables are built on the remote host. Remove all deployment steps.

Add a new run configuration on the remote host. For me, it's called "Custom Executable (on Evergreen)". Add the executable path on the remote host. I use: /home/purism/simbuild/simple-cam. "Local executable" can stay empty. Environment changes take effect as expected.

Debugger

When I tried debugging at first, I was dropped into disassembly mode. That is clearly not optimal. A lot of the appeal of a debugger is seeing the source code being debugged.

Fixing this involves some extra steps. First of all, this affects only some binaries. For me, what was not picked up was what I installed manually: the simple-cam binary, and libcamera. To fix this, you need to find the correct mapping between the files on the local computer and the paths embedded in the remote executable.

This method worked for me: I added a new breakpoint at "main" by function name, and started debugging. Make sure to select the correct configuration: it's on the left panel, above the "Run" button. Select your project, the kit you just made, and the remote executable.

Then, I opened the debugger log via View → Views → Debugger Log. Put the cursor in the "Command" prompt while the program is paused on disassembly, and type "bt". In the right pane you'll get something like:

1918bt
>&"bt\n"
>~"#0  main () at ../simple-cam/simple-cam.cpp:139\n"
>1918^done

This shows the path embedded in the binary that QtCreator (or GDB?) failed to find. Now you need to map it to a path on your local file system. Open Tools → Options → Debugger → General, and add the paths to the "Source Path Mapping" table. Here, I enter "../simple-cam" as "source path", and my local path to "simple-cam" as "target path".

Keep in mind that I redacted part of the local paths.

You can stop the debugging now, and start it again. Now you should see the sources for the paths you configured. You should see something like this in the "Application Output" tab on the bottom:

18:09:20: Checking available ports...
18:09:21: Found 101 free ports.
18:09:21: Starting gdbserver --multi :10000...
18:09:21: Debugging starts
Listening on port 10000
Remote debugging from host ::ffff:10.42.0.1, port 44146
Process /home/purism/simbuild/simple-cam created; pid = 22866
File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead.

Congratulations, you are now debugging your Librem 5 from your workstation!

Debugging symbols

If you need to debug a call into a library that isn't part of your application, you may see that this setup balks. It either ignores the call, drops down to assembly, or displays "??" in the stack trace.

This is a telltale symbol of missing debugging information for the library that the remote host is executing. But even if you install the debugging symbols and sources (in Fedora it's dnf debuginfo-install [name], in Debian it's hell), on the next run you get… no change.

It turns out that the debug symbols and sources must be available on the host side of the debugging connection. Let's get the symbols over:

scp -r purism@10.42.0.185:/usr/lib/debug /somewhere/l5_debug/

and hook them up to GDB by adding a new line to Tools → Options → Debugger → GDB → Additional Startup Commands:

set debug-file-directory /somewhere/l5_debug/debug

Now you'll be able to see symbol names, but you still can't inspect the sources of the library. Sadly, you have to find the sources of the package on the remote end, and hook them up yourself in the "Source Path Mapping" table, like we did a couple paragraphs earlier. Copy if needed, no automation here.

Limitations

This relies on git to push your changes to the remote host, so you must commit your changes if you want to run them. Otherwise the code shown may not match the code executed.

It could be adjusted for plain files, but this is what I have now.

When you try to debug libraries provided by the OS, make real sure that you're using the correct sources. Debugging something when the cursor is randomly one line off is hard to notice, and it makes for a frustrating debugging session.

Jazda: Rust on my bike

This blog post is an improved version of an impromptu talk I gave at FrOSCon two weeks ago.

I like to do things that don't quite make sense. One of them is putting rust on my bicycle.

This is my bicycle, and it's made out of steel. It has some rust already, but it's not the kind of "rust" I want to talk about.

I put Rust the programming language on my bike.

Bicycle computer

I bought my first bicycle computer before I could even program. It was a simple mechanical device that is mounted near the wheel. It counts wheel rotations, and displays a distance.

My next upgrade was an electronic device, one unlike what you could buy today. Those have more functionality: they may display distance, speed, and time travelled. I used one for over a decade, and it was possibly the best spent 20 EUR in my life.

Two things happened between then and now. One is that I learned to code. The other is that the battery ran out. Instead of replacing the battery, I came to the obvious conclusion: I can code, so I can make a better one!

Hardware

I've been struggling to design the hardware for my bike computer for years. Sadly, I suck at hardware design. I can't solder, I can't design a plastic case, I can't build a device that won't shake apart during a most peaceful ride. It became clear that if I want a bike computer, I should start with existing hardware.

Meanwhile, the situation in the outside world slowly changed. Smartphones gained popularity, Internet of Things followed, smart watches started utilizing the new tiny and powerful components. Bicycle computers started taking advantage of the new powers too.

I'm much better at programming than at hardware, so I stopped to consider: should I take advantage of that influx of hardware, and adapt an existing device to my needs?

I could slap a smartphone on the handlebars, write an application and call it a day. Or I could find a smaller device, more like the bicycle computer I retired, and use it as the base for my software. But which path should I choose?

There are many cyclists in the world. There are those who commute to work every day, others who go on family trips, some cycle to deliver things, and yet some like to race. They have different goals and needs while cycling, and each is interested in something different from a cycling computer.

This time, I'm building one that makes me happy. Which of my needs as a cyclist can a bike computer help with?

I usually cycle in places I roughly know, so I don't need a map, or a big, fragile screen to display one. I want to time my rides, so I want a screen that's big enough to show a few numbers in a big font, and the bike computer must be sturdy enough when I push the tempo over rocky trails. When I cycle, I leave my phone at home, so I won't miss anything if the bike computer is 100% offline. I draw on maps for OpenStreetMap, so I want some sensors like GPS, and a place to store their data. Thankfully, handling sensors and statistics doesn't require much computational power.

A smartphone meets those needs, but has some important downsides. Most smartphones aren't well readable in sunlight. They have short battery life – days compared to weeks – and they are not very resilient compared to other gadgets. I often get caught in the rain, and I'm pretty sure too much road dust or violent shaking is not healthy for smartphones – even if they don't come flying out of the harness.

The conclusion is pretty clear: I would be better served by a dedicated device. And so I started my search.

However, I'm only a single hacker, with limited time and abilities. I want to build a bike computer from scratch, I have another need – as a hacker, not cyclist: the device must be simple enough so that I can hack on it myself.

And I did see some interesting examples, like one based on Linux. I rejected this one purely because I was afraid about the hardware complexity – if the manufacturer decided they needed a full-blown OS, then there must be a lot of hardware to manage there.

Finally, I found one. The Bangle.jS 2 smart watch checked most of the boxes: lots of sensors, Bluetooth Low Energy, a screen readable in the sunlight, and even a GPS receiver! It only has one button, and it can't make sounds, but it was reverse engineered, and ready to flash with custom software. It was then or never: if I didn't use this as my base, I would probably never finish the bike computer project.

Rust

There are lots of choices in the embedded space. Operating systems like Nuttx, Espruino, Mbed OS, Riot, Zephyr. Or even running on bare metal. But my goal was clear:

I wrote enough C code to know I don't want to use it. The 2 Rust-enabled options were Riot and bare metal using rust-embedded crates.

Actually, there were 3 options. At the last moment, I discovered Tock, an OS written entirely in Rust. It has an advantage over all other options (except Espruino): it's a pre-emptive operating system, able to load applications at runtime.

And loading new apps is a standard function of your computer, of your smart phone, and some smart watches. Why isn't this standard on bike computers yet? Puzzling, but if the sports gadgets manufacturers won't do that, I gladly will.

Tock project

Applications made for Tock are native code, can be written in C or Rust. Because of the multiprocessing architecture of Tock, we can split functionality into apps, like a speed meter, or a clock, and not worry how buggy they are: they may crash all they want, but they are separated from each other, so a crashing clock won't bring your down speed display.

Imagine a future where people load applications from the internet on the bike computer. Being able to just ignore a crashing app will be absolutely necessary.

But this kind of safety comes with a cost. You have to write a lot more code to abstract hardware resources. Instead of writing just one device driver, you actually need to write 3 pieces: one kernel driver, one userspace driver, and one multiplexer.

Status

Despite the slower pace, I managed to put Tock OS on the Bangle.js 2 hardware rather quickly. I started with a demo displaying speed:

The demo is part of the Jazda project (which is what I called the bike computer), and it shows that the display stack is working, and that the GPS stack is working.

There's still lots of work before the grand vision can be realized. The main parts are:

• Bluetooth support
• Concurrency: there are compiler shortcomings that prevent this part of the OS from really gaining speed
• Communication between apps
• Communication with a computer
• Publishing apps online: a website, payment service

Jazda started as a hobby project, so I never forget about fun projects:

• Seismos, the sensor collection file system
• Ray-graphics, the SDF-based graphics library
• Skitram, the unpublished time-series data compression library

GPS vs BLE

Currently, the greatest shortcoming of Jazda is the lack of Bluetooth Low Energy. This is a wireless protocol normally used by bicycle sensors. What Jazda is using right now is GPS readings. Unfortunately, GPS is rather energy-hungry, which means the device can only display speed for 5 hours before turning off, and the readouts aren't very accurate, either.

Tock has a strict policy of not allowing direct hardware access, so I can't just snatch an external BLE stack. If I did that, I would be stuck maintaining the result myself, because it wouldn't be accepted upstream. Alternatively, I could write a BLE stack myself, but Bluetooth is notoriously difficult to implement correctly.

For now, I'll just keep working on the other parts.

Innovation

Thankfully, speed readouts are overrated. There are plenty of other things that can be done without them. I collected some ideas:

• drawing a situational map
• wheelie detection
• surface roughness scanning for OpenStreetMap
• sonar mode for chasing a recorded ride
• ???
• profit

Some of them come from myself, some were suggestions I heard. Perhaps I won't be able to implement them all, but that's fine, because Jazda is open source software. Anyone is allowed to implement any crazy idea without signing an NDA and without the need to find a job at a sports equipment company. Having a shower thought and some perseverance is all that's necessary.

Community

But it's always easier to hack when other people can help, so you're invited to join our chat on Matrix or IRC: #jazda:libera.chat . You can also reach out to me there to order development kits for Jazda. Those come with a unique breakout board to make it easy to reflash the Bangle.js 2 smart watch with the Jazda firmware.

If you're a hardware hacker, you're especially welcome. Perhaps you could help us build a future version of Jazda, without any shortcomings… and without useless heart sensors ;)

Why Jazda?

This is a cleaned up transcript of a lightning talk I gave at the Gulaschprogrammiernacht 20.

---

Have you ever had a project that you worked on for so long that you forgot why you're doing it? I was just asked "why are you doing this?" about a project I started 10 years ago, and on which I really started working this year. The project is called Jazda [https://jazda.org].

Why do I do it? It's an open source bicycle computer. There are a few of them, but not many. The general idea of a bicycle computer has been explored in so many ways. There's Garmin, which is almost a monopoly. You can also buy one at Kaufland for 10 euro. So why do you need another one?

If you take a fancy one like the Garmin, you'll find that it's not programmable. You cannot do it, you're not allowed to do it. If you take the Kaufland bike computer, it's just too simple. If you try to mount your phone on the bike, it's bulky and inconvenient. So yeah, I wanted something that I could program to get the four freedoms of software and to experiment with it.

There are other options. There are smart watches, like the PineTime, or Bangle.js. There is the Cyclotron, which is a simple bike computer that's also open source, including custom hardware. But I don't actually want any of those. Why not? First off, a bike computer needs to be readable in sunlight. Most smart watches can't do that. Bangle.js 2 can, but JavaScript? Sorry, I'm not a fan. The Cyclotron is the closest to what I want, but since I came all the way here, then maybe… let's not stop here?

If I use a simple single-purpose device, and connect it to the computer with a flasher to load software, it would be too technical, I would be basically the only person in the world interested in using it. What if I took the next step? What about a bicycle computer with apps?

Why should a bicycle computer be more difficult to use than an Apple phone or watch, or your Android? And what if you had a nice SDK to program it? Yeah, that's something to do.

Basically, this is the idea with Jazda. The project is still early, still not there, the goal has not been reached yet.

But you can already buy them from me, so I'm not still in the weeds, but there's still a lot of work required.

---

At the conference, my talk ended by inviting folks to talk to me and describing my hangout spot. It was a little awkward to explain.

Online, it's easier. I hang out on the IRC/Matrix channel #jazda:libera.chat . Stop by sometime!

Git-botch-email

I don't really like git-send-email. I avoid projects that use it, if I can.

It could have been much better, but I (thankfully) use it rarely enough that I can't come up with its list of sins on the spot, and I don't end up getting my arguments acknowledged.

Today (2021-09-24) I have the dreadful need to send a patch to the Linux kernel, and I'm going to make the most of it by lining up my thoughts. Hopefully someone takes it to heart and creates a git-sends-email-better.

Email is all right

Don't get me wrong, I like the decentralization aspect of email, I like that it's based on open standards, that I can use an arbitrary client to send it, and that I don't have to run random code on my computer just to submit or review patches. Forges fail at those, to various extents.

But git adds its own quirks

I have a tree with my commit on top: it changed 3 lines. I want someone to include my changes on top of the official tree. Let's assume that I read the project's contribution guide, and I know that I need to use git-send-email, and I also know where to send the change. Let's begin.

$ git send-email 
No patch files specified!
git send-email [options] <file | directory | rev-list options >
[...]

"patch files"? I thought git operated on the basis of commits and trees. Why would I want to send a patch file using git? Anyway, there's a "rev-list" possiblity, so we can proceed with that:

$ git send-email 00082f898de21fd5ebb28dc561c173f6fde8e44a
/tmp/7yZAWWiApC/0001-media-imx-Fix-rounding.patch
To whom should the emails be sent (if anyone)?

I answer the questions, and then I get:

Send this email? ([y]es|[n]o|[e]dit|[q]uit|[a]ll):

Lol, no. It's my first submission, and I want to review it. Possibly by giving it to someone else first. I'm only human, and I make mistakes.

But where's the "save" option?

Okay, never mind. Maybe there is a "save" option further along the way. After all, I didn't give git any email access yet – I just want a dry run. It's not going to throw away the edits it offers you to perform, right?

Send this email? ([y]es|[n]o|[e]dit|[q]uit|[a]ll): y
sendmail: Cannot open mail:25
Died at /usr/libexec/git-core/git-send-email line 1497, <FIN> line 3.

…Actually, it totallydid throw them away. Thankfully I didn't make any edits (j/k, the first time in my life I tried to send a patch, I followed a tutorial and already wrote a heap of text at this step. That hurt).

Do you know how the cat keeps meowing but you have no idea what it wants? If only it could speak human language.

If only git-send-email could speak human language.

Some searching later, it turns out that, indeed, git wanted to have access to my SMTP server to send the email itself.

Git sending emails

Let's get back to the save fiasco. I have a perfectly cromulent email client which can import .mbox or .eml files, and send them on. It's already configured for my email server, it can sign my emails, I trust it with my passwords, and it's customized to save sent emails to IMAP.

Why can't I save my git patch, and load it into my email client, to send it with my GPG signature?

Why should I trust git with full access to my email outbox? Why shouldn't I be able to use my usual Mail User Agent (MUA) to do the sending in a way that I pre-approved, respecting the security level I want to maintain, including TLS versions? Email is based on standards, after all.

I just hope Git doesn't handle the email password itself, given its track record.

Oh, and don't forget that if you have a fancy multi-identity setup in your MUA, you have to duplicate it with git-send-email. Sure, git has a global config file. But If I configured that to my work account, then I'm one mistake away from sending a personal contribution to some other random repo from my work email. Double difficulty if you contribute under multiple different contexts.

So I'm stuck at having to configure every repo which demands git-send-email separately (use --local instead of --global. With git-send-email, opsec is harder.

Just give me the damn save already!

Acceptance

Okay, I guess I have little choice in the matter. Sigh. Let's change the email to point to myself as the addressee. This will obviously not be a dry run for the final email because of the address mismatch, but hopefully nothing starts on fire when we change the addressee later.

Great, it seems to look … odd. I was not expecting the patch to be part of the message. How am I supposed to download and apply it? By copying and pasting? It makes sense: those who reply can easily add their comments inline. I guess I can live with it. As long as I don't change the contents of the patch, it should be possible to extract.

Intermission

But wait, git-send-email allows you to edit the message before sending, and mess it up in any way you want?!?

Send this email? ([y]es|[n]o|[e]dit|[q]uit|[a]ll): y
OK. Log says:
[...]
Result: 250

LOL, git-send-email apparently doesn't do any validation. I hope that email didn't actually get forwarded anywhere. I put in "none" in the address field, but I dont really know well enough how git-send-email works. I've seen it add addresses. That wouldn't have worried me with my MUA.

Back to the draft

Oh, right, maybe I can actually edit the draft from my MUA this time! If git-send-email doesn't try to stop me from being an idiot, I have no reason to use it now.

I copied the email from inbox to drafts, and added the timely message.

Intermission 2

Sometimes it's convenient to annotate patches with some notes that are not meant to be included in the commit message. [...] Such messages can be written below the three dashes "---" that are in every patch after the commit message.

So, if I want to include extra context inside my email, I should cram it in between lines of computer-readable text and hope for the best?

The changes from 451a7b7815d0b pass 2^n to round_up, while n is needed.

Fixes: 451a7b7815d0b
---
Hi, I tested the patch on the Librem 5.
drivers/staging/media/imx/imx-media-utils.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/> imx/imx-media-utils.c
index 5128915a5d6f..a2d8fab32a39 100644

That must be the best way ever invented to ensure people accidentally alter the wrong thing.

Applying the patch

I was hoping not to have to go through the ordeal, but I don't want to bother my co-workers with my non-work blog. I'm being too nice.

I was joking when I said "copy-paste" the email content, and yet, this is how Stack Overflow deals with it. And while I'm finding a bunch of guides for sending, that's the only answer dealing with receiving git-send-email results. Alas.

Surprisingly, it worked-for-me.

A couple exchanged emails later, and… I need to change the commit contents. Since the change was small, I just winged it, and altered the email directly, but if I actually had to go through the entire ordeal of git-send-email, I'd lose my changes again.

Revisions

Finally, my email went to review. Of course, it didn't come out unscathed. I had to submit a second version, which consisted of multiple commits.

Here again, git-send-email showed how being careful results in paper cuts: each commit is sent as a separate email. That would have been okay of I could just load them into my draft folder and send from there, but, as we established earlier, the only way to do that is to send the email to myself. That means I have to change the recipient list for each email in the series manually. What botheration!

More tool problems

Even after the patch was sent successfully, it's still less than a commit. It does not preserve the history of how the author got there. The base tree is discarded, and only the diff remains. Without an external convention, the reviewer does't even know which tree to apply the patch on, much less where it was originally tested. It's impossible to merge trees using git-send-patch either. There's not much "git" in it, really, because commits and trees are what git is made of.

Linux problems

There's a separate class of problems that are not the consequence of the tool, but the consequence of the culture which is often associated with the tool. It's fixable without any software changes, but it needs changes to wetware – which is probably even more difficult.

Here's a loose list, based on the Linux kernel:

How to find the correct address to direct the patches? How to find the correct tree to apply the patch on? Magical incantations are sometimes required like "Signed-off-by:" that have a meaning other than its constituent words (no, it doesn't mean your email is signed). Those things are documented, but they are not universal, and not obvious. And they are not difficult in the same way as writing a kernel patch is difficult – those are pure bureaucracy overhead, which does not have any bearing on code quality.

More overhead is in supporting ancient clients that can't handle MIME or compression. Those are banned in the kernel, instead of fixing clients to handle compression and inline disposition. Another exercise to support inflexible tools at the expense of human effort.

And the mother of all pet peeves: hard-wrapping of commit messages. Do you prefer to read prose by incessant scrolling, or with nonsense lines? Random example from the Linux kernel mailing list:

Monospaced text hard-wrapped wider than the display, reader scrolls every line horizontally

This is what reading a patch would look like on the Librem 5 – depending on how you prefer to suffer – and it's also what any other pre-formatted text looks like. Including your commit messages. Have mercy and don't do hard wrapping.

It can be better!

Those are, in the end, not unsurmountable problems, at least not all of them. When used together with a competent email client, git-send-email does its job passably. The only paper cut is not being able to export the emails without sending them.

It gets worse if there's no competent email client. Then the task of editing, reviewing, saving for later and coming back, and fixing mistakes falls entirely on the same tool. As far as I can tell, it's rather tedious, and the ability to get an overview is rather poor. There's no way to save and resume at all.

I'm not going to give it a score better than "passable" even with this papered over: it remains a fact that git-send-email almost encourages the submitter to accidentally mess up the patch by entering text in the wrong place. Until the payload is clearly delineated from the cover letter (as an inline attachment?), this cannot be solved.

---

Some time later

One of my patches was eventually accepted. The rest have been picked up by some poor soul who has been trying to get them upstreamed for the past 2 months.

git-send-email didn't get in the way any more, but I still messed up. I missed some feedback in my inbox, and thought the patches were completely forgotten, until the new person stepped in. You might blame my way of reading email, and you would be right: my email inbox is a mess. I've been trying to fix it for the past 5 years with little success. Meanwhile, I rarely lose feedback when it's placed on a web page, because it organizes conversations in a sensible manner.

Thankfully, that problem is not inherent to email, and has been solved by sourcehut. It closes the gap between an email-only workflow and a web-only forge by providing a single contact point, and displaying the knowledge (including historical data) in a decent manner. My analysis wouldn't be complete without mentioning it, and I ask projects using git-send-email to adopt it or something similar: I can't handle naked emails, and I'm not the only one!

But, being an internet service, sourcehut doesn't fix the problems in git-send-email. Perhaps they could come up with an improve version of that tool? I sure hope they could.

A potion of experience

You may notice that this blog has just been enriched, especially if you're following my Mastodon account. See that thing all the way past the article? It's a comments section. Cause I want to have a conversation with my readers, rather than keep shouting into the void.

You can surely see it's pretty basic.

I made it myself. See, I had my eye for a while on this elixir that would give me some network development experience, and I decided to use it. I had to supplement it with some extra stuff to get anything more than just experience, but I'll talk about that later.

Glug glug

It's not a mystery: it's just Elixir. It's a language running on the BEAM virtual machine, and taking advantage of the Erlang/OTP runtime. OTP has fascinated me for a long time, with its "crash-only" approach, a concurrent take on the Actor model, and aggressive immutability.

In short, it's a programming paradigm outside of the orthodoxy I've known. And what better source of new insights is there than an unusual point of view?

Disclaimer: I'm still rather new to Elixir, so if I mess up some terminology, let me know… in the comments ;)

Level up

The feature of Elixir that I liked the most comes from Erlang/OTP: it's the actor model. Your program gets split into… uh, "applications", which operate as independent threads, and can exchange messages. For example, the web module sends a message to the email module to let me know I should approve a new comment, and doesn't even wait for an answer.

If the email sender crashes, the web service keeps churning on, and only the email component gets restarted. And I don't even need to care about the restart logic!

That's already cool, but what if I want extra logic in the email sender? I don't want to be spammed by a lot of emails when my blog reaches the front page of Hacker News, but rather have the email module wait for 5 minutes between notifications.

Immutability then changes the rules of the game: you can't just save the time of the last email in a local variable, and handle messages in a loop. There's a kind of a local database for each application, where data can be explicitly stored. One thing comes to another, and I implemented a state machine to deal with the notifications.

And that's where I levelled up: I stated using state machines, which exchange messages to cause transitions. It's an excellent organization to debug, because you can serialize the state of your module, and see how exactly external events cause changes in the state. If you add a little more effort to turn side effects into more messages, you can perform "in vitro" state transformations as part of your test suite.

Bitter taste

Elixir is not all good though. The one especially bittersweet part is how much metaprogramming it allows. The basic syntax ends up being simple (or so I'm told), but once you actually start using Elixir, expect surprising syntax constructs. Here are a couple of examples that keep confusing me.

Most statements are contained between do and end, like this:

if true do
  x = x + 1
  y = y + 1
end

But not stuff inside branches of a case expression:

case foo do
    true ->
      x = x + 1
      y = y + 1
    false ->
      x = x
end

I feel uneasy each time I write this. What's the delimiter marking the end of the true branch? Note that indentation doesn't matter here.

---

Another annoying property of some Elixir libraries is spooky action at a distance, where the reasons for doing something are implicit and hidden away behind layers of abstraction. Here's an excerpt from the Plug library tutorial:

defmodule Example.Router do
  use Plug.Router

  alias Example.Plug.VerifyRequest

  plug Plug.Parsers, parsers: [:urlencoded, :multipart]
  plug VerifyRequest, fields: ["content", "mimetype"], paths: ["/upload"]
  plug :match
  plug :dispatch

  get "/" do
    send_resp(conn, 200, "Welcome")
  end

  get "/upload" do
    send_resp(conn, 201, "Uploaded")
  end

  match _ do
    send_resp(conn, 404, "Oops!")
  end
end

Take a look at the plug lines. They will take care of encoding and verification. It's nifty because you don't have to worry, just slap those in. It's confusing because if you're a newbie and want to stay near the basics, those constitute a barrier. It's opaque, because there doesn't seem to be a syntactical opening to declare some calls to be affected by those filters.

In the end, I avoided the problem by never using such clever tricks, at the cost of having less learning material (not that the material with tricks taught me anything).

---

In the end, it comes out as a mostly positive coding experience, and I'll choose Elixir over Django in the future.

But coding the web app is not all.

Hangover

After coding, I had to deploy it on my server somehow, or else experience is all I get. This grew to be a full half of the experience, and a half I'd rather do without.

Let me start with Ansible.

I don't like it.

It disappoints me in one crucial area: it does not compose. I can't easily create Ansible instructions to deploy Beng on a pristine system for developers, and then reuse the same instructions to deploy it on my infra alongside other pages. Isn't that what programming languages are good at? Executing batches of instructions differing by parameters? Perhaps I'm too much of an Ansible noob, but I haven't found the necessary flexibility there.

So I wrote a couple idempotent shell scripts to replace Ansible.

INTERMISSION

Back to Elixir - it turns out that the binaries need a certain version of the Erlang runtime to be present on the destination system. CentOS 7 didn't have the same version as my development machine. I couldn't build them in a CentOS container either.

Long story short, I gave up on CentOS and went with Nix to build the comments app.

The upside is that if I want, NixOS can take over a lot of what I needed Ansible for: building the software, configuring it, installing dependencies. The downside is, when I came back to the app 6 months later, my Nix package utterly and completely doesn't build. I'm not so sure about employing Nix for server config duties now.

END INTERMISSION

My shell scripts were still needed to move the data and configs from the development machine to the server. Step by step, they grew into something bigger. Something monstrous. Something like… Ansible? But with some neat features that Ansible doesn't have:

• my system can deploy stuff in Docker containers or SSH hosts. Ansible needs SSH to function.
• My system supports actual conditionals.

Sadly, it doesn't compose much better. I still have some hardcoded things I don't want to share with the world, so it will remain unpublished for now.

Aftermath

The Elixir app and the deployment sweat were a good lesson in humility. Originally, I estimated the whole thing to take a week. It took two weeks of intense work across several months. Last week I estimated the deployment portion to take an evening. That alone took a week. But now I have something to show for it, and the lessons I learned from Elixir levelled me up as a programmer, so… I guess it was worth it.

Browser tab archaeology

Tab hoarding is a hot topic these days. There are people who feel bad about it. Even the name "hoarding" is negatively coloured. Obviously, the internet is making a big deal out of using tabs as bookmarks, because it seems like we have multiple bookmarking options, and they all suck in practice.

But I'm shameless about it: browser tabs are my reading list, and browser windows are how I keep groups of links on a single topic that might be useful one day. At least until and unless a better solution emerges.

Sediments

And I hope it does, cause my tab count keeps increasing! Oh, the times when I used to have 120 tabs! Long lost in the murky waters of history, tabs buried under further 100 layers of sedimentary URLs!

If only I was an archaeologist, I would order an expedition to find the gems I buried and forgot about. "Oh, that bunch of tabs comes from the period when I wanted to learn Haskell! A very valuable finding!"

If I was a tab geologist, I would order a vertical cut through the layers of rocks, look at the patterns, and guess the macro-scale processes that unconsciously guide my tab hoarding today.

Actually, maybe I am a tab geologist!

Simple Tab Groups

I have this folder on my computer. It contains the history of my open tabs – daily snapshots going back two years. Here's an example file called stg-backup-2022-01-23@drive4ik.json:

{
    "version": "4.5.2",
    "groups": [
        {
            "id": 82,
            "title": "default",
            [...]
            "tabs": [
                {
                    "url": "https://en.wikibooks.org/wiki/Haskell",
                    "title": "Haskell - Wikibooks, open books for an open world"
                },

You see, I'm not the only one who uses tabs extensively. There's a lot of Firefox extensions to make them more powerful, and the one I settled on is called Simple Tab Groups. That's where the snapshots come from.

Excavation

Equipped with the historical snapshots and a string of Python, I dug into my own past. The entire 2 years stood in front of me, and revealed some facts that I might not have liked to know. The worst of them all is that I keep descending. When the first measurement was taken, I had merely 120 tabs open, and I peaked at 298. After days of concerted effort, I'm back to 205.

You can see it clearly here, on the age diagram:

It shows the general progression from prehistory on the left to modernity on the right. On the vertical scale, each non-white pixel is one open tab (yes, tabs are attached to the top of the image, fight me). Each day has a different colour assigned to it, and a tab opened on that day stays that color for its entire life.

Okay, that's not exactly right. Those dark horizontal streaks and dots – I think those are URLs I opened again later. Perhaps the front page of a news site, or simple the empty new tab.

As you can see, tabs opened in ancient times are assigned black, and later, differently colored layers gradually accumulate.

Clusters

While the smooth colors show the long term layering, they don't show the patterns: do tabs settle down one by one, or do they get forgotten in batches?

Another colouring helps answer this. Here, neighbouring days have contrasting colours:

And the answer is… a single tab can get forgotten, but about as often as a bunch of tabs. I wonder if I'm special in this regard.

Prehistory

But the last view still didn't shine any light on the dreaded dark layer. Thankfully, while dating is impossible, we can still analyze the material each tab was made of. i decided to colour each domain a different colour. Here's the result:

It's curious to see that there is clustering here too. What could the wide bands correspond to? A Wikipedia binge? Opening lots of comment threads on a social network? That kind of makes sense, until you notice that each wide band has a different colour – different domain! They correspond to some sorts of deep dives into topics I guess.

Statistics

A side effect of having access to that data is that I can also take some simple statistics. It turns out that, over 641 days,

• I visited 3659 URLs
• across 1213 domains,
• my last tab was usually something on Hacker News,
• and I beat the record of tab hoarding on 2022-01-28 with 298 tabs open.

Pickaxe and brush

I'm not one to hoard the tools of my trade, however. Any aspiring introspector can take advantage of what I did here.

You'll need Python 3 and the Pillow library. Then, download this script, and run it:

python3 tas.py ~/Downloads/my-tabs/ my_group image_name

Remember to replace my_group with the name of the tab group you wish to dig into!

So, are there any other Simple Tab Groups users here? Please show us what your hoarding looks like. If not, see you in another year!

Rust on Maemo

Everyone wants to run Rust on their phone, right?

But it's not so easy when your pocket device is the Nokia N900. You'd think that I should be using the Librem 5, but my pockets disagree. Actually, I agree with them, too, at least as long as I use the device to collect GPS tracks.

The phone is over 10 years old by this point, and it's not open, which means it's running outdated software. Nokia stopped supporting Maemo around 2011, and the last community update happened a couple of years ago too. Despite that, I'd like to run some of my own code on it. The canonical way to do it is to use the SDK via a Docker image, but it's rather clumsy.

GNU libc

My first attempt to compile code for the N900 was to use a recent version of Debian, running on an emulated ARM CPU. It didn't go far: after copying to the phone, the program failed to run. Glibc is too old.

Easy, I thought! I can always package up glibc together with the program via static linking. As I tried it out, I was greeted with a crash message along the lines of:

This kernel is too old.

Jolly, so Linux version 2.6.28 doesn't meet modern standards. It seems my road is blocked, and this was just a "hello world" C program.

Musl

A while later, I realized that the kernel complaint probably came from glibc as well. But Rust doesn't have to use glibc, it can also use musl as the C library! Let's check Rust's platform support page. The CPU on the N900 supports the ARMv7 instruction set, so we're looking for armv7-unknown-linux-. And here we go: apart from armv7-unknown-linux-gnueabi, indicating the usage of glibc, there's also armv7-unknown-linux-musleabi for musl.

Looking closer at musl's FAQ page and the section on requirements:

Linux 2.6 or later.

Awesome! That means our ancient kernel is supported.

Rust has the ability to cross-compile, which makes testing the solution easy. I picked up [my earlier project] related to the N900, added a .cargo/config file, and typed in the following commands:

dnf install -y gcc # needed for syn at compile time
dnf install -y lld # no armel gcc toolchain for Fedora, so use clang linker cause it does armel out of the box
curl https://sh.rustup.rs -sSf | sh -s -- -t armv7-unknown-linux-musleabi
cd /mnt/src/grconverter
source $HOME/.cargo/env
cargo build --target armv7-unknown-linux-musleabi
cp target/armv7-unknown-linux-musleabi/debug/read /mnt/n900 # copy onto the phone

…and it's alive!

$ ./read MyDocs/gpsrecorder/gpstrack-20210804-140132.gpsr | head
Header(
    Header {
        magic: [
            71,
            80,
            83,
            82,
        ],
        format: 3,
        time: 1628078492,

Or at least half-alive. Linking to external libraries is still an open question, but the standard library works, which means pure-Rust crates are good to go.

Happy hacking!

Reverse engineering a pen

Like a lot of people, I have a favorite pen. I ran across it randomly, but I fell in love with it instantly. It glides across paper with barely any pressure, leaves strong, confident, and uniform streaks, and it's built like a tank to boot. But, unlike most pens that are worthy of appreciation, this pen is really many pens – it's disposable.

While the ink is bright an saturated, and there's a selection of colours, the Uni-Ball Eye is not meant to be refilled. To someone trying to reduce my waste production, using up one perfectly good pen every year is completely unacceptable. Knowing tricks back from the bad days of ink jet printers, I bought a vial of fountain pen ink, a syringe, and I set out to refill the carcass of my most recent pen.

Knowing that this is a blog post about refilling a pen, you might already guess that it wasn't as easy as it sounds.

Sesame open

Before refilling, I destroyed a previous used up carcass to find best ways to inject liquid into the pen. It took me a while with the scissors – the outer shell is really thick – but eventually I found out that the back end has a thinner wall, and a needle did the job.

I also dissected the funny comb-like part near the tip that always looked like it filled with a little bit of ink. It's actually a second ink vessel, whose role I discover while refilling. Surprisingly, this vessel is not connected to the tip. There's a wick going from the tip straight to the big compartment, inside the tubular, comb-like second vessel.

Here's a diagram:

(The walls are much thicker in reality.)

Shooting it up

For the first refill, I simply poked a hole in the back of the pen, placed the conjoined pen and syringe horizontally, and squirted in a few drops of the liquid. The wick became flooded with ink, and I could write again! But… all the ink slowly seeped into the comb container, leaving the wick dry after a minute. Even worse, two "nostrils" near the tip readily released ink as long as the comb was full. What's going on?

This comb container on a new pen is already partially filled. The red streaks are ink.

An obvious culprit is gravity, but the ink wasn't flowing all at once, so there must be more to it.

The comb shape drew my attention. Being composed mostly of walls, it gives the liquid plenty of tight spaces to which it can stick. Did the manufacturer intend to make use of the capillary effect? It would help liquid flow into the comb.

There was the open question of whether the pen was originally airtight, but adding pressure into considerations would make the problem too complicated, so I followed the next obvious lead.

Capillary effect

If I could prevent the comb compartment from filling up, ink would stay in the main vessel, and I would write indefinitely. The only difference, apart from the opened end, between my refill and the original, was the ink used. Perhaps the original ink was tuned specifically for this pen design. Hoping to alter the liquid to be less eager to climb up the comb, I tried a couple mixes with glycerin and water.

Nosebleed

I tried those tests also with the syringe hole plugged. That's when something unusual happened: the nostrils produced bubbles of air, like a kid who dropped an ice cream. Clearly, that was not just gravity's fault: air doesn't flow out under its own weight.

The story of fountain pens leaking inside airplanes came to my mind. Perhaps this pen had similar considerations?

Air pressure

I had reasons to believe that yes, they do. First off, why would a pen need "nostrils" connecting to the ink compartment, if not to equalize pressure? Coupled with the comb compartment, this made sense. When the pen is in an upright position, ink flows towards the tip, and there's a pocket of air near the butt. That air would expand when surrounding air pressure drops, and push out ink from the main vessel. The comb area would contain the excess, and the comb structure itself would prevent it from flowing out.

Every time I tried to use the pen in the morning, it cried me tears of ink, flourishing my handwriting in a special way. Pressure is the hint. Mornings are cold. My hands are warm. Gases expand with temperature. Air pushes out ink. It all makes sense!

Flawed methodology

Sadly, my tests so far were all flawed: I never allowed the comb to dry up so far, and if the pressure theory was right, then the pen could work correctly only when the comb had plenty of empty space. I came up with a new procedure:

• use the pen until all the ink was released – via the tip or the nostrils
• carefully plug the injection point.

If my theory was right, then the following would happen:

• the liquid would have enough surface tension to not flood the comb after a few hours
• the wick would stay submerged in ink, and never dry up
• writing on cold mornings would make the comb fill up a little
• but it would not be enough to fill up the comb and cause spills.

Experiments

My daily companion when researching this article: the ink blot.

The plan didn't work out flawlessly, and every attempt was about as fun as watching ink dry. That's because using up all ink took up ages, and the ink blots added a special flourish to my handwriting. I made several failures to plug the injection point: it turns out that just sticking a needle in there is not enough, and ink flows out the back. What's good enough is glueing it up using some plastic foil and liquid latex.

While that works to keep the ink out the back end, it turns out that the ink still flows out the front! That invalidates my theory. Or does it?

I didn't want to seal the pen irreversibly in case that I needed to adjust something, so the end was probably letting air seep in a little. I left the pen overnight in different positions, and, after watching the ink in the comb change, concluded that gravity was responsible for the failure this time.

Without any better ideas, I went ahead and mixed in some glycerin into the ink vessel, which seems to have stopped the outflow. Success!

Inkblots

That last experiment proved the minimal changes needed for the Uni-Ball Eye to be reuseable:

• use ink with about 10% glycerin
• refill from the back while the comb compartment is not full
• plug in the refill hole afterwards.

The comb compartment is probably a way to avoid leaks due to pressure.

The top pen in this picture is the refilled one. In reality, both inks look darker.

The ink I used is not as intense as the original one, and it doesn't flow so easily, but on the other hand, it's a bit thinner, and still flows decently enough. It can work as a cheap alternative to a new pen, which normally costs about 3 EUR per piece.

Lessons learned

Turns out that there's a good deal of thought and complexity embedded in every day objects. Physical objects are take more effort to understand than as software. You need to control your environment, retrying the experiment can take a lot of time, and you may get yourself dirty in the middle. None of this will be a surprise to an experimental scientist, of course.

It's still a joy to explore, and the results are going to be a fair bit easier to explain to a random human – the outcome is satisfyingly concrete!

Start in the middle

It's a piece of advice I read years ago in the depths of the 'Net, which I can't find despite best efforts any more. It's surprising, because it's such an obvious guidance! Especially for those easily distracted, or prone to losing motivation.

TL;DR: When you work on something new, jump into the thick of it. Don't let yourself be sidetracked, and attack directly what interests you.

Storybuilding

Homer put Odysseus in media res as Odyssey begins: he's trying to return home after being shipwrecked. There's little backstory, or discussion about motivations. You focus on the current state, and wonder: how did he end up so? What challenges will he encounter? Can he succeed? You're hooked, and keep reading to see the questions answered.

It's good news for the distracted creator, because those are the same questions you may ask yourself when you have an awesome idea for a new project. What are the challenges? Can the idea succeed? You can get hooked on your own project by cutting out the fluff, and jumping straight into it.

Focus

No one has built Skynet yet. If you are overly ambitious, chances are that you will have to build a lot of supporting infrastructure. Instead, choose the core element and stick to it. If you want to take pictures of all the graffitti in your town, don't bother with setting up a web site, just take photos. If your idea is to program a fancy reverb effect, don't bother turning it into a library.

In fact, don't bother with all the surroundings. Before you write all the READMEs, set up a documentation site, and publish the library on your favourite web site, you're risking that you run out of energy to actually do the thing you set out to do.

Side tracks

Sometimes you might want to build something that is so original that no one even built the tools you could use. A spaceship out of water bottles? Perhaps needs heavy duty glue, and all the glues you found are suboptimal. Suddenly, you're spending days researching chemistry, and the spaceship project is not moving anywhere.

Everything is cool if you enjoy your glue side project, but if your only motivation is to build a rocket, then perhaps you need to cut your losses and use one of the store glues. Get back to the middle: the spaceship, before you start hating the project! Once you nail the central part, either you're going to have so much motivation you'll deal with the side projects, or you will blissfully not care any more. Win-win!

Software

The article is written about general projects, but software is especially prone to it, because there's so much complexity and potential challenges. I wouldn't have finished fluw if I didn't take this approach. Thankfully, knowledge stays, and the more you build, the easier it will be to choose tools and minimize the time spent not in the middle of things.

Protagonist

You're the protagonist in your life, and challenge you create for yourself are stories you live. So take advice from the art of writing, start in the middle, and chances are that you're going to enjoy more what you do.

When being a nerd paid off

A few days ago, my knowledge of obscure computing facts surprisingly saved the day! Surprisingly, because sticking to garden variety best principles would have left me worrying. What was the deal? It's about a deleted photo on my camera.

---

It was a sunny day, so I copied photos off my camera onto the computer, and set out with a sense of journey and a clean storage. Or so I thought: when I stopped to take a photo, my camera complained about full storage, so I selected "delete all photos". While the screen showed "erasing", I realized in horror: I took a cute picture of a dilapidated wagon already! And I'm not going back there again to take another one.

Common advice in those situations says: don't use the storage medium under no circumstances, and attempt recovery on a computer. Then you might be lucky and keep your deleted data from being overwritten.

This advice is right, but the trip was just beginning, and I was not ready to stop taking photos. Thankfully, my nerdy sense told me that I have a very good chance to get the picture back even if I take more photos – as long as I don't go crazy with them. That's due to some secrets that I learned about technology:

• computers (and therefore digital cameras) are made by humans, and humans like simplicity,
• modern storage is logically structured a bit like a cassette tape (or a photo film),
• "deleted" data does not actually get cleared until it's overwritten by new data.

You might be surprised to learn about the similarity of storage to tape, but it's there on hard drives, actual tape drives, SD cards, and SSDs. The structure exposed to the programmer is of a long string of small pieces of data (sectors), each of which has a number, starting from 0, almost like houses on the street.

Accessing sectors (by writing or reading) is simplest when it's done in order. We're only humans, and why use an elaborate scheme if we can just move from one to the next? And so, the reading head slides steadily on the tape, the film in a camera moves by one frame after each photo, the postman visits houses in order. And it's not just out of convenience: it's clearly the fastest when there's a moving physical object involved. Surprisingly, writing in order is even the fastest way to write data on SSDs, which have no moving parts!

Append only

This must be coupled with an observation: cameras – or at least my camera – only add new pictures onto the SD card (our "tape"), and when deletions happen, the entire contents of the card are marked as deleted. Just like in a film camera: once a photo is taken, it's taken. We can replace the entire tape, but we don't replace a single picture with a blank.

This makes life easier, because imagine if we had a tape where we could erase already taken pictures. At first it's okay: we take 30 pics, and we hit the end. Then we erase pictures number 7, 2, 17, and 22. To take another picture, we would have to fiddle with the tape to find an empty spot, then rewind it to the right position (make sure it's not overlapping!), and do it again until the tape is full again. That's no fun. It's easier if pictures come one after another.

And while I don't have any evidence, I assume that's how my digital camera works, too. It's not a general purpose computer, so it doesn't have to deal with a lot of deletion. I can safely assume that the picture of the wagon (which was the last picture on my SD card before it filled up) will not occupy the same sectors as the first 10 pictures I take after the card is erased, and that it will not get overwritten.

With a peace of mind (and ready to be proven wrong), I continued to take photos of the trip. When back home, I fired up testdisk. Lo and behold, I recovered the picture!

The real world

Okay, maybe I just got lucky. In reality, there's one thing that changes size. For SD cards, it's usually the File Allocation Table. It contains the numbers of sectors which contain pieces of each file, along with the names, and a bunch of extra info. As enough files and directories get added, it probably changes in size.

Another complication is the mysterious "PRIVATE" directory. I presume this contains only hibernation data to make the camera start faster, and I'm not seeing any reason for it to change size at all (what kind of generated data does a camera need anyway?).

In the end, what matters is not exactly that sectors get written from 0 up. What matters is that there is a defined order of writing which doesn't change between full card erasures. That's much more difficult to verify.

FAT32

FAT32 doesn't actually mark sectors as "deleted". It marks the entire file as deleted, without permanently erasing any of its information until it's needed. Testdisk knows this, and it makes recovery a breeze. The entire operation took 5 minutes.

CAUTION

This is all based on high level guessing, so don't rely on this. I made peace with losing the deleted picture when I decided to keep shooting new ones. Basic advice is still best: if you accidentally delete something important, immediately stop using the storage, do not mount the file system, and make a block level copy. And why don't you restore it from a backup anyway?

Linux kernel flame graphs

This post is a nothing-burger. All I wanted was to have those words together on the Internet: "Linux kernel flame graph profiling", because I spent about an hour looking for something I already knew would take 2 minutes to execute.

So, to whoever comes to ask the same question as I did back then, the solution is not perf, it' also not trace-cmd, but instead it's eBPF-based. It's bcc. Read the description of the profile tool. TL;DR:

./bcc/tools/profile.py -dfK > profile.flame
flamegraph.pl < profile.flame > flame.svg

Conveniently, Fedora has both flamegraph and the bcc library (not the tools) in the repos already.

This is what comes out the other end:

As you can see, my system is spending a lot of time in the kernel, messing with btrfs. Then it overheats, causing it to spend even more time in the kernel, due to idle injection. Then, it gets really sluggish, making the operator look for kernel profiling tools, and the operator finally writes a blog post about this.

The bug looks like this report.

Nothing-burger

If you thought I had something valuable to say, then… Well okay, maybe I shouldn't write blog posts if they are useless to anyone but me. You win. Give me a moment, I'll come up with something insightful to say.

---

Maybe you're not familiar with flame graphs, but they are a really nice way to check for performance issues. They show the amount of time the resource (in this case the CPU) is busy with each task. The width of the graph is 100% time, and the width of the different bars is the portion of time occupied by something.

This flame graph does not distinguish between different runs of the same task. If we were monitoring tasks done by Joe over the day, and he'd spend the morning slicing potatoes, then chopped wood, and then sliced potatoes again, this kind of a graph would only distinguish two tasks: slicing potatoes and chopping wood.

This makes sense when you consider why the bars are stacked vertically: each bar on top of another is a sub-task. If a task is composed of a hundred sub-tasks, presenting them together makes the picture clear. When we care about optimization, then it's useful to know how the total time spent peeling compares to the time spent cutting. The times for each potato aren't so relevant. It doesn't even matter a lot how we order the results on the left-right axis!

It's especially relevant for computers, where flame graphs may be used to get an idea how much time is spent in a procedure that may be called a thousand times. While the graph won't say how many times it was executed, it will show where it was called from.

Bonus

This is about the kernel, which is notoriously hard to debug, so let me share my most useful snippets for debugging. Those use trace-cmd, and give extra information without the need to recompile.

1. Prints the names of functions (with some exceptions like inlining) along the execution path:

sudo trace-cmd record -p function_graph -g start_procedure -F ./program_under_test

2. Prints the call stack whenever the chosen procedure is executed:

sudo trace-cmd record -p function_graph -g procedure_under_test -n printk -n dev_printk_emit -F ./program_under_test

View results with trace-cmd report.

Simulating fluwids

Nerd sniping. It doesn't apply only to mathematical puzzles, and the victim doesn't have to be another person. It's not uncommon to nerd snipe oneself with a nifty programming idea. Especially one that looks simple enough to turn into a working demo in a matter of hours, if not minutes.

This is a story of how I decided to have a working fluid simulation later in the evening.

---

This article uses MathML for rendering equations. If you don't see any, consider using a browser that can display maths, like Firefox.

Divergence

The idea entered my consciousness as I was daydreaming about cells floating in liquids, to give Breedmatic a biological special effect. As I considered what makes things flow, I was struck with a realization: a simple flowing fluid can be modelled with one equation!

If we simulate the fluid flow as a vector field F, then any volume V inside it satisfies our first equation:

{\iint }_{S}F\mathrm{n̂}\cdot dS=0

where n̂ is the normal vector to the surface S.

In simpler words, we can slice the fluid into any volume V, but the outflow of fluid is always going to be 0.

This itself is derived from the divergence formula:

div(F)=\underset{V\to 0}{lim}\frac{{\iint }_{S}F\mathrm{n̂}\cdot dS}{V}

Divergence is essentially the measure of the amount of "fluid" coming out from a given volume. Except in this case, the volume is infinitely small, and divergence is defined at any point inside the vector field.

Can you see how simple and powerful the concept is? Fluid into = fluid out of. We could easily create a grid representation of a 2D fluid that upholds this property. Start with a trivial grid, then subdivide it cleverly… until a large and pretty flowing vector field is formed. Piece of cake, that shouldn't take more than 1 hour of work.

The grid

I took inspiration for the computer representation of the map from wind forecasts. Like this example from meteo.pl:

This map uses wind barbs to indicate wind strength and direction in each cell of the underlying grid.

If we use square cells, it becomes a straightforward computer analogue of a vector field, with one important difference: the cell size inside a vector field is infinitely small. Computers can't do that (easily), but it's good enough for me.

The grid keeps the direction of flow in every cell.

Our first equation still needs to be upheld here, and not just in an abstract form. The easiest way to do that is to use one of the many formulas approximating divergence on a grid that we can find online. They have the advantage of being simple to calculate, operating on just 4 neighboring cells. Instead of writing them down, I will show what they calculate:

Divergence is approximated by 4 components, one for each field. Each says how much the flow points outward of the middle of the group of 4 cells, and then they are summed up.

Our goal is to keep the inwardness of the arrows equal to their outwardness. Simple, right?

---

Writing this project, I made a couple of mathematical assertions that I can't be bothered or am not able to prove. One of those is wrong, can you spot it before the grand reveal?

---

Assertion 1 enter stage right.

That if divergence = 0 in every possible group of 4 neighboring cells on the grid, then it's not possible to find any set of cells where divergence ≠ 0.

I think it's reasonable, because groups of 4 neighboring cells are overlapping, and cover the entire fluid together with their divergence = 0 guarantee. If it's false, then divergence could appear on a larger scale without appearing on a smaller scale first.

Correction: There is a hidden mistake here, that nevertheless doesn't affect algorithms as described in this post. Imagine a grid of cells where those in odd rows flow left, and those in even rows flow right. Taking each pair, the divergence is 0, but take 3 of them, and there will be 2 of one kind and 1 of the other. Unbalanced, and diverging! A better formulation should state:

Assertion 1 (corrected)

If divergence = 0 in every possible group of 4 neighboring cells on the grid, then it's not possible to find any set of cells where divergence ≠ 0, if that set is made of non-overlapping groups of 4 neighboring cells.

If we consider only pieces with 0 divergence, then the collection of them obviously has 0 divergence too.

Thanks to Roman for pointing it out.

Building the grid

We have an equation which is defined on a 4-group. How do we create a grid of size, say, 16×16 cells while maintaining the equation? Obviously, divide and conquer.

Start with a single 4-group. Insert some flow between the cells. Split it while maintaining the equation, and then insert some extra flow. Rinse, repeat.

Unrolling this step by step, we start with a group of 4 undisturbed cells of fluid flow.

There is no flow, so divergence = 0 and our requirement is maintained. So far so good. But how do we introduce some flow into it? A fluid that doesn't flow makes for a boring simulation indeed.

Introducing stir. Notice that divergence will not be changed if the fluid keeps flowing in circles, no matter how fast, so let's do that:

Flow in a stirred 4-group.

Great, now let's split it and cross the boundary from a 2×2 grid into a 4×4 world. We need new assertions here:

Assertion 2: if a single cell is split into 4 cells all of the same flow, the resulting 4-group has 0 divergence.

This seems straightforward to me. First, a single cell in our model cannot have any divergence. Fluid comes in, fluid goes out. Making it bigger doesn't change anything, the flow is still straight through.

Assertion 3: when a grid of 0 divergence can be split into 4-cells by turning each cell into 4 identical ones, the resulting divergence stays 0.

This one reaches a bit farther, but it's essentially the same principle: making a small thing bigger doesn't change the basic properties of the whole.

This is what the grid looks like after splitting.

---

With assertions stated, we can do the stirring again:

We stir around each middle point, each with a different force.

But now we hit another assertion!

Assertion 4: Stirring a 4-group does not affect the divergence of any overlapping 4-groups.

This one sounds less reasonable. Let's quickly check the basic cases with stir magnitude = a.

First, is overlapping on the side enough to disturb the neighbors? The divergence before is indexed with 0, and after applying neighbor's stir with 1.

The green 4-group and the one to the left share 2 cells. The one to the left was stirred with force a.

di{v}_1=di{v}_0+a-(-a)=di{v}_0

The extra flow comes in, but it leaves again. What about overlapping on the corner?

di{v}_1=di{v}_0+0=di{v}_0

Here, the flow is perpendicular to the center between the 4 cells, so it never even enters the equation. Great!

Assertion upheld, we have all the pieces to divide and conquer. Both splitting and introducing stir is ours to have. It's time to implement the algorithm and enjoy the results.

The reckoning

After all this thinking, armed with the above knowledge and confidence, writing the code seemed like a formality.

Not so. As I progressed through stages of stirring and splitting, a red light furiously engaged. CODE RED! CODE RED! INTEGRATION TESTS FAILING. It was the 4x4 grid. And the 8×8 grid. The divergence scanner reported anomalies never seen on smaller scales. Manual checks revealed that the checks were not just due to inaccuracies that computer calculations always make. Divergence was in places almost as strong as the average cell's flow!

But how? Where did I go wrong?

After some head-wall interactions, I extracted the offending 4-group. Half of it was the edge, which I added with an unchangeable flow of 0, the other half came from splitting a cell with a positive flow equal f. This is what it looked like:

Edge of the grid can never have any flow other than 0, so it's marked in gray.

div=a+b+c+d=-f\ne 0

As you can see, the divergence comes out to -f. This proves that Assertion 3 is wrong. Splitting can affect flow after all, on the boundary between cells.

But maybe I could salvage it? Find a better way to split? Come up with necessary equations? So I have tried, but never found any set of equations I could write down. I had to give up hope and search for another way.

At that point, the clock struck midnight.

Try again

Burning with even more nerd-snipedness than before, my attention turned to overanalysis. Accurate splitting got me in trouble. And the splitting comes from the way my grid looks. What are other representations of flows inside a vessel? Is there one where splitting flows is easy?

There is one. But it does not hold directions like a weather map. It resembles a series of barriers instead, tracking flows between, not inside cells. And divergence is easily calculated, too, for each region that would have been a cell in the previous model:

div=a+b+c+d

Can it be stirred?

When the stirring flow crosses into a cell, the same amount leaves the cell, leaving its divergence unchanged.

And what does splitting look like?

Here, the added walls are marked in lighter gray. Arrows have been replaced by their values.

Almost good. The values are not defined, but they are also not horribly floating. Let's try limiting the unknowns by simply forcing the boundary values to be halved.

The flows at 4 barriers inside still need to be calculated, but now they don't depend on anything outside this small piece. But now another piece of picture is revealed. We can calculate how much surplus fluid each subcell receives from the outside by comparing its 2 boundary edges:

s₁ = (b - a)/2 and so on.

If we distribute the surplus across the subcells, we can calculate the flows between them! But it's not so easy either. Notice that we can stir inside our 4-group, and that won't affect how much surplus fluid each cell has. That gives us some freedom we don't necessarily need.

Waterfall

One way to estimate flows between subcells is to look at the surplus, and consider the cells to be a waterfall. Where would the surplus flow? Downwards, from the highest point.

Calculating this is quite straightforward if we know which cell has the highest surplus. The trick to do that is arranging cells in a circle, where consecutive ones share a barrier: abcd, and a again. If we create an array of abcdabc, we can always find the index of the highest value and start a new array there.

The actual algorithm practically writes itself, and is left as an exercise for the reader. It can be somewhat freeform, but if it eliminates divergence, it's good enough. My version is available in fluw.

Simulation

The new approach knows the same tricks as the old one: stirring and splitting. And it can do them without mistakes, too. But the representation is different. It's a network of barriers:

which is not so easy to use in simulations. Thankfully, we can transform it into a grid by summing up barrier flows as components of the flow vector.

\mathrm{v⃗}=a-c,b-d

And we can also run our previous divergence detector, which… detects no divergence!

Video of the fluid simulation.

Mission complete! After several evenings of work, I ended up with fluw, which you can run on your computer too.

---

Author's note: The events didn't happen exactly as I described them here, but I prefer to spare the readers from reality which is way more boring and less dramatic than that.

A beautiful blossom of engineering excellence

Have you ever created something perfect? Simple and functional, robust, and with a core that needs no further adjustment?

I have, recently. I named it odwal. I would have chosen a more positive name if I had known the peace of mind it bestowed upon me in the last several months.

Odwal is perfectly underhanded. It doesn't look special, and I use it only to decrypt and mount my offline backup drives. But it takes care of all the tedium of the process: it deals with failures, and it stops the drives after I'm done.

It was not entirely my invention. The core perfectness comes from the idea of a dependency graph, known to many from the make tool. Whereas make is only concerned with creating resources, odwal takes care of releasing them in the right order too.

This snippet takes care of mounting and unmounting my backup:

(defstep backup_hdd
  (BlockStorage
    args (id "ata-redacted") ; filename from /dev/disk/by-id
  )
)

(defstep decrypted
    (EncryptedBlock
        args (
            path parent.0.path
            name "full1"
         )
         parents (backup_hdd)
    )
)

(defstep full_partitions
    (Partitions
       args (path parent.0.path)
       parents (decrypted)
    )
)

(Mount
    args (
        path "/dev/mapper/full1p1"
        destination "/backups/full"
    )
    parents (full_partitions)
)

First, I connect the USB enclosure to my computer. Then, I run odwal:

# python3 -m odwal ./odwal.to hold Mount:/dev/mapper/full1p1
[here odwal checks for the drive]
Enter passphrase for /dev/sdc:
[here odwal mounts the partition]
Press enter to tear down

Odwal asks me for the decryption password while preparing the mount, and then it gracefully waits until I no longer need it. Once I'm finished, I press enter, and odwal spins down the drive.

Even if I plug in the wrong drive, I don't have to worry. Odwal won't touch it. Even if I mess up the configuration, no harm is done. All I need to do is run odwal clear, and the world is good again.

The relief of using this compared to a bunch of shell scripts is impossible to describe. And it's only 400 lines of Python!

I hope one day I can bring my backup process halfway to the level of delight odwal represents.

My objective function will blow it up

I have a preference for playing god. Build a system, observe, disrupt, see what new equilibrium it reaches. This manifests in weirdest places: gardening, cleaning, feeding the birds, computer games. Here's the story of how it turned me into a loss function and how you could become one too.

Computer games define the main thread today. It all played out in the span of mere weeks before publishing the post.

Fuel

It started from an unexpected angle this time: not Black and White, where you literally play god to your people, nor Civilization where your actions determine the prosperity of a city. Instead, what set me on the path to explosion was Crimsonland, namely the freeware 2002 version. As I mourned that it's not playable under Linux, I considered making a Libre clone. However, after succumbing to a few more waves of monsters, I also concluded that it's perfect at what it does, therefore any potential attempt to replicate it would end up a great disappointment. No point trying. I let my attention wander.

Oxidizer

Creatures is be a name you might have heard in the '90s and early '00s. A series of games that could be as sophisticated tamagotchi, they put you in charge of the well-being of humanoid creatures called Norns. I stumbled across a gameplay video, which showed some gameplay, including sharing tips about punishing and rewarding Norns for their behaviour, breeding them, showing the contents of the neural network that makes their brains, and analyzing the chemicals flowing through their bodies. Turns out they have genes too!

To me, that moved the game from "being a parent" to "playing god" territory. I immediately tried it out. Turns out that playing a nurturing god is only exciting when there's something going on, and the Norns learn and explore rather slowly. As I let them be, as they walked back and forth on a second screen, my attention went astray again…

Detonator

My next chance encounter was the link to Bevy, the game engine written in Rust. Nice, I've been wanting to learn to use an Entity Component System for a while, and I'm unwilling to deal with C. I never made games because of how tedious basic graphics are, but if I had anything to use it for, I would give it a try.

Fire in the hole!

And then it all clicked.

What if I took Crimsonland and gave the monsters a set of genes – simple enough to let them sense the enemy and walk – and culled them using the protagonist's weapons, it would be artificial selection!

Selection by gun

What's so cool about that? It's that the programmer is no longer directly in control. A well executed evolutionary algorithm may take you to unexpected places. It may be undesired when it subverts your goals and do something you didn't want it to – there was a group who tried to teach a simulated human to throw a ball, but ended up with one running with it instead (sadly, I lost the link). It may be neutral when the shape of its solution is just weird, or, like I hope to see here, it may bring surprise and variety into the game. That's on top of the fact that I don't really know how to program a smart AI for a monster myself.

Now that we know what evolution is good for, what are the pieces it requires to take off?

First, we need a bunch of creatures: the gene pool. That's a given both in the virtual world and on farms. Evolution does not work on individuals, but on groups, and it's the group of individuals that keeps improving.

Next, in order for the group to improve, there needs to be a way to know the "better" ones from "worse" ones. For a farmer, this may be the number of laid eggs. For a shooter, let's start with the rule that every creature (genome) that manages to score a bite gets a "goodness" point. That's also called the "objective function".

It's no use knowing what is better if we don't do something about. We want improvement in our gene pool, so genomes scoring higher will more often be the source of newly spawned creatures, and those less bitey will be underrepresented. That keeps the balance on the high side.

But those spawned creatures will not just be genetic copies of their ancestors. If they were, our gene pool would only be able to lose genetic variety: some heritage lines (the unsuccessful ones) would die off forever, leaving emptiness in its place. So we need to restore some of that variety. Instead of spawning exact clones, we give them a chance to mutate! Less bitey mutations rarely score high, and so they rarely reproduce. Little harm done. However, better monsters score high often, and they will spawn copies more often – gene pool improvement!

We end up with a dominating loop: creature -> bite -> mutate -> better creature -> bite -> mutate. There are other paths which arrive at a dead end, like: creature -> bite -> mutate -> worse creature -> fail, or bad creature -> fail -> not breed. As long as the loop dominates, however, we're improving.

There, now we are on the path to breed ultimate killing machines, and I can play god with an entire species of virtual beings.

Breedmatic

The result of that all of thinking is a prototype game Breedmatic. Adhering to my principle of starting in the middle, from the most interesting part, I went on to create a bunch of genetically modifiable monsters. They are very simple: sense, turn, walk. The genes describe how they turn relative to their target: their "brain". There's one hidden part of the story though: breeding and maintaining a healthy population.

It turns out that implementing a good genetic algorithm takes a lot of time. There are considerations regarding the size of gene pools, the rates of mutation of different parts of the genome, the selection of the best objective function, and even the strategy for preserving diversity: should reproduction be asexual? Diploid? How many sexes? That is a lot of sessions of shooting monsters. Hands get tired, progress is slow. In a moment of inspiration, I decided to sidestep the problem with… an evolutionary algorithm. As of version 0.2, the shooter itself is driven by evolution, in much similar way as the monsters.

But they start blank, so you can watch the shooters over hundreds of attempts going from barely moving through phases of spinning randomly to somewhat competent, and even sometimes coming up with weird moves which end up scattering the otherwise accurate laser, which end up being nonsensically effective against the incoming hordes.

A video of an evolved shooter that actually kinda gets it. Seizures help when shooting lasers, apparently.

Playing god has never been so comfy: no need to even move a finger.

Is this fun tho?

Sure it is. Watching the struggle of each and every shooter, I can't help but pray to the random number generator gods for luck in the next draw. "May it be 55's offspring next! It was turning the right way, just opposite!". The chance of hitting gold with the brain connections keeps me on the edge of the seat, and I lose track of time. Every session, there's a chance that a previously unseen combination appears – and the brain is only made of 7 neurons!

But you promised I could be the objective function!

And so you can! The most recent version from git lets you take over control. The monsters are not bred for variety, and the gameplay is basic though, so it's rather boring. At least compared to watching evolution go, so I have a better idea going in that direction.

Did you play Massive Chalice? It combines a tactical game with the management of a royal lineage. There are not many games of this sort, and the premise is effectively the same as mine here: to end up with a genetically superior population. Following the inspiration, future versions of Breedmatic will aim to put the player – not the algorithm – in the role of the gene pool manager. Hopefully that's more fun than shooting questionably intelligent monsters.

Evolve

Breedmatic is open, so if you are dismayed with the intelligence or shape of the creatures, feel free to fork it or contribute. Perhaps we can share ideas regarding the actual gameplay. The prototype could still evolve in many directions!

Otherwise, watch this space. While the next post may not be about evolution, I always have some crazy project in progress.

Geo in GL part 1: Flattening Earth

It's my guilty pleasure to flatten the Earth. To put it on various planar surfaces: pages of paper, scrolls, billboards, screens. Many prominent people attempted that before me: Mercator and Cassini, just to name two.

Yes, I'm talking about maps and projections.

This is the first part of the story of how I used OpenGL shaders to project the Earth onto the flat computer screen. Here I introduce the reader to the basics of projections, write about the scope of my project, and then choose a projection that fits the scope.

Warning: if you believe the Earth is already flat, this blog post won't offer anything interesting for you. You might still be interested in the following parts in the series.

What's the point?

I read someone online say shaders are bad for drawing maps: they aren't precise enough to distinguish distances a meter apart! Of course, I don't believe everything online, so I decided to check it myself, while refreshing my OpenGL knowledge, while playing with maps.

Which brings me to the next point.

I like maps

While there are many representations of the real world, maps feel the most real. Some models turn the real into an unrecognizeable mess of abstract concepts. Maps are elegant, because they copy the real thing, just smaller. They aim to be recognizeable and familiar, so making them is always visually satisfying.

Distortions

Maps are not perfect, though. They are not scale models, and they contain some abstract parts: a mountain represented on a map will be as flat as the paper or screen it's on. If it's marked, it will be either a drawing of a mountain, or a squiggle of height lines.

Most maps are also not globes. Our planet is round, and that means a flat map can't capture it accurately. Like a flattened orange peel, the map will get distorted: there will be tears or stretched places. The different ways to flatten the Earth to fit on a map are called projections.

This is also not a globe. It's just its picture! The picture is called orthographic projection.

The scope

There are multitudes of projections available. To narrow it down, it helps to define what kind of maps are interesting in this project. With that in mind, the initial version of the program will

• display only small areas: not more than 100km in size. The smaller the area, the flatter the Earth surface, so distortions are smaller.
• Load a GPS track and display it as a squiggle. My GPS tracks are usually only a few hours long, and fit in the 100km size limit.
• Display nothing else. I don't want to get carried away and never prove what I set out to prove.

Projection choices

The last big decision was to choose a projection.

WGS84

Normally, we're used to seeing geographical coordinates of a point looking like: "54,5°N 23,7°E". This representation seems obvious. We live on a sphere, and so we use spherical coordinates. However, the prime meridian, and the precise measurement of the Earth axis, among oteher things, need to be standardized, so this representation is called WGS84. Since this is what GPS is using, it would be trivial to just plot the points from track on the screen as they are. However, the results would end up rather unappetizing:

The distortions are huge! Can you see the grid sizes? Places at the equator are much smaller than at the poles, and the poles are extremely stretched sideways too – if you look at the globe, the grid is composed of wedges close to the poles. This would be a very unpleasant map to look at, even on our small scale.

Mercator

The stretching is easily fixable by applying a varying stretching factor across the vertical position. This gives us the Mercator projection:

Image copyright Strebe, CC-BY-SA 3.0 Unported.

This projection is easy to calculate, and it covers most of the world, so it's very popular on the Internet. Most online maps use it, including OpenStreetMap's own osm.org. However, it's still rather bad: the stretching factor approaches infinity as we approach the poles, so the map doesn't have a top and bottom edge. Instead it's just cut off somewhere. The other problem is that the areas at the equator are still much smaller than near poles.

Something different

It seems that both of those projections are quite bad at some places. But they are also both quite good at certain places: in this case, at the equator. So why don't we choose the best projection for our data?

That's what paper maps do. Unlike general purpose computer map viewers, paper maps are often limited to a small area on the Earth. They can focus on making this area undistorted, and they can ignore all the misshapen mess that's not printed on the paper. I deliberately chose to limit my program's area of interest to a 100km radius to take advantage of this.

For this, we need a projection that's not general like WGS84, or like the Mercator projection, but instead focussed on the area in question. To look at the North Pole in an undistorted way, we could choose the The North Pole Lambert Azimuthal Equal Area projection:

Here, I decided to increase my difficulty: I will not make an assumption about where on the Earth my GPS track is. It could be Europe, the Equator, or the Arctic. Instead, I will try to hit the bullseye and make sure that my projection favors the area I'm displaying, no matter what it is. For that, I need to find a family of projections that I can adjust.

Thankfully, people have come up with plenty of different ways of projecting:

Various projections: Alaska, Mollweide, Winkel I, van der Grinten (I), Gauss-Schreiber Transverse Mercator, Tobler-Mercator, International Map of the World Polyconic, Oblique Cylindrical Equal Area. (Images courtesy PROJ.)

Some of them are tweakable, to name a few: azimuthal equidistant, Cassini, gnomonic.

Most of them preserve shapes over small areas rather well, so the choice is not that important. Nevertheless, I'm not an expert, so it's possible that I'm underestimating distortions. To stay on the safe side, I choose the azimuthal equidistant, because it promises that distances from the central point can be measured with a ruler. I used it previously with an overlaid kilometer grid and it worked well.

Here's the projection in proj form, so you can apply it in QGIS. Don't forget to adjust lat_0 and lon_0 to your area of interest!

+proj=aeqd +lat_0=54 +lon_0=24 +x_0=0 +y_0=0 +a=6371000 +b=6371000 +units=m +no_defs

Next steps

I have the projection, but I still don't know the math behind it, and don't have any code. Stay tuned for the next part!

Wayland and input methods

Wayland is gradually getting the ability to support input methods natively. Actually, it's plural "abilities", because there are several pieces related to the functionality. Working in this area, I had to explain this to newcomers so often that I decided to write this blog post instead, to explain to everyone what's going on here, once and for all.

Text input

Quick recap. The purpose of an input method is twofold: to give applications text from the user, and to recognize when and what kind of text is expected.

The most basic thing to do that under Wayland is the text-input protocol. It takes text from the compositor, and gives it to applications. It lets applications tell the compositor when and what kind of text they need. The protocol doesn't worry about the user, instead leaving that to the compositor.

Input method

The compositor can take two paths in order to let user input the text. Either it takes the burden of communication on its own, by handling input itself, or it can delegate that task to some other program.

In Wayland, the input-method protocol was designed to help. It is very similar to text-input, because it lets a program send text to the compositor, and allows the compositor to tell the program what kind of text is needed. Notice the inversion! This time, the program is special. It communicates with the user, and then gives the text to the compositor. The compositor is then typically going to send the text onward to the currently focused application using text-input, creating a chain: special program → compositor → focused application.

This protocol has place for some additional responsibilities, too. Because there is typically only one application using this protocol, it can do things which would not work with multiple applications. One of them is grabbing the keyboard, known to CJKV language users. Input-method allows the special program to ask the compositor to send it all keyboard presses ("exclusive grab"). Taking over the keyboard makes it possible to send the text "你好" when "nihao" is typed. The other extension is meant for creating a special pop up window, which the compositor places next to the text field, and which can be used to show typing completions.

Virtual keyboard

Input methods support would have been complete here, if all we cared about was text. However, the world is not so simple, and we have to deal with additional categories of input before being useful:

• text in legacy applications which don't support text-input, and
• triggering actions which would normally need a keyboard.

Both of them can be addressed by using a keyboard. But what if we're using a tablet computer, a TV, game console, or a phone, and there isn't one to speak of? We can address this issue by emulating button presses. Again, there are two basic ways to address this. The compositor can come up with something on its own, or it can delegate the task to another program.

The protocol virtual-keyboard is designed for programs which want to tell the compositor to issue "fake" keyboard button press events.

Together

A fully-fledged input method program will be a Wayland client using the input-method protocol for submitting text, but also supporting virtual-keyboard for submitting actions, and as a fallback for legacy applications.

A compositor would ferry text around between the input method program and whichever application is focused. It would also carry synthetic keyboard events from the input method program to the focused application.

An application consuming text would support text-input, and it would send enable and disable events whenever a text input field comes into focus or becomes unfocused. It would also accept keyboard events.

Legacy applications won't send enable and disable, even when a text field is focused, and the user ready to type. When that happens, the compositor and the input method won't realize that text should be submitted now. If the input method uses an on-screen keyboard, it could remain hidden! Because of that, it's best to always make sure the user can bring up the input method and input text, which would then be delivered as keyboard events (which are always supported by applications).

Current state

As of 2020-08-15, the latest versions of relevant protocols are:

text-input-unstable-v3

Accepted in wayland-protocols. Designed by me, based on gtk-text-input by Carlos Garnacho, and on text-input-unstable-v1.

input-method-unstable-v2

Used in wlroots. Designed by me, based on text-input-unstable-v3 and input-method-unstable-v1.

virtual-keyboard-unstable-v1

Used in wlroots. Designed by me, based on the wl_keyboard interface.

Future

There are still some topics open. The most important one is about fixing the deficiencies in text-input, and updating input-method to match. Another one is regarding whether virtual-keyboard is even worth the effort, considering how it stirs up some conflict with Wayland's design.

Less important is implementing the additional features of input-method.

There's also the exploratory idea of designing a protocol dedicated to submitting actions like "undo", "submit", "next field", but not text, in order to eliminate the need to emulate keys in modern keyboard.

Word embedding fun

I'm terrible at naming things. When others come up with slick user names like "el_duderino", or "pseudolus", I struggle to come up with anything better than "blob123". Right now I'm busy setting up a new project which needs a name, and I struggle with inspiration.

What if there was a machine to transform terrible ideas into good ones? Take a trope, and tweak it a little to get something original?

Word vectors

I'm not going to pretend that I know exactly what the term "word embedding" means. What matters is that words in a human language can be associated to vectors in a linear space. With a little thinking, we can leverage that property to find words with similar meanings, or solve analogies like this one:

What is to woman as king is to man?

Of course, it's "queen". The example is the classic one, so please read this because I couldn't explain it better than that article.

Okay, so what about the trope? Let's try with a political meme:

What's the anarchist version of "fully automated luxury gay space communism"?

Playing with words

To answer the question, I installed gensim from pip, and fetched some ready-made data. Using Python3 interactively, I started this way:

import gensim.downloader as api
corpus = api.load('text8')
from gensim.models.word2vec import Word2Vec
model = Word2Vec(corpus)

Now that I had a vector space with words, I started playing with it, translating the meme word by word. If "fully" is a "communism" term, what's its "anarchism" version?

>>> model.wv.similar_by_vector(-model.wv['communism'] + model.wv['anarchism'] + model.wv['fully'])
[('fully', 0.7874691486358643), ('correctable', 0.5339317321777344), ('strictly', 0.533452033996582), ('wholly', 0.5306737422943115), ('totally', 0.5066695213317871), ('properly', 0.4988959729671478), ('locally', 0.4917019009590149), ('entirely', 0.4908182919025421), ('statically', 0.48547613620758057), ('completely', 0.4838995933532715)]

Turns out it's still "fully". Going all the way, we get "fully introductory ornamental gay space humanism". That's a bit surprising! According to the model, "communist" is to "communism" as "anarchist" is to "humanism". I blame it on the model being relatively small.

Readable encyclopedic excellent lesbian topological altruism

This is the best nugget I got out of the text8 corpus. It seems to be giving a nice selection of words with similar meanings, although changing the political option doesn't seem to do much. Other examples of computer-generated silliness include "perfectly analytical aquarium bisexual seti individualist" and "fully extensible cheddar gay space anarchism". I'm going to keep "fully extensible cheddar" for a different project…

Using a larger model

Seeing the obvious drawbacks in accuracy and lack of sensitivity to political stances, I tried to use a serious model, based on the Wikipedia. Gensim doesn't disappoint in that department either:

model = api.load("fasttext-wiki-news-subwords-300")

After coming back from the tea break, we are in a 300-dimensional model, containing the condensed knowledge of Wikipedia. Let's see how it fares:

>>> model.wv.similar_by_vector(-model.wv['communist'] + model.wv['anarchist'] + model.wv['communism'])
[('anarchism', 0.883374810218811), ('anarchist', 0.8050785064697266), ('Anarchism', 0.7585680484771729), ('anarcho-syndicalism', 0.7496938705444336), ('anarcho-communism', 0.7432658672332764), ('anarchists', 0.7426955699920654), ('anarcho-socialism', 0.7400932312011719), ('anarchistic', 0.7102389931678772), ('anarcho-capitalism', 0.7090942859649658), ('anarchisms', 0.7069689035415649)]

Good! It could derive that the "anarchism" equivalent of "communist" is "anarchist".

>>> model.wv.similar_by_vector(-model.wv['communism'] + model.wv['anarchism'] + model.wv['fully'])
[('fully', 0.8348307609558105), ('fullly', 0.6537811756134033), ('fuly', 0.6530433893203735), ('thoroughly', 0.6464489102363586), ('completely', 0.6422995328903198), ('properly', 0.6398820877075195), ('adequately', 0.6317397356033325), ('fully-', 0.6239138841629028), ('comprehensively', 0.6175973415374756), ('partially', 0.606654167175293)]

Uh, there's a bit of repetition there. What about other words?

>>> model.wv.similar_by_vector(-model.wv['communism'] + model.wv['anarchism'] + model.wv['space'])
[('space', 0.8099461793899536), ('sub-space', 0.6615338325500488), ('spaces', 0.6537838578224182), ('work-space', 0.6258285045623779), ('non-space', 0.6244523525238037), ('space--and', 0.6012811064720154), ('workspace', 0.6001818776130676), ('WPspace', 0.5913711786270142), ('space-', 0.5863816738128662), ('space--', 0.5790454745292664)]

Disappointing :( I had to dig through a lot of results to get anything not related to the original word. Here are some examples: "adequately anarchist high-end lesbian column-free libertarianism", "completely computerized boutique-style heterosexual hypersphere anti-capitalism".

Different APIs

There's another API to create analogies: it's the most_similar call. It works like this:

>>> model.most_similar(positive=['anarchism', 'space'], negative=['communism'])
[('sub-space', 0.6117153763771057), ('spaces', 0.5981306433677673), ('work-space', 0.5736098885536194), ('non-space', 0.5651520490646362), ('workspace', 0.5563109517097473), ('space--and', 0.542439341545105), ('WPspace', 0.5411180257797241), ('space-', 0.5368467569351196), ('meta-space', 0.5321565866470337), ('space--', 0.5256682634353638)]

The results are a little different, although I don't know why. Perhaps it's about a different way of measuring similarity.

Word salad

Using models to reveal some hidden properties of political options didn't work out, but the smaller model was a decent thesaurus. I suspect this is due to less words, creating more varied results near the best answer. I don't know how the 2 APIs differ. While I expected the similarity function to work based on Euclidean distance, it seems to be based on cosine distance instead. Perhaps that's the reason the anchor word sometimes ends up in results as well.

All in all, I think word vectors are going to end up being a nice toy, when some weirdness is needed for brainstorming. Let "aquarium" be an example: it's far out in terms of adjectives, but it is indeed not entirely unrelated to "luxury".

It's not about keyboards

My work on Squeekboard and the corresponding Wayland protocols is open and public. That lets people come and contribute, and give feedback. Those people come with different expectations about the result of my work, and there's one that looks enticing but doesn't really correspond with what my work is about. It causes lots of confusion, so I would like to dispel it.

Squeekboard's goal is not to be a keyboard.

Instead, I designed Squeekboard and the pieces around it to support text composition first and foremost. While a keyboard is one method of composing text, it usually does other things as well:

• playing musical notes
• turning things on and off
• shooting virtual monsters
• moving within virtual space
• instructing the computer to do arbitrary actions.

Squeekboard doesn't want to do any of those things, unless they help composing text. To understand why, let's look at the basic properties of keyboards and Squeekboard. Keyboards

• have labels which don't change
• have keys which stay in place and retain their shape forever
• have keys which sense force
• have keys which provide precise tactile feedback.

Meanwhile, Squeekboard:

• occupies a changeable area of the screen
• has pixels which change color but not shape
• has touch points which overlap the display
• has touch points that capture "pressure".

As you can see, they are very different beasts, with different constraints and abilities

It's just an app!

Squeekboard has some ability to emulate the keyboard buttons and register touches, so in principle it could share the goals of a keyboard. The key to understanding why this is not a useful useful goal is to realize that all of Squeekboard's unique abilities are the same as of any other graphical program.

That gives us a new constraint: Squeekboard should not duplicate the functionality of the application it controls.

A physical keyboard has it easy: touch feedback and less hand movement always adds value. But what can an on-screen one offer? Let's think about the interactions the user has with an application.

The user might want to close or resize the application. That's easy, the compositor takes care of that. The user might want to issue an application-specific action, like rendering a scene, or moving the camera. Arguably, this is best handled by the application, which can present the relevant information. Commonly encountered actions like saving the file, copying or opening preferences are usually supported in a built-in way too. What does that leave us?

Text input.

Few applications give user an interface to enter text directly, making it the obvious responsibility of Squeekboard as an input method.

Extra powers

Dropping the preconception that an on-screen input method is a keyboard opens new possibilities for text input. Changing button shapes and numbers, presenting useful information in labels, offering corrections and suggestions, gestures, handwriting recognition, embedded input methods, and sending text directly to the application (not as button presses) are not things that can be done with just a matrix of buttons.

While Squeekboard doesn't support many of those features, the groundwork for most of them is included in the Wayland protocols developed alongside it.

Looking past the historically proven idea of a keyboard opens a different view on how touch-based human-computer interfaces should work, and Squeekboard is there to make best use of the new possibilities.

Why I avoid Cargo: dependency versions

Ever since I started making use of Cargo to build the Rust pieces of Squeekboard, I've hit a wall whenever I needed to add something nontrivial to the build process. I haven't documented them before, so whenever I complained about Cargo, I ended up looking ridiculous without having anything to show for the complaints.

Last week I spent three days solving a problem with building Squeekboard that should have been solveable in 30 minutes, in large part due to Cargo.

Buster and Bullseye

Squeekboard is a crucial part of the software stack of the Librem5 mobile phone. It's primary goal is to fit in well in PureOS, which powers the phone. PureOS is in turn based on Debian, and inherits a lot of its practices, which are followed by projects related to the Librem5, including Squeekboard.

One such practice is that Rust crates are vendored by Debian, and not by the application author. They are installable with apt, and end up in a local repository in /usr/share/cargo/registry.

Librem5's base operating system is moving from Debian 10 to Debian 11. I was asked to make Squeekboard work on both in the transition period. Debian 10 ships with gtk-0.5.0, whereas Debian 11 ships with gtk-0.7.0, which contain some incompatibilities, and Squeekboard's build process must adjust to them, depending on where the build happens.

Piece of cake: there's one variable, which needs to be turned into one decision. I did this a million times. Little did I know Cargo hates simple solutions.

What's the version we're using?

It's not unusual for projects to support two versions of the same dependency. Perhaps the versions come from different vendors. C programs don't have trouble with this:

#include <gtk.h>
#if GTK_VERSION==0.5
  // old stuff
#endif

Rustc won't know dependency versions by itself, but Cargo should turn it to something like this:

#[cfg(dependency.gtk.version = "0.5.*")]
use gio::SimpleActionExt; // Doesn't exist in later versions of gtk

But… I haven't managed to find any equivalent. It makes sense, Cargo can pull several copies of the same crate, and I think their pieces can even be used in the same files if one is not careful. So there's no good reason this should have worked. Fair, let's try something else.

What's the available version?

If the compilation process can't tell us what versions we're dealing with, perhaps we can check that before compilation. In Meson, we'd do something like this:

gtk-rs = dependency("gtk-rs")
if gtk.version().startswith("0.5.")
      add_project_arguments('--cfg old_gtk', language : 'rust')
endif

And then, in Rust:

#[cfg("old_gtk")]
use gio::SimpleActionExt; // Doesn't exist in later versions of gtk

Now the only remaining thing is to create the dependency lookup procedure. While Squeekboard is Debian-focused, building on non-Debian systems is still important, so it must build with crates.io. Thankfully, we have cargo search. Let's try with a vendored registry:

root@c684b7b31b07:/# CARGO_HOME=/mnt/build/eekboard/debian/cargo/ cargo search gtk
error: dir /usr/share/cargo/registry does not support API commands.
Check for a source-replacement in .cargo/config.

Oh no. We can forget about it. I'm not willing to write a tool that searches for crates in all the ways that Cargo supports, and I'm honestly boggled that Cargo doesn't properly do it itself.

As a bonus, the output of cargo search just doesn't make sense. I'm looking for the regex crate, which is at version "1.3.9":

$ cargo search regex=1
combine-regex-1 = "1.0.0"      # Re-export of regex 1.0 letting combine use both 0.2 and 1.0
webforms = "0.2.2"             # Provides form validation for web forms

Totally useless :(. But I still have a few tricks up my sleeve.

Use a build flag

Isn't this obvious? Let's skip all that dependency detection, and just order the build system to use one, in the dumbest fashion possible. The way you'd do it with Meson:

if get_option("legacy") == true
  gtk-rs = dependency("gtk-rs", version=0.5)
  add_project_arguments('--cfg old_gtk', language : 'rust')
else
  gtk-rs = dependency("gtk-rs", version>0.5)
endif
squeekboard = executable('squeekboard',
  dependencies: [gtk-rs],
)

Add to that the conditional in the Rust file, and we're done. Cargo is not Meson, but we can call it with flags too, and then instruct it to use the right dependency. Right?

Wrong.

While Cargo allows choosing dependency versions, they are selected based on target, not on flags. You can use:

[target.'cfg(unix)'.dependencies]
openssl = "1.0.1"
[target.'cfg(not(unix))'.dependencies]
openssl = "1.0.0"

You can remember that the cfg() syntax supports features:

[target.'cfg(feature="legacy")'.dependencies]
gtk="0.5.*"
[target.'cfg(not(feature="legacy"))'.dependencies]
gtk="0.7.*"

but then you see this:

# cargo build
warning: Found `feature = ...` in `target.'cfg(...)'.dependencies`. This key is not supported for selecting dependencies and will not work as expected. Use the [features] section instead: https://doc.rust-lang.org/cargo/reference/features.html

And when you follow that web page, you learn that you can specify other dependencies, but not other versions. Foiled again!

Again, I'm boggled why such a basic piece of functionality is working in such a complicated and restrictive way. Wouldn't it be easier to abolish the weird feature = [dep] syntax and instead let foo.dependencies work, with all the fine-grained control over what the dependencies are?

Embedded crates

But I didn't stop there. If the deps can't be specified the usual way, then let's get there through a back door. If I can't choose a version, I will choose a crate. 0.5 turns into a crate called "legacy", and 0.7 into one called "current". My Cargo.toml looked like this:

[dependencies]
current = {path="deps/current", optional=true}
old = {path="deps/old", optional=true}

[features]
legacy = ["old"]

The deps/old/Cargo.toml contained the actual version:

[dependencies]
gtk-rs = "0.5.*"

and the current one had 0.7.*. When we choose the crate, we choose the dependency with it! So clever! There's no way it won't work!

root@c684b7b31b07:~/foo/foo# CARGO_HOME=/mnt//build/eekboard/debian/cargo/ cargo build
error: failed to select a version for the requirement `gtk = "0.5.*"`
  candidate versions found which didn't match: 0.7.0
  location searched: directory source `/usr/share/cargo/registry` (which is replacing registry `https://github.com/rust-lang/crates.io-index`)
required by package `old v0.1.0 (/root/foo/foo/deps/old)`
    ... which is depended on by `foo v0.1.0 (/root/foo/foo)`

What!? This cannot be! Why oh why?

My only guess is that cargo pulls all the dependencies indiscriminately, regardless of whether it actually needs them or not. Obviously, this entire shebang is because we don't have one of them! Why is Cargo missing the point of choosing dependency versions so hard?

Nuclear option

With this in mind, there's only one solution left. If Cargo is greedy enough to snatch everything it sees, then we'll just not let it know there are possibly any other dependencies. We'll generate a Cargo.toml after we know which dependency we need, and we'll never let Cargo know about the other dependency. The details of the generation are quite straightforward: just change depedencies based on build flags. The build scripts get complicated, though. Now they must include --manifest-path. This looks trivial, but this changes the root of the crate, so now we need to give it the path to the sources again:

[lib]
name = "rs"
path = "@path@/src/lib.rs"
crate-type = ["staticlib", "rlib"]

# Cargo can't do autodiscovery if Cargo.toml is not in the root.
[[bin]]
name = "test_layout"
path = "@path@/src/bin/test_layout.rs"

[[example]]
name = "test_layout"
path = "@path@/examples/test_layout.rs"

But it works. Finally.

Sadly, we sacrificed autodetection of tests and binaries, as well as distro-agnosticism.

You're doing it all wrong!

I'm sure some of you will see the struggle and consider it self-inflicted. "Why didn't you try X?", "You're fighting the tool because your model is outdated!", "This is not a bug". Some of them will be valid criticisms, and I'm going to address those I could think of.

Use multiple Cargo.lock versions

I could have used "*" as the dependency version, and instead rely on two versions of Cargo.lock to select the actual dependency I want. That is troublesome.

I don't want to know exactly what versions of Rust crates (or any other deps really) the upstream distro is shipping. This is why I use distros in the first place: they remove some burden of deps auditing from me. All I want to know is the minor version for API compatibility reasons. However, Cargo.lock forces me to care about dependencies by asking for a hash of each. I would have to find out what dependency is available, and feed Cargo.lock with its hash. That hits the same problem with detecting what we have again.

Vendor your crates yourself

Squeekboard is just one project out of many in PureOS, and all the others follow Debian's best practices: use Debian's software versions whenever possible. On one hand, this is designed to set right expectations, on the other it relegates the responsibility for auditing and updating to Debian, as explained in the previous argument.

In this light, I'm already letting Debian vendor my crates, and that won't change.

Cargo is designed for crates.io, not for distributions or local repositories

I'm not sure if this is Cargo's explicit goal to be useful with crates.io, but my experience says other sources are in practice playing catch-up. If I relied exclusively on crates.io, I would have no problems.

However, the build policy in PureOS requires builds to happen without a network connection. That means we can't use crates.io. We've already eliminated vendoring before, and so we're stuck with some form of a local repository, which clearly isn't well-supported by Cargo.

Insights

My insight from this adventure is that Cargo doesn't prioritize users who want to control their own sources of software at the moment. It's inflexible in the way that favours crates.io, where the implicit assumption is that all possible crates and versions are available. After all, it crashes when a non-available version is specified even if unused.

Cargo also doesn't want application developers to offload dependency checking. It's recommended to commit Cargo.toml together with the application, but there's no provision to ignore it when doing a build with local versions of the same crates, like when you do in distro builds.

Cargo is not composable the way other build systems are. When building a C program, artifacts in the form of shared or static libraries are placed in a well-known directory. Cargo creates an opaque directory structure for compiled crates in the target directory, which cannot be used by another build system later due to lack of documentation. Sadly, this means that when Cargo fails to solve a need, there's no alternative.

The same creates a network effect, where Cargo is de facto the only tool that builds useful Rust programs. It's difficult to the point of pointlessness to "build" serde and gtk-rs separately, and then "link" them with the main program using manual rustc calls.

Creating composable artifacts would undermine Cargo's monopoly and allow a better integration with distributions as well as other programming languages in my opinion. Build systems should not infect all software they touch.

From C to exec: Part 3

In Previous parts, we followed a story of a piece of code transformed into a linked executable. The center stage was taken by symbols, around which the entire process of linking happens. They are the crucial component that lets us put programs fro multiple pieces. However, we watched what happens to symbols after they are there, but we haven't seen how to control their existence, and how to solve problems we may encounter.

The third part focuses on practical problem solving related to symbols and C.

Emitting symbols in C

You might remember the term "translation unit". It's just a more convenient name for "a thing that is compiled together", the bunch of source files that result in a single object file. This simple definition will be used to refer to the sources as opposed to the object file.

C does not support namespaces, so whenever you define a function under the same name in a different context (e.g. a logging function, an error handler or a container operation), they truly take the same name. This can lead to trouble. Let's introduce a new set of files: floats.c:

#include <stdio.h>
void print_number(float n) {
  printf("Float %f\n", n);
}

void multiply_by_tau(int number) {
  print_number(6.68 * number);
}

Then float.h:

void multiply_by_tau(int number);

Finally, main.c:

#include <stdio.h>
#include "float.h"

void print_number(int number) {
  printf("Integer %d\n", number);
}

void power_of_two(int number) {
  print_number(2 << number);
}

int main(void) {
  multiply_by_tau(100);
  power_of_two(10);
  return 0;
}

In this example, we end up having two different functions called print_number. One is in the translation unit with main.c, the other in float.c. They are only called from the relevant translation unit, so we should be fine, right?

# gcc -c main.c -o main.o
# gcc -c float.c -o float.o
# nm main.o | grep print_number
0000000000000000 T print_number
# nm float.o | grep print_number
0000000000000000 T print_number
# gcc float.o main.o -o main
/usr/bin/ld: main.o: in function `print_number':
main.c:(.text+0x0): multiple definition of `print_number'; float.o:float.c:(.text+0x0): first defined here
collect2: error: ld returned 1 exit status

Not really. We emitted a symbol in both of them, and the linker doesn't know which one to choose. Oops. We have to tell the compiler explicitly that each of those functions will only ever be called from the same translation unit. C defines a keyword static for this purpose. Let's check if it works. float.c becomes:

#include <stdio.h>
static void print_number(float n) {
  printf("Float %f\n", n);
}

void multiply_by_tau(int number) {
  print_number(6.68 * number);
}

And main.c now looks like this:

#include <stdio.h>
#include "float.h"

static void print_number(int number) {
  printf("Integer %d\n", number);
}

void power_of_two(int number) {
  print_number(2 << number);
}

int main(void) {
  multiply_by_tau(100);
  power_of_two(10);
  return 0;
}

And the compilation:

# gcc -c float.c -o float.o
# gcc -c main.c -o main.o
# gcc float.o main.o -o main
# ./main
Float 668.000000
Integer 2048
# nm main.o
0000000000000045 T main
                 U multiply_by_tau
0000000000000022 T power_of_two
                 U printf
0000000000000000 t print_number
# nm float.o
0000000000000024 T multiply_by_tau
                 U printf
0000000000000000 t print_number

Success! We turnedprint_number into a local symbol, (t is lowercase) which is ignored during linking.

Keywords

C defines keywords for controlling linkage type. Those are:

• extern for external linkage, i.e. emitting a symbol
• static for internal linkage, i.e. limiting usage to the current translation unit
• no keyword defaults to extern

However, functions aren't the only items that can be assigned symbols. While any kind of memory area can get a symbol, global variables are the other most common usage. Let's see it all in action in linkage.c:

int v_def = 0;
extern int v_ext = 0;
static int v_stat = 0;

void f_def(void) {};
extern void f_ext(void) {};
static void f_stat(void) {};

Looking at the symbols shows us:

# gcc -c linkage.c -o linkage.o
linkage.c:2:12: warning: ‘v_ext’ initialized and declared ‘extern’
    2 | extern int v_ext = 0;
      |            ^~~~~
# nm linkage.o
0000000000000000 T f_def
0000000000000007 T f_ext
000000000000000e t f_stat
0000000000000000 B v_def
0000000000000004 B v_ext
0000000000000008 b v_stat

As expected, the default is to emit a symbol, and static makes the symbol local. But there's an unexpected warning from the compiler, suggesting that extern on a variable does more than just emit a symbol for it. Let's try to apply compiler's advice and not assign any value invariable.c:

extern int v_ext;

Looking at symbols:

# gcc -c variable.c -o variable.o
# nm variable.o
#

…nothing? Let's try something else in variable.c:

extern int v_ext;

void use_variable(void) {
  v_ext = 0;
}

Symbols:

# gcc -c variable.c -o variable.o
# nm variable.o
0000000000000000 T use_variable
                 U v_ext

There it is, undefined! Without the body of the variable, the compiler took extern to mean "not defined here" instead of "emit a symbol". No wonder that we didn't see its trace when we didn't try to make use of it. This is similar to function prototypes we covered earlier.

Notice that U printed by the nm program does not mean "symbol of the undefined kind present". Instead, it means "the relocation table calls for this symbol, but we don't have it". Here, nm mixes up the symbol table and the relocations table, which can be confusing.

Using variables

Variables with the extern symbol may sound confusing at first, but they work the same way functions do. Let's take an example of three files. First is main_var.c

#include <stdio.h>
#include "var.h"

int main(void) {
  set_var(10);
  printf("var is %d\n", var);
}

The file var.h:

extern int var;
extern void set_var(int v);

And finally, var.c:

int var = 5;
void set_var(int v) {
  var = v;
}

Compiling them shows us that both the set_var function and the var variable are needed by main_var.c, and provided by var.c:

# gcc -c main_var.c -o main_var.o
# gcc -c var.c -o var.o
# gcc main_var.o var.o -o main_var
# ./main_var 
var is 10
# nm main_var.o
0000000000000000 T main
                 U printf
                 U set_var
                 U var
# nm var.o
0000000000000000 T set_var
0000000000000000 D var

As you can see, we managed to modify a variable from a different file using a function from a different file, and then read it out directly. This mechanism is commonly used to give programs access to memory managed by a library.

Header file trouble

There is a situation is when a function clashes with itself. It sounds silly, but this can happen if the same function creates a symbol in multiple translation units.

This usually happens when a function lives inside a header of a library we're using. Headers are not actually restricted to holding function prototypes, but instead they are fully fledged pieces of C code, verbosely included in the source file. Let's create source.c:

const int before;
#include "header.h"
const int after;

And header.h:

const int inside;

After running the C preprocessor, which happens as the first compilation step, we get:

$ gcc -E source.c
# 1 "source.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 31 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 32 "<command-line>" 2
# 1 "source.c"
const int before;
# 1 "header.h" 1
const int inside;
# 3 "source.c" 2
const int after;

We can see here that the text from the header was indeed included in place of the #include directive.

Now, why would anyone actually want to place a function in the header?

As you can see, the inclusion of a full function inside header means that it will land in the translation unit of the calling file, instead of the providing file, unlike in our library examples. Here we don't even have a separate "providing" file. We can see that our single translation unit contains what the header provided:

# gcc -c source.c -o source.o
# nm source.o
0000000000000004 C after
0000000000000004 C before
0000000000000004 C inside

The most obvious purpose for placing a function in the same translation unit as the caller is optimization. Making a function call is slow. In broad strokes, the compiled code usually saves a generic set of working data, as part of issuing a call and restores it as part of returning from it. But this doesn't make sense if our function is relatively simple. Sometimes it's more efficient to copy the contents of the function instead of calling it. The name for that is inlining.

A call to a function in a dynamic library can't be inlined automatically. That would require some advanced processing at the time our program is starting. A call to a function in a static library can sometimes be inlined. That is called link-time optimization, and it's done at linking time as a sort of additional, slow compilation step.

On the other hand, a call to a function in the same translation unit can be inlined as part of the compilation process even before the linking, and without much additional penalty. If we place the body of our function in the header, this is exactly what we get, regardless of the kind of library we create in our other translation units.

But what does this have to do with a symbol clashing with itself? Let's introduce an example of a library that must store something for bookkeeping, but doesn't want to be a burden for the calling code. Starting with store.c:

int calls_made = 0;

Corresponding store.h:

extern int calls_made;

void log_call(void) {
  calls_made += 1;
}

Now, user.c:

#include <stdio.h>
#include "store.h"

void print_and_bump(void) {
  puts("Caller print");
  log_call();
}

Its user.h:

void print_and_bump(void);

Finally, clash.c:

#include <stdio.h>
#include "store.h"
#include "user.h"

int main(void) {
  puts("main print");
  log_call();
  print_and_bump();
  return 0;
}

Let's try compiling and linking this project:

# gcc -c store.c -o store.o
# gcc -c user.c -o user.o
# gcc -c clash.c -o clash.o
# gcc store.o user.o clash.o -o clash
/usr/bin/ld: clash.o: in function `log_call':
clash.c:(.text+0x0): multiple definition of `log_call'; user.o:user.c:(.text+0x0): first defined here
collect2: error: ld returned 1 exit status

That's an error! Remember that we placed the log_call function into the header file. That means that every translation unit including the header file will get a copy. We can see that this is what happened:

# nm store.o | grep log_call
# nm user.o | grep log_call
0000000000000000 T log_call
# nm clash.o | grep log_call
0000000000000000 T log_call
# gcc -E user.c
[...]
# 1 "store.h"
extern int calls_made;

void log_call(void) {
  calls_made += 1;
}
# 3 "user.c" 2
[...]
# gcc -E clash.c
[...]
# 1 "store.h"
extern int calls_made;

void log_call(void) {
  calls_made += 1;
}
# 3 "clash.c" 2
[...]

The solution is straightforward. Let's change store.h to show this:

extern int calls_made;

static void log_call(void) {
  calls_made += 1;
}

Linking test:

# gcc -c user.c -o user.o
# gcc -c clash.c -o clash.o
# gcc store.o user.o clash.o -o clash
# nm clash.o | grep log_call
0000000000000000 t log_call
# nm user.o | grep log_call
0000000000000000 t log_call

That did it! Our static function's symbol turned local, and stopped being a duplicate.

Inline woes

You might know the C99 keyword inline already. It's meant to be a suggestion to the compiler that the function should be copied instead of called. It turns out that it affects linking.

Taking the example from above and modifying it slightly, we get a new store.h:

extern int calls_made;

inline void log_call(void) {
  calls_made += 1;
}

If you think that inline implies no need for any sort of linking, you may be surprised:

# gcc -std=c99 -c user.c -o user.o
# gcc -std=c99 -c clash.c -o clash.o
# gcc store.o user.o clash.o -o clash
/usr/bin/ld: user.o: in function `print_and_bump':
user.c:(.text+0xf): undefined reference to `log_call'
/usr/bin/ld: clash.o: in function `main':
clash.c:(.text+0xf): undefined reference to `log_call'
collect2: error: ld returned 1 exit status
# nm user.o | grep log_call
                 U log_call

I'm not going to dive into what exactly happened here, but it boils down to C99 designers wanting to avoid too many copies when inlining isn't possible. A symbol may still be needed for the function, as explained by Jens Gustedt. That explanation offers a solution in the form of placing the following line into exactly one translation unit:

extern inline void log_call(void);

Alternatively, use the static version, although it may offer a different rate of success in inlining or emit too many copies.

Bad declaration

Remember that symbols don't carry type information? One of the most confusing problems is using something believing it's one type when in reality it's another. For a quick example, see a botched version check in file version.c:

const char version[] = "10.1";

The header version.h has a mistake! The variable is declared as int instead of char:

extern const int version;

The file check.c is using the declaration from the header:

#include <stdio.h>
#include "version.h"

int main(void) {
  printf("Version %d\n", version);
  return 0;
}

And when it tries to read the version variable…

# gcc -c version.c -o version.o
# gcc -c check.c -o check.o
# gcc version.o check.o -o check
# ./check
Version 825110577

That doesn't look right… Nowhere during the entire process was the type of the actual variable compared to the one declared in the header. The compiler believed what was the header file, the linker matched the symbol simply by name, and the mistake only appeared when we tried to execute the program.

This time we got off easily, with an obvious problem. But the bugs can crash your code outright – if the type in the mistaken definition is larger than the actual type, or be subtle, if the wrong type is only slightly bigger and causes unrelated memory corruption, or when an int is taken for a float.

Your C++ compiler can catch some of those errors:

# g++ -c check.c -o check.o
# g++ -c version.c -o version.o
# g++ version.o check.o -o check
/usr/bin/ld: check.o: in function `main':
check.c:(.text+0x6): undefined reference to `version'
collect2: error: ld returned 1 exit status
# nm version.o
0000000000000000 r _ZL7version
# nm check.o
0000000000000000 T main
                 U printf
                 U version

Unfortunately, C offers no solution but to always keep the headers correct!

Summary

Those are the most common but annoying problems you may encounter which stem from the way C and symbols interact. If you're intrigued about some other linking problem, let me know about it. You can find me on the "about me" page.

Glossary

Important terms this episode:

• inlining – copying a function's body instead of making a call
• link-time optimization (LTO) – attempting to inline functions while linking, across translation unit boundaries

From C to exec: Part 2

The last part explored static linking, but stopped before actually executing our executable. If you think this was all the linking, you're in for a surprise!

This post is best followed using the code from the previous part.

Act 1: Unexpected symbol

Scene 1: Undefined?

Did you notice the lonely symbol in the last act? Let's call it forth:

# gcc -c -o printer.o printer.c
# ar rcs libprinter.a printer.o
# nm libprinter.a

printer.o:
0000000000000000 T print_hello
                 U puts

That's a lonely "Undefined" symbol! And where did it even come from? Remember printer.c:

#include <stdio.h>
void print_hello(void) {
  puts("Hello World!\n");
}

We call puts, but we never define it… Previously, when we tried to use a symbol that we didn't define, gcc refused to finish the linking, but here it works:

# gcc hello.o libprinter.a -o hello
# ./hello
Hello World!

What gives?

Scene 2: Deus ex library

Before we solve the mystery, let me throw another hint towards the solution: puts is hidden in a library! More specifically, it's in libc:

# nm /lib64/libc.so.6 | grep puts
[…]
0000000000073280 W puts
[…]

Here, "W" means it's a "weak" but defined symbol.

We know where the lost symbol is, and where it's missing from. But we still don't know the connection between the two places! The newest trace we have is the /lib64/libc.so.6 file...

Scene 3: Dynamic libraries

We're familiar with static libraries already. They have the .a file ending, and they are an ingredient for the executable, and are no longer used after linking. But another variety exists. Like static libraries, it contain symbols, but its usefulness extends beyond linking time, and into the actual execution. They are called dynamic libraries, and their file names traditionally end with .so, for "shared object".

# file /lib64/libc.so.6
/lib64/libc.so.6: symbolic link to libc-2.29.so
# file /lib64/libc-2.29.so
/lib64/libc-2.29.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=51377236ad01808e26404c98faa41d72f11a46a5, for GNU/Linux 3.2.0, not stripped, too many notes (256)

libc.so.6 is one of them.

Scene 4: Reconstruction

If we want to make progress on the story of the misplaced puts, we need to understand dynamic libraries. Time to create our own!

We already created a static library before, so let's use the same procedure, but make a dynamic one this time:

# gcc -fPIC -c printer.c -o printer.o
# gcc -shared printer.o -o libprinter.so
# file libprinter.so
libprinter.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=dd2595216f9c10b317ab3dbece8450a31fcaf672, not stripped
# nm libprinter.so | grep hello
0000000000001109 T print_hello

The procedure is similar to building a static library, except packaging with ar was replaced by another linking step (with -shared), making it look more like creating an executable. The object file gets created in a slightly different way, with the -fPIC flag being obligatory.

Most importantly, we see our print_hello symbol as present! Let's link the whole program together:

# gcc hello.o libprinter.so -o hello_dynamic
# file hello_dynamic
hello_dynamic: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=5a8e0d151089682720545349da87b330ab089805, not stripped

So far so good, this looks just like our previous static binary. Let's make sure all is fine.

# nm hello_dynamic | grep hello
                 U print_hello
# ./hello_dynamic
./hello_dynamic: error while loading shared libraries: libprinter.so: cannot open shared object file: No such file or directory

Uh oh! We lost print_hello along the way, and the program now tries to open the shared library we created. The one that contains print_hello! Could there be a connection? What made the program try to load a file if all we do is printing text?

Scene 5: Dynamic linker

The culprit here is the dynamic linker. It's part of the operating system responsible for loading programs. Remember the interpreter part of our file descriptions?

[...] dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2 [...]

The interpreter's responsibility is to execute executables. It sets up basic resources, links them together if needed (!), and finally gives control to the program by running the entry point (in C that ends up being the function corresponding to the main symbol).

You may be surprised to see linking here again. Didn't we do enough of that already? We linked object files together with the dynamic library, and the dynamic library needed to be linked itself. Why again?

Well, that's because we chose to create a dynamic library instead of a static one. The static library's selling point is that its code is injected (at the time of linking) into the executable. The dynamic library's point is that its code is injected (at the time of execution) into the running program. The former option provides some degree of certainty that the program doesn't change, while the latter gives some flexibility: the dynamic library can get changed and updated without the need to recreate the executable.

When the interpreter complains about cannot open shared object file, it means that it needed the library for the linking step, but couldn't find it in the standard location. The ldd command lists all dynamic libraries that need to be linked together before running the executable:

# ldd ./hello_dynamic
        linux-vdso.so.1 (0x00007ffc35be9000)
        libprinter.so => not found
        libc.so.6 => /lib64/libc.so.6 (0x00007fc874215000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fc874419000)

Our shared library is not found, because it's not in one of the standard paths. Thankfully, we can tell the dynamic linker to look for extra libraries in the current directory:

# LD_LIBRARY_PATH=`pwd` ldd ./hello_dynamic
        linux-vdso.so.1 (0x00007ff5f4e32000)
        libprinter.so => /home/rhn/libprinter.so (0x00007ff5f4e28000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ff5f4c26000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff5f4e33000)
# LD_LIBRARY_PATH=`pwd` ./hello_dynamic
Hello World!

Great, we directed the computer to find our lost print_hello symbol inside a shared library!

Scene 6: Standard libraries

The list coming from ldd is suspicious. In our final step using gcc, we linked hello.o together with libprinter.so. Where did the other dynamic libraries on the list come from?

The answer is: kernel and standard libraries.

If you look at our statically linked executable, you will see the same list:

# ldd ./hello
        linux-vdso.so.1 (0x00007ffeda3d9000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f4e3293a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f4e32b3e000)

The libraries here are required for the execution of virtually all programs, therefore gcc includes them implicitly. If you remember all the way back, we found puts in one of them: in libc.so.6. It turns out that all our executables do eventually link with that library, just well after the executable is created. That's why we were allowed to use puts even without knowing where it was defined.

Final words

With this, you should have a pretty good understanding of where linking happens on a modern computer. Linking still has a lot of quirks when working with C — after all, we may want some control over what we mark as symbols and how. That will et covered in a future part.

Glossary

• interpreter: the software that executes executables
• standard library: libraries containing basic functionality that most programmers don't need to think about

From C to exec

As the idea of having a technical blog grew in my mind and came closer to reality, it became obvious that it needs to host some kind of useful content apart from fluff and opinions. For this, I asked a few of my many technical friends:

What can I write about for you?

The suggestion that I liked the most was to write about the process that turns a C source file into an executable. The topic seems quite opaque, and indeed I don't recall seeing many ways to get into it without already having some background knowing about linking, translation units, and symbols.

The goal of this post is to establish this background for a reader who has a passing familiarity with C.

The seasoned reader should be warned that this is not a rigorous summary of the C standard, but rather a gentle introduction to the practical world of linking ELF files on GNU/Linux with GCC. Simplicity prevails, so while the C standard gives some room for different results, we'll explore only the case of unoptimized code and GCC version 9.

Act 1: Hello World

If you've dealt with C before, you're probably familiar with the hello.c file.

#include <stdio.h>
int main(void) {
  puts("Hello World!");
  return 0;
}

The process of turning code into an executable usually consists of at least 2 main steps: compilation and static linking. Depending on how you've usually done your programming, you may be familiar with different ways of performing those two steps. Let's take a look at two most popular ones.

Scene 1: Executable

This unsuspecting source file is about to undergo transformations that push it through the entire pipeline, changing it beyond recognition into a dynamic executable. Prepare your terminals:

# gcc hello.c -o hello
# ./hello
Hello World!
# file hello
hello.o: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=c1b39511769ab152611cd8ad83983dc724333b7a, not stripped

Congratulations, we've turned code into an excutable!

Scene 2: Linking

But wait, wasn't this supposed to be 2 steps? Indeed, gcc does them both at the same time when not passed the -c flag. Let's try again:

# gcc -c hello.c -o hello.o
# ./hello.o
bash: ./hello.o: Permission denied
# file hello.o
hello.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

That's not the same kind of file as before! What we got is an object file instead: a compiled version of the source code, without any linking applied. Notice the "relocatable" instead of "executable".

Let's link it now:

# gcc hello.o -o hello
# ./hello
Hello World!

That's an executable again!

You may wonder: what happened? how is the intermediate file different from the executable? why is it useful? Let's move on to the second act...

Act 2: Symbols

Here we discover why linking is useful and necessary.

Our project is growing more complicated. It's composed of two source files now.

hello.c morphs:

#include "print.h"
int main(void) {
  print_hello();
  return 0;
}

printer.c contains:

#include <stdio.h>
void print_hello(void) {
  puts("Hello World!");
}

Finally, they are joined by print.h:

void print_hello(void);

Scene 1: Object files

This is a more complicated project. A function from one file is referenced in another! And there's a new header file as well. How do we turn this into an executable? Of course, we could just use gcc without the -c flag, and hope that it figures out everything on its own. But that's not why we're here, so let's go step by step:

This compiles the printer.c file into an object file printer.o:

gcc -c printer.c -o printer.o

So far so good. This produces the hello.o file, which is compiled hello.c:

gcc -c hello.c -o hello.o

Wait a minute… hello.c calls a function in printer.c, but neither the command, nor hello.c reference printer.c anywhere! All they have is the name print_hello, but not the body of the function. How does the compiler know what call to insert?

Turns out that the compiler doesn't actually need to resolve all names to function bodies immediately. What it does instead is use symbols to step around the problem. When a source file containing a function (or a global variable) named print_hello gets compiled, the resulting object file (or a library) gets a symbol called print_hello. On the other hand, any object file using that symbol creates a reference to it (in the relocation table), to be filled later: at linking time.

Let's see it for ourselves:

# nm printer.o
0000000000000000 T print_hello
                 U puts

According to man nm, T means the symbol is defined and in the Text section. What about the other object file?

# nm hello.o
0000000000000000 T main
                 U print_hello

As expected, print_hello is referenced, but U for "Undefined". It's simply not in this file.

To fill in the vacancy, let's do the final step, and link both files together:

# gcc hello.o print.o -o hello
# nm hello
[...]
0000000000401126 T main
0000000000401136 T print_hello
[...]

There are many more symbols, but the one we're interested in is there: print_hello is now present. Notice that main is present as well, meaning that we have the bodies of both of our source files in the same compiled file now.

The linker has performed relocation. If you remember from before, the object file type returned by the file command was relocatable. Our binary is now executable, meaning that we can't repeat the same operation again to add some symbols we forgot about any more. But the linker will not let you forget anything anyway:

# gcc hello.o -o hello
/usr/bin/ld: hello.o: in function `main':
hello.c:(.text+0x5): undefined reference to `print_hello'
collect2: error: ld returned 1 exit status

Scene 2: Header files

The other anomaly introduced in this example is the header file. It contains only one line:

void print_hello(void);

What is it for? Suppose we didn't have it. The only information about print_hello would come from its call, which looks like this:

  print_hello();

Can you guess the type of the function? Well, sort of: it takes no arguments, but it may return anything. And if we made a mistake, like this:

  print_hello("world");

a compiler guessing the type of arguments would happily go on trusting what we wrote, instead of warning us of the extra argument. The line in the header file, called prototype, informs the compiler about which functions are available, and what type they have.

As an aside, you might notice that this was already partially covered by the linker: when function print_hello was unavailable in our example, we received am "undefined reference" message from the linker. Couldn't we get rid of headers and let the linker catch those errors? Indeed, gcc-c++ uses some additional information to store function type:

# cp printer.c printer.cpp
# g++ -c printer.cpp -o printerpp.o
# nm printerpp.o 
                 U puts
0000000000000000 T _Z11print_hellov
# c++filt _Z11print_hellov
print_hello()

This is called symbol mangling, and C compilers don't generally use it, leaving us with obligatory prototypes, and, in practice, header files.

Despite this, C++ still requires prototypes inside the header files, for reasons that are not clear to me. I suspect compatibility with C, or even IDE suggestions.

Scene 3: Static libraries

Static libraries are used when additional functionality provided by third party is needed. They also provide symbols, and may depend on other libraries. If this sounds familiar, it's because it is! That's the same core functionality as object files.

In fact, if you look at a shared library on Linux (file name ending with .a), it's just an ar archive containing multiple object files.

# ar -t /usr/lib64/libm-2.29.a
s_lib_version.o
s_matherr.o
s_signgam.o
fclrexcpt.o
fgetexcptflg.o
fraiseexcpt.o
[...]

Let's turn our printer into a static library and see for ourselves:

# gcc -c -o printer.o printer.c
# ar rcs libprinter.a printer.o
# nm libprinter.a

printer.o:
0000000000000000 T print_hello
                 U puts
# gcc hello.o libprinter.a -o hello
# ./hello
Hello World!

Intermission

Those steps are sufficient if you need to create a binary for the bare metal, for example for an Arduino with an AVR microcontroller, or if you're creating a basic operating system yourself. The executable needs to be provided in the final form, taking minimal advantage of libraries present on the system. Anything with an operating system permitting dynamic (shared) libraries (like Windows or Linux) is likely to have a 3rd step, which happens immediately before execution: dynamic linking. We will cover that in a future chapter, together with various gotchas related to how C emits symbols.

Glossary

compilation: turning a source representation of a program into machine code

• executable: a runnable program
• header file: a file in the C language containing, among others, prototypes
• linking: combining different object files together
• object file: a compiled form of the source
• prototype: a piece of text telling the compiler about the existence of a certain symbol and its type
• relocation: filling symbols from one object file into others
• relocation table: a list of references to missing symbols
• source code: text written in a human-readable form, e.g. in the C language
• static library: a collection of object files
• symbol: a piece of program that can be accessed from another object file

Brutalist blogging

I started this website in order to share bits of my knowledge. While I don't know everything, I have some workflows and opinions on a range of topics, from systems programming, through user interfaces, to collaboration. I hope that some of my insights will be interesting to others.

One of my insights is that computers are under-utilized. They are powerful, yet few people are able to command their power. The deluge of programming resources seems to be losing against the dwindling cues to do something novel with the machines. Today's phones neither come with BASIC, nor Turbo Pascal, apps don't lend themselves to modification via replacing config or art files, and Excel is not as ubiquitous as on desktops. The end result is closer to reading someone's books of magic than writing them.

As I started writing this Markdown document, I noticed that I was on the verge of the magical realm, and it was up to me to leave cracks in the magical armor, through which the inner workings of the post could be seen. I took inspiration from brutalist architecture, which tries to be structurally honest:

Whatever has been said about honest use of materials, most modern buildings appear to be made of whitewash or patent glazing, even when they are made of concrete or steel. Hunstanton appears to be made of glass, brick, steel and concrete, and is in fact made of glass, brick, steel and concrete. Water and electricity do not come out of unexplained holes in the wall, but are delivered to the point of use by visible pipes and manifest conduits. One can see what Hunstanton is made of, and how it works, and there is not another thing to see except the play of spaces.

In a parallel between architecture and computing, I want my work to stand out against mainstream technology by being more honest transparent, and less secretive and magical. Brutalist blogging, if you will.

The first property of this blog is its simplicity. Minimal CSS, HTML, little scripting, and no minification make it human-inspectable. I intend to add more transparency as time goes on: each entry's raw CommonMark source can be accessed by adding .md to the URL. And finally, the HTML renderer preserves some of the source markup, like # for headings, > for quotes, to let the reader see the inner workings even if it never occurred to them to ask about them.

As a strong believer in the notion that computers should empower their users, I hope that my little contribution helps dispel the magic.