# Why I avoid Cargo: dependency versions
Ever since I started making use of Cargo to build the Rust pieces of Squeekboard, I've hit a wall whenever I needed to add something nontrivial to the build process. I haven't documented them before, so whenever I complained about Cargo, I ended up looking ridiculous without having anything to show for the complaints.
Last week I spent three days solving a problem with building Squeekboard that should have been solveable in 30 minutes, in large part due to Cargo.
## Buster and Bullseye
Squeekboard is a crucial part of the software stack of the Librem5 mobile phone. It's primary goal is to fit in well in PureOS, which powers the phone. PureOS is in turn based on Debian, and inherits a lot of its practices, which are followed by projects related to the Librem5, including Squeekboard.
One such practice is that Rust crates are vendored by Debian, and not by the application author. They are installable with `apt`, and end up in a local repository in `/usr/share/cargo/registry`.
Librem5's base operating system is moving from Debian 10 to Debian 11. I was asked to make Squeekboard work on both in the transition period. Debian 10 ships with gtk-0.5.0, whereas Debian 11 ships with gtk-0.7.0, which contain some incompatibilities, and Squeekboard's build process must adjust to them, depending on where the build happens.
Piece of cake: there's one variable, which needs to be turned into one decision. I did this a million times. Little did I know Cargo hates simple solutions.
## What's the version we're using?
It's not unusual for projects to support two versions of the same dependency. Perhaps the versions come from different vendors. C programs don't have trouble with this:
```
#include <gtk.h>
#if GTK_VERSION==0.5
// old stuff
#endif
```
Rustc won't know dependency versions by itself, but Cargo should turn it to something like this:
```
#[cfg(dependency.gtk.version = "0.5.*")]
use gio::SimpleActionExt; // Doesn't exist in later versions of gtk
```
But… I haven't managed to find any equivalent. It makes sense, Cargo can pull several copies of the same crate, and I think their pieces can even be used in the same files if one is not careful. So there's no good reason this should have worked. Fair, let's try something else.
## What's the available version?
If the compilation process can't tell us what versions we're dealing with, perhaps we can check that before compilation. In Meson, we'd do something like this:
```
gtk-rs = dependency("gtk-rs")
if gtk.version().startswith("0.5.")
add_project_arguments('--cfg old_gtk', language : 'rust')
endif
```
And then, in Rust:
```
#[cfg("old_gtk")]
use gio::SimpleActionExt; // Doesn't exist in later versions of gtk
```
Now the only remaining thing is to create the dependency lookup procedure. While Squeekboard is Debian-focused, building on non-Debian systems is still important, so it must build with crates.io. Thankfully, we have `cargo search`. Let's try with a vendored registry:
```
root@c684b7b31b07:/# CARGO_HOME=/mnt/build/eekboard/debian/cargo/ cargo search gtk
error: dir /usr/share/cargo/registry does not support API commands.
Check for a source-replacement in .cargo/config.
```
Oh no. We can forget about it. I'm not willing to write a tool that searches for crates in all the ways that Cargo supports, and I'm honestly boggled that Cargo doesn't properly do it itself.
As a bonus, the output of cargo search just doesn't make sense. I'm looking for the regex crate, which is at version "1.3.9":
```
$ cargo search regex=1
combine-regex-1 = "1.0.0" # Re-export of regex 1.0 letting combine use both 0.2 and 1.0
webforms = "0.2.2" # Provides form validation for web forms
```
Totally useless :(. But I still have a few tricks up my sleeve.
## Use a build flag
Isn't this obvious? Let's skip all that dependency detection, and just order the build system to use one, in the dumbest fashion possible. The way you'd do it with Meson:
```
if get_option("legacy") == true
gtk-rs = dependency("gtk-rs", version=0.5)
add_project_arguments('--cfg old_gtk', language : 'rust')
else
gtk-rs = dependency("gtk-rs", version>0.5)
endif
squeekboard = executable('squeekboard',
dependencies: [gtk-rs],
)
```
Add to that the conditional in the Rust file, and we're done. Cargo is not Meson, but we can call it with flags too, and then instruct it to use the right dependency. Right?
Wrong.
While Cargo allows choosing dependency versions, they are selected based on target, not on flags. You can use:
```
[target.'cfg(unix)'.dependencies]
openssl = "1.0.1"
[target.'cfg(not(unix))'.dependencies]
openssl = "1.0.0"
```
You can remember that the `cfg()` syntax supports features:
```
[target.'cfg(feature="legacy")'.dependencies]
gtk="0.5.*"
[target.'cfg(not(feature="legacy"))'.dependencies]
gtk="0.7.*"
```
but then you see this:
```
# cargo build
warning: Found `feature = ...` in `target.'cfg(...)'.dependencies`. This key is not supported for selecting dependencies and will not work as expected. Use the [features] section instead: https://doc.rust-lang.org/cargo/reference/features.html
```
And when you follow that web page, you learn that you can specify other dependencies, but not other versions. Foiled again!
Again, I'm boggled why such a basic piece of functionality is working in such a complicated and restrictive way. Wouldn't it be easier to abolish the weird `feature = [dep]` syntax and instead let `foo.dependencies` work, with all the fine-grained control over what the dependencies are?
## Embedded crates
But I didn't stop there. If the deps can't be specified the usual way, then let's get there through a back door. If I can't choose a version, I will choose a crate. 0.5 turns into a crate called "legacy", and 0.7 into one called "current". My Cargo.toml looked like this:
```
[dependencies]
current = {path="deps/current", optional=true}
old = {path="deps/old", optional=true}
[features]
legacy = ["old"]
```
The `deps/old/Cargo.toml` contained the actual version:
```
[dependencies]
gtk-rs = "0.5.*"
```
and the current one had `0.7.*`. When we choose the crate, we choose the dependency with it! So clever! There's no way it won't work!
```
root@c684b7b31b07:~/foo/foo# CARGO_HOME=/mnt//build/eekboard/debian/cargo/ cargo build
error: failed to select a version for the requirement `gtk = "0.5.*"`
candidate versions found which didn't match: 0.7.0
location searched: directory source `/usr/share/cargo/registry` (which is replacing registry `https://github.com/rust-lang/crates.io-index`)
required by package `old v0.1.0 (/root/foo/foo/deps/old)`
... which is depended on by `foo v0.1.0 (/root/foo/foo)`
```
What!? This cannot be! Why oh why?
My only guess is that cargo pulls all the dependencies indiscriminately, regardless of whether it actually needs them or not. Obviously, this entire shebang is because we *don't* have one of them! Why is Cargo missing the point of choosing dependency versions so hard?
## Nuclear option
With this in mind, there's only one solution left. If Cargo is greedy enough to snatch everything it sees, then we'll just not let it know there are possibly any other dependencies. We'll generate a Cargo.toml after we know which dependency we need, and we'll never let Cargo know about the other dependency. The details of the generation are quite straightforward: just change depedencies based on build flags. The build scripts get complicated, though. Now they must include `--manifest-path`. This looks trivial, but this changes the root of the crate, so now we need to give it the path to the sources again:
```
[lib]
name = "rs"
path = "@path@/src/lib.rs"
crate-type = ["staticlib", "rlib"]
# Cargo can't do autodiscovery if Cargo.toml is not in the root.
[[bin]]
name = "test_layout"
path = "@path@/src/bin/test_layout.rs"
[[example]]
name = "test_layout"
path = "@path@/examples/test_layout.rs"
```
But it works. Finally.
Sadly, we sacrificed autodetection of tests and binaries, as well as distro-agnosticism.
## You're doing it all wrong!
I'm sure some of you will see the struggle and consider it self-inflicted. "Why didn't you try X?", "You're fighting the tool because your model is outdated!", "This is not a bug". Some of them will be valid criticisms, and I'm going to address those I could think of.
### Use multiple Cargo.lock versions
I could have used "*" as the dependency version, and instead rely on two versions of Cargo.lock to select the actual dependency I want. That is troublesome.
I don't want to know exactly what versions of Rust crates (or any other deps really) the upstream distro is shipping. This is why I use distros in the first place: they remove some burden of deps auditing from me. All I want to know is the minor version for API compatibility reasons. However, Cargo.lock forces me to care about dependencies by asking for a hash of each. I would have to find out what dependency is available, and feed Cargo.lock with its hash. That hits the same problem with detecting what we have again.
### Vendor your crates yourself
Squeekboard is just one project out of many in PureOS, and all the others follow Debian's best practices: use Debian's software versions whenever possible. On one hand, this is designed to set right expectations, on the other it relegates the responsibility for auditing and updating to Debian, as explained in the previous argument.
In this light, I'm already letting Debian vendor my crates, and that won't change.
### Cargo is designed for crates.io, not for distributions or local repositories
I'm not sure if this is Cargo's explicit goal to be useful with crates.io, but my experience says other sources are in practice playing catch-up. If I relied exclusively on crates.io, I would have no problems.
However, the build policy in PureOS requires builds to happen without a network connection. That means we can't use crates.io. We've already eliminated vendoring before, and so we're stuck with some form of a local repository, which clearly isn't well-supported by Cargo.
## Insights
My insight from this adventure is that Cargo doesn't prioritize users who want to control their own sources of software at the moment. It's inflexible in the way that favours crates.io, where the implicit assumption is that all possible crates and versions are available. After all, it crashes when a non-available version is specified even if unused.
Cargo also doesn't want application developers to offload dependency checking. It's recommended to commit Cargo.toml together with the application, but there's no provision to ignore it when doing a build with local versions of the same crates, like when you do in distro builds.
Cargo is not composable the way other build systems are. When building a C program, artifacts in the form of shared or static libraries are placed in a well-known directory. Cargo creates an opaque directory structure for compiled crates in the `target` directory, which cannot be used by another build system later due to lack of documentation. Sadly, this means that when Cargo fails to solve a need, there's no alternative.
The same creates a network effect, where Cargo is de facto the only tool that builds useful Rust programs. It's difficult to the point of pointlessness to "build" serde and gtk-rs separately, and then "link" them with the main program using manual `rustc` calls.
Creating composable artifacts would undermine Cargo's monopoly and allow a better integration with distributions as well as other programming languages in my opinion. Build systems should not infect all software they touch.