Rust: Building dependencies with isolated dependency graphs

Rust’s cargo is opinionated as to how dependencies are resolved and selected.

When two dependencies in a programs build depend on the same crate with different patch versions, cargo will resolve to a single version so that only one version of the crate is built into the program.

 ┌──────────────────┐  ┌──────────────────┐  
 │stellar-xdr 21.0.0│  │stellar-xdr 21.0.1│  
 └──────────────────┘  └──────────────────┘  
                                ▲            
           ┌────────────────────┤            
        ┌──┴──┐              ┌──┴──┐         
        │rlib1│              │rlib2│         
        └─────┘              └─────┘         
           ▲                    ▲            
           │                    │            
           │      ┌──────┐      │            
           └──────┤bridge├──────┘            
                  └──────┘

This behavior is desirable most of the time, however when attempting to depend on multiple versions of a library for consensus, where even the smallest change in behavior is a risk, it is preferred to have the dependency graphs of each library resolved independently so that over time they resolve consistently and previous versions of the library continue to use the same dependencies that it did in past builds. In the diagram above that would mean rlib1 would continue to resolve and use stellar-xdr 21.0.0.

Cargo doesn’t support retaining unique dependency graphs for each dependency, but we can using Rust’s rlibs crate-type and rustc directly.

In the following example on GitHub is a program that builds two versions of a library and each librarys’ dependency graph to rlibs, and imports those graphs into the final program:

https://github.com/leighmcculloch/cargo-rlib-separate-dep-tree-example

In the example the bridge program imports the rlib1 and rlib2 crates that are the same crate that at different versions dependended on a different version of the stellar-xdr crate.

rlib1 depends on stellar-xdr 21.0.0
rlib2 depends on stellar-xdr 21.0.1

Normally Cargo would merge the stellar-xdr dependencies and import only one of them to service the requirements of the two rlib crates. In the example the rlib crates are built in isolation and so the version resolution occurs in isolation.

When bridge is built it uses the predetermined and prebuilt rlibs for both the directly imported dependencies rlib1 and rlib2 as well as the transitive dependency stellar-xdr and its dependencies.

 ┌──────────────────┐  ┌──────────────────┐ 
 │stellar-xdr 21.0.0│  │stellar-xdr 21.0.1│ 
 └──────────────────┘  └──────────────────┘ 
           ▲                    ▲           
           │                    │           
        ┌──┴──┐              ┌──┴──┐        
        │rlib1│              │rlib2│        
        └─────┘              └─────┘        
           ▲                    ▲           
           │                    │           
           │      ┌──────┐      │           
           └──────┤bridge├──────┘           
                  └──────┘

How

The steps to replicating the example’s setup:

Set the crate-type of the dependency to rlib (e.g. rlib1/Cargo.toml, rlib2/Cargo.toml)
```
[lib]
crate-type = ["rlib"]
```
Build the rlibs (e.g. Makefile)
```
$ cargo build --release
```

Provide the rlibs both direct and deps to rustc via the Cargo config (e.g. bridge/.cargo/config.toml)

[build]
rustflags = """
--extern rlib1=../rlib1/target/release/librlib.rlib
--extern rlib2=../rlib2/target/release/librlib.rlib
-L dependency=../rlib1/target/release/deps
-L dependency=../rlib2/target/release/deps"""

Don’t import the rlibs (e.g. bridge/Cargo.toml)
Use the extern libs like any other dependency (e.g. bridge/src/main.rs)

Test

To test this setup and example, run the make test command in the example repo. It will build the rlib crates, then build the bridge program and execute it.

The output will show the output of calling functions on each rlib that output data from each respective stellar-xdr crate that was different between the patch versions.

$ make test
cd rlib1 && cargo build --release
   Compiling rlib v1.0.0 (rlib1)
    Finished `release` profile [optimized] target(s) in 0.45s
cd rlib2 && cargo build --release
   Compiling rlib v2.0.0 (rlib2)
    Finished `release` profile [optimized] target(s) in 0.06s
cd bridge && cargo rustc --release -- \
                --extern rlib1=../rlib1/target/release/librlib.rlib \
                --extern rlib2=../rlib2/target/release/librlib.rlib \
                -L dependency=../rlib1/target/release/deps \
                -L dependency=../rlib2/target/release/deps
   Compiling bridge v0.1.0 (bridge)
    Finished `release` profile [optimized] target(s) in 0.14s
./bridge/target/release/bridge
rlib1: 26
rlib2: 25

Cargo logo by the Rust Foundation is licensed under CC BY 4.0