Scaling number of supported platforms. Cross compilation of the rust code with bazel. Part 5

 

This is the 5th part of Basel cross-compilation series.

Previous posts:

  1. What? Do I really need to configure C++ compiler to compile rust?
  2. Compilation from macOS(Intel) to linux x86_64 finaly works. 
  3. Cross compilation for ARM v7. 
  4. Fetching toolchains at build time.

 

All the build files in the previous steps looked pretty much the same, the only differences were:

  • URLs
  • exec architecture
  • target architecture
  • some compiler flags (like paths)

Of cause we don't want to support a separate build file for every host and target combination. The way around it maybe to generate it from the template.

The easiest way would be to use string formatting int the *.bzl file.

So we create `toolchains/cpp_toolchains.bzl`, and store the BUILD file content in the variable there:

BUILD_FILE_TEMPLATE = """
load("@bazel_tools//tools/cpp:unix_cc_toolchain_config.bzl", "cc_toolchain_config")
load("@bazel_tools//tools/build_defs/cc:action_names.bzl", "ACTION_NAMES")

BIN_PREFIX = "{bin_prefix}"

cc_toolchain_config(
    name = "{target_triple}_toolchain_config",
    abi_libc_version = "unknown",
    abi_version = "unknown",
    compiler = "gcc",
    cpu = "{target_cpu}",
    host_system_name = "local",
    target_libc = "unknown",
    target_system_name = "{target_triple}",
    toolchain_identifier = "{target_triple}",
    tool_paths = {{
        "ar": BIN_PREFIX + "ar",
        "cpp": BIN_PREFIX + "g++",
        "gcc": BIN_PREFIX + "gcc",
        "dwp": BIN_PREFIX + "dwp",
        "gcov": BIN_PREFIX + "gcov",
        "ld": BIN_PREFIX + "ld",
        "nm": BIN_PREFIX + "nm",
        "objcopy": BIN_PREFIX + "objcopy",
        "objdump": BIN_PREFIX + "objdump",
        "strip": BIN_PREFIX + "strip",
        "llvm-cov": BIN_PREFIX + "llvm-cov",
    }},
    compile_flags = [
        {compile_flags}
    ],
    link_libs = [
      "-lstdc++",
    ],
)

filegroup(
    name = "all",
    srcs = glob(["**/**"]),
)

cc_toolchain(
    name = "{target_triple}_toolchain",
    # We tell the toolchain, that for every command, all the files are needed
    # This is not ideal, but simple.
    all_files = ":all",
    compiler_files = ":all",
    dwp_files = ":all",
    linker_files = ":all",
    objcopy_files = ":all",
    strip_files = ":all",
    toolchain_config = ":{target_triple}_toolchain_config",
)

toolchain(
    name = "{target_triple}",
    exec_compatible_with = [
        "@platforms//os:{exec_os}",
        "@platforms//cpu:{exec_cpu}",
    ],
    target_compatible_with = [
        "@platforms//os:{target_os}",
        "@platforms//cpu:{target_cpu}",
    ],
    toolchain = ":{target_triple}_toolchain",
    toolchain_type = "@rules_cc//cc:toolchain_type",
)
"""
 
The parameters for each template we store in the other variable:
TOOLCHAIN_CONFIGS = {
    "aarch64-linux-musl": {
    "target_triple": "aarch64-linux-musl",
    "target_cpu": "aarch64",
    "target_os": "linux",
    "exec_os": "macos",
    "exec_cpu": "x86_64",
    "url":
    "https://github.com/messense/homebrew-macos-cross-toolchains/releases/download/v11.2.0-1/aarch64-unknown-linux-musl-x86_64-darwin.tar.gz",
    "sha256": "4fbe95500c327828b437f380bb15851e9a8126cf95180fbf15b76b78e0322ae3",
    "strip_prefix": "aarch64-unknown-linux-musl",
    "bin_prefix": "bin/aarch64-linux-musl-",
    "compile_flags": """
      "-nostdinc++",
      "-isystem",
      "external/aarch64-linux-musl/aarch64-unknown-linux-musl/include/c++/11.2.0",
      "-isystem",
      "external/aarch64-linux-musl/aarch64-unknown-linux-musl/include/c++/11.2.0/aarch64-unknown-linux-musl",
      "-isystem",
      "external/aarch64-linux-musl/aarch64-unknown-linux-musl/sysroot/usr/include/",
    """,
    }
  }
 
We now can download and register toolchain in the macros:
def register_cpp_toolchain(target_triple):
  config = TOOLCHAIN_CONFIGS[target_triple]
  http_archive(
    name = target_triple,
    sha256 = config["sha256"],
    urls = [config["url"]],
    strip_prefix = config["strip_prefix"],
    build_file_content = BUILD_FILE_TEMPLATE.format(**config),
  )
  native.register_toolchains("@{0}//:{0}".format(target_triple))
 
And now we call this macro in the WORKSPACE:
load("//toolchains:cpp_toolchains.bzl", "register_cpp_toolchain")
register_cpp_toolchain("aarch64-linux-musl")

We can notice that we create repository with a name same as target_triple. In this case it would be aarch64-linux-musl that can conflict with other repositories. We better rename it to something that signals, that it is a C++ toolchain repository. 

def register_cpp_toolchain(target_triple):
  config = TOOLCHAIN_CONFIGS[target_triple] 
  processed_config = dict(config)
    
  processed_config["repo_name"] = "cc_toolchain_{target_triple}".format(**config)

Since we generate the repository name outside of config now, we need to modify the compile_flags we can do it dynamically:
TOOLCHAIN_CONFIGS = {
   "aarch64-linux-musl": {
   ...
   "compile_flags_template": """    
   "-nostdinc++",
   "-isystem", "external/{repo_name}/aarch64-unknown-linux-musl/include/c++/11.2.0",
   "-isystem", "external/{repo_name}/aarch64-unknown-linux-musl/include/c++/11.2.0/aarch64-unknown-linux-musl",
   "-isystem", "external/{repo_name}/aarch64-unknown-linux-musl/sysroot/usr/include/",
   """,
   },
}
      
def register_cpp_toolchain(target_triple):
   ...
  
   processed_config["compile_flags"] = config["compile_flags_template"].format(**processed_config)

Now we can add to the Workspace file calls:
load("//toolchains:cpp_toolchains.bzl", "register_cpp_toolchain")
register_cpp_toolchain("aarch64-linux-musl")
register_cpp_toolchain("x86_64-linux-musl") # NEW
register_cpp_toolchain("armv7-linux-musleabihf") # NEW

And cross compilation wold be possible for all 3 platforms.
But we want to cross-compile from the linux too.

In order to do this, we add another layer into the "TOOLCHAIN_CONFIGS" mapping:
TOOLCHAIN_CONFIGS = {
  "x86_64-macos": {
  ...     
  },
     
  "x86_64-linux": {        
  ... 
  }   
}
 
The key is a execution platform, and the content would be configs for different target platforms.

We convert our register_cpp_toolchain macro into a helper macro, that accepts the execution platform as well:
def _register_cpp_toolchain(exec_platform, target_triple):
  config = TOOLCHAIN_CONFIGS[exec_platform][target_triple]
  processed_config = dict(config)
  processed_config["repo_name"] = "cc_toolchain_{target_triple}-{exec_platform}".format(
    exec_platform=exec_platform, **config)
  processed_config["compile_flags"] = config["compile_flags_template"].format(**processed_config)
 
Note, that now we added the exec platform to the generated repository name, otherwise bazel will not work correctly. (Fairly speaking, I would expect that it would crash on the cration of the same repositry second time, but it actually sort of works but incorrectly, and crashes later while resolving toolchains).
 
And the register_cpp_toolchain function now just registers both toolchains:
def register_cpp_toolchain(target_triple):
  _register_cpp_toolchain("x86_64-macos", target_triple) 
  _register_cpp_toolchain("x86_64-linux", target_triple)
Now we can test that the cross-compilation works. Most probably some include flags have to be adjusted, depending on exact toolchain, used. But generally we are done with toolchains. It works now.
 
 

Links:

 https://bazel.build/rules/lib/globals/bzl#repository_rule

Comments

Post a Comment

Popular posts from this blog

Compilation from macOS(Intel) to linux x86_64 finaly works. Cross compilation of the rust code with bazel. Part 2.

This is the continuation of cross-compilation  topic that I've started in previous part What? Do I really need to configure C++ compiler to compile rust? Cross compilation of the rust code with bazel. Part 1. Since we need to add C++ cross-compilation toolchain, let's add the pure C++ project. This will help us to test the C++ toolchain separately I've added a typical C++ hello world impliemntaion and a BUILD file for it: cc_binary ( name = "hello-world" , srcs = [ "main.cpp" ], ) And our goal is to make it compilable for linux by command: bazel build //cpp/hello_world:hello-world --platforms=//:x86_64-linux --incompatible_enable_cc_toolchain_resolution If we just try it, we get and error,  (23:01:17) ERROR: /private/var/tmp/_bazel_evgenypetrov/e83d964deb47c21842622d1b7ffa55fc/external/bazel_tools/tools/cpp/BUILD:58:19: in cc_toolchain_alias rule @baze l_tools//tools/cpp:current_cc_toolchain: Unable to find a CC toolchain using toolchain resolution. ...
Read more

Cross compilation for ARM v7. Cross compilation of the rust code with bazel. Part 3.

This is the third part of the series "Cross compilation of rust with bazel". Previous topics: What? Do I really need to configure C++ compiler to compile rust? Cross compilation of the rust code with bazel. Part 1. Compilation from macOS(Intel) to linux x86_64 finaly works. Cross compilation of the rust code with bazel. Part 2.   Next challenge is to cross-compile for armv7. This part we did together with my colleague Dima and we did it on linux. First thing, that we need is to add the target platform into our root BUILD.bazel platform ( name = "armv7-linux-musleabihf" , constraint_values = [ "@platforms//cpu:armv7" , "@platforms//os:linux" , ], ) And rust toolchain for the new target in the WORKSPACE file: rust_register_toolchains ( edition = "2021" , extra_target_triples = [ "x86_64-unknown-linux-musl" , "armv7-unknown-linux-musleabihf" , # NEW ], ) And as we remember from the part 1 ,...
Read more