When testing software on Linux, I use servers with various architectures, such as Intel, AMD, Arm, etc. Even when I have a Linux machine [1] that meets my testing requirements, I still need to perform many steps:
- Download and install necessary software
- Verify if there are new test software packages on the build server
- Obtain and set up yum repositories required for dependency packages
- Download and install new test software packages (based on step 2)
- Obtain and set up necessary SSL certificates
- Set up the testing environment, obtain required Git repositories, change configurations, restart daemons, etc.
- Do other necessary tasks
Automating with Scripts#
These steps are so routine that it becomes necessary to automate them and save the scripts in a central location (such as a file server) where they can be downloaded when needed. For this purpose, I wrote a 100-120 line Bash shell script that handles all the configurations for me (including error checking). This script simplifies my workflow in the following way:
- Configure a new Linux system (supporting the architecture for testing)
- Log in to the system and download the automated shell script from the central location
- Run it to configure the system
- Start testing
Learning Go Language#
I have wanted to learn Go language for some time, and converting my beloved shell script into a Go program seemed like a good project to get started and help me get acquainted. Its syntax seemed simple, and after trying out some test programs, I began to improve my knowledge and familiarize myself with the Go standard library.
I spent a week writing Go programs on my laptop. I frequently tested the programs on my x86 server, fixing errors and making the program robust, and everything went smoothly.
I continued to rely on my shell script until I fully transitioned to the Go program. Then, I pushed the binary file to the central file server so that every time I configured a new server, all I had to do was get the binary file, set the executable flag, and run the binary file. I was satisfied with the early results:
$ wget http://file.example.com/<myuser>/bins/prepnode
$ chmod +x ./prepnode
$ ./prepnode
Then, a problem occurred.
In the second week, I allocated a new server from the resource pool. As usual, I downloaded the binary file, set the executable flag, and ran the binary file. But this time, it failed with a strange error:
$ ./prepnode
bash: ./prepnode: cannot execute binary file: Exec format error
$
At first, I thought maybe I didn't set the executable flag successfully. But it was set as expected:
$ ls -l prepnode
-rwxr-xr-x. 1 root root 2640529 Dec 16 05:43 prepnode
What happened? I made no changes to the source code, and the compilation didn't raise any errors or warnings. The previous run was successful, so I carefully examined the error message "format error."
I checked the format of the binary file, and everything seemed fine:
$ file prepnode
prepnode: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
I quickly ran the following command to identify the architecture of the configured test server and the platform the binary was trying to run on. It was an Arm64 architecture, but the binary file I compiled (on my x86 laptop) was in x86-64 format:
$ uname -m
aarch64
The First Lesson for Script Writers in Compilation#
Before that, I had never considered this situation (although I knew about it). I mainly studied scripting languages (usually Python) and shell scripts. Bash shell and Python interpreters can be used on most Linux servers of any architecture. Everything went smoothly before.
But now I'm dealing with Go, a compiled language that generates executable binary files. The compiled binary file consists of machine code or assembly instructions specific to a particular architecture, which is why I received the format error. Previously, the shell and Python interpreters handled the underlying machine code or architecture-specific instructions for me.
Cross-compiling with Go#
I checked the documentation for Golang and found that to generate an Arm64 binary file, all I needed to do was set two environment variables before running the go build command to compile the Go program.
GOOS refers to the operating system, such as Linux, Windows, BSD, etc., while GOARCH refers to the architecture on which the program is to be built.
$ env GOOS=linux GOARCH=arm64 go build -o prepnode_arm64
After building the program, I ran the file command again, and this time it showed ARM AArch64 instead of the previous x86. So now I could build binary files for different architectures on my laptop.
$ file prepnode_arm64
prepnode_arm64: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, not stripped
I copied the binary file from my laptop to the ARM server. Now running the binary file (with the executable flag set) didn't produce any errors:
$ ./prepnode_arm64 -h
Usage of ./prepnode_arm64:
-c Clean existing installation
-n Do not start test run (default true)
-s Use stage environment, default is qa
-v Enable verbose output
What about other architectures?#
x86 and Arm are two of the five architectures supported by the software I'm testing, and I was concerned that Go might not support other architectures. But that's not the case. You can check the architectures supported by Go:
$ go tool dist list
Go supports multiple platforms and operating systems, including:
AIX
Android
Darwin
Dragonfly
FreeBSD
Illumos
ios
Js/wasm
JavaScript
Linux
NetBSD
OpenBSD
Plan 9
Solaris
Windows
To find the specific Linux architectures it supports, run:
$ go tool dist list | grep linux
As shown in the output below, Go supports all the architectures I'm using. Although x86_64 is not listed, AMD64 is compatible with x86-64, so you can generate AMD64 binary files that can run on x86 architecture:
$ go tool dist list | grep linux
linux/386
linux/amd64
linux/arm
linux/arm64
linux/mips
linux/mips64
linux/mips64le
linux/mipsle
linux/ppc64
linux/ppc64le
linux/riscv64
linux/s390x
Handling All Architectures#
Generating binary files for all the architectures I'm testing is as simple as writing a small shell script on my x86 laptop:
#!/usr/bin/bash
archs=(amd64 arm64 ppc64le ppc64 s390x)
for arch in ${archs[@]}
do
env GOOS=linux GOARCH=${arch} go build -o prepnode_${arch}
done
$ file prepnode_*
prepnode_amd64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=y03MzCXoZERH-0EwAAYI/p909FDnk7xEUo2LdHIyo/V2ABa7X_rLkPNHaFqUQ6/5p_q8MZiR2WYkA5CzJiF, not stripped
prepnode_arm64: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=q-H-CCtLv__jVOcdcOpA/CywRwDz9LN2Wk_fWeJHt/K4-3P5tU2mzlWJa0noGN/SEev9TJFyvHdKZnPaZgb, not stripped
prepnode_ppc64: ELF 64-bit MSB executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), statically linked, Go BuildID=DMWfc1QwOGIq2hxEzL_u/UE-9CIvkIMeNC_ocW4ry/r-7NcMATXatoXJQz3yUO/xzfiDIBuUxbuiyaw5Goq, not stripped
prepnode_ppc64le: ELF 64-bit LSB executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), statically linked, Go BuildID=C6qCjxwO9s63FJKDrv3f/xCJa4E6LPVpEZqmbF6B4/Mu6T_OR-dx-vLavn1Gyq/AWR1pK1cLz9YzLSFt5eU, not stripped
prepnode_s390x: ELF 64-bit MSB executable, IBM S/390, version 1 (SYSV), statically linked, Go BuildID=faC_HDe1_iVq2XhpPD3d/7TIv0rulE4RZybgJVmPz/o_SZW_0iS0EkJJZHANxx/zuZgo79Je7zAs3v6Lxuz, not stripped
Now, whenever I configure a new machine, I run the following wget command to download the binary file for the specific architecture, set the executable flag, and then run it:
$ wget http://file.domain.com/<myuser>/bins/prepnode_<arch>
$ chmod +x ./prepnode_<arch>
$ ./prepnode_<arch>
Why?#
You might wonder why I didn't stick with shell scripting or port the program to Python instead of using a compiled language to avoid these troubles. Well, there are always trade-offs, and in doing so, I wouldn't have learned about Go's cross-compiling capabilities and how the program works at the low level when executed on a CPU. In computing, there are always trade-offs, but never let them hinder your learning.
References#
Cross-compiling made easy with Golang [1]
Cross-compiling with Golang [2]