Add arm6v (Rpi0/1) build to gh action #276
Conversation
| linux-armv6-rpi) | ||
| # Add the RPI toolchain in order to use the Rpi toolchain | ||
| # See https://github.com/mdirkse/rust_armv6 | ||
| git clone --depth=1 https://github.com/raspberrypi/tools.git "rpi_tools" |
There was a problem hiding this comment.
I was taking a look at this now, and I don't understand this part. Installing the gcc-arm-linux-gnueabihf Debian package, like we already do for amrv7, and then configuring cargo to target arm instead of armv7 seems to work fine. The cross-compiler comes with Rust, so the extra cargo config is only for the linker -- and the linker in use shouldn't really matter when targeting v6 or v7, I'd think.
I've tried this out in #277 . Could you give the binary produced by that a try? (And I've also fixed a couple of minor cosmetic issues, and ensured the v6 binary is actually made part of the release. But kept your name, obviously!)
There was a problem hiding this comment.
Hi,
I've not looked into details too much but trusted @mdirkse's note in the readme here saying :
Cargo knows how to cross-compile to ARMv6, but we need the linker from the official Raspberry PI toolchain (installed in the image). To use it, make sure your project has a .cargo/config file with at least the following content:
I guess the resulting build is indeed different as I'm getting a different output when comparing respectively the build from PR#277 and the one resulting from a build with Rpi tools linker (endbasic-linux-armv6-rpi.zip :
# PR277 build
$ file endbasic
endbasic: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, BuildID[sha1]=25d224ceb0bfabcebbcbf45eb596d214bd88088e, for GNU/Linux 3.2.0, not stripped
# Using Rpi tools linker
$file endbasic
endbasic: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, not stripped
I am getting a segfault trying to run the PR#277 build on real hw.
When I have more time I'll try to provide more infos if you need me to.
EDIT: I realise now that although I mention a link to https://github.com/mdirkse/rust_armv6 in the script, I did not otherwise credit @mdirkse for this part of the script on which my PR is in part based, so maybe there should be more mentionned about it ?
There was a problem hiding this comment.
The comment is right in that you need a linker, but I'm surprised you need that specific linker. I'd hope the one that comes with Debian for cross-compilation is "good enough" as it has been for armv7.
I did look at the differences too but I hoped they wouldn't matter. "pie" is obviously a difference just based on the file type, which then has implications on the required shared libraries:
dev:~/os/endbasic> arm-linux-gnueabihf-readelf -d endbasic-bad | grep Shared.lib
0x00000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x00000001 (NEEDED) Shared library: [libm.so.6]
0x00000001 (NEEDED) Shared library: [libc.so.6]
0x00000001 (NEEDED) Shared library: [ld-linux-armhf.so.3]
dev:~/os/endbasic> arm-linux-gnueabihf-readelf -d endbasic-good | grep Shared.lib
0x00000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x00000001 (NEEDED) Shared library: [librt.so.1]
0x00000001 (NEEDED) Shared library: [libpthread.so.0]
0x00000001 (NEEDED) Shared library: [libm.so.6]
0x00000001 (NEEDED) Shared library: [libdl.so.2]
0x00000001 (NEEDED) Shared library: [libc.so.6]
0x00000001 (NEEDED) Shared library: [ld-linux-armhf.so.3]
But they look... sane?
TBH I don't even know if the armv7 binary works today. The GH Actions builds it on ubuntu-latest, but that's different than Raspbian. What OS are you running it on?
There was a problem hiding this comment.
Sorry about answer spamming :) I'm putting this in a new answer for visibility.
I just noticed this in the rpi tools readme :
Note: if building for Pi0/1 using --with-arch=armv6 --with-float=hard --with-fpu=vfp is recommended (and matches the default flags of the toolchains included here).
Maybe that's the culprit ?
There was a problem hiding this comment.
I tried running the 'good' builds on the official 32bits bookworm lite rpios, and then on some custom buildroot based distro, it works fine on those.
FWIW, here is a strace of the 'bad' build:
xecve("/mnt/endbasic", ["/mnt/endbasic"], 0xbee84e60 /* 10 vars */) = 0
brk(NULL) = 0x15f1000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/libgcc_s.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\0\0\0\0004\0\0\0"..., 512) = 512
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH, STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_att0
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6fd0000
mmap2(NULL, 127284, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb6fb0000
mmap2(0xb6fce000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1d000) = 0xb6fce000
close(3) = 0
openat(AT_FDCWD, "/lib/libm.so.6", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\0\0\0\0004\0\0\0"..., 512) = 512
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH, STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_att0
mmap2(NULL, 303120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb6f65000
mmap2(0xb6fae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x48000) = 0xb6fae000
close(3) = 0
openat(AT_FDCWD, "/lib/libc.so.6", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\349\2\0004\0\0\0"..., 512) = 512
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH, STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_att0
mmap2(NULL, 1307188, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb6e25000
mmap2(0xb6f58000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x132000) = 0xb6f58000
mmap2(0xb6f5b000, 37428, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb6f5b000
close(3) = 0
set_tls(0xb6fd1620) = 0
set_tid_address(0xb6fd11a8) = 111
set_robust_list(0xb6fd11ac, 12) = 0
rseq(0xb6fd1780, 0x20, 0, 0xe7f5def3) = 0
mprotect(0xb6f58000, 8192, PROT_READ) = 0
mprotect(0xb6fae000, 4096, PROT_READ) = 0
mprotect(0xb6fce000, 4096, PROT_READ) = 0
mprotect(0xa0d000, 155648, PROT_READ) = 0
mprotect(0xb6ff3000, 8192, PROT_READ) = 0
ugetrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---
+++ killed by SIGSEGV +++
Segmentation fault
So a SEGV_MAPERR error ( i.e: /* address not mapped to object */) 🤔
Is there any debugging output I can provide that could help track the issue ?
As for the armv7 builds, I should be able to test those on a rpi3 next week.
TL;DR
LongreadHi, I did some tests on Qemu (using this setup guide) using: simulating :
I tried running :
BullseyeOn Bullseye, only the build using the rpi tools linker works. After checking, the GLIBC version on RpiOS is 2.31 : and it looks like the Here are some more informations about the builds dependencies, running BookwormOn Bookworm, the ARMV7 build works, but the ARMV6 build without rpi tools linker does not work. Ldd version is 2.36 : |
|
Hmm, so the glibc version inconsistencies... I could expect (and I've recently been having to deal with that problem at work too, heh). Maybe the only reasonable path here is yours, which is to actually use rpi-tools for both builds just "to be safe" and try to avoid the system-provided libraries. But could you reproduce the crash you were previously seeing? In any case, thanks for digging into this and the qemu instructions! I'll try to play with it too. And it'd be awesome to have a run test in the GH scripts to validate that the binaries are good, but I can imagine that it's going to be way too slow and annoying to automate. |
|
Hi, TL;DR
A few more tests : Real HW, Raspberry Pi B rev 2.0, RaspiOS Bookworm, linux 6.1.0 :Only build with rpi tools works Real HW, Raspberry Pi A rev 2.0, buildroot system, linux 6.6.28 :Only build with rpi tools works Segfault is occuring here. QemuI did some further tests with qemu, simulating a rpi0/1 this time, with both bullseye and bookworm to see if I could reproduce the crash/segfault you mentionned. I had to use a workaround documented here to avoid a kernel panic at boot : BullseyeBookworm |
2 participants
In relation to #275, this adds armv6 builds to the gh workflow.
Don't merge a0d3a72, it was just a push to test/trigger the actions. It looks like it builds fine :
https://github.com/ABelliqueux/endbasic/actions/runs/14943362855