build: auto-detect the presence of the openat2() syscall

[mmayer]

Let configure detect if the openat2() syscall is supported by the kernel headers we are building against. Do not attempt to use openat2() if support is not present.

Users can still disable using the openat2() syscall manually if so desired.

[mmayer]

This is for issue 924.

[satmandu]

Note that this doesn't fix make check:

gcc -I. -I. -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W  -c tls.c -o tls.o
gcc -I. -I. -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W  -c t_stub.c -o t_stub.o
gcc -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W -flto=auto -o tls tls.o syscall.o util2.o t_stub.o lib/compat.o lib/snprintf.o lib/permstring.o lib/sysxattrs.o  -lacl -lpopt -llz4 -lzstd -lxxhash -lcrypto 
gcc -I. -I. -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W  -c getgroups.c -o getgroups.o
gcc -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W -flto=auto -o getgroups getgroups.o -lacl -lpopt -llz4 -lzstd -lxxhash -lcrypto 
gcc -I. -I. -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W  -c getfsdev.c -o getfsdev.o
gcc -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W -flto=auto -o getfsdev getfsdev.o -lacl -lpopt -llz4 -lzstd -lxxhash -lcrypto 
gcc -I. -I. -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W  -c testrun.c -o testrun.o
gcc -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W -flto=auto -o testrun testrun.o
gcc -I. -I. -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W  -c trimslash.c -o trimslash.o
gcc -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W -flto=auto -o trimslash trimslash.o syscall.o util2.o t_stub.o lib/compat.o lib/snprintf.o -lacl -lpopt -llz4 -lzstd -lxxhash -lcrypto 
gcc -I. -I. -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W  -c t_unsafe.c -o t_unsafe.o
gcc -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W -flto=auto -o t_unsafe t_unsafe.o syscall.o util1.o util2.o t_stub.o lib/compat.o lib/snprintf.o lib/wildmatch.o -lacl -lpopt -llz4 -lzstd -lxxhash -lcrypto 
gcc -I. -I. -I./zlib -O3 -pipe -ffat-lto-objects -fPIC -fuse-ld=mold  -flto=auto -flto=auto -DHAVE_CONFIG_H -Wall -W  -c t_chmod_secure.c -o t_chmod_secure.o
t_chmod_secure.c:22:10: fatal error: linux/openat2.h: No such file or directory
   22 | #include <linux/openat2.h>
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:72: t_chmod_secure.o] Error 1

[mmayer]

It works for me. I just tried again after "make distclean".

$ ls -l rsync -rwxrwxr-x 1 mmayer mmayer 2276384 May 29 10:04 rsync

This is with Ubuntu 18.04.

Did you regenerate configure.sh after applying the patch? What does config.log have to say about detecting the presence of SYS_openat2?

You should see something like this.

[...]
checking whether to enable rolling-checksum SIMD optimizations... yes (x86_64)
checking if assembler accepts noexecstack... yes
checking for openat2... no
checking for broken largefile support... no
checking for special C compiler options needed for large files... no
[...]

And in config.log:

[...]
configure.sh:6286: checking for openat2
configure.sh:6305: gcc -c -g -O2 -DHAVE_CONFIG_H -Wall -W  conftest.c >&5
conftest.c: In function 'main':
conftest.c:72:9: error: 'SYS_openat2' undeclared (first use in this function); did you mean 'SYS_openat'?
 int i = SYS_openat2
         ^~~~~~~~~~~
         SYS_openat
conftest.c:72:9: note: each undeclared identifier is reported only once for each function it appears in
conftest.c:72:5: warning: unused variable 'i' [-Wunused-variable]
 int i = SYS_openat2
     ^
[...]

[mmayer]

But I see what you mean re: "make check". Let me take a look. ... That's because t_chmod_secure.c only has

#if defined(__linux__)

instead of

#if defined(__linux__) && defined(HAVE_OPENAT2)

Let me add that change to the pull request.

[mmayer]

BTW, make check was already failing in this situation before my patch. So that is nothing new.

[mmayer]

My pull request is now updated to fix t_chmod_secure.c (i.e. make check) as well.

[tridge]

Unfortunately we also need runtime detection as android blocks the syscall for openat2 with seccomp, which signals the process on attempted openat2() calls, so we need runtime fallback. AOSP refused to fix.

[satmandu]

Having runtime detection makes sense. In part also because if built against newer kernel headers but run on a system with an older kernel, we'll want to make sure the functionality is actually available before using it.

[cyphar]

openat2 is one of the newer syscalls that have uniform syscall numbers for basically all architectures, so you could just define __NR_openat2 yourself as a fallback with very little effort.

Compile-time detection of syscalls doesn't really make sense in the modern world of containers and seccomp, it makes far more sense to check this at runtime anyway. To be fair, modern glibc has wrappers for openat2 finally, so it makes more sense now than it did a few years ago, but I would still recommend just using your own syscall wrappers -- the Linux syscall ABI is guaranteed to never change, and if you want to make use of future struct open_how extensions it is probably easier to do it directly.

[tridge]

the Linux syscall ABI is guaranteed to never change

well, sort of ... when vendors like android use seccomp to block system calls (like they do for openat2) then from userspace point of view the ABI has changed .... sort of!

[cyphar]

I had a chat with @tridge and normally sandboxes use SECCOMP_RET_ERRNO(ENOSYS) which means that the ABI works but apparently Android does SECCOMP_RET_TRAP?! (This is completely crazy, by the way -- this would even break glibc which depends on ENOSYS for fallback detection!!).

That's really painful -- and you can't detect it at build-time either! (Maybe Android should have some #define SYSCALL_<N>_IS_A_LANDMINE 1 for stuff like this. 🤦)

[tridge]

@mmayer @cyphar I've added in android.c which does the horrible signal catch to avoid the seccomp nonsense in android