Comments

3360 Comments

It looks like this is crashing on startup on KDE (where akonadi uses it) on F44 and Rawhide, so I'm -1ing this update and the F42 one to ensure we don't push the bug there.

https://bugzilla.redhat.com/show_bug.cgi?id=2439809

This is failing tests because of a version problem. The previous update was kernel-6.20.0-0.rc0.260212gc22e26bd0906e.3.fc45 . You didn't bump the 0 before rc0, or rc0 itself, and this was built on the same day, so the date component is the same. That means we wind up just comparing the git commit, and it looks like 'c' is considered newer than '3', so this is 'older' than the current stable update.

Either fix the NVR (bump the initial 0?) or do another snapshot tomorrow, I guess.

aha, well that'd certainly explain the test failures, I guess. Didn't know bind depended on libuv!

I'm curious: what's wrong with it? In openQA it seems the FreeIPA replication tests fail consistently, but that seems a bit odd. I was assuming it must be a blip as I thought libuv likely had nothing to do with FreeIPA replication, but it was a very consistent result, failed five times across prod and stg...

I've re-run the tests now, let's see what happens.

So the test failures here are to do with the lua change, I think. It seems like what happens is:

  • We set up a buildroot repo and a side repo with this rpm-6.0.1-4.fc45 build in it, and update the system; it correctly updates to this version of rpm and lua-libs 5.5.0-1.fc45
  • Later in the test we do dnf -y group install base-x and, for some reason, that decides to downgrade lua-libs to 5.4.8-5.fc44 and hence rpm to 6.0.1-3.fc45
  • The 'check no package from the update has an older version than what's in the update install' step of the test fails because rpm was downgraded

The tricky bit is why does dnf decide to downgrade lua-libs. I suspect it's because something in base-x requires lua 5.4 (probably liblua-5.4.so()(64bit)). dnf has two choices how to solve that:

  1. Keep lua-libs-5.5.0 and rpm-6.0.1-4, install lua5.4-libs-5.5.0
  2. Downgrade lua-libs to 5.4.8-5 (which it can still find in the 'rawhide' repo, currently, as there's been no compose since it was pushed stable) and rpm to 6.0.1-3

...and it picks 2) because it involves installing fewer packages, which dnf's (well, libsolv's) heuristics tend to like, I think.

If I'm right about that it's probably not a genuine problem, because once there's a new Rawhide compose and the older lua disappears from the rawhide repo, option 2 won't be available and it'll have to do option 1. So I think the thing to do would be to re-run the tests after the next Rawhide compose is done and synced, and see what happens.

pygobject3 was rebuilt separately with the tuned fix, so removing the older build from this update.

oh, we have f44-backgrounds, but this didn't update the deps of the package from f43-backgrounds to f44-backgrounds so it doesn't work.

I guess we also need f44-backgrounds.

This seems to be missing gnome-session:

Dependency problems in the tested packages themselves:
package: gdm-1:50~beta-1.fc45.x86_64 from file:///mnt/update_repo
  gnome-session >= 50~alpha
  gnome-session-wayland-session >= 50~alpha

This got pushed because critpath generation for F44 is broken so it wasn't considered critpath any more so it wasn't gated :| we'll have to hope it works OK. openQA tests on 44 are a bit busted right now, I will try to fix things up in the morning and re-run tests to find out where we stand.

OK, we figured that out too. The fix is pushed and ready to build once branching is done, at least if we can figure out any signing issues or whatever.

We'll have to rebuild everything for Rawhide too as a separate update, of course.

OK, we (mainly @catanzaro) figured that out. Koji is down for branching ATM. Once it's back up we need to backport that, and also do an xdg-desktop-portal-gnome build with the crash fix in it - either backport the subproject update somehow (additional source tarball, I guess) or get a 50.beta release of x-d-p-g.

Sigh, so, latest problem: gnome-shell is crashing on aarch64 due to an undefined symbol:

Feb 05 10:34:26 fedora gnome-shell[1221]: JS ERROR: GIRepository.InvokeError: Could not locate meta_backend_set_keymap_layout_group_async: 'meta_backend_set_keymap_layout_group_async': /lib64/libmutter-18.so.0: undefined symbol: meta_backend_set_keymap_layout_group_async
                                          _promisify@resource:///org/gnome/gjs/modules/core/overrides/Gio.js:468:9
                                          @resource:///org/gnome/shell/ui/environment.js:37:5
Feb 05 10:34:26 fedora gnome-shell[1221]: Execution of main.js threw exception: Module resource:///org/gnome/shell/ui/init.js threw an exception

no idea as of yet why it's undefined on aarch64 only (apparently). Can't debug ATM because Koji is down for branching.

Fix for the aarch64 gdm issue building on this side tag as https://koji.fedoraproject.org/koji/taskinfo?taskID=141935681 . It'll be pulled in whenever @xhorak is ready to re-generate the update with other new builds.

yeah, confirmed. I've filed an upstream issue for now. Can poke more on this tomorrow.

Aha. With gdm debug logging I'm seeing this:

Feb 04 21:06:22 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gdm[1362]: Gdm: GdmLocalDisplayFactory: Checking if udev has settled enough to support graphics.
Feb 04 21:06:22 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gdm[1362]: Gdm: GdmLocalDisplayFactory: Found secondary PCI graphics adapter, not proceeding yet.
Feb 04 21:06:22 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gdm[1362]: Gdm: GdmLocalDisplayFactory: Found secondary PCI graphics adapter, not proceeding yet.
Feb 04 21:06:22 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gdm[1362]: Gdm: GdmLocalDisplayFactory: udev has not settled enough for graphics.
Feb 04 21:06:22 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gdm[1362]: Gdm: GdmLocalDisplayFactory: udev is still settling, so not creating display yet

so I'm suspecting this is gonna be https://gitlab.gnome.org/GNOME/gdm/-/merge_requests/343 . I'm just re-confirming ATM (I've only seen this after a 'systemctl restart gdm.service' so far, confirming now that it happens even on the normal automatic start of gdm).

so, for the aarch64 thing, I managed to hack stuff up so I can ssh into the test VM while it's in the state where GDM isn't appearing. but I'm not sure what to do now I'm in.

There's a gdm process, there's no gnome-shell process. this is all there is for gdm.service logs:

root@ibm-p8-kvm-03-guest-02:/home/test# journalctl -b -u gdm
Feb 04 20:47:18 fedora systemd[1]: Starting gdm.service - GNOME Display Manager...
Feb 04 20:47:18 fedora systemd[1]: Started gdm.service - GNOME Display Manager.

This is a backtrace of the running gdm process:


Thread 5 (Thread 0xffff82d6ed20 (LWP 1104) "pool-spawner"):
#0  syscall () at ../sysdeps/unix/sysv/linux/aarch64/syscall.S:38
No locals.
#1  0x0000ffff916ff104 in g_cond_wait_impl (cond=0xaaaadab35f68, mutex=0xaaaadab35f60) at ../glib/gthread-posix.c:1026
        saved_errno = 0
        res = <optimized out>
        sampled = 0
#2  g_cond_wait (cond=0xaaaadab35f68, mutex=0xaaaadab35f60) at ../glib/gthread.c:1686
No locals.
#3  0x0000ffff91696bb4 in g_async_queue_pop_intern_unlocked (queue=0xaaaadab35f60, wait=1, end_time=-1) at ../glib/gasyncqueue.c:376
        retval = <optimized out>
        __func__ = "g_async_queue_pop_intern_unlocked"
#4  0x0000ffff91705dfc in g_thread_pool_spawn_thread (data=<optimized out>) at ../glib/gthreadpool.c:297
        spawn_thread_data = <optimized out>
        thread = 0x0
        error = 0x0
        name = "pool-0\000\000\000\000\000\000\000\000\000"
#5  0x0000ffff91704dac in g_thread_proxy (data=0xaaaadab35fd0) at ../glib/gthread.c:893
        thread = 0xaaaadab35fd0
        __func__ = "g_thread_proxy"
#6  0x0000ffff911bf49c in start_thread (arg=0xffff82d6ed20) at pthread_create.c:454
        ret = <optimized out>
        pd = 0xffff82d6ed20
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {32, 8454144, 17, 281474915781232, 281473116270816, 281472876866848, 281473122390016, 281473117519872, 281473124806688, 281472868417536, 281472876864976, 16226067375394241941, 0, 16226067375062035249, 0, 0, 0, 0, 0, 0, 0, 0}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
        prevstate = <optimized out>
#7  0x0000ffff9122a11c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone3.S:75
No locals.

Thread 4 (Thread 0xffff8255ed20 (LWP 1105) "gmain"):
#0  __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/aarch64/syscall_cancel.S:50
No locals.
#1  0x0000ffff911bb6a0 in __internal_syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:49
        result = <optimized out>
        pd = 0xffff8255ed20
        ch = <optimized out>
#2  0x0000ffff911bb6f0 in __syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:75
        r = <optimized out>
#3  0x0000ffff9121f32c in __GI_ppoll (fds=fds@entry=0xffff7c000880, nfds=nfds@entry=1, timeout=<optimized out>, timeout@entry=0x0, sigmask=sigmask@entry=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:42
        tval = {tv_sec = 9208057360, tv_nsec = 281472868410240}
#4  0x0000ffff916d1488 in ppoll (__fds=0xffff7c000880, __nfds=1, __timeout=0x0, __ss=0x0) at /usr/include/bits/poll2.h:101
No locals.
#5  g_main_context_poll_unlocked (priority=<optimized out>, context=0xaaaadab302f0, timeout_usec=<optimized out>, fds=0xffff7c000880, n_fds=1) at ../glib/gmain.c:4811
        spec = {tv_sec = 9, tv_nsec = 208057360}
        spec_p = 0x0
        ret = <optimized out>
        errsv = <optimized out>
        poll_func = <optimized out>
#6  g_main_context_iterate_unlocked (context=context@entry=0xaaaadab302f0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4485
        max_priority = 2147483647
        timeout_usec = -1
        some_ready = <optimized out>
        nfds = 1
        allocated_nfds = <optimized out>
        fds = 0xffff7c000880
        begin_time_nsec = <optimized out>
#7  0x0000ffff916d1548 in g_main_context_iteration (context=0xaaaadab302f0, may_block=may_block@entry=1) at ../glib/gmain.c:4556
        retval = <optimized out>
#8  0x0000ffff916d15b0 in glib_worker_main (data=<optimized out>) at ../glib/gmain.c:6764
No locals.
#9  0x0000ffff91704dac in g_thread_proxy (data=0xaaaadab36440) at ../glib/gthread.c:893
        thread = 0xaaaadab36440
        __func__ = "g_thread_proxy"
#10 0x0000ffff911bf49c in start_thread (arg=0xffff8255ed20) at pthread_create.c:454
        ret = <optimized out>
        pd = 0xffff8255ed20
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {32, 8454144, 17, 281474915780976, 281473116270816, 281472868412704, 281473122390016, 281473117519872, 281473124806688, 281472859963392, 281472868410832, 16226067375394241941, 0, 16226067375070620465, 0, 0, 0, 0, 0, 0, 0, 0}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
        prevstate = <optimized out>
#11 0x0000ffff9122a11c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone3.S:75
No locals.

Thread 3 (Thread 0xffff81d4ed20 (LWP 1107) "gdbus"):
#0  __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/aarch64/syscall_cancel.S:50
No locals.
#1  0x0000ffff911bb6a0 in __internal_syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:49
        result = <optimized out>
        pd = 0xffff81d4ed20
        ch = <optimized out>
#2  0x0000ffff911bb6f0 in __syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:75
        r = <optimized out>
#3  0x0000ffff9121f32c in __GI_ppoll (fds=fds@entry=0xffff740008c0, nfds=nfds@entry=2, timeout=<optimized out>, timeout@entry=0x0, sigmask=sigmask@entry=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:42
        tval = {tv_sec = 9242695720, tv_nsec = 281472859956112}
#4  0x0000ffff916d1488 in ppoll (__fds=0xffff740008c0, __nfds=2, __timeout=0x0, __ss=0x0) at /usr/include/bits/poll2.h:101
No locals.
#5  g_main_context_poll_unlocked (priority=<optimized out>, context=0xaaaadab42fa0, timeout_usec=<optimized out>, fds=0xffff740008c0, n_fds=2) at ../glib/gmain.c:4811
        spec = {tv_sec = 9, tv_nsec = 242695720}
        spec_p = 0x0
        ret = <optimized out>
        errsv = <optimized out>
        poll_func = <optimized out>
#6  g_main_context_iterate_unlocked (context=0xaaaadab42fa0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4485
        max_priority = 2147483647
        timeout_usec = -1
        some_ready = <optimized out>
        nfds = 2
        allocated_nfds = <optimized out>
        fds = 0xffff740008c0
        begin_time_nsec = <optimized out>
#7  0x0000ffff916d1648 in g_main_loop_run (loop=0xaaaadab43190) at ../glib/gmain.c:4695
        self = <optimized out>
        __func__ = "g_main_loop_run"
#8  0x0000ffff9158ca04 in gdbus_shared_thread_func (user_data=0xaaaadab44820) at ../gio/gdbusprivate.c:284
        data = 0xaaaadab44820
#9  0x0000ffff91704dac in g_thread_proxy (data=0xaaaadab431b0) at ../glib/gthread.c:893
        thread = 0xaaaadab431b0
        __func__ = "g_thread_proxy"
#10 0x0000ffff911bf49c in start_thread (arg=0xffff81d4ed20) at pthread_create.c:454
        ret = <optimized out>
        pd = 0xffff81d4ed20
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {32, 8454144, 11, 281474915781280, 281473116270816, 281472859958560, 281473122390016, 281473117519872, 281473124806688, 281472851509248, 281472859956688, 16226067375394241941, 0, 16226067375112497969, 0, 0, 0, 0, 0, 0, 0, 0}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
        prevstate = <optimized out>
#11 0x0000ffff9122a11c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone3.S:75
No locals.

Thread 2 (Thread 0xffff8153ed20 (LWP 1109) "GDM userdb work"):
#0  __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/aarch64/syscall_cancel.S:54
No locals.
#1  0x0000ffff911bb6a0 in __internal_syscall_cancel (a1=a1@entry=8, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=0, a6=a6@entry=8, nr=nr@entry=22) at cancellation.c:49
        result = <optimized out>
        pd = 0xffff8153ed20
        ch = <optimized out>
#2  0x0000ffff911bb6f0 in __syscall_cancel (a1=a1@entry=8, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=0, a6=a6@entry=8, nr=nr@entry=22) at cancellation.c:75
        r = <optimized out>
#3  0x0000ffff9122a404 in __GI_epoll_pwait (epfd=epfd@entry=8, events=<optimized out>, maxevents=<optimized out>, timeout=<optimized out>, set=set@entry=0x0) at ../sysdeps/unix/sysv/linux/epoll_pwait.c:40
No locals.
#4  0x0000ffff9122a47c in epoll_wait (epfd=epfd@entry=8, events=<optimized out>, maxevents=<optimized out>, timeout=<optimized out>) at ../sysdeps/unix/sysv/linux/epoll_wait.c:32
No locals.
#5  0x0000ffff918adba0 in epoll_wait_usec (fd=8, events=<optimized out>, maxevents=12, timeout=<optimized out>) at ../src/libsystemd/sd-event/sd-event.c:4576
        msec = <optimized out>
        r = <optimized out>
        epoll_pwait2_absent = <optimized out>
#6  process_epoll (e=0xffff78000880, timeout=<optimized out>, threshold=9223372036854775807, ret_min_priority=<synthetic pointer>) at ../src/libsystemd/sd-event/sd-event.c:4599
        n_event_queue = 2
        m = <optimized out>
        n_event_max = 12
        min_priority = 9223372036854775807
        something_new = false
        r = <optimized out>
        __PRETTY_FUNCTION__ = <optimized out>
        __func__ = <optimized out>
#7  sd_event_wait (e=<optimized out>, e@entry=0xffff78000880, timeout=<optimized out>, timeout@entry=18446744073709551615) at ../src/libsystemd/sd-event/sd-event.c:4722
        epoll_min_priority = <optimized out>
        child_min_priority = <optimized out>
        threshold = 9223372036854775807
        r = <optimized out>
        __func__ = "sd_event_wait"
#8  0x0000ffff918aedd8 in sd_event_run (e=<optimized out>, e@entry=0xffff78000880, timeout=timeout@entry=18446744073709551615) at ../src/libsystemd/sd-event/sd-event.c:4871
        r = 0
        _ref = 0xffff78000880
        __func__ = "sd_event_run"
        __PRETTY_FUNCTION__ = <optimized out>
#9  0x0000ffff918aef5c in sd_event_loop (e=<optimized out>) at ../src/libsystemd/sd-event/sd-event.c:4899
        r = <optimized out>
        _ref = <optimized out>
        __func__ = "sd_event_loop"
#10 0x0000aaaad673b79c in gdm_dynamic_user_varlink_worker (data=0xaaaadab399f0) at ../daemon/gdm-dynamic-user-store.c:721
        ctx = 0xaaaadab399f0
        event = 0xffff78000880
        server = 0xffff78000c20
        r = <optimized out>
#11 0x0000ffff91704dac in g_thread_proxy (data=0xaaaadab47f10) at ../glib/gthread.c:893
        thread = 0xaaaadab47f10
        __func__ = "g_thread_proxy"
#12 0x0000ffff911bf49c in start_thread (arg=0xffff8153ed20) at pthread_create.c:454
        ret = <optimized out>
        pd = 0xffff8153ed20
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {32, 8454144, 0, 281474915779120, 281473116270816, 281472851504416, 281473122390016, 281473117519872, 281473124806688, 281472843055104, 281472851502544, 16226067375394241941, 0, 16226067375121083185, 0, 0, 0, 0, 0, 0, 0, 0}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
        prevstate = <optimized out>
#13 0x0000ffff9122a11c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone3.S:75
No locals.

Thread 1 (Thread 0xffff919e3020 (LWP 1096) "gdm"):
#0  __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/aarch64/syscall_cancel.S:50
No locals.
#1  0x0000ffff911bb6a0 in __internal_syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:49
        result = <optimized out>
        pd = 0xffff919e3020
        ch = <optimized out>
#2  0x0000ffff911bb6f0 in __syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:75
        r = <optimized out>
#3  0x0000ffff9121f32c in __GI_ppoll (fds=fds@entry=0xaaaadab46150, nfds=nfds@entry=3, timeout=<optimized out>, timeout@entry=0x0, sigmask=sigmask@entry=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:42
        tval = {tv_sec = 60640780280, tv_nsec = 281474915782336}
#4  0x0000ffff916d1488 in ppoll (__fds=0xaaaadab46150, __nfds=3, __timeout=0x0, __ss=0x0) at /usr/include/bits/poll2.h:101
No locals.
#5  g_main_context_poll_unlocked (priority=<optimized out>, context=0xaaaadab3d0f0, timeout_usec=<optimized out>, fds=0xaaaadab46150, n_fds=3) at ../glib/gmain.c:4811
        spec = {tv_sec = 60, tv_nsec = 640780280}
        spec_p = 0x0
        ret = <optimized out>
        errsv = <optimized out>
        poll_func = <optimized out>
#6  g_main_context_iterate_unlocked (context=0xaaaadab3d0f0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4485
        max_priority = 2147483647
        timeout_usec = -1
        some_ready = <optimized out>
        nfds = 3
        allocated_nfds = <optimized out>
        fds = 0xaaaadab46150
        begin_time_nsec = <optimized out>
#7  0x0000ffff916d1648 in g_main_loop_run (loop=0xaaaadab455d0) at ../glib/gmain.c:4695
        self = <optimized out>
        __func__ = "g_main_loop_run"
#8  0x0000aaaad672c7e4 in main (argc=<optimized out>, argv=<optimized out>) at ../daemon/main.c:298
        main_loop = 0xaaaadab455d0
        context = <optimized out>
        error = 0x0
        res = <optimized out>
        entries = {{long_name = 0xaaaad6760b00 "fatal-warnings", short_name = 0 '\000', flags = 0, arg = G_OPTION_ARG_NONE, arg_data = 0xaaaad67805dc <fatal_warnings>, description = 0xaaaad6760b10 "Make all warnings fatal", arg_description = 0x0}, {long_name = 0xaaaad6760b28 "timed-exit", short_name = 0 '\000', flags = 0, arg = G_OPTION_ARG_NONE, arg_data = 0xaaaad67805e8 <do_timed_exit>, description = 0xaaaad6760b38 "Exit after a time (for debugging)", arg_description = 0x0}, {long_name = 0xaaaad6758528 "version", short_name = 0 '\000', flags = 0, arg = G_OPTION_ARG_NONE, arg_data = 0xaaaad67805d8 <print_version>, description = 0xaaaad6760b60 "Print GDM version", arg_description = 0x0}, {long_name = 0x0, short_name = 0 '\000', flags = 0, arg = G_OPTION_ARG_NONE, arg_data = 0x0, description = 0x0, arg_description = 0x0}}
        print_version = 0
        fatal_warnings = 0
        do_timed_exit = 0
quit
Detaching from program: /usr/bin/gdm, process 1096
[Inferior 1 (process 1096) detached]

what else can I do to figure this out?

Figured out the xdg-desktop-portal-gnome crash (thanks Elliott), we need https://gitlab.gnome.org/GNOME/xdg-desktop-portal-gnome/-/merge_requests/249 , either backported somehow (tricky as it's a subproject) or just a new upstream release of x-d-p-gnome to package.

aarch64 thing is still a mystery.

So I'm seeing two remaining issues. One, both printing tests fail because the print dialog doesn't open. That looks like it's caused by xdg-desktop-portal crashing (there's repeated crashes logged in the journal). I'll backtrace that soon.

Two, after installing the update and rebooting on aarch64, we fail to reach GDM, and we also can't switch to a VT. If I make the system boot to runlevel 3, though, it does get to a VT fine. The journal from the failed boot seems not to give any indication of the problem, though :(