Figured out the xdg-desktop-portal-gnome crash (thanks Elliott), we need https://gitlab.gnome.org/GNOME/xdg-desktop-portal-gnome/-/merge_requests/249 , either backported somehow (tricky as it's a subproject) or just a new upstream release of x-d-p-gnome to package.

aarch64 thing is still a mystery.

so, for the aarch64 thing, I managed to hack stuff up so I can ssh into the test VM while it's in the state where GDM isn't appearing. but I'm not sure what to do now I'm in.

There's a gdm process, there's no gnome-shell process. this is all there is for gdm.service logs:

root@ibm-p8-kvm-03-guest-02:/home/test# journalctl -b -u gdm
Feb 04 20:47:18 fedora systemd[1]: Starting gdm.service - GNOME Display Manager...
Feb 04 20:47:18 fedora systemd[1]: Started gdm.service - GNOME Display Manager.

This is a backtrace of the running gdm process:

Thread 5 (Thread 0xffff82d6ed20 (LWP 1104) "pool-spawner"):
#0  syscall () at ../sysdeps/unix/sysv/linux/aarch64/syscall.S:38
No locals.
#1  0x0000ffff916ff104 in g_cond_wait_impl (cond=0xaaaadab35f68, mutex=0xaaaadab35f60) at ../glib/gthread-posix.c:1026
        saved_errno = 0
        res = <optimized out>
        sampled = 0
#2  g_cond_wait (cond=0xaaaadab35f68, mutex=0xaaaadab35f60) at ../glib/gthread.c:1686
No locals.
#3  0x0000ffff91696bb4 in g_async_queue_pop_intern_unlocked (queue=0xaaaadab35f60, wait=1, end_time=-1) at ../glib/gasyncqueue.c:376
        retval = <optimized out>
        __func__ = "g_async_queue_pop_intern_unlocked"
#4  0x0000ffff91705dfc in g_thread_pool_spawn_thread (data=<optimized out>) at ../glib/gthreadpool.c:297
        spawn_thread_data = <optimized out>
        thread = 0x0
        error = 0x0
        name = "pool-0\000\000\000\000\000\000\000\000\000"
#5  0x0000ffff91704dac in g_thread_proxy (data=0xaaaadab35fd0) at ../glib/gthread.c:893
        thread = 0xaaaadab35fd0
        __func__ = "g_thread_proxy"
#6  0x0000ffff911bf49c in start_thread (arg=0xffff82d6ed20) at pthread_create.c:454
        ret = <optimized out>
        pd = 0xffff82d6ed20
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {32, 8454144, 17, 281474915781232, 281473116270816, 281472876866848, 281473122390016, 281473117519872, 281473124806688, 281472868417536, 281472876864976, 16226067375394241941, 0, 16226067375062035249, 0, 0, 0, 0, 0, 0, 0, 0}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
        prevstate = <optimized out>
#7  0x0000ffff9122a11c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone3.S:75
No locals.

Thread 4 (Thread 0xffff8255ed20 (LWP 1105) "gmain"):
#0  __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/aarch64/syscall_cancel.S:50
No locals.
#1  0x0000ffff911bb6a0 in __internal_syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:49
        result = <optimized out>
        pd = 0xffff8255ed20
        ch = <optimized out>
#2  0x0000ffff911bb6f0 in __syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:75
        r = <optimized out>
#3  0x0000ffff9121f32c in __GI_ppoll (fds=fds@entry=0xffff7c000880, nfds=nfds@entry=1, timeout=<optimized out>, timeout@entry=0x0, sigmask=sigmask@entry=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:42
        tval = {tv_sec = 9208057360, tv_nsec = 281472868410240}
#4  0x0000ffff916d1488 in ppoll (__fds=0xffff7c000880, __nfds=1, __timeout=0x0, __ss=0x0) at /usr/include/bits/poll2.h:101
No locals.
#5  g_main_context_poll_unlocked (priority=<optimized out>, context=0xaaaadab302f0, timeout_usec=<optimized out>, fds=0xffff7c000880, n_fds=1) at ../glib/gmain.c:4811
        spec = {tv_sec = 9, tv_nsec = 208057360}
        spec_p = 0x0
        ret = <optimized out>
        errsv = <optimized out>
        poll_func = <optimized out>
#6  g_main_context_iterate_unlocked (context=context@entry=0xaaaadab302f0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4485
        max_priority = 2147483647
        timeout_usec = -1
        some_ready = <optimized out>
        nfds = 1
        allocated_nfds = <optimized out>
        fds = 0xffff7c000880
        begin_time_nsec = <optimized out>
#7  0x0000ffff916d1548 in g_main_context_iteration (context=0xaaaadab302f0, may_block=may_block@entry=1) at ../glib/gmain.c:4556
        retval = <optimized out>
#8  0x0000ffff916d15b0 in glib_worker_main (data=<optimized out>) at ../glib/gmain.c:6764
No locals.
#9  0x0000ffff91704dac in g_thread_proxy (data=0xaaaadab36440) at ../glib/gthread.c:893
        thread = 0xaaaadab36440
        __func__ = "g_thread_proxy"
#10 0x0000ffff911bf49c in start_thread (arg=0xffff8255ed20) at pthread_create.c:454
        ret = <optimized out>
        pd = 0xffff8255ed20
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {32, 8454144, 17, 281474915780976, 281473116270816, 281472868412704, 281473122390016, 281473117519872, 281473124806688, 281472859963392, 281472868410832, 16226067375394241941, 0, 16226067375070620465, 0, 0, 0, 0, 0, 0, 0, 0}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
        prevstate = <optimized out>
#11 0x0000ffff9122a11c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone3.S:75
No locals.

Thread 3 (Thread 0xffff81d4ed20 (LWP 1107) "gdbus"):
#0  __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/aarch64/syscall_cancel.S:50
No locals.
#1  0x0000ffff911bb6a0 in __internal_syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:49
        result = <optimized out>
        pd = 0xffff81d4ed20
        ch = <optimized out>
#2  0x0000ffff911bb6f0 in __syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:75
        r = <optimized out>
#3  0x0000ffff9121f32c in __GI_ppoll (fds=fds@entry=0xffff740008c0, nfds=nfds@entry=2, timeout=<optimized out>, timeout@entry=0x0, sigmask=sigmask@entry=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:42
        tval = {tv_sec = 9242695720, tv_nsec = 281472859956112}
#4  0x0000ffff916d1488 in ppoll (__fds=0xffff740008c0, __nfds=2, __timeout=0x0, __ss=0x0) at /usr/include/bits/poll2.h:101
No locals.
#5  g_main_context_poll_unlocked (priority=<optimized out>, context=0xaaaadab42fa0, timeout_usec=<optimized out>, fds=0xffff740008c0, n_fds=2) at ../glib/gmain.c:4811
        spec = {tv_sec = 9, tv_nsec = 242695720}
        spec_p = 0x0
        ret = <optimized out>
        errsv = <optimized out>
        poll_func = <optimized out>
#6  g_main_context_iterate_unlocked (context=0xaaaadab42fa0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4485
        max_priority = 2147483647
        timeout_usec = -1
        some_ready = <optimized out>
        nfds = 2
        allocated_nfds = <optimized out>
        fds = 0xffff740008c0
        begin_time_nsec = <optimized out>
#7  0x0000ffff916d1648 in g_main_loop_run (loop=0xaaaadab43190) at ../glib/gmain.c:4695
        self = <optimized out>
        __func__ = "g_main_loop_run"
#8  0x0000ffff9158ca04 in gdbus_shared_thread_func (user_data=0xaaaadab44820) at ../gio/gdbusprivate.c:284
        data = 0xaaaadab44820
#9  0x0000ffff91704dac in g_thread_proxy (data=0xaaaadab431b0) at ../glib/gthread.c:893
        thread = 0xaaaadab431b0
        __func__ = "g_thread_proxy"
#10 0x0000ffff911bf49c in start_thread (arg=0xffff81d4ed20) at pthread_create.c:454
        ret = <optimized out>
        pd = 0xffff81d4ed20
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {32, 8454144, 11, 281474915781280, 281473116270816, 281472859958560, 281473122390016, 281473117519872, 281473124806688, 281472851509248, 281472859956688, 16226067375394241941, 0, 16226067375112497969, 0, 0, 0, 0, 0, 0, 0, 0}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
        prevstate = <optimized out>
#11 0x0000ffff9122a11c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone3.S:75
No locals.

Thread 2 (Thread 0xffff8153ed20 (LWP 1109) "GDM userdb work"):
#0  __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/aarch64/syscall_cancel.S:54
No locals.
#1  0x0000ffff911bb6a0 in __internal_syscall_cancel (a1=a1@entry=8, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=0, a6=a6@entry=8, nr=nr@entry=22) at cancellation.c:49
        result = <optimized out>
        pd = 0xffff8153ed20
        ch = <optimized out>
#2  0x0000ffff911bb6f0 in __syscall_cancel (a1=a1@entry=8, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=0, a6=a6@entry=8, nr=nr@entry=22) at cancellation.c:75
        r = <optimized out>
#3  0x0000ffff9122a404 in __GI_epoll_pwait (epfd=epfd@entry=8, events=<optimized out>, maxevents=<optimized out>, timeout=<optimized out>, set=set@entry=0x0) at ../sysdeps/unix/sysv/linux/epoll_pwait.c:40
No locals.
#4  0x0000ffff9122a47c in epoll_wait (epfd=epfd@entry=8, events=<optimized out>, maxevents=<optimized out>, timeout=<optimized out>) at ../sysdeps/unix/sysv/linux/epoll_wait.c:32
No locals.
#5  0x0000ffff918adba0 in epoll_wait_usec (fd=8, events=<optimized out>, maxevents=12, timeout=<optimized out>) at ../src/libsystemd/sd-event/sd-event.c:4576
        msec = <optimized out>
        r = <optimized out>
        epoll_pwait2_absent = <optimized out>
#6  process_epoll (e=0xffff78000880, timeout=<optimized out>, threshold=9223372036854775807, ret_min_priority=<synthetic pointer>) at ../src/libsystemd/sd-event/sd-event.c:4599
        n_event_queue = 2
        m = <optimized out>
        n_event_max = 12
        min_priority = 9223372036854775807
        something_new = false
        r = <optimized out>
        __PRETTY_FUNCTION__ = <optimized out>
        __func__ = <optimized out>
#7  sd_event_wait (e=<optimized out>, e@entry=0xffff78000880, timeout=<optimized out>, timeout@entry=18446744073709551615) at ../src/libsystemd/sd-event/sd-event.c:4722
        epoll_min_priority = <optimized out>
        child_min_priority = <optimized out>
        threshold = 9223372036854775807
        r = <optimized out>
        __func__ = "sd_event_wait"
#8  0x0000ffff918aedd8 in sd_event_run (e=<optimized out>, e@entry=0xffff78000880, timeout=timeout@entry=18446744073709551615) at ../src/libsystemd/sd-event/sd-event.c:4871
        r = 0
        _ref = 0xffff78000880
        __func__ = "sd_event_run"
        __PRETTY_FUNCTION__ = <optimized out>
#9  0x0000ffff918aef5c in sd_event_loop (e=<optimized out>) at ../src/libsystemd/sd-event/sd-event.c:4899
        r = <optimized out>
        _ref = <optimized out>
        __func__ = "sd_event_loop"
#10 0x0000aaaad673b79c in gdm_dynamic_user_varlink_worker (data=0xaaaadab399f0) at ../daemon/gdm-dynamic-user-store.c:721
        ctx = 0xaaaadab399f0
        event = 0xffff78000880
        server = 0xffff78000c20
        r = <optimized out>
#11 0x0000ffff91704dac in g_thread_proxy (data=0xaaaadab47f10) at ../glib/gthread.c:893
        thread = 0xaaaadab47f10
        __func__ = "g_thread_proxy"
#12 0x0000ffff911bf49c in start_thread (arg=0xffff8153ed20) at pthread_create.c:454
        ret = <optimized out>
        pd = 0xffff8153ed20
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {32, 8454144, 0, 281474915779120, 281473116270816, 281472851504416, 281473122390016, 281473117519872, 281473124806688, 281472843055104, 281472851502544, 16226067375394241941, 0, 16226067375121083185, 0, 0, 0, 0, 0, 0, 0, 0}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
        prevstate = <optimized out>
#13 0x0000ffff9122a11c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone3.S:75
No locals.

Thread 1 (Thread 0xffff919e3020 (LWP 1096) "gdm"):
#0  __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/aarch64/syscall_cancel.S:50
No locals.
#1  0x0000ffff911bb6a0 in __internal_syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:49
        result = <optimized out>
        pd = 0xffff919e3020
        ch = <optimized out>
#2  0x0000ffff911bb6f0 in __syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=8, a6=a6@entry=0, nr=nr@entry=73) at cancellation.c:75
        r = <optimized out>
#3  0x0000ffff9121f32c in __GI_ppoll (fds=fds@entry=0xaaaadab46150, nfds=nfds@entry=3, timeout=<optimized out>, timeout@entry=0x0, sigmask=sigmask@entry=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:42
        tval = {tv_sec = 60640780280, tv_nsec = 281474915782336}
#4  0x0000ffff916d1488 in ppoll (__fds=0xaaaadab46150, __nfds=3, __timeout=0x0, __ss=0x0) at /usr/include/bits/poll2.h:101
No locals.
#5  g_main_context_poll_unlocked (priority=<optimized out>, context=0xaaaadab3d0f0, timeout_usec=<optimized out>, fds=0xaaaadab46150, n_fds=3) at ../glib/gmain.c:4811
        spec = {tv_sec = 60, tv_nsec = 640780280}
        spec_p = 0x0
        ret = <optimized out>
        errsv = <optimized out>
        poll_func = <optimized out>
#6  g_main_context_iterate_unlocked (context=0xaaaadab3d0f0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4485
        max_priority = 2147483647
        timeout_usec = -1
        some_ready = <optimized out>
        nfds = 3
        allocated_nfds = <optimized out>
        fds = 0xaaaadab46150
        begin_time_nsec = <optimized out>
#7  0x0000ffff916d1648 in g_main_loop_run (loop=0xaaaadab455d0) at ../glib/gmain.c:4695
        self = <optimized out>
        __func__ = "g_main_loop_run"
#8  0x0000aaaad672c7e4 in main (argc=<optimized out>, argv=<optimized out>) at ../daemon/main.c:298
        main_loop = 0xaaaadab455d0
        context = <optimized out>
        error = 0x0
        res = <optimized out>
        entries = {{long_name = 0xaaaad6760b00 "fatal-warnings", short_name = 0 '\000', flags = 0, arg = G_OPTION_ARG_NONE, arg_data = 0xaaaad67805dc <fatal_warnings>, description = 0xaaaad6760b10 "Make all warnings fatal", arg_description = 0x0}, {long_name = 0xaaaad6760b28 "timed-exit", short_name = 0 '\000', flags = 0, arg = G_OPTION_ARG_NONE, arg_data = 0xaaaad67805e8 <do_timed_exit>, description = 0xaaaad6760b38 "Exit after a time (for debugging)", arg_description = 0x0}, {long_name = 0xaaaad6758528 "version", short_name = 0 '\000', flags = 0, arg = G_OPTION_ARG_NONE, arg_data = 0xaaaad67805d8 <print_version>, description = 0xaaaad6760b60 "Print GDM version", arg_description = 0x0}, {long_name = 0x0, short_name = 0 '\000', flags = 0, arg = G_OPTION_ARG_NONE, arg_data = 0x0, description = 0x0, arg_description = 0x0}}
        print_version = 0
        fatal_warnings = 0
        do_timed_exit = 0
quit
Detaching from program: /usr/bin/gdm, process 1096
[Inferior 1 (process 1096) detached]

what else can I do to figure this out?

Aha. With gdm debug logging I'm seeing this:

Feb 04 21:06:22 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gdm[1362]: Gdm: GdmLocalDisplayFactory: Checking if udev has settled enough to support graphics.
Feb 04 21:06:22 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gdm[1362]: Gdm: GdmLocalDisplayFactory: Found secondary PCI graphics adapter, not proceeding yet.
Feb 04 21:06:22 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gdm[1362]: Gdm: GdmLocalDisplayFactory: Found secondary PCI graphics adapter, not proceeding yet.
Feb 04 21:06:22 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gdm[1362]: Gdm: GdmLocalDisplayFactory: udev has not settled enough for graphics.
Feb 04 21:06:22 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gdm[1362]: Gdm: GdmLocalDisplayFactory: udev is still settling, so not creating display yet

so I'm suspecting this is gonna be https://gitlab.gnome.org/GNOME/gdm/-/merge_requests/343 . I'm just re-confirming ATM (I've only seen this after a 'systemctl restart gdm.service' so far, confirming now that it happens even on the normal automatic start of gdm).

yeah, confirmed. I've filed an upstream issue for now. Can poke more on this tomorrow.

Could you update gnome-shell straight to the 50.beta, please? It had been released only few hours ago, thus it was not ready before you created this update. It contains important changes for gnome-software, which would be good to have ready for the next week (and even if not, the 50.beta is supposed to be released this week in general, thus getting the latest bits is better) .

Fix for the aarch64 gdm issue building on this side tag as https://koji.fedoraproject.org/koji/taskinfo?taskID=141935681 . It'll be pulled in whenever @xhorak is ready to re-generate the update with other new builds.

This update has been unpushed.

This update was unpushed as part of Fedora 44 Mass Branching because it failed gating tests. Please resubmit these changes for F44 and/or F45/Rawhide.

OK, we (mainly @catanzaro) figured that out. Koji is down for branching ATM. Once it's back up we need to backport that, and also do an xdg-desktop-portal-gnome build with the crash fix in it - either backport the subproject update somehow (additional source tarball, I guess) or get a 50.beta release of x-d-p-g.

OK, we figured that out too. The fix is pushed and ready to build once branching is done, at least if we can figure out any signing issues or whatever.

We'll have to rebuild everything for Rawhide too as a separate update, of course.

This got pushed because critpath generation for F44 is broken so it wasn't considered critpath any more so it wasn't gated :| we'll have to hope it works OK. openQA tests on 44 are a bit busted right now, I will try to fix things up in the morning and re-run tests to find out where we stand.