Comments

1155 Comments

I explained and fixed the issue @robatino described above here and here. Not sure if we need to do anything for the change to gating.yaml to take effect, though.

the openQA failures here can be ignored, they're due to the update not containing any x86_64 or noarch packages. Telling openQA not to run if there are no packages for it to test is actually kind of awkward and it happens fairly rarely so I just let it be.

karma

As the openQA results show, this breaks FreeIPA server deployment. See https://bugzilla.redhat.com/show_bug.cgi?id=1857043 (same bug in Rawhide) for more details.

karma

As the openQA results show, this breaks FreeIPA server deployment. See https://bugzilla.redhat.com/show_bug.cgi?id=1857043 (same bug in Rawhide) for more details.

karma

per openQA results, FreeIPA server deployment is broken with this update.

karma

per openQA results, FreeIPA server deployment is broken with this update.

As discussed on IRC, that's because I adjusted how the tests work recently, and if you schedule the current test code with an older scheduler you'll have issues because variables the current test code expects to be set (specifically UP1REL, in this case) won't be set by the older scheduler code. Please update your fedora_openqa checkout :)

I've re-scheduled the affected flavors with the current scheduler code, the tests should pass now.

karma

welp, this'll do it

OK, this looks like it has another unfulfilled dep too :/. It needs bind-dyndb-ldap >= 11.3-1 , and that update was only just pushed to testing a day ago. This can't go stable before that does; we should disable autopush (for both karma and time).

openQA tests failed (you can't see this ATM because of the data centre switch, but I can!) because the selinux-policy this depends on is not pushed stable yet. I'll re-run them when it is.

the openQA failure here is just because the update is so large it causes the test VM to run out of disk space. it's not a bug in the update really. I could hack it up and re-run it but it doesn't seem worth bothering because there's no reason this update would actually break FreeIPA.

karma

+1ing this as we need to push it stable because the Firefox update already went stable :(

karma

indeed openQA tests are passing now, looking good.

karma

the bug affects upgrades from F31. working on update within F32 doesn't mean the bug isn't a problem. Since I'm fairly clear on what's going on now and it does seem like a reason not to push this update stable as-is, gonna give it -1 now.

I did a scratch build with the proposed change and tested it. It passed.

OK, so I played around with this locally, using a couple of test packages, foo and bar. I built a foo-1-1 which is just a completely empty package. bar has bar and bar-udev binary packages (also completely empty), with bar-udev requiring bar = %{version}-%{release}.

I built a bar-1-1 where the bar subpackage obsoletes foo < 2-1 and provides foo = %{version}-%{release}, and a bar-1-2 where the obsoletes and provides are in the bar-udev subpackage. I put those packages in a side repo and ran dnf update, and it reproduces the problem: dnf refuses to do anything because it can't install both bar-1-1 and bar-udev-1-2.

I played around with some possible solutions. An explicit Conflicts: bar < 1-2 in bar-udev-1-2 doesn't help. But an explicit Obsoletes: bar < 1-2 in bar-udev-1-2 does help: it makes the upgrade work in the intended way, by only installing bar-udev-1-2.

I was worried it would cause a problem if you have bar-1-1 but not bar-udev-1-1 installed - I was worried it would pull in bar-udev-1-2 on update. But it doesn't seem to:

[root@adam noarch]# dnf --disablerepo=* install /home/adamw/local/repo/x86_64/bar-1-1.noarch.rpm ... Installed: bar-1-1.noarch

Complete!
[root@adam noarch]# dnf --disablerepo=* --enablerepo=side --refresh update
Private side repo                                                                     2.9 MB/s | 3.0 kB     00:00    
Dependencies resolved.
======================================================================================================================
 Package                   Architecture                 Version                      Repository                  Size
======================================================================================================================
Upgrading:
 bar                       noarch                       1-2                          side                       6.1 k

Transaction Summary
======================================================================================================================
Upgrade  1 Package

Total size: 6.1 k

so I think this is a plausible solution: adding Obsoletes: systemd < 245.6-1 to the systemd-udev subpackage. Can you see any other possible issues with that? If not, do you mind if I try it? Thanks!

The change was made a while back, but there's been no update in the interim. The last systemd to hit Bodhi was systemd-245.4-1.fc32, which is the build that's in the frozen stable F32 repo, the one this update is conflicting with. No other systemd update has been issued for F32 since.

I think the problem is that dnf can "see" both systemd-245.4-1.fc32 and systemd-udev-245.6-1.fc32 - the former in the fedora repo, the latter in updates-testing (or in the openQA test, in the repo called advisory, which is where we put the update under test). If the update was pushed stable this would still be the case - systemd-245.4-1.fc32 would not disappear, it would still be in the fedora repo and still visible to dnf. I believe dnf's behaviour for obsoletes is "greedy": it will try to pull in every different package it can see that obsoletes an installed package. So it tries to pull in the older systemd as well as the newer systemd-udev, and that's obviously problematic.

We don't see this problem in Rawhide because we don't have this "two different builds available in different repos" wrinkle there. We always only have the one main repo for Rawhide, dnf only sees one build of systemd at a time there.

I'm going to test if having the newer systemd-udev explicitly conflict with or obsolete the older systemd will help here...

openQA failure here needs a bit of investigation. It's probably some kind of bug somewhere, not sure if it's an issue with the package deps or with dnf. I'm going to disable autopush on this update so it doesn't get autopushed tonight, and will look into it tomorrow if no-one else beats me to it.

This update has been unpushed.

karma

Yup, broken. openQA hit the same crash dwmw2 hit, in FreeIPA server deployment.

BZ#1831086 softhsm use-after-free on process exit