obsolete

passt-0^20251209.gc3f1ba7-1.fc44

FEDORA-2025-3d462f0aaf created by sbrivio a month ago for Fedora 44

Automatic update for passt-0^20251209.gc3f1ba7-1.fc44.

Changelog
* Tue Dec  9 2025 Stefano Brivio <sbrivio@redhat.com> - 0^20251209.gc3f1ba7-1
- Fix build on Fedora 43, selinux_requires_min not available on Copr builders
- use %selinux_requires_min macro, drop overlapping dependencies
- use regex instead of SELinux template
- Upstream changes: https://passt.top/passt/log/?qt=range&q=2025_09_19.623dbf6..2025_12_09.c3f1ba7

This update was automatically created

a month ago

This update's test gating status has been changed to 'waiting'.

a month ago

@alexwill I see the same failure here as FEDORA-2025-93b4eb64c3, but now pasta doesn't terminate anymore, it just handles the connection failure as it should.

I can't reproduce this locally, that same curl command works for me with the same container image. How can we debug this? The debug logs I was asking for on https://bodhi.fedoraproject.org/updates/FEDORA-2025-93b4eb64c3#comment-4473565 would help here as well.

This update's test gating status has been changed to 'failed'.

a month ago

Hmm, yeah, I can't immediately reproduce either. Theory: perhaps this only fails if we've run the podman bats test suite first? Let me check that out.

Do you have any idea what would "localhost" resolve to, in this case? Is it intended that these tests run while systemd seems to be starting services?

I can't get the debug log thing to work. If I use /tmp/pasta.log the command runs, but then I can't upload /tmp/pasta.log later because of systemd private /tmp I think? The file doesn't 'exist' so far as the root user is concerned. If I try and use /var/tmp/pasta.log or /home/testman/pasta.log I get "permission denied". I can't do it as root because pasta networking is only supported for rootless. So, uh, bit stuck. I'll poke at it more tomorrow.

It's not the bats tests, it still works for me in a local VM after running the bats tests. You're probably right to think about networking; the networking setup in the test is different from how it would be in a typical test VM, it's using tap networking with a static IP config (this is so we can have another VM connect to the web server and make sure that works). I can figure out what localhost resolves to tomorrow, it's a bit late now for me.

I was suggesting /tmp/pasta.log because other locations, in general, are prevented by SELinux. You can either set SELinux to permissive mode or use another directory if /tmp doesn't work for you, say, /var/run/user/1000/containers/networks/rootless-netns/.

I'm meanwhile trying to figure this out as well, but I have no updates so far.

The only (intended) difference I can see in the behaviour between this version of passt (providing pasta(1)) and the previous one is the result of fixing https://bugs.passt.top/show_bug.cgi?id=113, that is, we now preserve specific IPv4 loopback addresses when forwarding inbound ports. It's not necessarily 127.0.0.1, IPv4 has the whole 127.0.0.0/8 as loopback network, unlike IPv6's ::1.

So, if localhost resolves to, say, 127.0.1.1, but the server in the container specifically listens to 127.0.0.1, you won't be able to reach the server using that other address. Maybe the issue here is somehow related to https://github.com/systemd/systemd/issues/39836?

This is blocking all the updates of passt for Fedora 41, 42, 43 and Rawhide, so let me know if there's anything I can do.

Oops, I keep mixing that up, that one should be just about forwarding outbound connections. Well, in any case, the issue I mentioned above might still be relevant.

Well, by setting SELinux to permissive I managed to get the debug log, does this help?

pasta 0^20251209.gc3f1ba7-1.fc44.x86_64: /usr/bin/pasta.avx2 (3171)
0.0012: info:    No interfaces with usable IPv6 routes
0.0012:          Failed to detect external interface for IPv6
0.0012: info:    IPv6: no external interface as template, use local mode
0.0019: info:    Template interface: enp0s3 (IPv4)
0.0019: info:    Namespace interface: enp0s3
0.0019: info:    MAC:
0.0019: info:        host: 9a:55:9a:55:9a:55
0.0019: info:    DHCP:
0.0019: info:        assign: 172.16.2.114
0.0019: info:        mask: 255.255.255.0
0.0019: info:        router: 172.16.2.2
0.0019: info:    DNS:
0.0019: info:        169.254.1.1
0.0019: info:    DNS search list:
0.0019: info:        .
0.0019: info:    NDP:
0.0019: info:        assign: ::
0.0019: info:        router: fe80::1
0.0019: info:        our link-local: fe80::1
0.0019: info:    DNS search list:
0.0019: info:        .
0.0065:          Sending initial ARP request for guest MAC address
0.0066:          Sending initial NDP NS request for guest MAC address
0.0074:          Using TCP RTO parameters, syn_retries: 6, syn_linear_timeouts: 4, rto_max: 120
0.0088:          SO_PEEK_OFF supported
0.0088:          TCP_INFO tcpi_snd_wnd field  supported
0.0088:          TCP_INFO tcpi_bytes_acked field  supported
0.0088:          TCP_INFO tcpi_min_rtt field  supported
0.0089:          neighbour notifier update: 172.16.2.2 / ae:aa:b8:2a:c2:4a
0.0089:          ARP announcement for 172.16.2.2 / ae:aa:b8:2a:c2:4a
0.0098:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
1.0446:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
1.0447:          tap: protocol 58, fe80::9f:d8ff:fecf:bc5c -> ff02::16 (1 packet)
1.0475:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
1.0475:          tap: protocol 58, fe80::9f:d8ff:fecf:bc5c -> ff02::16 (1 packet)
1.6206:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
1.6206:          tap: protocol 58, fe80::9f:d8ff:fecf:bc5c -> ff02::16 (1 packet)
1.8765:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
1.8766:          tap: protocol 58, fe80::9f:d8ff:fecf:bc5c -> ff02::16 (1 packet)
4.7565:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
11.9885:          pasta: epoll event on netlink neighbour notifier socket 0 (events: 0x00000001)
11.9885:          neighbour notifier update: 172.16.2.2 / ae:aa:b8:2a:c2:4a
12.0302:          pasta: epoll event on listening TCP socket 6 (events: 0x00000001)
12.0302:          Flow 0 (NEW): FREE -> NEW
12.0303:          Flow 0 (INI): NEW -> INI
12.0303:          Flow 0 (INI): HOST [::1]:43182 -> [::]:8080 => ?
12.0303:          Flow 0 (TGT): INI -> TGT
12.0303:          Flow 0 (TGT): HOST [::1]:43182 -> [::1]:8080 => SPLICE [::]:0 -> [fe80::9f:d8ff:fecf:bc5c]:80
12.0303:          Flow 0 (TCP connection (spliced)): TGT -> TYPED
12.0303:          Flow 0 (TCP connection (spliced)): HOST [::1]:43182 -> [::1]:8080 => SPLICE [::]:0 -> [fe80::9f:d8ff:fecf:bc5c]:80
12.0304:          Flow 0 (TCP connection (spliced)): event at tcp_splice_connect:384
12.0304:          Flow 0 (TCP connection (spliced)): SPLICE_CONNECT
12.0304:          Flow 0 (TCP connection (spliced)): Couldn't connect socket for splice: Invalid argument
12.0304:          Flow 0 (TCP connection (spliced)): flag at tcp_splice_conn_from_sock:453
12.0304:          Flow 0 (TCP connection (spliced)): RCVLOWAT_ACT_1
12.0304:          Flow 0 (TCP connection (spliced)): TYPED -> ACTIVE
12.0304:          Flow 0 (TCP connection (spliced)): HOST [::1]:43182 -> [::1]:8080 => SPLICE [::]:0 -> [fe80::9f:d8ff:fecf:bc5c]:80
12.0305:          Flow 0 (TCP connection (spliced)): CLOSED
12.0305:          Flow 0 (FREE): ACTIVE -> FREE
12.0305:          Flow 0 (FREE): HOST [::1]:43182 -> [::1]:8080 => SPLICE [::]:0 -> [fe80::9f:d8ff:fecf:bc5c]:80
12.5005:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)

The console error in this case was "curl: (52) Empty reply from server", btw, not "Connection reset by peer". Not sure if that's significant.

btw, the fact that the tests are failing indicates very strongly that it really is this update that causes the problem. We have this set up quite robustly in openQA. If a test is failing on a given update but not on other updates for the same release, it's pretty much certain that update is the one triggering the failure.

And it looks like localhost resolves to 127.0.0.1 (IPv4) or ::1 (IPv6).

Thanks for the log! It definitely helps.

What's a bit unexpected in the setup is that the host has no IPv6 configured at all, and yet localhost resolves to ::1 instead of 127.0.0.1.

At that point, given that we have no IPv6 address to copy, we just use a link-local address (that's the "local mode" we introduced a while ago to cover cases such as this one or https://bugs.passt.top/show_bug.cgi?id=50). But if we connect to a link-local address (assuming it's correct) in the container, we need to pass the interface number along in connect(), and it looks like we don't do that. I'll look into this next.

Here's the same log file from a successful run (on Rawhide, from a test of another update):

pasta 0^20250919.g623dbf6-1.fc44.x86_64: /usr/bin/pasta.avx2 (3164)
0.0013: info:    No interfaces with usable IPv6 routes
0.0013:          Failed to detect external interface for IPv6
0.0026: info:    Template interface: enp0s3 (IPv4)
0.0027: info:    Namespace interface: enp0s3
0.0027: info:    MAC:
0.0027: info:        host: 9a:55:9a:55:9a:55
0.0027: info:    DHCP:
0.0027: info:        assign: 172.16.2.114
0.0027: info:        mask: 255.255.255.0
0.0027: info:        router: 172.16.2.2
0.0027: info:    DNS:
0.0027: info:        169.254.1.1
0.0027: info:    DNS search list:
0.0027: info:        .
0.0110:          Sending initial ARP request for guest MAC address
0.0144:          SO_PEEK_OFF supported
0.0145:          TCP_INFO tcpi_snd_wnd field  supported
0.0145:          TCP_INFO tcpi_bytes_acked field  supported
0.0145:          TCP_INFO tcpi_min_rtt field  supported
0.0161:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
0.2658:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
0.4498:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
1.2899:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
1.6417:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
5.3217:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)
12.0340:          pasta: epoll event on listening TCP socket 6 (events: 0x00000001)
12.0340:          Flow 0 (NEW): FREE -> NEW
12.0341:          Flow 0 (INI): NEW -> INI
12.0341:          Flow 0 (INI): HOST [127.0.0.1]:41490 -> [0.0.0.0]:8080 => ?
12.0341:          Flow 0 (TGT): INI -> TGT
12.0341:          Flow 0 (TGT): HOST [127.0.0.1]:41490 -> [0.0.0.0]:8080 => SPLICE [0.0.0.0]:0 -> [172.16.2.114]:80
12.0341:          Flow 0 (TCP connection (spliced)): TGT -> TYPED
12.0341:          Flow 0 (TCP connection (spliced)): HOST [127.0.0.1]:41490 -> [0.0.0.0]:8080 => SPLICE [0.0.0.0]:0 -> [172.16.2.114]:80
12.0343:          Flow 0 (TCP connection (spliced)): event at tcp_splice_connect:391
12.0343:          Flow 0 (TCP connection (spliced)): SPLICE_CONNECT
12.0343:          Flow 0 (TCP connection (spliced)): TYPED -> ACTIVE
12.0343:          Flow 0 (TCP connection (spliced)): HOST [127.0.0.1]:41490 -> [0.0.0.0]:8080 => SPLICE [0.0.0.0]:0 -> [172.16.2.114]:80
12.0343:          pasta: epoll event on connected spliced TCP socket 49 (events: 0x00000004)
12.0343:          Flow 0 (TCP connection (spliced)): event at tcp_splice_connect_finish:336
12.0343:          Flow 0 (TCP connection (spliced)): SPLICE_ESTABLISHED
12.0343:          Flow 0 (TCP connection (spliced)): event at tcp_splice_sock_handler:502
12.0343:          Flow 0 (TCP connection (spliced)): 82 from read-side call
12.0344:          Flow 0 (TCP connection (spliced)): 82 from write-side call (passed 1048576)
12.0357:          pasta: epoll event on connected spliced TCP socket 49 (events: 0x00000001)
12.0357:          Flow 0 (TCP connection (spliced)): 8743 from read-side call
12.0358:          Flow 0 (TCP connection (spliced)): 8743 from write-side call (passed 1048576)
12.2690:          pasta: epoll event on connected spliced TCP socket 145 (events: 0x00002001)
12.2690:          Flow 0 (TCP connection (spliced)): event at tcp_splice_sock_handler:509
12.2690:          Flow 0 (TCP connection (spliced)): FIN_RCVD_0
12.2691:          Flow 0 (TCP connection (spliced)): 0 from read-side call
12.2691:          Flow 0 (TCP connection (spliced)): -1 from write-side call (passed 1048576)
12.2692:          Flow 0 (TCP connection (spliced)): event at tcp_splice_sock_handler:613
12.2692:          Flow 0 (TCP connection (spliced)): FIN_SENT_1
12.2696:          pasta: epoll event on connected spliced TCP socket 49 (events: 0x00002011)
12.2697:          Flow 0 (TCP connection (spliced)): event at tcp_splice_sock_handler:509
12.2697:          Flow 0 (TCP connection (spliced)): FIN_RCVD_1
12.2697:          Flow 0 (TCP connection (spliced)): 0 from read-side call
12.2697:          Flow 0 (TCP connection (spliced)): -1 from write-side call (passed 1048576)
12.2698:          Flow 0 (TCP connection (spliced)): event at tcp_splice_sock_handler:613
12.2698:          Flow 0 (TCP connection (spliced)): FIN_SENT_0
12.2698:          Flow 0 (TCP connection (spliced)): flag at tcp_splice_sock_handler:634
12.2698:          Flow 0 (TCP connection (spliced)): RCVLOWAT_ACT_1
12.2699:          Flow 0 (TCP connection (spliced)): CLOSED
12.2699:          Flow 0 (FREE): ACTIVE -> FREE
12.2699:          Flow 0 (FREE): HOST [127.0.0.1]:41490 -> [0.0.0.0]:8080 => SPLICE [0.0.0.0]:0 -> [172.16.2.114]:80
14.0897:          pasta: epoll event on /dev/net/tun device 9 (events: 0x00000001)

I'm no networking expert, but I think on Fedora there's always some baseline level of IPv6 functionality unless you try really hard to get rid of it. I'm not sure you can get an lo that doesn't report itself as having address ::1 for e.g. We don't explicitly configure IPv6 in the openQA tests, but we also don't explicitly try to "turn it off".

Right, yes, IPv6 addresses and routes not being configured at all doesn't imply that IPv6 is disabled (which Fedora doesn't do, of course).

Anyway, in the previous run you showed, we disable IPv6 altogether in the container as we don't find any host interface with IPv6 configured.

That's because of https://bugs.passt.top/show_bug.cgi?id=128, but I didn't think it was desirable... so I fixed it.

That revealed another issue or two: we connect to a link-local address instead of ::1 in the container, which looks wrong, and when we do that (which might be correct in some cases), we pass an invalid address (missing interface) to connect().

Filed https://bugs.passt.top/show_bug.cgi?id=181 meanwhile.

Stand-alone reproducer:

$ pasta -4 --config-net -- sh -c 'pasta --trace --config-net -t 1024 -- nc -l 1024 & { sleep 1; : | nc ::1 1024; }'

So just to make sure I'm following: the reason the openQA test hit this, but you and I couldn't reproduce, is that in the openQA test we set up a static IPv4 config only on the 'real' network interface, so it has no IPv6 addresses - but lo still has an IPv6 address? Whereas in our attempts to reproduce we were just using default DHCP networking, which gives the 'real' network interface an IPv6 address even if it's not globally addressable?

In my attempts, I was using IPv6 hosts with a global unicast address and everything. In your attempt, I'm not sure, but I guess so.

Here's the fix by the way: https://archives.passt.top/passt-dev/20251210070257.1775739-3-david@gibson.dropbear.id.au/. I'm making a new upstream release right now.

And about the openQA environment: yes, pasta(1) doesn't see any IPv6 address and turns on "local mode" for IPv6, https://passt.top/builds/latest/web/passt.1.html#lbAT.

Previously, we would just disable IPv6 in that case, but as a result of https://bugs.passt.top/show_bug.cgi?id=128, now the container has basic IPv6 connectivity and we try to use it, but forgot about scope_id for connect() in this particular case.

This update has been obsoleted by passt-0^20251210.gd04c480-1.fc44.

a month ago

The new fix worked! See FEDORA-2025-9ae8fe3dad and https://openqa.fedoraproject.org/tests/4060148.

Thanks a lot for all the support. This was an actual and serious issue that was prevented, https://bugs.passt.top/show_bug.cgi?id=181.

Thanks for the prompt response!


Please log in to add feedback.

Metadata
Type
unspecified
Karma
0
Signed
Content Type
RPM
Test Gating
Autopush Settings
Unstable by Karma
-3
Stable by Karma
disabled
Stable by Time
0 days
Thresholds
Minimum Karma
+0
Minimum Testing
0 days
Dates
submitted
a month ago
in testing
a month ago

Automated Test Results