misc/57508: Automated test case ifconfig_up_down_ipv4 fails intermittently on some architectures

[jspath55%gmail.com@localhost]

>Number:         57508
>Category:       misc
>Synopsis:       Automated test case ifconfig_up_down_ipv4 fails intermittently on some architectures
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    misc-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jul 06 20:15:00 +0000 2023
>Originator:     Jim Spath
>Release:        10.0_BETA
>Organization:
>Environment:
NetBSD etc.etc 10.0_BETA NetBSD 10.0_BETA (GENERIC) #0: Sun Feb 12 12:39:37 UTC 2023  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd64 x86_64 AMD 686-class NetBSD

>Description:
This test case fails intermittently on some architectures I tested but not all.
Several tests were chosen from the full test suite after observing inconsistent pass/fail results over many runs.
Tests were run periodically from cron, timed to avoid overlapping runs while completing as many attempts as feasible based on the system speed.

The set being reported in July 2023 includes failure rates in the one-percent ranges, to be followed by other tests that fail more frequently.

The results for systems used:

System Passes Fails

amd64 1484 22
Pi3 270 1
Pi4 783 8
Pi0W 597 0
Pi02W 886 2
i386 (Intel Atom) 701 9


Example failure message (snippet):

tc-so:  linkstr: bus1
tc-so:Executing command [ rump.sysctl -w net.inet.ip.dad_count=5 ]
tc-so:Executing command [ rump.ifconfig shmif0 inet 10.0.0.1/24 up ]
tc-so:Executing command [ rump.ifconfig shmif0 ]
tc-so:Executing command [ rump.ifconfig shmif0 ]
tc-so:Executing command [ rump.ifconfig -w 10 ]
tc-so:Executing command [ rump.ifconfig shmif0 ]
tc-so:Executing command [ rump.ifconfig shmif0 down ]
tc-so:Executing command [ rump.ifconfig shmif0 ]
tc-so:Executing command [ rump.ifconfig shmif0 ]
tc-so:Executing command [ rump.ifconfig shmif0 up ]
tc-so:Executing command [ rump.ifconfig shmif0 ]
tc-so:Executing command [ rump.ifconfig -w 10 ]
tc-se:Fail: incorrect exit status: 1, expected: 0
tc-se:stdout:
tc-se:
tc-se:stderr:
tc-se:rump.ifconfig: timed out




Example pass message (snippet):


tps-count: 1
tp-start: 1686817064.650528, net/if/t_ifconfig, 1
tc-start: 1686817064.650602, ifconfig_up_down_ipv4
tc-so:Executing command [ rump_server -lrumpnet -lrumpnet_net -lrumpnet_netinet -lrumpnet_shmif unix://./r1 ]
tc-so:Executing command [ rump.sysctl -q -w net.inet.arp.debug=1 ]
tc-so:unix://./r1
tc-so:Executing command [ rump.ifconfig shmif0 create ]
tc-so:Executing command [ rump.ifconfig shmif0 linkstr bus1 ]
tc-so:unix://./r1 shmif0
[...]
tc-so:unix://./r1
tc-end: 1686817089.586699, ifconfig_up_down_ipv4, passed
tp-end: 1686817089.587922, net/if/t_ifconfig
info: time.end, Thu Jun 15 08:18:09 UTC 2023


>How-To-Repeat:
$ atf-run net/if/t_ifconfig:ifconfig_up_down_ipv4

>Fix:
Unknown.