pkg/56817: python deadlock when building samba4 for netbsd32

To: pkg-manager%netbsd.org@localhost,gnats-admin%netbsd.org@localhost,pkgsrc-bugs%netbsd.org@localhost
Subject: pkg/56817: python deadlock when building samba4 for netbsd32
From: mlelstv%netbsd.org@localhost
Date: Sat, 7 May 2022 06:20:01 +0000 (UTC)
>Number:         56817
>Category:       pkg
>Synopsis:       python deadlock when building samba4 for netbsd32
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat May 07 06:20:01 +0000 2022
>Originator:     Michael van Elst
>Release:        NetBSD 9.2_STABLE
>Organization:
	
>Environment:
	
	
System: NetBSD victory.netbsd.org 9.2_STABLE NetBSD 9.2_STABLE (GENERIC64) #16: Fri Jan 21 02:56:58 CET 2022 mlelstv@gossam:/home/netbsd9/obj.evbarm64-el/home/netbsd9/src/sys/arch/evbarm/compile/GENERIC64 evbarm
Architecture: aarch64
Machine: evbarm
>Description:
Building samba in a earmv6hf or earmv7hf chroot fails as the waf
tool, written in python, deadlocks.

 |         \-+- 22533 9997 /usr/pkg/bin/python3.9 /pbulk/work/net/samba4/work/samba-4.15.6/buildtools/bin/waf confi
 |           |--- 03473 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 04018 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 05348 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 07638 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 08883 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 11853 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 12165 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 12166 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 12360 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 14247 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 16072 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 17717 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 18183 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 21811 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           |--- 26858 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20
 |           \--- 27455 9997 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 20

The children are all single-threaded workers waiting in pipe_rd:

USER   PID %CPU %MEM   VSZ  RSS TTY   STAT STARTED    TIME COMMAND
9997 27455  0.0  0.0 13472 6904 pts/1 I+   Thu03PM 0:00.17 /usr/pkg/bin/python3.9 -c #! /usr/bin/env python\n# encoding: utf-8\n# Thomas Nagy, 2016-2018 (ita)\n\nimport os, sys, traceback, base64, signal\ntry:\n\timport cPickle\nexcept ImportError:\n\timport pickle as cPickle\n\ntry:\n\timport subprocess32 as subprocess\nexcept ImportError:\n\timport subprocess\n\ntry:\n\tTimeoutExpired = subprocess.TimeoutExpired\nexcept AttributeError:\n\tclass TimeoutExpired(Exception):\n\t\tpass\n\ndef run():\n\ttxt = sys.stdin.readline().strip()\n\tif not txt:\n\t\t# parent process probably ended\n\t\tsys.exit(1)\n\t[cmd, kwargs, cargs] = cPickle.loads(base64.b64decode(txt))\n\tcargs = cargs or {}\n\n\tif not 'close_fds' in kwargs:\n\t\t# workers have no fds\n\t\tkwargs['close_fds'] = False\n\n\tret = 1\n\tout, err, ex, trace = (None, None, None, None)\n\ttry:\n\t\tproc = subprocess.Popen(cmd, **kwargs)\n\t\ttry:\n\t\t\tout, err = proc.communicate(**cargs)\n\t\texcept TimeoutExpired:\n\t\t\tif k
 wargs.get('start_new_session') and hasattr(os, 'killpg'):\n\t\t\t\tos.killpg(proc.pid, signal.SIGKILL)\n\t\t\telse:\n\t\t\t\tproc.kill()\n\t\t\tout, err = proc.communicate()\n\t\t\texc = TimeoutExpired(proc.args, timeout=cargs['timeout'], output=out)\n\t\t\texc.stderr = err\n\t\t\traise exc\n\t\tret = proc.returncode\n\texcept Exception as e:\n\t\texc_type, exc_value, tb = sys.exc_info()\n\t\texc_lines = traceback.format_exception(exc_type, exc_value, tb)\n\t\ttrace = str(cmd) + '\\n' + ''.join(exc_lines)\n\t\tex = e.__class__.__name__\n\n\t# it is just text so maybe we do not need to pickle()\n\ttmp = [ret, out, err, ex, trace]\n\tobj = base64.b64encode(cPickle.dumps(tmp))\n\tsys.stdout.write(obj.decode())\n\tsys.stdout.write('\\n')\n\tsys.stdout.flush()\n\nwhile 1:\n\ttry:\n\t\trun()\n\texcept KeyboardInterrupt:\n\t\tbreak\n\n 


The last thing that happened was an attempt to build a config test
for setuidx. The build directory only has the source file, so it
is likely that no worker did ever invoke the compiler.

| ...
| #include <stdbool.h>
| #include <stdarg.h>
|  int main(void) { void *__x = (void *)setuidx; return (int)__x; return 0; }
| 
| <==
| [1/2] Compiling bin/.conf_check_650d04c71d35f36b6df490067b0b8987/test.c


Here is a backtrace from the main process with 383 threads, threads 2..382 look like the first.

Thread 1 (LWP 2630 of process 22533):
#0  0xf63b9b4c in ___lwp_park60 () from /usr/lib/libc.so.12
#1  0xf64aa9a8 in pthread_cond_timedwait () from /usr/lib/libpthread.so.1
#2  0xf6698900 in PyThread_acquire_lock_timed () from /usr/pkg/lib/libpython3.9.so.1.0
#3  0xf66ee77c in acquire_timed () from /usr/pkg/lib/libpython3.9.so.1.0
#4  0xf66ee950 in lock_PyThread_acquire_lock () from /usr/pkg/lib/libpython3.9.so.1.0
#5  0xf6589f88 in method_vectorcall_VARARGS_KEYWORDS () from /usr/pkg/lib/libpython3.9.so.1.0
#6  0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#7  0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#8  0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#9  0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#10 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#11 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#12 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#13 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#14 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#15 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#16 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#17 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#18 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#19 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#20 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#21 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
[m#22 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#23 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#24 0xf65829a8 in method_vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#25 0xf657f9f4 in PyVectorcall_Call () from /usr/pkg/lib/libpython3.9.so.1.0
#26 0xf66ee3c8 in t_bootstrap () from /usr/pkg/lib/libpython3.9.so.1.0
#27 0xf6698184 in pythread_wrapper () from /usr/pkg/lib/libpython3.9.so.1.0
#28 0xf64ac8f8 in ?? () from /usr/lib/libpthread.so.1
#29 0xf63b9988 in __mknod50 () from /usr/lib/libc.so.12
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

...

Thread 383 (LWP 1 of process 22533):
#0  0xf63b9b4c in ___lwp_park60 () from /usr/lib/libc.so.12
#1  0xf64aa9a8 in pthread_cond_timedwait () from /usr/lib/libpthread.so.1
#2  0xf6698900 in PyThread_acquire_lock_timed () from /usr/pkg/lib/libpython3.9.so.1.0
#3  0xf66ee77c in acquire_timed () from /usr/pkg/lib/libpython3.9.so.1.0
#4  0xf66ee950 in lock_PyThread_acquire_lock () from /usr/pkg/lib/libpython3.9.so.1.0
#5  0xf6589f88 in method_vectorcall_VARARGS_KEYWORDS () from /usr/pkg/lib/libpython3.9.so.1.0
#6  0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#7  0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#8  0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#9  0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#10 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#11 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#12 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#13 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#14 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#15 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#16 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#17 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#18 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#19 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#20 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#21 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#22 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#23 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#24 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#25 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#26 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#27 0xf657fa4c in PyVectorcall_Call () from /usr/pkg/lib/libpython3.9.so.1.0
#28 0xf6716650 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#29 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#30 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#31 0xf6582a64 in method_vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#32 0xf657fa4c in PyVectorcall_Call () from /usr/pkg/lib/libpython3.9.so.1.0
#33 0xf6716650 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#34 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#35 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#36 0xf657fa4c in PyVectorcall_Call () from /usr/pkg/lib/libpython3.9.so.1.0
#37 0xf6716650 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#38 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#39 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#40 0xf6582a64 in method_vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#41 0xf67166f0 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#42 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#43 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#44 0xf67166f0 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#45 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#46 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#47 0xf67166f0 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#48 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#49 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#50 0xf657f9f4 in PyVectorcall_Call () from /usr/pkg/lib/libpython3.9.so.1.0
#51 0xf6716650 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#52 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#53 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#54 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#55 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#56 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#57 0xf6716694 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#58 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#59 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#60 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#61 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#62 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#63 0xf657f9f4 in PyVectorcall_Call () from /usr/pkg/lib/libpython3.9.so.1.0
#64 0xf6716650 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#65 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#66 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#67 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#68 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#69 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#70 0xf6716694 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#71 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#72 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#73 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#74 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#75 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#76 0xf6582a64 in method_vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#77 0xf67181e8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#78 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#79 0xf657fd14 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#80 0xf67169c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#81 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#82 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#83 0xf6716694 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#84 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#85 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#86 0xf6716694 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#87 0xf670fa9c in function_code_fastcall () from /usr/pkg/lib/libpython3.9.so.1.0
#88 0xf657fe44 in _PyFunction_Vectorcall () from /usr/pkg/lib/libpython3.9.so.1.0
#89 0xf67181e8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.9.so.1.0
#90 0xf664eb08 in _PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#91 0xf664f0b4 in _PyEval_EvalCodeWithName () from /usr/pkg/lib/libpython3.9.so.1.0
#92 0xf664f124 in PyEval_EvalCodeEx () from /usr/pkg/lib/libpython3.9.so.1.0
#93 0xf664f160 in PyEval_EvalCode () from /usr/pkg/lib/libpython3.9.so.1.0
#94 0xf6688e30 in run_eval_code_obj () from /usr/pkg/lib/libpython3.9.so.1.0
#95 0xf6688f38 in run_mod () from /usr/pkg/lib/libpython3.9.so.1.0
#96 0xf668bd88 in pyrun_file () from /usr/pkg/lib/libpython3.9.so.1.0
#97 0xf668bf34 in PyRun_SimpleFileExFlags () from /usr/pkg/lib/libpython3.9.so.1.0
#98 0xf66a85cc in Py_RunMain () from /usr/pkg/lib/libpython3.9.so.1.0
#99 0xf66a8b24 in Py_BytesMain () from /usr/pkg/lib/libpython3.9.so.1.0
#100 0x08b40894 in _start ()



>How-To-Repeat:
Build samba4 in a netbsd32 chroot on aarch64.

>Fix:
	

>Unformatted:
Prev by Date: Re: pkg/48595 (www/firefox on NetBSD/macppc)
Next by Date: Re: pkg/56817: python deadlock when building samba4 for netbsd32
Previous by Thread: Re: pkg/56611 (Current ./bootstrap fails on Cygwin-3.3.3)
Next by Thread: Re: pkg/56817: python deadlock when building samba4 for netbsd32
Indexes:
Home | Main Index | Thread Index | Old Index