[src/trunk]: src/sys/arch/m68k/m68k in pmap_enter_ptpage(), if we are allowed...

[chs <chs%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/ca98aff6314e
branches:  trunk
changeset: 368738:ca98aff6314e
user:      chs <chs%NetBSD.org@localhost>
date:      Sun Jul 31 17:11:41 2022 +0000

description:
in pmap_enter_ptpage(), if we are allowed to fail then fail rather than
waiting for memory to be available.  when we are mapping an anon or uobj page
then we will be holding the lock for that page owner, and sleeping to wait
for memory with a page owner lock held is illegal because the pagedaemon
can wait for that lock, which will lead to deadlock.  fixes PR 56932.

diffstat:

 sys/arch/m68k/m68k/pmap_motorola.c |  8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)

diffs (29 lines):

diff -r 610958a094e9 -r ca98aff6314e sys/arch/m68k/m68k/pmap_motorola.c
--- a/sys/arch/m68k/m68k/pmap_motorola.c        Sun Jul 31 13:49:23 2022 +0000
+++ b/sys/arch/m68k/m68k/pmap_motorola.c        Sun Jul 31 17:11:41 2022 +0000
@@ -1,4 +1,4 @@
-/*     $NetBSD: pmap_motorola.c,v 1.76 2022/04/16 18:15:21 andvar Exp $        */
+/*     $NetBSD: pmap_motorola.c,v 1.77 2022/07/31 17:11:41 chs Exp $        */
 
 /*-
  * Copyright (c) 1999 The NetBSD Foundation, Inc.
@@ -119,7 +119,7 @@
 #include "opt_m68k_arch.h"
 
 #include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: pmap_motorola.c,v 1.76 2022/04/16 18:15:21 andvar Exp $");
+__KERNEL_RCSID(0, "$NetBSD: pmap_motorola.c,v 1.77 2022/07/31 17:11:41 chs Exp $");
 
 #include <sys/param.h>
 #include <sys/systm.h>
@@ -2585,6 +2585,10 @@
                                           va - vm_map_min(kernel_map),
                                           NULL, UVM_PGA_ZERO)) == NULL) {
                        rw_exit(uvm_kernel_object->vmobjlock);
+                       if (can_fail) {
+                               pmap->pm_sref--;
+                               return ENOMEM;
+                       }
                        uvm_wait("ptpage");
                        rw_enter(uvm_kernel_object->vmobjlock, RW_WRITER);
                }