Re: High priority xcall with an arbitrary IPL_SOFT*

[Ryota Ozaki <ozaki-r%netbsd.org@localhost>]

On Mon, Jan 29, 2018 at 1:08 PM, Ryota Ozaki <ozaki-r%netbsd.org@localhost> wrote:
> Hi,
>
> This proposal extends xcall to allow to specify an IPL of the high
> priority xcall. Currently the high priority xcall uses only a softint
> of IPL_SOFTSERIAL. It doesn't work if the caller depends on
> a different softint IPL.
>
> I'm now working on stability improvements of the network stack
> under load. psref_target_destroy can take longer time than we desire
> under load at worst, say 10 or 20 seconds. The cause of the delay
> is xcall and the delay can be mitigated by changing xcall to use
> the high priority. However it breaks atomicity of psref because
> psrefs used in the network stack normally run with IPL_SOFTNET.
> One workaround is to use IPL_SOFSERIAL for such psrefs but we should
> avoid using abnormal IPL as much as possible.
>
> So I propose the above enhancement to solve the issue.
>
> This is a patch implements the feature:
>   http://www.netbsd.org/~ozaki-r/xcall-multi-ipl.diff
> The next patch applies it to psref:
>   http://www.netbsd.org/~ozaki-r/xcall-multi-ipl-psref.diff
> The patch updates xcall.9
>   http://www.netbsd.org/~ozaki-r/xcall-multi-ipl-man.diff

I forgot to mention about API. By the change, there will be three
usages of xcall.

Low priority:
  xc_broadcast(0, ...);
High priority:
  xc_broadcast(XC_HIGHPRI, ...);  // Use a softint of IPL_SOFTSERIAL
High priority with IPL:
  xc_broadcast(XC_HIGHPRI_IPL(ipl), ...);  // Use a softint of ipl

The change doesn't break the kernel API/ABI.

  ozaki-r