Re: percpu_foreach() does not execute remotely

[Taylor R Campbell <campbell+netbsd-tech-kern%mumble.net@localhost>]

> Date: Tue, 28 Jan 2020 18:54:59 -0800
> From: Jason Thorpe <thorpej%me.com@localhost>
> 
> Something like this.  I haven't tested this yet, but I will shortly.
> Only high-pri xcalls are supported in this -- to make low-pri xcalls
> work, you'd need to change how percpu_cpu_swap() interlocks with
> percpu_foreach().  High-pri doesn't need to take the
> percpu_swap_lock because percpu_cpu_swap() protects its critical
> section with splhigh().

Why is there a restriction to priority and why is percpu_swap_lock
relevant?  I'm probably missing something else that should be obvious,
but why not just...

struct percpu_foreach_xcall {
	percpu_callback_t	cb;
	void			*cookie;
};

static void
percpu_foreach_xc(void *vpercpu, void *vpfx)
{
	struct percpu *percpu = vpercpu;
	struct percpu_foreach_xcall *pfx = vpfx;
	void *ptr;

	ptr = percpu_getref(percpu);
	(*pfx->cb)(ptr, pfx->cookie, curcpu());
	percpu_putref(percpu);
}

void
percpu_foreach_xcall(struct percpu *percpu, unsigned xcflags,
    percpu_callback_t cb, void *cookie)
{
	struct percpu_foreach_xcall pfx = { .cb = cb, .cookie = cookie };
	CPU_INFO_ITERATOR cii;
	struct cpu_info *ci;

	for (CPU_INFO_FOREACH(cii, ci))
		xc_wait(xc_unicast(xcflags, &percpu_foreach_xc, percpu, &pfx));
}