Re: Atomic migration chains, balancing fixes.

[Mindaugas Rasiukevicius <rmind%NetBSD.org@localhost>]

Andrew Doran <ad%netbsd.org@localhost> wrote:
> On Tue, May 20, 2008 at 02:41:40PM +0100, Mindaugas Rasiukevicius wrote:
> 
> > http://www.netbsd.org/~rmind/midle_1.diff
> 
> @@ -264,6 +264,9 @@ sched_dequeue(struct lwp *l)
> 
> +       if (spc->spc_migrating == 0)
> +               spc->spc_migrating = NULL;
> +
> 
> Should that '0' be 'l'?   

Indeed.

> @@ -268,40 +268,26 @@ cpu_xc_offline(struct cpu_info *ci)
> 
>         LIST_FOREACH(l, &alllwp, l_list) {
> -               if (l->l_cpu != ci || (l->l_pflag & LP_BOUND))
> +               /* Checks could be performed without lock */
> +               if (l->l_cpu != ci || (l->l_pflag & LP_BOUND) != 0)
>                         continue;
> +               lwp_lock(l);
> 
> Since cpu offline is not performed often, maybe it's better to take the lock
> to be safe?

OK.

> What I mean is: in comparison to the hardware penalty a thread would pay to
> move its cache and VM footprint from one CPU to another, switching to the
> local idle loop is likely to be cheap. So it's unlikely to be a large
> component of the cost involved in migrating a thread.

Understood. I was thinking about overall effect, eg. periodical balancing.

After some glancing at the approaches, that temporary "migration state" when
LWP is in the migration chain does not look very nice.
Migration via idle loop seems to be less risky. If you are OK with the patch,
I will commit it shortly.

-- 
Best regards,
Mindaugas
www.NetBSD.org