Subject: Re: [PATCH] netfilter: use per-cpu recursive lock (v11)

From: Lai Jiangshan <[email-masked]>
Date: Tue, 21 Apr 2009 14:52:30 +0800

Stephen Hemminger wrote:
>> xt_info_rdlock_bh() called recursively here will enter the
>> critical region without &__get_cpu_var(xt_info_locks)->lock.
> 
> NO spin_lock_bh always does a preempt_disable
> 
>   xt_info_rdlock_bh            (depth = -1)
> +1         preempt_disable
>            spin_lock_bh
> +1             preempt_disable
> -1         preempt_enable_no_resched
> ---
> +1
> 
> Second call preempt_count=1   (depth = 0)
>        xt_info_rdlock_bh
> +1         preempt_disable
> -1         preempt_enable_no_resched
> ---
> 

I think I had made you mistook my email.

The preempt_count is correct, but I thought about lock-save:
we must hold &__get_cpu_var(xt_info_locks)->lock
when we enter the read-side critical region.

--------------------
xt_info_rdlock_bh() (depth = -1)
  preempt_disable()
  depth++
  ==========>interrupt here
  ==========>
  ==========>xt_info_rdlock_bh() (depth = 0)
  ==========>  preempt_disable()
  ==========>  depth++
  ==========>  preempt_enable_no_resched()
  ==========>
  ==========>enter the read-side critical region *without* lock.
  ==========>  it may get trashy data.
  ==========>
  ==========>xt_info_rdunlock_bh()
  ==========>
  ==========>interrupt return.
  spin_lock_bh()
  preempt_enable_no_resched()

enter the read-side critical region *with* lock.

xt_info_rdunlock_bh().
----------------------

So I asked:
> +void xt_info_rdlock_bh(void)
> > +{
> > +	struct xt_info_lock *lock;
> > +
> > +	preempt_disable();
> > +	lock = &__get_cpu_var(xt_info_locks);
> > +	if (likely(++lock->depth == 0))

So what happen when xt_info_rdlock_bh() called recursively here?

> > +		spin_lock_bh(&lock->lock);
> > +	preempt_enable_no_resched();
> > +}
> > +EXPORT_SYMBOL_GPL(xt_info_rdlock_bh);
> > +

----------
Is this OK? (Now I suppose we can enter the read-side critical region
in irq context)

void xt_info_rdlock_bh(void)
{
	unsigned long flags;
	struct xt_info_lock *lock;

	local_irq_save(flags);
	lock = &__get_cpu_var(xt_info_locks);
	if (likely(++lock->depth == 0))
		spin_lock_bh(&lock->lock);
	local_irq_restore(flags);
}

Lai