From: ebiederm@xmission•com (Eric W. Biederman)
To: Hans Schillstrom <hans@schillstrom•com>
Cc: netdev@vger•kernel.org, Daniel Lezcano <daniel.lezcano@free•fr>
Subject: Re: Race condition when creating multiple namespaces?
Date: Mon, 11 Apr 2011 17:27:35 -0700 [thread overview]
Message-ID: <m1ei58co08.fsf@fess.ebiederm.org> (raw)
In-Reply-To: <201104112301.46776.hans@schillstrom.com> (Hans Schillstrom's message of "Mon, 11 Apr 2011 23:01:46 +0200")
Hans Schillstrom <hans@schillstrom•com> writes:
> Hello
> I'v been strugling with this for some time now
>
> When creating multiple namespaces using lxc-start, un-initialized network namespace parts will be called by the new process in the namespace.
> ex. when using conntrack or ipvsadm to quickly, (a sleep 2 "solves" the problem).
> (From what I can see syscall clone() is used in lx-start i.e. do_fork will be called later on.)
> Actually I was debugging ip_vs when closing multiple ns when I fell into this one.
>
> I have a loop that create 33 containers whith lxc-start ... -- test.sh
> the first thing the new conatiner does in test.sh is
> #!/bin/bash
> iptables -t mangle -A PREROUTING -m conntrack --ctstate RELATED,ESTABLISHED -j CONNMARK --restore-mark
> nc -l -p1234
>
> This results in NULL ptr in ip_conntrack_net_init(struct *net)
Ouch!
> and in anoither test test.sh looks like this
> #!/bin/bash
> ipvsadm --start-daemon=master --mcast-interface=lo
> nc -l -p1234
>
> And this results in an uniitialized spinlock in ip_vs_sync
>
> I put a printk in nsproxy: copy_namespaces() and could see a dozens of them
> before anything appears from ipvs or conntrack.
>
> My feeling is that when you start up user processes in a new name space,
> all kernel related init should have been done (you should not need to add a sleep to get it working)
>
> All test made by using todays net-next-2.6 (2.6.39-rc1)
>
> Note:
> That neither conntrack or ip_vs modules where loaded,
> if modules where loaded before creating new namespaces it all works...
>
> Finally the question,
> Should it really work to load modules within a namespace ,
> that is a part of netns ?
>From an implementation point of view kernel modules are not in a
namespace, so there should be no difference between being in a namespace
and loading a kernel networking module and not being in a namespace and
loading in a kernel module.
It does sound like you have hit a module loading race, and perhaps
a race that is confined to network namespaces.
My head is in another problem so I won't be able to look at this for
a bit. But if you are getting into ip_conntrack_net_init with
a NULL network namespace something spectacularly bad is happening.
In particular it looks like you must be hitting a bug in for_each_net.
Which would pretty much have to be a race in adding or removing from
net_namespace_list.
I took a quick skim through the code and whenever we modify the
net_namespace we hold but the net_mutex and inside it the rtnl_lock so I
don't immediate see how you could be getting a NULL net into
ip_conntrack_net_init.
Is there a codepath besides register_pernet_subsys that is calling
ip_conntrack_net_init?
Do you have any local modifications that could be messing up register_pernet_subsys?
Eric
next prev parent reply other threads:[~2011-04-12 0:27 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-04-11 21:01 Race condition when creating multiple namespaces? Hans Schillstrom
2011-04-12 0:27 ` Eric W. Biederman [this message]
2011-04-12 4:56 ` Hans Schillstrom
2011-04-14 20:46 ` Hans Schillstrom
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=m1ei58co08.fsf@fess.ebiederm.org \
--to=ebiederm@xmission$(echo .)com \
--cc=daniel.lezcano@free$(echo .)fr \
--cc=hans@schillstrom$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox