* Can we move device drivers into user-space? @ 2012-02-23 4:56 Jidong Xiao 2012-02-23 15:57 ` Cong Wang 2012-02-24 15:19 ` Jidong Xiao 0 siblings, 2 replies; 68+ messages in thread From: Jidong Xiao @ 2012-02-23 4:56 UTC (permalink / raw) To: Kernel development list Hi, I am just curious. Since the concept user-space device drivers has been proposed for several years, and some related projects and research papers have demonstrated the feasibility of of moving device drivers into use space. In particular, this paper: Tolerating Malicious Device Drivers in Linux. http://pdos.csail.mit.edu/papers/sud:usenix10.pdf In this paper, existing device driver code need not to be changed, which should help the idea to be applied in practice. The advantage and disadvantage of move device drivers into use space of both obvious: Advantage: Since most of kernel bugs are caused by device drivers issues, moving device drivers into user space can reduce the impact of device driver bugs. From security perspective, the system can be more secure and robust if most device drivers are working in user space. Disadvantage: At least, existing techniques as well as the above paper showed a relatively high overhead. So is it mainly because the high overhead that prevents the user-space device drivers ideas being accepted in Linux? Regards -Jidong ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-23 4:56 Can we move device drivers into user-space? Jidong Xiao @ 2012-02-23 15:57 ` Cong Wang 2012-02-23 16:34 ` Jidong Xiao 2012-02-24 15:19 ` Jidong Xiao 1 sibling, 1 reply; 68+ messages in thread From: Cong Wang @ 2012-02-23 15:57 UTC (permalink / raw) To: Jidong Xiao; +Cc: Kernel development list On 02/23/2012 12:56 PM, Jidong Xiao wrote: > Hi, Hi, > So is it mainly because the high overhead that prevents the user-space > device drivers ideas being accepted in Linux? > At least UIO drivers are already in Linux kernel, see drivers/uio/. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-23 15:57 ` Cong Wang @ 2012-02-23 16:34 ` Jidong Xiao 2012-02-23 20:48 ` david 0 siblings, 1 reply; 68+ messages in thread From: Jidong Xiao @ 2012-02-23 16:34 UTC (permalink / raw) To: Cong Wang; +Cc: Kernel development list On Thu, Feb 23, 2012 at 10:57 AM, Cong Wang <xiyou.wangcong@gmail.com> wrote: > On 02/23/2012 12:56 PM, Jidong Xiao wrote: >> >> Hi, > > > Hi, > > >> So is it mainly because the high overhead that prevents the user-space >> device drivers ideas being accepted in Linux? >> > > At least UIO drivers are already in Linux kernel, see drivers/uio/. > Oh, so does it make sense to move existing device drivers into user space? For example, move most of the stuff located under drivers/usb into user-space? -Jidong ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-23 16:34 ` Jidong Xiao @ 2012-02-23 20:48 ` david 2012-02-23 21:01 ` Jidong Xiao 2012-02-23 21:18 ` Roland Dreier 0 siblings, 2 replies; 68+ messages in thread From: david @ 2012-02-23 20:48 UTC (permalink / raw) To: Jidong Xiao; +Cc: Cong Wang, Kernel development list On Thu, 23 Feb 2012, Jidong Xiao wrote: >> >> At least UIO drivers are already in Linux kernel, see drivers/uio/. >> > > Oh, so does it make sense to move existing device drivers into user > space? For example, move most of the stuff located under drivers/usb > into user-space? Why would you want to? What advantage are you looking to gain from all the effort? David Lang ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-23 20:48 ` david @ 2012-02-23 21:01 ` Jidong Xiao 2012-02-24 18:21 ` Mauro Carvalho Chehab 2012-02-23 21:18 ` Roland Dreier 1 sibling, 1 reply; 68+ messages in thread From: Jidong Xiao @ 2012-02-23 21:01 UTC (permalink / raw) To: david; +Cc: Cong Wang, Kernel development list On Thu, Feb 23, 2012 at 3:48 PM, <david@lang.hm> wrote: > On Thu, 23 Feb 2012, Jidong Xiao wrote: > >>> >>> At least UIO drivers are already in Linux kernel, see drivers/uio/. >>> >> >> Oh, so does it make sense to move existing device drivers into user >> space? For example, move most of the stuff located under drivers/usb >> into user-space? > > > Why would you want to? What advantage are you looking to gain from all the > effort? > Since device drivers are a significant source of bugs in OS. Moving them to user space can reduce the impact of these bugs. Otherwise, why UIO, or drivers/uio, is accepted by mainstream Linux kernel. -Jidong ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-23 21:01 ` Jidong Xiao @ 2012-02-24 18:21 ` Mauro Carvalho Chehab 2012-02-25 15:10 ` Eduard - Gabriel Munteanu 2012-02-25 15:31 ` Richard Yao 0 siblings, 2 replies; 68+ messages in thread From: Mauro Carvalho Chehab @ 2012-02-24 18:21 UTC (permalink / raw) To: Jidong Xiao; +Cc: david, Cong Wang, Kernel development list Em Thu, 23 Feb 2012 16:01:56 -0500 Jidong Xiao <jidong.xiao@gmail.com> escreveu: > On Thu, Feb 23, 2012 at 3:48 PM, <david@lang.hm> wrote: > > On Thu, 23 Feb 2012, Jidong Xiao wrote: > > > >>> > >>> At least UIO drivers are already in Linux kernel, see drivers/uio/. > >>> > >> > >> Oh, so does it make sense to move existing device drivers into user > >> space? For example, move most of the stuff located under drivers/usb > >> into user-space? > > > > > > Why would you want to? What advantage are you looking to gain from all the > > effort? > > > Since device drivers are a significant source of bugs in OS. Moving > them to user space can reduce the impact of these bugs. Otherwise, why > UIO, or drivers/uio, is accepted by mainstream Linux kernel. Moving a buggy driver to userspace won't fix the bug. You're just moving it from one place to another place. Also, the code will likely require changes to work on userspace, so, the chances are that you're actually introducing more bugs. The impact of the bug won't also reduce, on most cases, as the userspace driver will very likely require root capabilities. Also, as driver talks directly with the hardware, an userspace block driver would have access to the raw disk data. So, even if you find a way for it to run unprivileged, it can still mangle the data written on the disk, and even have a malicious code there that adds or allows to add a malware at the disk partitions. That's said, there are much more eyes inspecting the kernel sources than on any other userspace project. So, the risk of a bad code to be inserted unnoticed at the Linux kernel is degrees of magnitude lower than on an userspace driver. So, I can't see any advantage on doing something like that. > > -Jidong > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- Cheers, Mauro --- Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 18:21 ` Mauro Carvalho Chehab @ 2012-02-25 15:10 ` Eduard - Gabriel Munteanu 2012-02-26 0:06 ` Mauro Carvalho Chehab 2012-02-25 15:31 ` Richard Yao 1 sibling, 1 reply; 68+ messages in thread From: Eduard - Gabriel Munteanu @ 2012-02-25 15:10 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Jidong Xiao, david, Cong Wang, Kernel development list On Fri, Feb 24, 2012 at 04:21:09PM -0200, Mauro Carvalho Chehab wrote: > Moving a buggy driver to userspace won't fix the bug. You're just moving > it from one place to another place. Also, the code will likely require changes > to work on userspace, so, the chances are that you're actually introducing more > bugs. Hi, It does provide isolation, see below. > The impact of the bug won't also reduce, on most cases, as the userspace driver > will very likely require root capabilities. Not as proposed, that's the point. For IOMMU-enabled systems, you can safely delegate an entire device to a userspace device and minimize privileged code. If I understand correctly, the performance impact is also minimal with respect to driver <-> device interaction. I'm not sure if driver <-> client might be problematic, but you can probably have the device DMA directly from/into client memory given the right mechanisms. This is currently employed by virtualization software to do PCI passthrough. The guest OS can directly control the hardware. Sure, you can't do it without proper hardware support. The question is how we can reuse existing code. > Also, as driver talks directly with the hardware, an userspace block driver > would have access to the raw disk data. So, even if you find a way for it to > run unprivileged, it can still mangle the data written on the disk, and > even have a malicious code there that adds or allows to add a malware at the > disk partitions. That's true, but it still makes sense for other drivers, say NIC drivers. Why should corrupting a network driver possibly result in total privilege escalation? > That's said, there are much more eyes inspecting the kernel sources than on any > other userspace project. So, the risk of a bad code to be inserted unnoticed at > the Linux kernel is degrees of magnitude lower than on an userspace driver. Those much more eyes have already missed important bugs in the past. No disrespect here, I'm just saying in many cases (like the one mentioned above) this approach almost eliminates the issue altogether. It's one reason we keep certain userspace out of the kernel. > So, I can't see any advantage on doing something like that. > Another advantage is you can debug and/or profile the driver more easily. Consider a failed takeover attempt that results in a core dump (which also wouldn't result in a complete DoS of the machine). Anyway, I'm not arguing "this is the way it should be done". After all, not all machines are able to handle such a setup. But don't throw the baby out with the water, it's worth considering ways to make things safer. Also, let's not label things like this one as "microkernel" or "academia" and totally reject them; instead consider whether it's practical given recent advancements. Regards, Eduard ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-25 15:10 ` Eduard - Gabriel Munteanu @ 2012-02-26 0:06 ` Mauro Carvalho Chehab 2012-02-26 0:29 ` Richard Yao 2012-02-26 1:58 ` Dr. David Alan Gilbert 0 siblings, 2 replies; 68+ messages in thread From: Mauro Carvalho Chehab @ 2012-02-26 0:06 UTC (permalink / raw) To: Eduard - Gabriel Munteanu Cc: Jidong Xiao, david, Cong Wang, Kernel development list Em 25-02-2012 13:10, Eduard - Gabriel Munteanu escreveu: > On Fri, Feb 24, 2012 at 04:21:09PM -0200, Mauro Carvalho Chehab wrote: >> Moving a buggy driver to userspace won't fix the bug. You're just moving >> it from one place to another place. Also, the code will likely require changes >> to work on userspace, so, the chances are that you're actually introducing more >> bugs. > > Hi, > > > It does provide isolation, see below. It depends on what you call isolation. Moving drivers to userspace will protect ring 0, this but won't prevent OS attacks. See below. > >> The impact of the bug won't also reduce, on most cases, as the userspace driver >> will very likely require root capabilities. > > Not as proposed, that's the point. For IOMMU-enabled systems, you can > safely delegate an entire device to a userspace device and minimize > privileged code. If I understand correctly, the performance impact is > also minimal with respect to driver <-> device interaction. I'm not sure > if driver <-> client might be problematic, but you can probably have the > device DMA directly from/into client memory given the right mechanisms. > > This is currently employed by virtualization software to do PCI > passthrough. The guest OS can directly control the hardware. > > Sure, you can't do it without proper hardware support. The question is > how we can reuse existing code. > >> Also, as driver talks directly with the hardware, an userspace block driver >> would have access to the raw disk data. So, even if you find a way for it to >> run unprivileged, it can still mangle the data written on the disk, and >> even have a malicious code there that adds or allows to add a malware at the >> disk partitions. > > That's true, but it still makes sense for other drivers, say NIC > drivers. Why should corrupting a network driver possibly result in total > privilege escalation? Bad NIC drivers could allow overriding firewall settings, making an external packet to look as if it were a local one. >> That's said, there are much more eyes inspecting the kernel sources than on any >> other userspace project. So, the risk of a bad code to be inserted unnoticed at >> the Linux kernel is degrees of magnitude lower than on an userspace driver. > > Those much more eyes have already missed important bugs in the past. Yes, nobody is perfect. But the probability that something passes on a 4000+ people review is lower than the probability of a bug on a piece of code where just one or two people are looking on it. > No disrespect here, I'm just saying in many cases (like the one mentioned > above) this approach almost eliminates the issue altogether. It doesn't. Security is based on 3 pilars: confidentiality, integrity, availability (CIA). Moving a driver from kernelspace to userspace will likely weak all 3: - it is easier to inspect the data, reducing confidentiality; - it is easier to forge data or to mangle, reducing integrity; - the driver availability will likely weak, as less people will likely be looking into it, and the driver doesn't need to have the generally stronger criteria adopted by the Kernel maintainers to accept it[1]. [1] Ok, if an userspace driver stops, the kernel can still run (this is not different than the great majority of kernel driver OOPSes: only the driver stops). Yet, if the driver is needed for normal operation of that system, having it stopped drops the system availability. Also, it is easier to replace an userspace driver than a kernel one, especially if the distro has some mechanism to taint non-signed drivers, or if module support is disabled. > It's one reason we keep certain userspace out of the kernel. > >> So, I can't see any advantage on doing something like that. >> > > Another advantage is you can debug and/or profile the driver more > easily. This is only partially true. The current debug tools available on the Kernel are not hard to deal with, and are not behind what's available on userspace. > Consider a failed takeover attempt that results in a core dump > (which also wouldn't result in a complete DoS of the machine). > > Anyway, I'm not arguing "this is the way it should be done". After all, > not all machines are able to handle such a setup. But don't throw the > baby out with the water, it's worth considering ways to make things > safer. Also, let's not label things like this one as "microkernel" or > "academia" and totally reject them; instead consider whether it's > practical given recent advancements. > > > Regards, > Eduard > ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 0:06 ` Mauro Carvalho Chehab @ 2012-02-26 0:29 ` Richard Yao 2012-02-27 11:31 ` Mauro Carvalho Chehab 2012-02-26 1:58 ` Dr. David Alan Gilbert 1 sibling, 1 reply; 68+ messages in thread From: Richard Yao @ 2012-02-26 0:29 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Eduard - Gabriel Munteanu, Jidong Xiao, david, Cong Wang, Kernel development list On Sat, Feb 25, 2012 at 7:06 PM, Mauro Carvalho Chehab <mchehab@redhat.com> wrote: > This is only partially true. The current debug tools available on the > Kernel are not hard to deal with, and are not behind what's available > on userspace. Is there documentation available for these tools? I would find that useful. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 0:29 ` Richard Yao @ 2012-02-27 11:31 ` Mauro Carvalho Chehab 0 siblings, 0 replies; 68+ messages in thread From: Mauro Carvalho Chehab @ 2012-02-27 11:31 UTC (permalink / raw) To: Richard Yao Cc: Eduard - Gabriel Munteanu, Jidong Xiao, david, Cong Wang, Kernel development list Em 25-02-2012 22:29, Richard Yao escreveu: > On Sat, Feb 25, 2012 at 7:06 PM, Mauro Carvalho Chehab > <mchehab@redhat.com> wrote: >> This is only partially true. The current debug tools available on the >> Kernel are not hard to deal with, and are not behind what's available >> on userspace. > > Is there documentation available for these tools? I would find that useful. See Documentation/. You'll see there lots of interesting things that could be used to debug a driver, including fixing performance issues on it: kdbg, remote console, perf, dynamic printk's, etc. Mauro. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 0:06 ` Mauro Carvalho Chehab 2012-02-26 0:29 ` Richard Yao @ 2012-02-26 1:58 ` Dr. David Alan Gilbert 2012-02-26 3:34 ` arts zhao 2012-02-27 11:29 ` Mauro Carvalho Chehab 1 sibling, 2 replies; 68+ messages in thread From: Dr. David Alan Gilbert @ 2012-02-26 1:58 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Eduard - Gabriel Munteanu, Jidong Xiao, david, Cong Wang, Kernel development list * Mauro Carvalho Chehab (mchehab@redhat.com) wrote: > Em 25-02-2012 13:10, Eduard - Gabriel Munteanu escreveu: > > On Fri, Feb 24, 2012 at 04:21:09PM -0200, Mauro Carvalho Chehab wrote: > >> Moving a buggy driver to userspace won't fix the bug. You're just moving > >> it from one place to another place. Also, the code will likely require changes > >> to work on userspace, so, the chances are that you're actually introducing more > >> bugs. > > <snip> > >> That's said, there are much more eyes inspecting the kernel sources than on any > >> other userspace project. So, the risk of a bad code to be inserted unnoticed at > >> the Linux kernel is degrees of magnitude lower than on an userspace driver. > > > > Those much more eyes have already missed important bugs in the past. > > Yes, nobody is perfect. But the probability that something passes on a 4000+ people > review is lower than the probability of a bug on a piece of code where just one > or two people are looking on it. That there are 4000+ people reading a driver is a big assumption; for common drivers I'd agree - one problem though is there are a lot of drivers for obscure hardware or old/dead hardware/protocols that frankly near to nobody cares about. Very few people read those drivers; yet sometimes they get built and distributed and someone then finds that since no one has looked at them they're full of holes, and given a malicious USB device for example, you can suddenly create one of these devices that only 3 people have bothered to read the source to - 5 years ago. (The Econet security bug recently would be an example of that). There is a line which says that things that really aren't used just shouldn't be built; but then there are things that are only used by a few people, and then ones only used by a few organisations - and it gets very difficult to say at what point you say just turn it off. Dave -- -----Open up your eyes, open up your mind, open up your code ------- / Dr. David Alan Gilbert | Running GNU/Linux | Happy \ \ gro.gilbert @ treblig.org | | In Hex / \ _________________________|_____ http://www.treblig.org |_______/ ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 1:58 ` Dr. David Alan Gilbert @ 2012-02-26 3:34 ` arts zhao 2012-02-27 11:29 ` Mauro Carvalho Chehab 1 sibling, 0 replies; 68+ messages in thread From: arts zhao @ 2012-02-26 3:34 UTC (permalink / raw) To: Dr. David Alan Gilbert Cc: Mauro Carvalho Chehab, Eduard - Gabriel Munteanu, Jidong Xiao, david, Cong Wang, Kernel development list, arts.linux Basically, I agree with your idea that taking most of device drivers away the kernel scope to user space. Then we can easily debug them and maintain them. However, some device drivers or some layer should be also still existing in kernel scope. such as pci/pcie low-lever driver.... To keep the overhead as lower as possible... thanks -wenke 2012/2/26 Dr. David Alan Gilbert <linux@treblig.org>: > * Mauro Carvalho Chehab (mchehab@redhat.com) wrote: >> Em 25-02-2012 13:10, Eduard - Gabriel Munteanu escreveu: >> > On Fri, Feb 24, 2012 at 04:21:09PM -0200, Mauro Carvalho Chehab wrote: >> >> Moving a buggy driver to userspace won't fix the bug. You're just moving >> >> it from one place to another place. Also, the code will likely require changes >> >> to work on userspace, so, the chances are that you're actually introducing more >> >> bugs. >> > > > <snip> > >> >> That's said, there are much more eyes inspecting the kernel sources than on any >> >> other userspace project. So, the risk of a bad code to be inserted unnoticed at >> >> the Linux kernel is degrees of magnitude lower than on an userspace driver. >> > >> > Those much more eyes have already missed important bugs in the past. >> >> Yes, nobody is perfect. But the probability that something passes on a 4000+ people >> review is lower than the probability of a bug on a piece of code where just one >> or two people are looking on it. > > That there are 4000+ people reading a driver is a big assumption; for common > drivers I'd agree - one problem though is there are a lot of drivers for obscure > hardware or old/dead hardware/protocols that frankly near to nobody cares about. > > Very few people read those drivers; yet sometimes they get built and distributed > and someone then finds that since no one has looked at them they're full of holes, > and given a malicious USB device for example, you can suddenly create one of these > devices that only 3 people have bothered to read the source to - 5 years ago. > (The Econet security bug recently would be an example of that). > > There is a line which says that things that really aren't used > just shouldn't be built; but then there are things that are only used > by a few people, and then ones only used by a few organisations - and > it gets very difficult to say at what point you say just turn it off. > > Dave > -- > -----Open up your eyes, open up your mind, open up your code ------- > / Dr. David Alan Gilbert | Running GNU/Linux | Happy \ > \ gro.gilbert @ treblig.org | | In Hex / > \ _________________________|_____ http://www.treblig.org |_______/ > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 1:58 ` Dr. David Alan Gilbert 2012-02-26 3:34 ` arts zhao @ 2012-02-27 11:29 ` Mauro Carvalho Chehab 1 sibling, 0 replies; 68+ messages in thread From: Mauro Carvalho Chehab @ 2012-02-27 11:29 UTC (permalink / raw) To: Dr. David Alan Gilbert Cc: Eduard - Gabriel Munteanu, Jidong Xiao, david, Cong Wang, Kernel development list Em 25-02-2012 23:58, Dr. David Alan Gilbert escreveu: > * Mauro Carvalho Chehab (mchehab@redhat.com) wrote: >> Em 25-02-2012 13:10, Eduard - Gabriel Munteanu escreveu: >>> On Fri, Feb 24, 2012 at 04:21:09PM -0200, Mauro Carvalho Chehab wrote: >>>> Moving a buggy driver to userspace won't fix the bug. You're just moving >>>> it from one place to another place. Also, the code will likely require changes >>>> to work on userspace, so, the chances are that you're actually introducing more >>>> bugs. >>> > > <snip> > >>>> That's said, there are much more eyes inspecting the kernel sources than on any >>>> other userspace project. So, the risk of a bad code to be inserted unnoticed at >>>> the Linux kernel is degrees of magnitude lower than on an userspace driver. >>> >>> Those much more eyes have already missed important bugs in the past. >> >> Yes, nobody is perfect. But the probability that something passes on a 4000+ people >> review is lower than the probability of a bug on a piece of code where just one >> or two people are looking on it. > > That there are 4000+ people reading a driver is a big assumption; for common > drivers I'd agree - one problem though is there are a lot of drivers for obscure > hardware or old/dead hardware/protocols that frankly near to nobody cares about. Drivers for dead hardware won't offer security risks: those drivers will not load, as such hardware won't be found at the machine. The same applies to old hardware drivers: they won't run on modern machines. Yet, even those old/dead drivers have people looking into it. I receive lots of patches from the janitors team touching those old stuff, fixing potential issues on them. If the userspace drivers become fragmented into hundreds of independent projects, I doubt that most of those projects would have a janitors or a security team. Yet, they can offer security risks. > Very few people read those drivers; yet sometimes they get built and distributed > and someone then finds that since no one has looked at them they're full of holes, > and given a malicious USB device for example, you can suddenly create one of these > devices that only 3 people have bothered to read the source to - 5 years ago. > (The Econet security bug recently would be an example of that). If a malicious person has physical access to the hardware, he can certainly compromise the machine even without an USB device: if he powers off the machine, a DoS is caused; if he stoles the hard disks, confidentiality is compromised; if he boots a live CD, he can insert malicious code there, etc. > There is a line which says that things that really aren't used > just shouldn't be built; but then there are things that are only used > by a few people, and then ones only used by a few organisations - and > it gets very difficult to say at what point you say just turn it off. True. If such organizations have high security requirements, they should be hardening the OS, disabling anything that allows to load a driver at runtime. The consulting services for it I've worked with in the past generally disables not only all unused drivers, but also any other drivers that would allow the usage of external media (USB, CD rom, memory stick drivers) and userspace ones. Regards, Mauro ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 18:21 ` Mauro Carvalho Chehab 2012-02-25 15:10 ` Eduard - Gabriel Munteanu @ 2012-02-25 15:31 ` Richard Yao 1 sibling, 0 replies; 68+ messages in thread From: Richard Yao @ 2012-02-25 15:31 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Jidong Xiao, david, Cong Wang, Kernel development list > The impact of the bug won't also reduce, on most cases, as the userspace driver > will very likely require root capabilities. This is incorrect. root capabilities can be isolated via security frameworks like SELinux. Ring 0 code cannot. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-23 20:48 ` david 2012-02-23 21:01 ` Jidong Xiao @ 2012-02-23 21:18 ` Roland Dreier 1 sibling, 0 replies; 68+ messages in thread From: Roland Dreier @ 2012-02-23 21:18 UTC (permalink / raw) To: david; +Cc: Jidong Xiao, Cong Wang, Kernel development list On Thu, Feb 23, 2012 at 12:48 PM, <david@lang.hm> wrote: > Why would you want to? What advantage are you looking to gain from all the > effort? If someone has a paragraph that starts "Advantage:" in their email, that is usually a good place to look for what they think the advantage of their idea is. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-23 4:56 Can we move device drivers into user-space? Jidong Xiao 2012-02-23 15:57 ` Cong Wang @ 2012-02-24 15:19 ` Jidong Xiao 2012-02-24 15:38 ` Greg KH 2012-02-24 15:58 ` Valdis.Kletnieks 1 sibling, 2 replies; 68+ messages in thread From: Jidong Xiao @ 2012-02-24 15:19 UTC (permalink / raw) To: Kernel development list; +Cc: Greg KH On Wed, Feb 22, 2012 at 11:56 PM, Jidong Xiao <jidong.xiao@gmail.com> wrote: > Hi, > > I am just curious. Since the concept user-space device drivers has > been proposed for several years, and some related projects and > research papers have demonstrated the feasibility of of moving device > drivers into use space. In particular, this paper: > > Tolerating Malicious Device Drivers in Linux. > http://pdos.csail.mit.edu/papers/sud:usenix10.pdf > > In this paper, existing device driver code need not to be changed, > which should help the idea to be applied in practice. > > The advantage and disadvantage of move device drivers into use space > of both obvious: > > Advantage: Since most of kernel bugs are caused by device drivers > issues, moving device drivers into user space can reduce the impact of > device driver bugs. From security perspective, the system can be more > secure and robust if most device drivers are working in user space. > Disadvantage: At least, existing techniques as well as the above paper > showed a relatively high overhead. > > So is it mainly because the high overhead that prevents the user-space > device drivers ideas being accepted in Linux? > Actually, my major concern is, since UIO has been accepted, then why don't we move all the rest device drivers into user space as well. As I understand, currently, some of device drivers are running on user space, while the other (or say the majority of) device drivers are running on kernel space, so why don't we maintain a consistent device drivers infrastructure, say, either all in user space, or all in kernel space. (Sure some critical device drivers still need to be kept in kernel space.) -Jidong ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 15:19 ` Jidong Xiao @ 2012-02-24 15:38 ` Greg KH 2012-02-24 16:38 ` Jidong Xiao 2012-02-24 17:07 ` Guenter Roeck 2012-02-24 15:58 ` Valdis.Kletnieks 1 sibling, 2 replies; 68+ messages in thread From: Greg KH @ 2012-02-24 15:38 UTC (permalink / raw) To: Jidong Xiao; +Cc: Kernel development list On Fri, Feb 24, 2012 at 10:19:36AM -0500, Jidong Xiao wrote: > On Wed, Feb 22, 2012 at 11:56 PM, Jidong Xiao <jidong.xiao@gmail.com> wrote: > > Hi, > > > > I am just curious. Since the concept user-space device drivers has > > been proposed for several years, and some related projects and > > research papers have demonstrated the feasibility of of moving device > > drivers into use space. In particular, this paper: > > > > Tolerating Malicious Device Drivers in Linux. > > http://pdos.csail.mit.edu/papers/sud:usenix10.pdf > > > > In this paper, existing device driver code need not to be changed, > > which should help the idea to be applied in practice. > > > > The advantage and disadvantage of move device drivers into use space > > of both obvious: > > > > Advantage: Since most of kernel bugs are caused by device drivers > > issues, moving device drivers into user space can reduce the impact of > > device driver bugs. From security perspective, the system can be more > > secure and robust if most device drivers are working in user space. > > Disadvantage: At least, existing techniques as well as the above paper > > showed a relatively high overhead. > > > > So is it mainly because the high overhead that prevents the user-space > > device drivers ideas being accepted in Linux? > > > > Actually, my major concern is, since UIO has been accepted, then why > don't we move all the rest device drivers into user space as well. As > I understand, currently, some of device drivers are running on user > space, while the other (or say the majority of) device drivers are > running on kernel space, so why don't we maintain a consistent device > drivers infrastructure, say, either all in user space, or all in > kernel space. (Sure some critical device drivers still need to be kept > in kernel space.) Feel free to create patches to do so, and handle all of the userspace changes needed in order to implement this. I think you haven't thought through the true reason we have device drivers, and why Linux isn't a microkernel... And I'd take exception to your "advantage:" line above, I don't believe that is true at all. Best of luck with your work, greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 15:38 ` Greg KH @ 2012-02-24 16:38 ` Jidong Xiao 2012-02-24 16:54 ` Greg KH 2012-02-27 11:17 ` Bernd Petrovitsch 2012-02-24 17:07 ` Guenter Roeck 1 sibling, 2 replies; 68+ messages in thread From: Jidong Xiao @ 2012-02-24 16:38 UTC (permalink / raw) To: Greg KH; +Cc: Kernel development list On Fri, Feb 24, 2012 at 10:38 AM, Greg KH <greg@kroah.com> wrote: > On Fri, Feb 24, 2012 at 10:19:36AM -0500, Jidong Xiao wrote: >> On Wed, Feb 22, 2012 at 11:56 PM, Jidong Xiao <jidong.xiao@gmail.com> wrote: >> > Hi, >> > >> > I am just curious. Since the concept user-space device drivers has >> > been proposed for several years, and some related projects and >> > research papers have demonstrated the feasibility of of moving device >> > drivers into use space. In particular, this paper: >> > >> > Tolerating Malicious Device Drivers in Linux. >> > http://pdos.csail.mit.edu/papers/sud:usenix10.pdf >> > >> > In this paper, existing device driver code need not to be changed, >> > which should help the idea to be applied in practice. >> > >> > The advantage and disadvantage of move device drivers into use space >> > of both obvious: >> > >> > Advantage: Since most of kernel bugs are caused by device drivers >> > issues, moving device drivers into user space can reduce the impact of >> > device driver bugs. From security perspective, the system can be more >> > secure and robust if most device drivers are working in user space. >> > Disadvantage: At least, existing techniques as well as the above paper >> > showed a relatively high overhead. >> > >> > So is it mainly because the high overhead that prevents the user-space >> > device drivers ideas being accepted in Linux? >> > >> >> Actually, my major concern is, since UIO has been accepted, then why >> don't we move all the rest device drivers into user space as well. As >> I understand, currently, some of device drivers are running on user >> space, while the other (or say the majority of) device drivers are >> running on kernel space, so why don't we maintain a consistent device >> drivers infrastructure, say, either all in user space, or all in >> kernel space. (Sure some critical device drivers still need to be kept >> in kernel space.) > > Feel free to create patches to do so, and handle all of the userspace > changes needed in order to implement this. > > I think you haven't thought through the true reason we have device > drivers, and why Linux isn't a microkernel... > > And I'd take exception to your "advantage:" line above, I don't believe > that is true at all. > > Best of luck with your work, > Although I was asking "can we" do something, I am not actually strongly in favor of either move or not move, as indeed it costs too much to do the moving job. But when you say "handle all of the userspace changes needed in order to implement this", if the device driver can be moved without modification (like the paper above shows), there should not be much changes required for user space applications. Also, if user space device drivers is a bad idea, why drivers/uio has been created and merged into mainline kernel? Regarding "And I'd take exception to your "advantage:" line above, I don't believe that is true at all", do you agree that a significant portion of kernel crash incidents are due to bugs in drivers? As to "Linux isn't a microkernel", even though the debate between Linux and microkernel have never stopped, it's hard to deny the fact that Linux is slowly slowly moving towards a microkernel structure: LKM, fuse, uio, etc. Regards Jidong ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 16:38 ` Jidong Xiao @ 2012-02-24 16:54 ` Greg KH 2012-02-24 17:06 ` Jidong Xiao ` (2 more replies) 2012-02-27 11:17 ` Bernd Petrovitsch 1 sibling, 3 replies; 68+ messages in thread From: Greg KH @ 2012-02-24 16:54 UTC (permalink / raw) To: Jidong Xiao; +Cc: Kernel development list On Fri, Feb 24, 2012 at 11:38:54AM -0500, Jidong Xiao wrote: > On Fri, Feb 24, 2012 at 10:38 AM, Greg KH <greg@kroah.com> wrote: > > On Fri, Feb 24, 2012 at 10:19:36AM -0500, Jidong Xiao wrote: > >> On Wed, Feb 22, 2012 at 11:56 PM, Jidong Xiao <jidong.xiao@gmail.com> wrote: > >> > Hi, > >> > > >> > I am just curious. Since the concept user-space device drivers has > >> > been proposed for several years, and some related projects and > >> > research papers have demonstrated the feasibility of of moving device > >> > drivers into use space. In particular, this paper: > >> > > >> > Tolerating Malicious Device Drivers in Linux. > >> > http://pdos.csail.mit.edu/papers/sud:usenix10.pdf > >> > > >> > In this paper, existing device driver code need not to be changed, > >> > which should help the idea to be applied in practice. Please note, that one of the strengths of Linux is that we CAN change driver code, and we do, which makes implementations like this nice from an academic point of view, but unrealistic from a real-world point of view. > >> Actually, my major concern is, since UIO has been accepted, then why > >> don't we move all the rest device drivers into user space as well. As > >> I understand, currently, some of device drivers are running on user > >> space, while the other (or say the majority of) device drivers are > >> running on kernel space, so why don't we maintain a consistent device > >> drivers infrastructure, say, either all in user space, or all in > >> kernel space. (Sure some critical device drivers still need to be kept > >> in kernel space.) > > > > Feel free to create patches to do so, and handle all of the userspace > > changes needed in order to implement this. > > > > I think you haven't thought through the true reason we have device > > drivers, and why Linux isn't a microkernel... > > > > And I'd take exception to your "advantage:" line above, I don't believe > > that is true at all. > > > > Best of luck with your work, > > > Although I was asking "can we" do something, I am not actually > strongly in favor of either move or not move, as indeed it costs too > much to do the moving job. Then why even discuss this at all? What is your goal here? If it is to have others do work based on an idea you pointed out, you are in the wrong place. > But when you say "handle all of the userspace changes needed in order > to implement this", if the device driver can be moved without > modification (like the paper above shows), there should not be much > changes required for user space applications. The paper shows one such implementation that purports to not need userspace changes. As we have yet to see any code, I remain unconvinced. > Also, if user space device drivers is a bad idea, why drivers/uio has > been created and merged into mainline kernel? UIO fits a real need for some types of devices, why wouldn't it be merged? You are trying to say it is to be used for all drivers, which is totally missing the point. > Regarding "And I'd take exception to your "advantage:" line above, I > don't believe that is true at all", do you agree that a significant > portion of kernel crash incidents are due to bugs in drivers? No I do not. If you refer to the references from the paper where they make that claim, they are talking about a different operating system than Linux. But, by virtue of the fact that the majority of the code running in your kernel is drivers, yes, odds are drivers will have a small majority of the bugs overall, given the percentages involved. However, the bugs-per-line-of-code for Linux drivers is _much_ less than other operating systems, especially given the fact that Linux drivers require much less lines of code overall than other operating systems (30% at the most for the majority of device types.) So I would like to see real numbers backing up your claim before I agree with it. > As to "Linux isn't a microkernel", even though the debate between > Linux and microkernel have never stopped, Um, who is having such a debate? We aren't, so I don't think the debate has ever started. > it's hard to deny the fact that Linux is slowly slowly moving towards > a microkernel structure: LKM, fuse, uio, etc. "LKM", what do you mean by that? FUSE and UIO fit the needs of some specific devices / filesystems, that is why Linux supports them. To claim that their presence indicates that Linux is becoming a microkernel is to show a lack of understanding what a real microkernel is. thanks, greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 16:54 ` Greg KH @ 2012-02-24 17:06 ` Jidong Xiao 2012-02-24 17:13 ` Greg KH 2012-02-24 17:10 ` Al Viro 2012-02-25 19:23 ` Jidong Xiao 2 siblings, 1 reply; 68+ messages in thread From: Jidong Xiao @ 2012-02-24 17:06 UTC (permalink / raw) To: Greg KH; +Cc: Kernel development list On Fri, Feb 24, 2012 at 11:54 AM, Greg KH <gregkh@linuxfoundation.org> wrote: > On Fri, Feb 24, 2012 at 11:38:54AM -0500, Jidong Xiao wrote: >> On Fri, Feb 24, 2012 at 10:38 AM, Greg KH <greg@kroah.com> wrote: >> > On Fri, Feb 24, 2012 at 10:19:36AM -0500, Jidong Xiao wrote: >> >> On Wed, Feb 22, 2012 at 11:56 PM, Jidong Xiao <jidong.xiao@gmail.com> wrote: >> >> > Hi, >> >> > >> >> > I am just curious. Since the concept user-space device drivers has >> >> > been proposed for several years, and some related projects and >> >> > research papers have demonstrated the feasibility of of moving device >> >> > drivers into use space. In particular, this paper: >> >> > >> >> > Tolerating Malicious Device Drivers in Linux. >> >> > http://pdos.csail.mit.edu/papers/sud:usenix10.pdf >> >> > >> >> > In this paper, existing device driver code need not to be changed, >> >> > which should help the idea to be applied in practice. > > Please note, that one of the strengths of Linux is that we CAN change > driver code, and we do, which makes implementations like this nice from > an academic point of view, but unrealistic from a real-world point of > view. > >> >> Actually, my major concern is, since UIO has been accepted, then why >> >> don't we move all the rest device drivers into user space as well. As >> >> I understand, currently, some of device drivers are running on user >> >> space, while the other (or say the majority of) device drivers are >> >> running on kernel space, so why don't we maintain a consistent device >> >> drivers infrastructure, say, either all in user space, or all in >> >> kernel space. (Sure some critical device drivers still need to be kept >> >> in kernel space.) >> > >> > Feel free to create patches to do so, and handle all of the userspace >> > changes needed in order to implement this. >> > >> > I think you haven't thought through the true reason we have device >> > drivers, and why Linux isn't a microkernel... >> > >> > And I'd take exception to your "advantage:" line above, I don't believe >> > that is true at all. >> > >> > Best of luck with your work, >> > >> Although I was asking "can we" do something, I am not actually >> strongly in favor of either move or not move, as indeed it costs too >> much to do the moving job. > > Then why even discuss this at all? What is your goal here? If it is to > have others do work based on an idea you pointed out, you are in the > wrong place. > >> But when you say "handle all of the userspace changes needed in order >> to implement this", if the device driver can be moved without >> modification (like the paper above shows), there should not be much >> changes required for user space applications. > > The paper shows one such implementation that purports to not need > userspace changes. As we have yet to see any code, I remain > unconvinced. > >> Also, if user space device drivers is a bad idea, why drivers/uio has >> been created and merged into mainline kernel? > > UIO fits a real need for some types of devices, why wouldn't it be > merged? You are trying to say it is to be used for all drivers, which > is totally missing the point. > >> Regarding "And I'd take exception to your "advantage:" line above, I >> don't believe that is true at all", do you agree that a significant >> portion of kernel crash incidents are due to bugs in drivers? > > No I do not. If you refer to the references from the paper where they > make that claim, they are talking about a different operating system > than Linux. But, by virtue of the fact that the majority of the code > running in your kernel is drivers, yes, odds are drivers will have a > small majority of the bugs overall, given the percentages involved. > However, the bugs-per-line-of-code for Linux drivers is _much_ less than > other operating systems, especially given the fact that Linux drivers > require much less lines of code overall than other operating systems > (30% at the most for the majority of device types.) So I would like to > see real numbers backing up your claim before I agree with it. > >> As to "Linux isn't a microkernel", even though the debate between >> Linux and microkernel have never stopped, > > Um, who is having such a debate? We aren't, so I don't think the debate > has ever started. > Here is the classic debate between Andrew S. Tanenbaum and Linus Torvalds: http://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_debate Regards Jidong ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 17:06 ` Jidong Xiao @ 2012-02-24 17:13 ` Greg KH 2012-02-24 17:21 ` Jidong Xiao 2012-02-25 2:33 ` Richard Yao 0 siblings, 2 replies; 68+ messages in thread From: Greg KH @ 2012-02-24 17:13 UTC (permalink / raw) To: Jidong Xiao; +Cc: Kernel development list On Fri, Feb 24, 2012 at 12:06:54PM -0500, Jidong Xiao wrote: > On Fri, Feb 24, 2012 at 11:54 AM, Greg KH <gregkh@linuxfoundation.org> wrote: > >> As to "Linux isn't a microkernel", even though the debate between > >> Linux and microkernel have never stopped, > > > > Um, who is having such a debate? We aren't, so I don't think the debate > > has ever started. > > > > Here is the classic debate between Andrew S. Tanenbaum and Linus Torvalds: > http://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_debate That was 20 years ago, to assume that anyone rational was still having that debate today is insane. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 17:13 ` Greg KH @ 2012-02-24 17:21 ` Jidong Xiao 2012-02-24 17:31 ` Greg KH 2012-02-25 2:33 ` Richard Yao 1 sibling, 1 reply; 68+ messages in thread From: Jidong Xiao @ 2012-02-24 17:21 UTC (permalink / raw) To: Greg KH; +Cc: Kernel development list On Fri, Feb 24, 2012 at 12:13 PM, Greg KH <gregkh@linuxfoundation.org> wrote: > On Fri, Feb 24, 2012 at 12:06:54PM -0500, Jidong Xiao wrote: >> On Fri, Feb 24, 2012 at 11:54 AM, Greg KH <gregkh@linuxfoundation.org> wrote: >> >> As to "Linux isn't a microkernel", even though the debate between >> >> Linux and microkernel have never stopped, >> > >> > Um, who is having such a debate? We aren't, so I don't think the debate >> > has ever started. >> > >> >> Here is the classic debate between Andrew S. Tanenbaum and Linus Torvalds: >> http://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_debate > > That was 20 years ago, to assume that anyone rational was still having > that debate today is insane. > You probably missed the second paragraph. "This subject was revisited in 2006 after Tanenbaum wrote a cover story for Computer magazine titled "Can We Make Operating Systems Reliable and Secure?".[3] While Tanenbaum himself has mentioned that he did not write the article to renew the debate on kernel design,[4] the juxtaposition of the article and an archived copy of the 1992 debate on the technology site Slashdot caused the subject to be rekindled.[5] Torvalds posted a rebuttal of Tanenbaum's arguments via an online discussion forum,[6] and several technology news sites began reporting the issue.[7] This prompted Jonathan Shapiro to respond that most of the field-proven reliable and secure computer systems use a more microkernel-like approach.[8]" ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 17:21 ` Jidong Xiao @ 2012-02-24 17:31 ` Greg KH 0 siblings, 0 replies; 68+ messages in thread From: Greg KH @ 2012-02-24 17:31 UTC (permalink / raw) To: Jidong Xiao; +Cc: Kernel development list On Fri, Feb 24, 2012 at 12:21:27PM -0500, Jidong Xiao wrote: > On Fri, Feb 24, 2012 at 12:13 PM, Greg KH <gregkh@linuxfoundation.org> wrote: > > On Fri, Feb 24, 2012 at 12:06:54PM -0500, Jidong Xiao wrote: > >> On Fri, Feb 24, 2012 at 11:54 AM, Greg KH <gregkh@linuxfoundation.org> wrote: > >> >> As to "Linux isn't a microkernel", even though the debate between > >> >> Linux and microkernel have never stopped, > >> > > >> > Um, who is having such a debate? We aren't, so I don't think the debate > >> > has ever started. > >> > > >> > >> Here is the classic debate between Andrew S. Tanenbaum and Linus Torvalds: > >> http://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_debate > > > > That was 20 years ago, to assume that anyone rational was still having > > that debate today is insane. > > > > You probably missed the second paragraph. You assume incorrectly. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 17:13 ` Greg KH 2012-02-24 17:21 ` Jidong Xiao @ 2012-02-25 2:33 ` Richard Yao 2012-02-25 4:28 ` Jidong Xiao 1 sibling, 1 reply; 68+ messages in thread From: Richard Yao @ 2012-02-25 2:33 UTC (permalink / raw) To: Greg KH; +Cc: Jidong Xiao, Kernel development list -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 02/24/12 12:13, Greg KH wrote: > On Fri, Feb 24, 2012 at 12:06:54PM -0500, Jidong Xiao wrote: >> On Fri, Feb 24, 2012 at 11:54 AM, Greg KH <gregkh@linuxfoundation.org> wrote: >>>> As to "Linux isn't a microkernel", even though the debate between >>>> Linux and microkernel have never stopped, >>> >>> Um, who is having such a debate? We aren't, so I don't think the debate >>> has ever started. >>> >> >> Here is the classic debate between Andrew S. Tanenbaum and Linus Torvalds: >> http://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_debate > > That was 20 years ago, to assume that anyone rational was still having > that debate today is insane. Jidong Xiao was correct in saying that there was a debate. It continues much like the Korean war does today. With that said, I suggest that Jidong Xiao switch to MINIX, GNU or his own OS based on one of the L4 kernels (SEL4 maybe?). -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJPSEiMAAoJELFAT5FmjZuE2fgP/jSLP8a1bloTJAuQhZScXaWS 7A9GOYKbtGE6WFaYvzzTNuEON+jK9hQqFAaVpW/GvIJLaQC5agdTzawVi1It/nIk 281FXUrFdZiQtEOArwb3sSxLcUgcPeEWaEG798wqM2Vc2LDAgKEaNh01RasH4di1 v78NRXxCsOfPtszlFbA6u3g/+sjNK2KXKjCPKayN8uzpihvIal+JQx8iunen/TbT 3kENQiT6mA1oP/r3/z19ayL0piKpm4YKZI4UwtGWT0QzwpPD69sVrcCS1ddF58O6 CVl+vsQl/8ptUnLZ115XujXas2y1CioUfkTiKezkGpz/roJoquKwUejamyBc+JHa 0Ph5cJBNg/tfiOcQDeM1GYsolfFVBy5Wvn4uh5R24bBi1nLl/YeAe/0eHY43cdoT mbusYNbBHYr9Bx6B+1Ehy0c+/hd5949kQqWLchi8INYsRUakXqnYQNEt3Jznkpr8 sj4Ga9iayEa3OnwLolLDvBmgLFAF4ZUDBiIFsV6QMrGosvm2DC3Cmk/+ZBwOhZkw lmLor34YJdVQ/0GivwxoGX12HlMQB7IBQiAnQUX5z+a9BOQTcNf4muXM/rs5cDk1 jbztbT9BbTKQVKeQMkZLewct+qEfBHuRA5KimoKL+jxmfZw5UqjBRhtR+U4T+C6A WTJMtPUD4Z08trnWHxRU =H2R7 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-25 2:33 ` Richard Yao @ 2012-02-25 4:28 ` Jidong Xiao 0 siblings, 0 replies; 68+ messages in thread From: Jidong Xiao @ 2012-02-25 4:28 UTC (permalink / raw) To: Richard Yao; +Cc: Greg KH, Kernel development list Thank you everyone! I opened this thread mainly because I was wondering whether it makes sense to move device drivers into user space. I had no clear idea, but I learnt a lot from the whole discussion between everyone, and I think I am not the only one who benefits from this discussion. Regards Jidong On Fri, Feb 24, 2012 at 9:33 PM, Richard Yao <ryao@cs.stonybrook.edu> wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 02/24/12 12:13, Greg KH wrote: >> On Fri, Feb 24, 2012 at 12:06:54PM -0500, Jidong Xiao wrote: >>> On Fri, Feb 24, 2012 at 11:54 AM, Greg KH <gregkh@linuxfoundation.org> > wrote: >>>>> As to "Linux isn't a microkernel", even though the debate between >>>>> Linux and microkernel have never stopped, >>>> >>>> Um, who is having such a debate? We aren't, so I don't think the debate >>>> has ever started. >>>> >>> >>> Here is the classic debate between Andrew S. Tanenbaum and Linus Torvalds: >>> http://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_debate >> >> That was 20 years ago, to assume that anyone rational was still having >> that debate today is insane. > Jidong Xiao was correct in saying that there was a debate. It continues > much like the Korean war does today. > > With that said, I suggest that Jidong Xiao switch to MINIX, GNU or his > own OS based on one of the L4 kernels (SEL4 maybe?). > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.18 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQIcBAEBAgAGBQJPSEiMAAoJELFAT5FmjZuE2fgP/jSLP8a1bloTJAuQhZScXaWS > 7A9GOYKbtGE6WFaYvzzTNuEON+jK9hQqFAaVpW/GvIJLaQC5agdTzawVi1It/nIk > 281FXUrFdZiQtEOArwb3sSxLcUgcPeEWaEG798wqM2Vc2LDAgKEaNh01RasH4di1 > v78NRXxCsOfPtszlFbA6u3g/+sjNK2KXKjCPKayN8uzpihvIal+JQx8iunen/TbT > 3kENQiT6mA1oP/r3/z19ayL0piKpm4YKZI4UwtGWT0QzwpPD69sVrcCS1ddF58O6 > CVl+vsQl/8ptUnLZ115XujXas2y1CioUfkTiKezkGpz/roJoquKwUejamyBc+JHa > 0Ph5cJBNg/tfiOcQDeM1GYsolfFVBy5Wvn4uh5R24bBi1nLl/YeAe/0eHY43cdoT > mbusYNbBHYr9Bx6B+1Ehy0c+/hd5949kQqWLchi8INYsRUakXqnYQNEt3Jznkpr8 > sj4Ga9iayEa3OnwLolLDvBmgLFAF4ZUDBiIFsV6QMrGosvm2DC3Cmk/+ZBwOhZkw > lmLor34YJdVQ/0GivwxoGX12HlMQB7IBQiAnQUX5z+a9BOQTcNf4muXM/rs5cDk1 > jbztbT9BbTKQVKeQMkZLewct+qEfBHuRA5KimoKL+jxmfZw5UqjBRhtR+U4T+C6A > WTJMtPUD4Z08trnWHxRU > =H2R7 > -----END PGP SIGNATURE----- > ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 16:54 ` Greg KH 2012-02-24 17:06 ` Jidong Xiao @ 2012-02-24 17:10 ` Al Viro 2012-02-25 19:23 ` Jidong Xiao 2 siblings, 0 replies; 68+ messages in thread From: Al Viro @ 2012-02-24 17:10 UTC (permalink / raw) To: Greg KH; +Cc: Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 08:54:48AM -0800, Greg KH wrote: > > As to "Linux isn't a microkernel", even though the debate between > > Linux and microkernel have never stopped, > > Um, who is having such a debate? We aren't, so I don't think the debate > has ever started. > > > it's hard to deny the fact that Linux is slowly slowly moving towards > > a microkernel structure: LKM, fuse, uio, etc. > > "LKM", what do you mean by that? At a guess, modules. As to WTF would those possibly have to do with microkernels, your guess is as good as mine... ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 16:54 ` Greg KH 2012-02-24 17:06 ` Jidong Xiao 2012-02-24 17:10 ` Al Viro @ 2012-02-25 19:23 ` Jidong Xiao 2012-02-25 20:55 ` Greg KH 2 siblings, 1 reply; 68+ messages in thread From: Jidong Xiao @ 2012-02-25 19:23 UTC (permalink / raw) To: Greg KH; +Cc: Kernel development list On Fri, Feb 24, 2012 at 11:54 AM, Greg KH <gregkh@linuxfoundation.org> wrote: > On Fri, Feb 24, 2012 at 11:38:54AM -0500, Jidong Xiao wrote: >> On Fri, Feb 24, 2012 at 10:38 AM, Greg KH <greg@kroah.com> wrote: >> > On Fri, Feb 24, 2012 at 10:19:36AM -0500, Jidong Xiao wrote: >> >> On Wed, Feb 22, 2012 at 11:56 PM, Jidong Xiao <jidong.xiao@gmail.com> wrote: >> >> > Hi, >> >> > >> >> > I am just curious. Since the concept user-space device drivers has >> >> > been proposed for several years, and some related projects and >> >> > research papers have demonstrated the feasibility of of moving device >> >> > drivers into use space. In particular, this paper: >> >> > >> >> > Tolerating Malicious Device Drivers in Linux. >> >> > http://pdos.csail.mit.edu/papers/sud:usenix10.pdf >> >> > >> >> > In this paper, existing device driver code need not to be changed, >> >> > which should help the idea to be applied in practice. > > Please note, that one of the strengths of Linux is that we CAN change > driver code, and we do, which makes implementations like this nice from > an academic point of view, but unrealistic from a real-world point of > view. > >> >> Actually, my major concern is, since UIO has been accepted, then why >> >> don't we move all the rest device drivers into user space as well. As >> >> I understand, currently, some of device drivers are running on user >> >> space, while the other (or say the majority of) device drivers are >> >> running on kernel space, so why don't we maintain a consistent device >> >> drivers infrastructure, say, either all in user space, or all in >> >> kernel space. (Sure some critical device drivers still need to be kept >> >> in kernel space.) >> > >> > Feel free to create patches to do so, and handle all of the userspace >> > changes needed in order to implement this. >> > >> > I think you haven't thought through the true reason we have device >> > drivers, and why Linux isn't a microkernel... >> > >> > And I'd take exception to your "advantage:" line above, I don't believe >> > that is true at all. >> > >> > Best of luck with your work, >> > >> Although I was asking "can we" do something, I am not actually >> strongly in favor of either move or not move, as indeed it costs too >> much to do the moving job. > > Then why even discuss this at all? What is your goal here? If it is to > have others do work based on an idea you pointed out, you are in the > wrong place. > >> But when you say "handle all of the userspace changes needed in order >> to implement this", if the device driver can be moved without >> modification (like the paper above shows), there should not be much >> changes required for user space applications. > > The paper shows one such implementation that purports to not need > userspace changes. As we have yet to see any code, I remain > unconvinced. > >> Also, if user space device drivers is a bad idea, why drivers/uio has >> been created and merged into mainline kernel? > > UIO fits a real need for some types of devices, why wouldn't it be > merged? You are trying to say it is to be used for all drivers, which > is totally missing the point. > >> Regarding "And I'd take exception to your "advantage:" line above, I >> don't believe that is true at all", do you agree that a significant >> portion of kernel crash incidents are due to bugs in drivers? > > No I do not. If you refer to the references from the paper where they > make that claim, they are talking about a different operating system > than Linux. But, by virtue of the fact that the majority of the code > running in your kernel is drivers, yes, odds are drivers will have a > small majority of the bugs overall, given the percentages involved. > However, the bugs-per-line-of-code for Linux drivers is _much_ less than > other operating systems, especially given the fact that Linux drivers > require much less lines of code overall than other operating systems > (30% at the most for the majority of device types.) So I would like to > see real numbers backing up your claim before I agree with it. > Hi, Greg, These two studies support my point. If the first one is too old, then the second one should be more convincing. To save your time, you can take a look at their conclusion first. An Empirical Study of Operating Systems Errors http://www.stanford.edu/~engler/metrics-sosp-01.pdf Faults in Linux: Ten Years Later http://pagesperso-systeme.lip6.fr/Suman.Saha/src/asplos11.pdf Regards Jidong ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-25 19:23 ` Jidong Xiao @ 2012-02-25 20:55 ` Greg KH 2012-02-25 23:43 ` Jidong Xiao 2012-02-26 22:46 ` Greg KH 0 siblings, 2 replies; 68+ messages in thread From: Greg KH @ 2012-02-25 20:55 UTC (permalink / raw) To: Jidong Xiao; +Cc: Kernel development list On Sat, Feb 25, 2012 at 02:23:07PM -0500, Jidong Xiao wrote: > Hi, Greg, > > These two studies support my point. If the first one is too old, then > the second one should be more convincing. To save your time, you can > take a look at their conclusion first. > > An Empirical Study of Operating Systems Errors > http://www.stanford.edu/~engler/metrics-sosp-01.pdf > > Faults in Linux: Ten Years Later > http://pagesperso-systeme.lip6.fr/Suman.Saha/src/asplos11.pdf This second paper proves my point, it's funny that you tried to use it to prove yours, you obviously must not have read the conclusion... Anyway, any paper that goes "look at all of these problems in the code!" and isn't instantly followed by patches fixing ALL of those problems by the authors of the paper, should be ignored as a troll masquerading as a "study". greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-25 20:55 ` Greg KH @ 2012-02-25 23:43 ` Jidong Xiao 2012-02-26 17:40 ` Greg KH 2012-02-26 22:46 ` Greg KH 1 sibling, 1 reply; 68+ messages in thread From: Jidong Xiao @ 2012-02-25 23:43 UTC (permalink / raw) To: Greg KH; +Cc: Kernel development list On Sat, Feb 25, 2012 at 3:55 PM, Greg KH <gregkh@linuxfoundation.org> wrote: > On Sat, Feb 25, 2012 at 02:23:07PM -0500, Jidong Xiao wrote: >> Hi, Greg, >> >> These two studies support my point. If the first one is too old, then >> the second one should be more convincing. To save your time, you can >> take a look at their conclusion first. >> >> An Empirical Study of Operating Systems Errors >> http://www.stanford.edu/~engler/metrics-sosp-01.pdf >> >> Faults in Linux: Ten Years Later >> http://pagesperso-systeme.lip6.fr/Suman.Saha/src/asplos11.pdf > > This second paper proves my point, it's funny that you tried to use it > to prove yours, you obviously must not have read the conclusion... > > Anyway, any paper that goes "look at all of these problems in the code!" > and isn't instantly followed by patches fixing ALL of those problems by > the authors of the paper, should be ignored as a troll masquerading as a > "study". > My point was "a significant portion of kernel crash incidents are due to bugs in drivers". You said no. I did *not* say bugs in device drivers are the dominant factor of kernel crashes/faults. So at least my point matches with the conclusion of the second paper. You can certainly say these academic studies are meaningless because they are not telling the whole story, but you can not deny the fact that because of the large code base, it is the almost impossible to eliminate all the bugs/problems from device drivers. That's why people are doing research to mitigate this problem, even though moving device drivers to user space may not be a good idea, or it is unrealistic in Linux, those researchers as well as their results deserve more respect. Everyone in the whole community, including kernel developers and researchers, shares the same goal, namely, improving the kernel code quality. Regards Jidong ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-25 23:43 ` Jidong Xiao @ 2012-02-26 17:40 ` Greg KH 0 siblings, 0 replies; 68+ messages in thread From: Greg KH @ 2012-02-26 17:40 UTC (permalink / raw) To: Jidong Xiao; +Cc: Kernel development list On Sat, Feb 25, 2012 at 06:43:58PM -0500, Jidong Xiao wrote: > On Sat, Feb 25, 2012 at 3:55 PM, Greg KH <gregkh@linuxfoundation.org> wrote: > > On Sat, Feb 25, 2012 at 02:23:07PM -0500, Jidong Xiao wrote: > >> Hi, Greg, > >> > >> These two studies support my point. If the first one is too old, then > >> the second one should be more convincing. To save your time, you can > >> take a look at their conclusion first. > >> > >> An Empirical Study of Operating Systems Errors > >> http://www.stanford.edu/~engler/metrics-sosp-01.pdf > >> > >> Faults in Linux: Ten Years Later > >> http://pagesperso-systeme.lip6.fr/Suman.Saha/src/asplos11.pdf > > > > This second paper proves my point, it's funny that you tried to use it > > to prove yours, you obviously must not have read the conclusion... > > > > Anyway, any paper that goes "look at all of these problems in the code!" > > and isn't instantly followed by patches fixing ALL of those problems by > > the authors of the paper, should be ignored as a troll masquerading as a > > "study". > > > My point was "a significant portion of kernel crash incidents are due > to bugs in drivers". You said no. I did *not* say bugs in device > drivers are the dominant factor of kernel crashes/faults. So at least > my point matches with the conclusion of the second paper. You can > certainly say these academic studies are meaningless because they are > not telling the whole story, but you can not deny the fact that > because of the large code base, it is the almost impossible to > eliminate all the bugs/problems from device drivers. That's why people > are doing research to mitigate this problem, even though moving device > drivers to user space may not be a good idea, or it is unrealistic in > Linux, those researchers as well as their results deserve more > respect. Everyone in the whole community, including kernel developers > and researchers, shares the same goal, namely, improving the kernel > code quality. I don't see much help from the majority of researchers toward actually helping change Linux for the better, their goals usually are just publishing a paper and then moving on. Yes, some stick around, and some people are working on making the intersection between researchers and real-life more condusive toward helping us out, but that is a hard task given that the goals of the two groups are usually totally different in the end. I suggest, if you have the time, to try to work with the researchers to resolve the problems that they find. They said that they found hundreds of problems. Why aren't those problems all now fixed? What kept them from getting fixed after they found them? Was it something that the community ignored, or did they not even try? Find out the root cause of that, and then, for their next paper, they might have something totally different to report... good luck, greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-25 20:55 ` Greg KH 2012-02-25 23:43 ` Jidong Xiao @ 2012-02-26 22:46 ` Greg KH 1 sibling, 0 replies; 68+ messages in thread From: Greg KH @ 2012-02-26 22:46 UTC (permalink / raw) To: Jidong Xiao; +Cc: Kernel development list On Sat, Feb 25, 2012 at 12:55:40PM -0800, Greg KH wrote: > On Sat, Feb 25, 2012 at 02:23:07PM -0500, Jidong Xiao wrote: > > Hi, Greg, > > > > These two studies support my point. If the first one is too old, then > > the second one should be more convincing. To save your time, you can > > take a look at their conclusion first. > > > > An Empirical Study of Operating Systems Errors > > http://www.stanford.edu/~engler/metrics-sosp-01.pdf > > > > Faults in Linux: Ten Years Later > > http://pagesperso-systeme.lip6.fr/Suman.Saha/src/asplos11.pdf > > This second paper proves my point, it's funny that you tried to use it > to prove yours, you obviously must not have read the conclusion... > > Anyway, any paper that goes "look at all of these problems in the code!" > and isn't instantly followed by patches fixing ALL of those problems by > the authors of the paper, should be ignored as a troll masquerading as a > "study". Oops, it was pointed out to me that one of the authors of that paper really is contributing patches, my appologies to them. Julia has probably fixed more bugs than almost anyone else in the kernel community. I did not mean to make light of her contributions at all, again my mistake. greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 16:38 ` Jidong Xiao 2012-02-24 16:54 ` Greg KH @ 2012-02-27 11:17 ` Bernd Petrovitsch 1 sibling, 0 replies; 68+ messages in thread From: Bernd Petrovitsch @ 2012-02-27 11:17 UTC (permalink / raw) To: Jidong Xiao; +Cc: Greg KH, Kernel development list On Fre, 2012-02-24 at 11:38 -0500, Jidong Xiao wrote: [...] > Although I was asking "can we" do something, I am not actually > strongly in favor of either move or not move, as indeed it costs too > much to do the moving job. You obviously missed a crucial point in the little corner "LKML" of the universe: You are (automatically, implicitly) expected to start the work and produce patches and keep on working on it and actually *do* the job. If only to show that you are serious, that the "ideas" are actually useful and you are not just trolling around. Possibly people will help you though. Quoting some studies is nice for reference (implying that they are relevant and applicable which should be checked before blindly believing in it) but most here have read too many "studies" to be impressed any number of them. That's usually worded in (and meant with) "show patches" or "try it out" or "talk is cheap, show me the code". Bernd -- Bernd Petrovitsch Email : bernd@petrovitsch.priv.at LUGA : http://www.luga.at ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 15:38 ` Greg KH 2012-02-24 16:38 ` Jidong Xiao @ 2012-02-24 17:07 ` Guenter Roeck 2012-02-24 17:17 ` Greg KH 1 sibling, 1 reply; 68+ messages in thread From: Guenter Roeck @ 2012-02-24 17:07 UTC (permalink / raw) To: Greg KH; +Cc: Jidong Xiao, Kernel development list On Fri, 2012-02-24 at 10:38 -0500, Greg KH wrote: > On Fri, Feb 24, 2012 at 10:19:36AM -0500, Jidong Xiao wrote: > > On Wed, Feb 22, 2012 at 11:56 PM, Jidong Xiao <jidong.xiao@gmail.com> wrote: > > > Hi, > > > > > > I am just curious. Since the concept user-space device drivers has > > > been proposed for several years, and some related projects and > > > research papers have demonstrated the feasibility of of moving device > > > drivers into use space. In particular, this paper: > > > > > > Tolerating Malicious Device Drivers in Linux. > > > http://pdos.csail.mit.edu/papers/sud:usenix10.pdf > > > > > > In this paper, existing device driver code need not to be changed, > > > which should help the idea to be applied in practice. > > > > > > The advantage and disadvantage of move device drivers into use space > > > of both obvious: > > > > > > Advantage: Since most of kernel bugs are caused by device drivers > > > issues, moving device drivers into user space can reduce the impact of > > > device driver bugs. From security perspective, the system can be more > > > secure and robust if most device drivers are working in user space. > > > Disadvantage: At least, existing techniques as well as the above paper > > > showed a relatively high overhead. > > > > > > So is it mainly because the high overhead that prevents the user-space > > > device drivers ideas being accepted in Linux? > > > > > > > Actually, my major concern is, since UIO has been accepted, then why > > don't we move all the rest device drivers into user space as well. As > > I understand, currently, some of device drivers are running on user > > space, while the other (or say the majority of) device drivers are > > running on kernel space, so why don't we maintain a consistent device > > drivers infrastructure, say, either all in user space, or all in > > kernel space. (Sure some critical device drivers still need to be kept > > in kernel space.) > > Feel free to create patches to do so, and handle all of the userspace > changes needed in order to implement this. > > I think you haven't thought through the true reason we have device > drivers, and why Linux isn't a microkernel... > > And I'd take exception to your "advantage:" line above, I don't believe > that is true at all. I second that. Worse, the real disadvantages are ignored. Crappy user-space code, code is kept proprietary, code is no longer submitted upstream, people don't care about implementing interrupts and instead implement polling loops, porting to later kernel versions is a pain. The list goes on and on. This is all more than annoying. I would prefer another approach: Fix the problematic drivers, and spend time researching how to improve forced module unload to a point where modules can be re-loaded after an OOPS. How about dropping UIO support from the kernel ? That would make more sense to me. Guenter ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 17:07 ` Guenter Roeck @ 2012-02-24 17:17 ` Greg KH 2012-02-24 17:47 ` Guenter Roeck 0 siblings, 1 reply; 68+ messages in thread From: Greg KH @ 2012-02-24 17:17 UTC (permalink / raw) To: Guenter Roeck; +Cc: Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 09:07:09AM -0800, Guenter Roeck wrote: > How about dropping UIO support from the kernel ? That would make more > sense to me. Again, UIO solves a real need, are you to tell the users of that code that somehow we are now not going to support them anymore? UIO was created when Thomas and I sat in the back of a conference presentation and saw, for the umpteenth time, a presentation by someone who was trying to write userspace drivers, and obviously didn't know what they were doing. UIO provides a framework that actually works (unlike all of the previous research papers were trying to do), and is used in real systems (laser welding robots!) every day, manufacturing things that you use and rely on. You remove UIO at the risk of pissing off those robots, the choice is yours, I know I'm not going to do it... greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 17:17 ` Greg KH @ 2012-02-24 17:47 ` Guenter Roeck 2012-02-24 18:34 ` Greg KH 2012-03-01 9:54 ` Thomas Gleixner 0 siblings, 2 replies; 68+ messages in thread From: Guenter Roeck @ 2012-02-24 17:47 UTC (permalink / raw) To: Greg KH; +Cc: Jidong Xiao, Kernel development list On Fri, 2012-02-24 at 12:17 -0500, Greg KH wrote: > On Fri, Feb 24, 2012 at 09:07:09AM -0800, Guenter Roeck wrote: > > How about dropping UIO support from the kernel ? That would make more > > sense to me. > > Again, UIO solves a real need, are you to tell the users of that code > that somehow we are now not going to support them anymore? > > UIO was created when Thomas and I sat in the back of a conference > presentation and saw, for the umpteenth time, a presentation by someone > who was trying to write userspace drivers, and obviously didn't know > what they were doing. > > UIO provides a framework that actually works (unlike all of the previous > research papers were trying to do), and is used in real systems (laser > welding robots!) every day, manufacturing things that you use and rely > on. > > You remove UIO at the risk of pissing off those robots, the choice is > yours, I know I'm not going to do it... I understand the background and reasoning, but ... I have seen UIO used for networking drivers, hwmon drivers, I2C bus master drivers (with matching I2C client drivers in user-space), mfd devices, and so on. I have seen existing kernel drivers re-implemented as UIO drivers. I have seen UIO drivers where the kernel part of the driver is larger than the entire driver written as kernel driver. I have seen UIO drivers using polling instead of interrupts "because it is faster than interrupts". Often, those drivers are then re-written for the next board (to support the same chip) because they were not written with HW-independence in mind and don't support HW abstraction. Yes, there may be real need for UIO in some cases, but all I have seen it used for so far is what I would call abuse, resulting in maintenance nightmares. Given the choice, I would be quite happy to piss off some robots. Call it a prejudice if you like ;). Guenter ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 17:47 ` Guenter Roeck @ 2012-02-24 18:34 ` Greg KH 2012-02-24 19:15 ` Henrik Rydberg 2012-03-01 9:54 ` Thomas Gleixner 1 sibling, 1 reply; 68+ messages in thread From: Greg KH @ 2012-02-24 18:34 UTC (permalink / raw) To: Guenter Roeck; +Cc: Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 09:47:48AM -0800, Guenter Roeck wrote: > > You remove UIO at the risk of pissing off those robots, the choice is > > yours, I know I'm not going to do it... > > I understand the background and reasoning, but ... > > I have seen UIO used for networking drivers, hwmon drivers, I2C bus > master drivers (with matching I2C client drivers in user-space), mfd > devices, and so on. And all of those uses are wrong. Go kick those developers and companies today. Hard. > I have seen existing kernel drivers re-implemented as UIO drivers. Where? By whom? > I have seen UIO drivers where the kernel part of the > driver is larger than the entire driver written as kernel driver. I have > seen UIO drivers using polling instead of interrupts "because it is > faster than interrupts". We can't prevent people from being stupid, can we? > Often, those drivers are then re-written for the next board (to support > the same chip) because they were not written with HW-independence in > mind and don't support HW abstraction. > > Yes, there may be real need for UIO in some cases, but all I have seen > it used for so far is what I would call abuse, resulting in maintenance > nightmares. Again, that's using the interface wrong, and foolishly. Don't do that. > Given the choice, I would be quite happy to piss off some robots. Call > it a prejudice if you like ;). You can't control how people use an interface. You will note that there are no stupid users of the UIO interface in the kernel today, we can't control what people do in the privacy of their own companies/homes, despite numerous governments trying to do so in the past. So yes, people will always do stupid, foolish things. And they were doing them before UIO came along, now they just have the chance to at least do those foolish things in a way that interfaces with the kernel in a semi-sane manner, not messing anything else in the kernel up. sorry, greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 18:34 ` Greg KH @ 2012-02-24 19:15 ` Henrik Rydberg 2012-02-24 19:26 ` Greg KH 0 siblings, 1 reply; 68+ messages in thread From: Henrik Rydberg @ 2012-02-24 19:15 UTC (permalink / raw) To: Greg KH; +Cc: Guenter Roeck, Jidong Xiao, Kernel development list > > Given the choice, I would be quite happy to piss off some robots. Call > > it a prejudice if you like ;). > > You can't control how people use an interface. You will note that there > are no stupid users of the UIO interface in the kernel today, we can't > control what people do in the privacy of their own companies/homes, > despite numerous governments trying to do so in the past. > > So yes, people will always do stupid, foolish things. And they were > doing them before UIO came along, now they just have the chance to at > least do those foolish things in a way that interfaces with the kernel > in a semi-sane manner, not messing anything else in the kernel up. So the question is; can the uio example be repeated in other areas, to bring more kernel power to userspace? No doubt, the kernel has fostered an unprecedented code-quality view, which, IMHO, is the bottom line. As long as that quality remains unchallenged, having all drivers in-kernel will remain the best choice. But, suppose some entity managed to keep that quality in userspace, would there be any reason to think of it as a bad idea? I think not. Henrik ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 19:15 ` Henrik Rydberg @ 2012-02-24 19:26 ` Greg KH 2012-02-24 20:10 ` Henrik Rydberg 0 siblings, 1 reply; 68+ messages in thread From: Greg KH @ 2012-02-24 19:26 UTC (permalink / raw) To: Henrik Rydberg; +Cc: Guenter Roeck, Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 08:15:35PM +0100, Henrik Rydberg wrote: > > > Given the choice, I would be quite happy to piss off some robots. Call > > > it a prejudice if you like ;). > > > > You can't control how people use an interface. You will note that there > > are no stupid users of the UIO interface in the kernel today, we can't > > control what people do in the privacy of their own companies/homes, > > despite numerous governments trying to do so in the past. > > > > So yes, people will always do stupid, foolish things. And they were > > doing them before UIO came along, now they just have the chance to at > > least do those foolish things in a way that interfaces with the kernel > > in a semi-sane manner, not messing anything else in the kernel up. > > So the question is; can the uio example be repeated in other areas, to > bring more kernel power to userspace? What exactly do you mean by "more kernel power"? You can write userspace char drivers, filesystems, usb drivers, usb gadget drivers, and lots of other things today with the interfaces we provide from the kernel. And even better yet, please show what you mean with patches. greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 19:26 ` Greg KH @ 2012-02-24 20:10 ` Henrik Rydberg 2012-02-24 20:16 ` Greg KH 0 siblings, 1 reply; 68+ messages in thread From: Henrik Rydberg @ 2012-02-24 20:10 UTC (permalink / raw) To: Greg KH; +Cc: Guenter Roeck, Jidong Xiao, Kernel development list > > > So yes, people will always do stupid, foolish things. And they were > > > doing them before UIO came along, now they just have the chance to at > > > least do those foolish things in a way that interfaces with the kernel > > > in a semi-sane manner, not messing anything else in the kernel up. > > > > So the question is; can the uio example be repeated in other areas, to > > bring more kernel power to userspace? > > What exactly do you mean by "more kernel power"? You can write > userspace char drivers, filesystems, usb drivers, usb gadget drivers, > and lots of other things today with the interfaces we provide from the > kernel. This is all good, but the thread was started for some generic reason not covered by those examples. The uio interface was pointed out because it brings pci(e) to userland, which was not (readily) accessible before. However, every driver that cannot be implemented in userspace is another example. I am not complaining about the kernel and its driver structure - on the contrary. I do, however, see a reason why constructing lower-level interfaces to userspace may be of benefit. The kernel is growing tremendously fast. Sooner or later, parts of the present driver responsibility will have to be split into smaller chunks. Why not place those chunks outside the kernel itself? > And even better yet, please show what you mean with patches. Sure. This particular thread seems to transcend patches, though. It does not hurt to vent public opinion every now and then. Henrik ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 20:10 ` Henrik Rydberg @ 2012-02-24 20:16 ` Greg KH 2012-02-24 20:37 ` Henrik Rydberg 0 siblings, 1 reply; 68+ messages in thread From: Greg KH @ 2012-02-24 20:16 UTC (permalink / raw) To: Henrik Rydberg; +Cc: Guenter Roeck, Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 09:10:27PM +0100, Henrik Rydberg wrote: > I am not complaining about the kernel and its driver structure - on > the contrary. I do, however, see a reason why constructing lower-level > interfaces to userspace may be of benefit. The kernel is growing > tremendously fast. Sooner or later, parts of the present driver > responsibility will have to be split into smaller chunks. Why not > place those chunks outside the kernel itself? I fail to understand why you think the growth of the kernel will require drivers to be split into chunks in the future. Are we somehow growing faster than is required / needed / wanted and outstripping the size of what we are allowed to take up on machines? greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 20:16 ` Greg KH @ 2012-02-24 20:37 ` Henrik Rydberg 2012-02-24 20:56 ` Greg KH 0 siblings, 1 reply; 68+ messages in thread From: Henrik Rydberg @ 2012-02-24 20:37 UTC (permalink / raw) To: Greg KH; +Cc: Guenter Roeck, Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 12:16:55PM -0800, Greg KH wrote: > On Fri, Feb 24, 2012 at 09:10:27PM +0100, Henrik Rydberg wrote: > > I am not complaining about the kernel and its driver structure - on > > the contrary. I do, however, see a reason why constructing lower-level > > interfaces to userspace may be of benefit. The kernel is growing > > tremendously fast. Sooner or later, parts of the present driver > > responsibility will have to be split into smaller chunks. Why not > > place those chunks outside the kernel itself? > > I fail to understand why you think the growth of the kernel will require > drivers to be split into chunks in the future. Are we somehow growing > faster than is required / needed / wanted and outstripping the size of > what we are allowed to take up on machines? How long can one grow and still claim to _be_ a kernel? The current (arbitrary) split between kernel and userland was made a long time ago. For a number of reasons, of course, but not least out of convenience. Projecting the current kernel growth ten years into the future, will there still not be another (arbitrary) convenient split? Even so, the main point was not whether a secondary split may or may not happen, but what will happen to the quality of the total system when/if that happens. I doubt the fate of userland drivers would be so heatedly discussed if everyone believed that work would be carried out with the same quality currently found in the kernel. Henrik ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 20:37 ` Henrik Rydberg @ 2012-02-24 20:56 ` Greg KH 2012-02-24 21:22 ` Henrik Rydberg 0 siblings, 1 reply; 68+ messages in thread From: Greg KH @ 2012-02-24 20:56 UTC (permalink / raw) To: Henrik Rydberg; +Cc: Guenter Roeck, Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 09:37:15PM +0100, Henrik Rydberg wrote: > On Fri, Feb 24, 2012 at 12:16:55PM -0800, Greg KH wrote: > > On Fri, Feb 24, 2012 at 09:10:27PM +0100, Henrik Rydberg wrote: > > > I am not complaining about the kernel and its driver structure - on > > > the contrary. I do, however, see a reason why constructing lower-level > > > interfaces to userspace may be of benefit. The kernel is growing > > > tremendously fast. Sooner or later, parts of the present driver > > > responsibility will have to be split into smaller chunks. Why not > > > place those chunks outside the kernel itself? > > > > I fail to understand why you think the growth of the kernel will require > > drivers to be split into chunks in the future. Are we somehow growing > > faster than is required / needed / wanted and outstripping the size of > > what we are allowed to take up on machines? > > How long can one grow and still claim to _be_ a kernel? You tell me. I know of lots of "kernels" that are far bigger than Linux, and people still consider them a kernel. > The current > (arbitrary) split between kernel and userland was made a long time > ago. For a number of reasons, of course, but not least out of > convenience. Projecting the current kernel growth ten years into the > future, will there still not be another (arbitrary) convenient split? Why? What problems are we having with our existing user/kernel split that would drive such a change? > Even so, the main point was not whether a secondary split may or may > not happen, but what will happen to the quality of the total system > when/if that happens. I doubt the fate of userland drivers would be so > heatedly discussed if everyone believed that work would be carried out > with the same quality currently found in the kernel. Quality is not the only reason we don't want userspace drivers for some types of devices. Realize what drivers provide to userspace today and you will understand that. thanks, greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 20:56 ` Greg KH @ 2012-02-24 21:22 ` Henrik Rydberg 2012-02-24 21:30 ` Ted Ts'o 0 siblings, 1 reply; 68+ messages in thread From: Henrik Rydberg @ 2012-02-24 21:22 UTC (permalink / raw) To: Greg KH; +Cc: Guenter Roeck, Jidong Xiao, Kernel development list > > How long can one grow and still claim to _be_ a kernel? > > You tell me. I know of lots of "kernels" that are far bigger than > Linux, and people still consider them a kernel. I would love to be enligthened. > > The current > > (arbitrary) split between kernel and userland was made a long time > > ago. For a number of reasons, of course, but not least out of > > convenience. Projecting the current kernel growth ten years into the > > future, will there still not be another (arbitrary) convenient split? > > Why? What problems are we having with our existing user/kernel split > that would drive such a change? Maintenance. Sure, as soon as an area grows too large for a single person, the current structure will ensure it divides so that the patch stream becomes manageable. We have already seen ample examples of that. But the overall structure of the kernel will become less and less manageable, and the likelihood of duplicates and maintenance problems will increase. At some point, intra-system interfaces will be introduced, such that several areas may coexist and expand individually. At that point, the convenience split will already have been made, and the subsystems may well reside in different places. > > Even so, the main point was not whether a secondary split may or may > > not happen, but what will happen to the quality of the total system > > when/if that happens. I doubt the fate of userland drivers would be so > > heatedly discussed if everyone believed that work would be carried out > > with the same quality currently found in the kernel. > > Quality is not the only reason we don't want userspace drivers for some > types of devices. Realize what drivers provide to userspace today and > you will understand that. I think I do understand how software interfaces and streamlining works, so I fail to get the point here. Thanks, Henrik ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 21:22 ` Henrik Rydberg @ 2012-02-24 21:30 ` Ted Ts'o 2012-02-24 22:14 ` Henrik Rydberg 0 siblings, 1 reply; 68+ messages in thread From: Ted Ts'o @ 2012-02-24 21:30 UTC (permalink / raw) To: Henrik Rydberg Cc: Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 10:22:38PM +0100, Henrik Rydberg wrote: > > Maintenance. Sure, as soon as an area grows too large for a single > person, the current structure will ensure it divides so that the patch > stream becomes manageable. We have already seen ample examples of > that. But the overall structure of the kernel will become less and > less manageable, and the likelihood of duplicates and maintenance > problems will increase. Not necessarily. The primary area of growth has been hardware support (i.e., drivers), and that scales quite well. The main issue is the grown of intra-system interfaces, and you haven't shown that this has happened in anything approaching the levels where we need to worry. If we add a new file system, like btrfs, that doesn't add a whole new set of VFS interfaces, and it's not like we need to worry about communication interfaces between btrfs and ext4.... - Ted ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 21:30 ` Ted Ts'o @ 2012-02-24 22:14 ` Henrik Rydberg 2012-02-24 22:20 ` Greg KH ` (2 more replies) 0 siblings, 3 replies; 68+ messages in thread From: Henrik Rydberg @ 2012-02-24 22:14 UTC (permalink / raw) To: Ted Ts'o, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 04:30:27PM -0500, Ted Ts'o wrote: > On Fri, Feb 24, 2012 at 10:22:38PM +0100, Henrik Rydberg wrote: > > > > Maintenance. Sure, as soon as an area grows too large for a single > > person, the current structure will ensure it divides so that the patch > > stream becomes manageable. We have already seen ample examples of > > that. But the overall structure of the kernel will become less and > > less manageable, and the likelihood of duplicates and maintenance > > problems will increase. > > Not necessarily. The primary area of growth has been hardware support > (i.e., drivers), and that scales quite well. > > The main issue is the grown of intra-system interfaces, and you > haven't shown that this has happened in anything approaching the > levels where we need to worry. If we add a new file system, like > btrfs, that doesn't add a whole new set of VFS interfaces, and it's > not like we need to worry about communication interfaces between btrfs > and ext4.... True, and the emergent interface (vfs) is even accessible in userspace, so all is good in that department for now. Conversely, you have not shown that this can go on forever. ;-) The main issue that set me off has been sufficiently diluted in the (selective) discussion so as to no longer make sense as a reply: At some point, in-tree or out-of-tree will no longer be distinguishable, and the question is if/when/why something would/should happen as a consequence. A matter of politics, if you will. It seemed like an honest question starting this thread, and I think it is fair to keep it in mind as such. Thanks. Henrik ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 22:14 ` Henrik Rydberg @ 2012-02-24 22:20 ` Greg KH 2012-02-24 22:49 ` Henrik Rydberg 2012-02-25 12:15 ` Theodore Tso 2012-02-26 4:56 ` Bobby Powers 2 siblings, 1 reply; 68+ messages in thread From: Greg KH @ 2012-02-24 22:20 UTC (permalink / raw) To: Henrik Rydberg Cc: Ted Ts'o, Guenter Roeck, Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 11:14:59PM +0100, Henrik Rydberg wrote: > The main issue that set me off has been sufficiently diluted in the > (selective) discussion so as to no longer make sense as a reply: At > some point, in-tree or out-of-tree will no longer be distinguishable, > and the question is if/when/why something would/should happen as a > consequence. A matter of politics, if you will. It seemed like an > honest question starting this thread, and I think it is fair to keep > it in mind as such. If something happens to require such a change in the future, we will deal with it, at that time. Remember the rule, "Linux is evolution, not intelligent design." greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 22:20 ` Greg KH @ 2012-02-24 22:49 ` Henrik Rydberg 2012-02-24 22:54 ` Greg KH 0 siblings, 1 reply; 68+ messages in thread From: Henrik Rydberg @ 2012-02-24 22:49 UTC (permalink / raw) To: Greg KH; +Cc: Ted Ts'o, Guenter Roeck, Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 02:20:34PM -0800, Greg KH wrote: > On Fri, Feb 24, 2012 at 11:14:59PM +0100, Henrik Rydberg wrote: > > The main issue that set me off has been sufficiently diluted in the > > (selective) discussion so as to no longer make sense as a reply: At > > some point, in-tree or out-of-tree will no longer be distinguishable, > > and the question is if/when/why something would/should happen as a > > consequence. A matter of politics, if you will. It seemed like an > > honest question starting this thread, and I think it is fair to keep > > it in mind as such. > > If something happens to require such a change in the future, we will > deal with it, at that time. Remember the rule, "Linux is evolution, not > intelligent design." Who "we" is is exactly the point. For the not-so-important record, evolution does actually create intelligent designs. Thanks, Henrik ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 22:49 ` Henrik Rydberg @ 2012-02-24 22:54 ` Greg KH 2012-02-24 23:14 ` Henrik Rydberg 0 siblings, 1 reply; 68+ messages in thread From: Greg KH @ 2012-02-24 22:54 UTC (permalink / raw) To: Henrik Rydberg Cc: Ted Ts'o, Guenter Roeck, Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 11:49:30PM +0100, Henrik Rydberg wrote: > On Fri, Feb 24, 2012 at 02:20:34PM -0800, Greg KH wrote: > > On Fri, Feb 24, 2012 at 11:14:59PM +0100, Henrik Rydberg wrote: > > > The main issue that set me off has been sufficiently diluted in the > > > (selective) discussion so as to no longer make sense as a reply: At > > > some point, in-tree or out-of-tree will no longer be distinguishable, > > > and the question is if/when/why something would/should happen as a > > > consequence. A matter of politics, if you will. It seemed like an > > > honest question starting this thread, and I think it is fair to keep > > > it in mind as such. > > > > If something happens to require such a change in the future, we will > > deal with it, at that time. Remember the rule, "Linux is evolution, not > > intelligent design." > > Who "we" is is exactly the point. The kernel development community will handle it. > For the not-so-important record, evolution does actually create > intelligent designs. For some things yes, for others (like your tailbone), perhaps not :) greg k-h ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 22:54 ` Greg KH @ 2012-02-24 23:14 ` Henrik Rydberg 0 siblings, 0 replies; 68+ messages in thread From: Henrik Rydberg @ 2012-02-24 23:14 UTC (permalink / raw) To: Greg KH; +Cc: Ted Ts'o, Guenter Roeck, Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 02:54:37PM -0800, Greg KH wrote: > On Fri, Feb 24, 2012 at 11:49:30PM +0100, Henrik Rydberg wrote: > > On Fri, Feb 24, 2012 at 02:20:34PM -0800, Greg KH wrote: > > > On Fri, Feb 24, 2012 at 11:14:59PM +0100, Henrik Rydberg wrote: > > > > The main issue that set me off has been sufficiently diluted in the > > > > (selective) discussion so as to no longer make sense as a reply: At > > > > some point, in-tree or out-of-tree will no longer be distinguishable, > > > > and the question is if/when/why something would/should happen as a > > > > consequence. A matter of politics, if you will. It seemed like an > > > > honest question starting this thread, and I think it is fair to keep > > > > it in mind as such. > > > > > > If something happens to require such a change in the future, we will > > > deal with it, at that time. Remember the rule, "Linux is evolution, not > > > intelligent design." > > > > Who "we" is is exactly the point. > > The kernel development community will handle it. That's nice to hear, it seems like a fairly bright bunch. > > For the not-so-important record, evolution does actually create > > intelligent designs. > > For some things yes, for others (like your tailbone), perhaps not :) Oh no, that's where the matrix cord is plugged in. :-) Cheers, Henrik ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 22:14 ` Henrik Rydberg 2012-02-24 22:20 ` Greg KH @ 2012-02-25 12:15 ` Theodore Tso 2012-02-26 9:54 ` Henrik Rydberg 2012-02-26 4:56 ` Bobby Powers 2 siblings, 1 reply; 68+ messages in thread From: Theodore Tso @ 2012-02-25 12:15 UTC (permalink / raw) To: Henrik Rydberg Cc: Theodore Tso, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list On Feb 24, 2012, at 5:14 PM, Henrik Rydberg wrote: > True, and the emergent interface (vfs) is even accessible in > userspace, so all is good in that department for now. Conversely, you > have not shown that this can go on forever. ;-) … and the performance is terrible --- as it must be. If you don't understand why this must be, then we're wasting our time having this conversation. And if you're the one wanting to make a change, then you are the one who has to provide the proof. Otherwise we're just talking about hypotheticals, and that way lies over-design. Why waste time engineering something that may never come to pass? Of course, your time is something you are free to spend however you like. So if you want to try implementing some of these grand plans of yours, show us the code…. -- Ted ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-25 12:15 ` Theodore Tso @ 2012-02-26 9:54 ` Henrik Rydberg 0 siblings, 0 replies; 68+ messages in thread From: Henrik Rydberg @ 2012-02-26 9:54 UTC (permalink / raw) To: Theodore Tso; +Cc: Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list > … and the performance is terrible --- as it must be. If you don't > understand why this must be, then we're wasting our time having this > conversation. You are right, I don't understand the assumptions leading to that conclusion. It does not seem to stop people from experimenting with filesystems that do no even reside within the same computer (hdfs). > And if you're the one wanting to make a change, then you are the one > who has to provide the proof. I know the drill, and I obviously already got hit on the head. ;-) > Otherwise we're just talking about > hypotheticals, and that way lies over-design. Why waste time > engineering something that may never come to pass? This is a seriously defensive attitude, given that the thread was hypothetical from the start. > Of course, your time is something you are free to spend however you > like. So if you want to try implementing some of these grand plans > of yours, show us the code…. If I put on rose-colored glasses and try really hard, I can almost interpret this as an invitation to share code. Thanks. Henrik ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 22:14 ` Henrik Rydberg 2012-02-24 22:20 ` Greg KH 2012-02-25 12:15 ` Theodore Tso @ 2012-02-26 4:56 ` Bobby Powers 2012-02-26 10:47 ` Henrik Rydberg 2 siblings, 1 reply; 68+ messages in thread From: Bobby Powers @ 2012-02-26 4:56 UTC (permalink / raw) To: Henrik Rydberg Cc: Ted Ts'o, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list On Fri, Feb 24, 2012 at 5:14 PM, Henrik Rydberg <rydberg@euromail.se> wrote: > On Fri, Feb 24, 2012 at 04:30:27PM -0500, Ted Ts'o wrote: >> On Fri, Feb 24, 2012 at 10:22:38PM +0100, Henrik Rydberg wrote: >> > >> > Maintenance. Sure, as soon as an area grows too large for a single >> > person, the current structure will ensure it divides so that the patch >> > stream becomes manageable. We have already seen ample examples of >> > that. But the overall structure of the kernel will become less and >> > less manageable, and the likelihood of duplicates and maintenance >> > problems will increase. >> >> Not necessarily. The primary area of growth has been hardware support >> (i.e., drivers), and that scales quite well. >> >> The main issue is the grown of intra-system interfaces, and you >> haven't shown that this has happened in anything approaching the >> levels where we need to worry. If we add a new file system, like >> btrfs, that doesn't add a whole new set of VFS interfaces, and it's >> not like we need to worry about communication interfaces between btrfs >> and ext4.... > > True, and the emergent interface (vfs) is even accessible in > userspace, so all is good in that department for now. Conversely, you > have not shown that this can go on forever. ;-) > > The main issue that set me off has been sufficiently diluted in the > (selective) discussion so as to no longer make sense as a reply: At > some point, in-tree or out-of-tree will no longer be distinguishable, Please explain how you would be unable to distinguish between a driver that lives in the kernel source tree, and one that does not. yours, Bobby > and the question is if/when/why something would/should happen as a > consequence. A matter of politics, if you will. It seemed like an > honest question starting this thread, and I think it is fair to keep > it in mind as such. > > Thanks. > Henrik > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 4:56 ` Bobby Powers @ 2012-02-26 10:47 ` Henrik Rydberg 2012-02-26 12:26 ` Richard Yao 2012-02-26 23:08 ` david 0 siblings, 2 replies; 68+ messages in thread From: Henrik Rydberg @ 2012-02-26 10:47 UTC (permalink / raw) To: Bobby Powers Cc: Ted Ts'o, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list > > The main issue that set me off has been sufficiently diluted in the > > (selective) discussion so as to no longer make sense as a reply: At > > some point, in-tree or out-of-tree will no longer be distinguishable, > > Please explain how you would be unable to distinguish between a driver > that lives in the kernel source tree, and one that does not. The SUD pointed to in the beginning of the thread is an example of this, but I was not thinking of it in quite so literal terms. Rather, I was imagining that as the kernel grows and the in-kernel interfaces matures, the amount of actual communication between different portions of the code diminishes. Code on opposite sides of a stable interface is, for all practical purposes, separated. Whether that code lives in-tree or out-of tree is then of little consequence. To try to prevent another flame war, let's make it clear that I am not saying that the most powerful in-kernel argument, that code can be changed, is unimportant. Maybe code, like so many other things, arranges itself in a scale-free critical fashion, which would forever warrant a monolithic approach. Maybe it would even make sense to have userspace join the same tree as well. There is however a frofoundly political aspect here, which cannot be expressed in terms of code. Also, in practise, breaking things down into manageable chunks is usually a good idea in the end. Thanks, Henrik ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 10:47 ` Henrik Rydberg @ 2012-02-26 12:26 ` Richard Yao 2012-02-26 14:23 ` Bernd Petrovitsch [not found] ` <365b85cee33d4f1aadc31336663de21c@HUBCAS2.cs.stonybrook.edu> 2012-02-26 23:08 ` david 1 sibling, 2 replies; 68+ messages in thread From: Richard Yao @ 2012-02-26 12:26 UTC (permalink / raw) To: Henrik Rydberg Cc: Bobby Powers, Ted Ts'o, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list On Sun, Feb 26, 2012 at 5:47 AM, Henrik Rydberg <rydberg@euromail.se> wrote: >> > The main issue that set me off has been sufficiently diluted in the >> > (selective) discussion so as to no longer make sense as a reply: At >> > some point, in-tree or out-of-tree will no longer be distinguishable, >> >> Please explain how you would be unable to distinguish between a driver >> that lives in the kernel source tree, and one that does not. > > The SUD pointed to in the beginning of the thread is an example of > this, but I was not thinking of it in quite so literal terms. Rather, > I was imagining that as the kernel grows and the in-kernel interfaces > matures, the amount of actual communication between different portions > of the code diminishes. Code on opposite sides of a stable interface > is, for all practical purposes, separated. Whether that code lives > in-tree or out-of tree is then of little consequence. > > To try to prevent another flame war, let's make it clear that I am not > saying that the most powerful in-kernel argument, that code can be > changed, is unimportant. Maybe code, like so many other things, > arranges itself in a scale-free critical fashion, which would forever > warrant a monolithic approach. Maybe it would even make sense to have > userspace join the same tree as well. There is however a frofoundly > political aspect here, which cannot be expressed in terms of > code. Also, in practise, breaking things down into manageable chunks > is usually a good idea in the end. I do not see what prevents an in-kernel context switch into a ring 3 context with a different process address space. Is it necessary to remove the code from the kernel tree before someone can do this? ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 12:26 ` Richard Yao @ 2012-02-26 14:23 ` Bernd Petrovitsch 2012-02-26 15:29 ` Henrik Rydberg [not found] ` <365b85cee33d4f1aadc31336663de21c@HUBCAS2.cs.stonybrook.edu> 1 sibling, 1 reply; 68+ messages in thread From: Bernd Petrovitsch @ 2012-02-26 14:23 UTC (permalink / raw) To: Henrik Rydberg Cc: Richard Yao, Bobby Powers, Ted Ts'o, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list On Son, 2012-02-26 at 07:26 -0500, Richard Yao wrote: > On Sun, Feb 26, 2012 at 5:47 AM, Henrik Rydberg <rydberg@euromail.se> wrote: [...] > > changed, is unimportant. Maybe code, like so many other things, > > arranges itself in a scale-free critical fashion, which would forever "Code arranges" not itself, but people arrange code the best they see fit. > > warrant a monolithic approach. Maybe it would even make sense to have And the Linux kernel is BTW not really that monolitic in the spirit of the definition of Tanenbaums book. It's just the category that sucks the least there. > > userspace join the same tree as well. There is however a frofoundly > > political aspect here, which cannot be expressed in terms of > > code. Also, in practise, breaking things down into manageable chunks > > is usually a good idea in the end. Yes, everybody learns that in the first lecture on programming. Please sent patches to make actually things better and not just talk. Thank you. > I do not see what prevents an in-kernel context switch into a ring 3 > context with a different process address space. Is it necessary to > remove the code from the kernel tree before someone can do this? No, and that's the real problem here: Everybody who might think it's a good idea may please implement - and thus propose - something concrete, try it out and comeback with experience and performance numbers - and not just try to come up with some theory and other misleading points (what political aspect?!) and ideas what others should do and why it might be better. Bernd -- Bernd Petrovitsch Email : bernd@petrovitsch.priv.at LUGA : http://www.luga.at ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 14:23 ` Bernd Petrovitsch @ 2012-02-26 15:29 ` Henrik Rydberg 0 siblings, 0 replies; 68+ messages in thread From: Henrik Rydberg @ 2012-02-26 15:29 UTC (permalink / raw) To: Bernd Petrovitsch Cc: Richard Yao, Bobby Powers, Ted Ts'o, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list > > > changed, is unimportant. Maybe code, like so many other things, > > > arranges itself in a scale-free critical fashion, which would forever > > "Code arranges" not itself, but people arrange code the best they see > fit. You seem to have missed the point of that sentence. Go pick on somebody else. > > > userspace join the same tree as well. There is however a frofoundly > > > political aspect here, which cannot be expressed in terms of > > > code. Also, in practise, breaking things down into manageable chunks > > > is usually a good idea in the end. > > Yes, everybody learns that in the first lecture on programming. Please > sent patches to make actually things better and not just talk. Thank > you. I actually do on a regular basis. I wish everybody did. Thanks. Henrik ^ permalink raw reply [flat|nested] 68+ messages in thread
[parent not found: <365b85cee33d4f1aadc31336663de21c@HUBCAS2.cs.stonybrook.edu>]
* Re: Can we move device drivers into user-space? [not found] ` <365b85cee33d4f1aadc31336663de21c@HUBCAS2.cs.stonybrook.edu> @ 2012-02-26 15:05 ` Richard Yao 2012-02-26 20:30 ` Ted Ts'o [not found] ` <09a5cca9cffb4300843f682be529e8ca@HUBCAS2.cs.stonybrook.edu> 0 siblings, 2 replies; 68+ messages in thread From: Richard Yao @ 2012-02-26 15:05 UTC (permalink / raw) To: Bernd Petrovitsch Cc: Henrik Rydberg, Bobby Powers, Ted Ts'o, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list >> I do not see what prevents an in-kernel context switch into a ring 3 >> context with a different process address space. Is it necessary to >> remove the code from the kernel tree before someone can do this? > > No, and that's the real problem here: Everybody who might think it's a > good idea may please implement - and thus propose - something concrete, > try it out and comeback with experience and performance numbers - and > not just try to come up with some theory and other misleading points > (what political aspect?!) and ideas what others should do and why it > might be better. It seems counterproductive to tell people to produce results without doing the brainstorming that such results require. If such discussion is not welcome here, then the FreeBSD project has a similar kernel design. Perhaps it would be more productive for those interested in this to brainstorm on their mailing list. If anything materializes, it could be for their kernel instead. Nothing is forcing anyone to do things with Linux. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 15:05 ` Richard Yao @ 2012-02-26 20:30 ` Ted Ts'o [not found] ` <09a5cca9cffb4300843f682be529e8ca@HUBCAS2.cs.stonybrook.edu> 1 sibling, 0 replies; 68+ messages in thread From: Ted Ts'o @ 2012-02-26 20:30 UTC (permalink / raw) To: Richard Yao Cc: Bernd Petrovitsch, Henrik Rydberg, Bobby Powers, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list On Sun, Feb 26, 2012 at 10:05:59AM -0500, Richard Yao wrote: > >> I do not see what prevents an in-kernel context switch into a ring 3 > >> context with a different process address space. Is it necessary to > >> remove the code from the kernel tree before someone can do this? > > > > No, and that's the real problem here: Everybody who might think it's a > > good idea may please implement - and thus propose - something concrete, > > try it out and comeback with experience and performance numbers - and > > not just try to come up with some theory and other misleading points > > (what political aspect?!) and ideas what others should do and why it > > might be better. > > It seems counterproductive to tell people to produce results without > doing the brainstorming that such results require. It's also counterproductive to tell kernel programmers how they should do what they are being paid how to do. If you just want to brainstorm possibilities where **you** will do all of the work, as opposed to asking the kernel development community to work on your pet theories, it's helpful if you more carefully label your poposals as such. > If such discussion is not welcome here, then the FreeBSD project has a > similar kernel design. Perhaps it would be more productive for those > interested in this to brainstorm on their mailing list. If anything > materializes, it could be for their kernel instead. Nothing is forcing > anyone to do things with Linux. My experience is that most researchers produce code which is not of production quality. It's very easy to produce a proof of concept that works well enough to publish a paper. Making something which is actually performant, is **hard**. The problem is that you don't get Ph.D.'s for counting cache line misses --- or for getting your implementation to the point where cache line misses matter in the first place. So the threat that Linux will miss out on some great breakthrough is something that is very hard for me to take seriously, speaking quite frankly. Usenix has awarded best paper awards to work that was done on obsolete kernels, where if the goal of that work was to produce production kernel code, was in fact horrible work. But if the goal of that work is to produce Ph.D. graduates, or to help the professor get tenure, was perhaps more successful. That being said, maybe you'd be happier engaging with the Minix 3 community. They are folks who think (unproven, theoretical) safety is more important than performance. And they've created a kernel which is a 140 times slower at process creation, ten times greater syscall overhead, and file copies that were two to ten times slower.[1] [1] http://lwn.net/Articles/220255/ And Prof. Tannenbaum is an academic, so his goals may be more closely aligned to yours. Best regards, - Ted ^ permalink raw reply [flat|nested] 68+ messages in thread
[parent not found: <09a5cca9cffb4300843f682be529e8ca@HUBCAS2.cs.stonybrook.edu>]
* Re: Can we move device drivers into user-space? [not found] ` <09a5cca9cffb4300843f682be529e8ca@HUBCAS2.cs.stonybrook.edu> @ 2012-02-26 21:25 ` Richard Yao 2012-02-26 21:35 ` Theodore Tso [not found] ` <10de0ef9fb5d44c08669191e12343a97@HUBCAS2.cs.stonybrook.edu> 0 siblings, 2 replies; 68+ messages in thread From: Richard Yao @ 2012-02-26 21:25 UTC (permalink / raw) To: Ted Ts'o, Richard Yao, Bernd Petrovitsch, Henrik Rydberg, Bobby Powers, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list Ted, I think you have me confused with Jidong Xiao. He is the one who suggested moving things to userspace. I happened to see the conversation, noticed people had missed a few points and added them in an attempt to be helpful. If you look at my emails to the mailing list, you would see that I had said that this work should be done elsewhere. The email to which you had replied was one that was meant to say that if people were set on having those interested in doing this to work with LInux, it should be okay for them to brainstorm on the LKML. With that said, please refrain from further outbursts. They hurt Linux. Yours truly, Richard Yao On Sun, Feb 26, 2012 at 3:30 PM, Ted Ts'o <tytso@mit.edu> wrote: > On Sun, Feb 26, 2012 at 10:05:59AM -0500, Richard Yao wrote: >> >> I do not see what prevents an in-kernel context switch into a ring 3 >> >> context with a different process address space. Is it necessary to >> >> remove the code from the kernel tree before someone can do this? >> > >> > No, and that's the real problem here: Everybody who might think it's a >> > good idea may please implement - and thus propose - something concrete, >> > try it out and comeback with experience and performance numbers - and >> > not just try to come up with some theory and other misleading points >> > (what political aspect?!) and ideas what others should do and why it >> > might be better. >> >> It seems counterproductive to tell people to produce results without >> doing the brainstorming that such results require. > > It's also counterproductive to tell kernel programmers how they should > do what they are being paid how to do. If you just want to brainstorm > possibilities where **you** will do all of the work, as opposed to > asking the kernel development community to work on your pet theories, > it's helpful if you more carefully label your poposals as such. > >> If such discussion is not welcome here, then the FreeBSD project has a >> similar kernel design. Perhaps it would be more productive for those >> interested in this to brainstorm on their mailing list. If anything >> materializes, it could be for their kernel instead. Nothing is forcing >> anyone to do things with Linux. > > My experience is that most researchers produce code which is not of > production quality. It's very easy to produce a proof of concept that > works well enough to publish a paper. Making something which is > actually performant, is **hard**. The problem is that you don't get > Ph.D.'s for counting cache line misses --- or for getting your > implementation to the point where cache line misses matter in the > first place. So the threat that Linux will miss out on some great > breakthrough is something that is very hard for me to take seriously, > speaking quite frankly. > > Usenix has awarded best paper awards to work that was done on obsolete > kernels, where if the goal of that work was to produce production > kernel code, was in fact horrible work. But if the goal of that work > is to produce Ph.D. graduates, or to help the professor get tenure, > was perhaps more successful. > > That being said, maybe you'd be happier engaging with the Minix 3 > community. They are folks who think (unproven, theoretical) safety is > more important than performance. And they've created a kernel which > is a 140 times slower at process creation, ten times greater syscall > overhead, and file copies that were two to ten times slower.[1] > > [1] http://lwn.net/Articles/220255/ > > And Prof. Tannenbaum is an academic, so his goals may be more closely > aligned to yours. > > Best regards, > > - Ted ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 21:25 ` Richard Yao @ 2012-02-26 21:35 ` Theodore Tso [not found] ` <10de0ef9fb5d44c08669191e12343a97@HUBCAS2.cs.stonybrook.edu> 1 sibling, 0 replies; 68+ messages in thread From: Theodore Tso @ 2012-02-26 21:35 UTC (permalink / raw) To: Richard Yao Cc: Theodore Tso, Bernd Petrovitsch, Henrik Rydberg, Bobby Powers, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list On Feb 26, 2012, at 4:25 PM, Richard Yao wrote: > Ted, > > I think you have me confused with Jidong Xiao. He is the one who > suggested moving things to userspace. I happened to see the > conversation, noticed people had missed a few points and added them in > an attempt to be helpful. If you look at my emails to the mailing > list, you would see that I had said that this work should be done > elsewhere. The email to which you had replied was one that was meant > to say that if people were set on having those interested in doing > this to work with LInux, it should be okay for them to brainstorm on > the LKML. The problem is that there are a huge number of crackpots out there, and Linux is popular enough that many of them tend to gravitate to the LKML. If they want to brainstorm, that's fine; but don't expect senior kernel developers to necessarily pay a lot of attention. It would be like people insisting on being able to brainstorm about whether it's possible to square the circle, or solve the halting problem, on some abstract math theorist's mailing list. It's possible they might be on the verge of a massive breakthrough, but when I was one of the postmaster of MIT, I got to field enough of these folks asking if we could make sure the chair of the math department would take a look at their proof, because they couldn't get anyone else to pay attention…. There is an old saying, "code talks, bullshit walks". If someone is known to have produced real code, then people are much more likely to take the time to respond. If someone hasn't yet produced real code, it's better if they try it, and show at least some kind of interesting result, before they try to waste a lot of people's time working on their pet theories. And if they want to go do this "value add" on the FreeBSD list, I'd encourage them to do it, except I don't think I would be thanked by the FreeBSD folks…. -- Ted > > With that said, please refrain from further outbursts. They hurt Linux. > > Yours truly, > Richard Yao > > On Sun, Feb 26, 2012 at 3:30 PM, Ted Ts'o <tytso@mit.edu> wrote: >> On Sun, Feb 26, 2012 at 10:05:59AM -0500, Richard Yao wrote: >>>>> I do not see what prevents an in-kernel context switch into a ring 3 >>>>> context with a different process address space. Is it necessary to >>>>> remove the code from the kernel tree before someone can do this? >>>> >>>> No, and that's the real problem here: Everybody who might think it's a >>>> good idea may please implement - and thus propose - something concrete, >>>> try it out and comeback with experience and performance numbers - and >>>> not just try to come up with some theory and other misleading points >>>> (what political aspect?!) and ideas what others should do and why it >>>> might be better. >>> >>> It seems counterproductive to tell people to produce results without >>> doing the brainstorming that such results require. >> >> It's also counterproductive to tell kernel programmers how they should >> do what they are being paid how to do. If you just want to brainstorm >> possibilities where **you** will do all of the work, as opposed to >> asking the kernel development community to work on your pet theories, >> it's helpful if you more carefully label your poposals as such. >> >>> If such discussion is not welcome here, then the FreeBSD project has a >>> similar kernel design. Perhaps it would be more productive for those >>> interested in this to brainstorm on their mailing list. If anything >>> materializes, it could be for their kernel instead. Nothing is forcing >>> anyone to do things with Linux. >> >> My experience is that most researchers produce code which is not of >> production quality. It's very easy to produce a proof of concept that >> works well enough to publish a paper. Making something which is >> actually performant, is **hard**. The problem is that you don't get >> Ph.D.'s for counting cache line misses --- or for getting your >> implementation to the point where cache line misses matter in the >> first place. So the threat that Linux will miss out on some great >> breakthrough is something that is very hard for me to take seriously, >> speaking quite frankly. >> >> Usenix has awarded best paper awards to work that was done on obsolete >> kernels, where if the goal of that work was to produce production >> kernel code, was in fact horrible work. But if the goal of that work >> is to produce Ph.D. graduates, or to help the professor get tenure, >> was perhaps more successful. >> >> That being said, maybe you'd be happier engaging with the Minix 3 >> community. They are folks who think (unproven, theoretical) safety is >> more important than performance. And they've created a kernel which >> is a 140 times slower at process creation, ten times greater syscall >> overhead, and file copies that were two to ten times slower.[1] >> >> [1] http://lwn.net/Articles/220255/ >> >> And Prof. Tannenbaum is an academic, so his goals may be more closely >> aligned to yours. >> >> Best regards, >> >> - Ted ^ permalink raw reply [flat|nested] 68+ messages in thread
[parent not found: <10de0ef9fb5d44c08669191e12343a97@HUBCAS2.cs.stonybrook.edu>]
* Re: Can we move device drivers into user-space? [not found] ` <10de0ef9fb5d44c08669191e12343a97@HUBCAS2.cs.stonybrook.edu> @ 2012-02-26 22:03 ` Richard Yao 2012-02-27 11:17 ` Bernd Petrovitsch 0 siblings, 1 reply; 68+ messages in thread From: Richard Yao @ 2012-02-26 22:03 UTC (permalink / raw) To: Theodore Tso Cc: Bernd Petrovitsch, Henrik Rydberg, Bobby Powers, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list > The problem is that there are a huge number of crackpots out there, and Linux is popular enough that many of them tend to gravitate to the LKML. If they want to brainstorm, that's fine; but don't expect senior kernel developers to necessarily pay a lot of attention. I think it is nice for people to be interested in kernel design and even if they have bad ideas, letting them discuss their ideas generally gives them the opportunity to learn why they are bad, which could generate good ideas in the process. microkernel designs are nice, but Linux is not the right basis for one. With that said, I suggest you refrain from being so quick to judge. It is a slippery slope that can lead you to consider anyone absent from a mental whitelist to be "a crackpot". It also causes problems when you misidentify someone because he made an attempt to help less knowledgeable people learn, rather than telling them with his vision of how things work. This can happen whenever you meet new people who would have been on your list had you known them better. Yours truly, Richard Yao ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 22:03 ` Richard Yao @ 2012-02-27 11:17 ` Bernd Petrovitsch 0 siblings, 0 replies; 68+ messages in thread From: Bernd Petrovitsch @ 2012-02-27 11:17 UTC (permalink / raw) To: Richard Yao Cc: Theodore Tso, Henrik Rydberg, Bobby Powers, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list On Son, 2012-02-26 at 17:03 -0500, Richard Yao wrote: [...] > I think it is nice for people to be interested in kernel design and > even if they have bad ideas, letting them discuss their ideas > generally gives them the opportunity to learn why they are bad, which > could generate good ideas in the process. Part of the learning experience is (or should be IMHO) to - at least - try it out in someway at some point and not only "discuss" it or throw ideas around. And that is: look for one target like a char driver or a filesystem or what ever one likes or seems fit and try to produce a somewhat working prototype. The performance point is probably valid but not every driver is a high performance issue. As for "filesystems", there is already FUSE around BTW. And IIRC there were similar efforts for block and/or char drivers. And of course, the "try it out" means primarily the OP - see the other mail. > microkernel designs are > nice, but Linux is not the right basis for one. Yes, that's the reason that everyone uses a microkernel today. SCNR .... And Mach got at times so fat so that people considered something else. Bernd -- Bernd Petrovitsch Email : bernd@petrovitsch.priv.at LUGA : http://www.luga.at ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 10:47 ` Henrik Rydberg 2012-02-26 12:26 ` Richard Yao @ 2012-02-26 23:08 ` david 2012-02-27 0:01 ` Henrik Rydberg 1 sibling, 1 reply; 68+ messages in thread From: david @ 2012-02-26 23:08 UTC (permalink / raw) To: Henrik Rydberg Cc: Bobby Powers, Ted Ts'o, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list On Sun, 26 Feb 2012, Henrik Rydberg wrote: >>> The main issue that set me off has been sufficiently diluted in the >>> (selective) discussion so as to no longer make sense as a reply: At >>> some point, in-tree or out-of-tree will no longer be distinguishable, >> >> Please explain how you would be unable to distinguish between a driver >> that lives in the kernel source tree, and one that does not. > > The SUD pointed to in the beginning of the thread is an example of > this, but I was not thinking of it in quite so literal terms. Rather, > I was imagining that as the kernel grows and the in-kernel interfaces > matures, the amount of actual communication between different portions > of the code diminishes. Code on opposite sides of a stable interface > is, for all practical purposes, separated. Whether that code lives > in-tree or out-of tree is then of little consequence. the point that you seem to be missing is that the interfaces between the different areas of the kernel are not stable, they change over time. When both sides of the interface are in the kernel, this is not a problem, both sides get changed, but if one side was out of the kernel, then you either can't make the change, or have a flag day change where both sides need to change in lock-step (and downgrading is hard as both sides need to change again) This is completely ignoring the performance and security aspects of userspace components vs kernel components. Ted is explaining the performance aspects well, but let's look at the security aspects as well. It's not just a case of "if something in userspace crashes, it doesn't crash the kerenl", it's also a case that "if you have a userspace component, then the kernel must sanity check the userspace interface to defend against rogue userspace". Doing these checks is not cheap (adding to performance overhead), and may not even be possible (how do you know if the command being sent to the SCSI bus is safe or not?) David Lang ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-26 23:08 ` david @ 2012-02-27 0:01 ` Henrik Rydberg 2012-02-27 0:53 ` david 0 siblings, 1 reply; 68+ messages in thread From: Henrik Rydberg @ 2012-02-27 0:01 UTC (permalink / raw) To: david Cc: Bobby Powers, Ted Ts'o, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list Hi David, > the point that you seem to be missing is that the interfaces between > the different areas of the kernel are not stable, they change over > time. The argument was based on the idea that they would stabilize over time. However, I realize this may not be true, which was also touched upon in a later reply. The heavy-tailed nature of large changes in open-source projects seems to put some hard numbers behind that claim [1]. > When both sides of the interface are in the kernel, this is > not a problem, both sides get changed, but if one side was out of > the kernel, then you either can't make the change, or have a flag > day change where both sides need to change in lock-step (and > downgrading is hard as both sides need to change again) Assuming the interfaces changes, this follows naturally, of course. > This is completely ignoring the performance and security aspects of > userspace components vs kernel components. Indeed. > Ted is explaining the performance aspects well, but let's look at > the security aspects as well. > > It's not just a case of "if something in userspace crashes, it > doesn't crash the kerenl", it's also a case that "if you have a > userspace component, then the kernel must sanity check the userspace > interface to defend against rogue userspace". Doing these checks is > not cheap (adding to performance overhead), and may not even be > possible (how do you know if the command being sent to the SCSI bus > is safe or not?) No doubt, an open-ended system has its own set of problems. At any given system size, the question is how this balances against a closed system. The assumption I made was that as the system grows, the balance would shift in favor of an open-ended system. This may not be the case at all, as you are saying. It would be nice to be able to see this in a quantitative manner if possible. Thanks for taking the time to respond. Henrik [1] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.91.7114 ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-27 0:01 ` Henrik Rydberg @ 2012-02-27 0:53 ` david 2012-02-27 9:07 ` Henrik Rydberg 0 siblings, 1 reply; 68+ messages in thread From: david @ 2012-02-27 0:53 UTC (permalink / raw) To: Henrik Rydberg Cc: Bobby Powers, Ted Ts'o, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list On Mon, 27 Feb 2012, Henrik Rydberg wrote: > Hi David, > >> the point that you seem to be missing is that the interfaces between >> the different areas of the kernel are not stable, they change over >> time. > > The argument was based on the idea that they would stabilize over > time. However, I realize this may not be true, which was also touched > upon in a later reply. The heavy-tailed nature of large changes in > open-source projects seems to put some hard numbers behind that claim [1]. What you are hearing from the long-time kernel developers is that they expect the interfaces to change over time. Remember, the linux kernel has been around for a long time, and as a result, the interfaces have gone through many iterations. There is no reason to believe that the current version is the final one, and the kernel developers now expect that the current interfaces are _not_ going to remain the same forever, they expect that new technology will drive changes in these interfaces. This is especially true for the types of things that device drivers need. Take for example locking. That is an implicit part of the interface to the device drivers, and is are area with a fairly high rate of change today. Anyone who claims that the current locking is 'correct' and can be depended on by an out-of-tree driver for even a couple of kernel releases is going to be either laughed at or ignored as being too ignorant to matter. Yes, you can look around and find interfaces in the kernel that have not changed for a long time, but I'll bet that for every example you find, someone else could find an example of an interface that had remained unchanged for just about as long, but has changed fairly recently. David Lang ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-27 0:53 ` david @ 2012-02-27 9:07 ` Henrik Rydberg 0 siblings, 0 replies; 68+ messages in thread From: Henrik Rydberg @ 2012-02-27 9:07 UTC (permalink / raw) To: david Cc: Bobby Powers, Ted Ts'o, Greg KH, Guenter Roeck, Jidong Xiao, Kernel development list > >The argument was based on the idea that they would stabilize over > >time. However, I realize this may not be true, which was also touched > >upon in a later reply. The heavy-tailed nature of large changes in > >open-source projects seems to put some hard numbers behind that claim [1]. > > What you are hearing from the long-time kernel developers is that > they expect the interfaces to change over time. That is coming through clearly. What I find interesting is that in addition to the authorative voice of Linux experience, there is physical evidence that the well-known phenomenon of self-organized criticality governs open-source code development at large. The collective stream of patches coming from individual decisions and real-world adaption shows the same behavior as avalanches, solar flares and earthquakes; there is always a bigger change around the corner. This critical phenomenon manifests itself through heavy-tailed distributions, as reported in reference [1]. To a physicist, this is a big deal. Coming back to interfaces, the conclusion is that a large code tree which exhibits these qualities is susceptible to large changes, and can therefore adapt better. This puts extra weight behind the notion that Linux is doing the Right Thing. The measurement by Wu and Holt in 2006 [1] has probably been succeeded many times over, perhaps someone else has pointers to more recent studies. > Remember, the linux kernel has been around for a long time, and as a > result, the interfaces have gone through many iterations. There is > no reason to believe that the current version is the final one, and > the kernel developers now expect that the current interfaces are > _not_ going to remain the same forever, they expect that new > technology will drive changes in these interfaces. This is > especially true for the types of things that device drivers need. Theory, experiment and experience seems to say the same thing, so one simply has to agree with this point. > Take for example locking. That is an implicit part of the interface > to the device drivers, and is are area with a fairly high rate of > change today. Anyone who claims that the current locking is > 'correct' and can be depended on by an out-of-tree driver for even a > couple of kernel releases is going to be either laughed at or > ignored as being too ignorant to matter. > > Yes, you can look around and find interfaces in the kernel that have > not changed for a long time, but I'll bet that for every example you > find, someone else could find an example of an interface that had > remained unchanged for just about as long, but has changed fairly > recently. Obviously there is no disagreement here anymore. And I wouldn't bet without first computing the probability of such a change. :-) Thanks. Henrik ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 17:47 ` Guenter Roeck 2012-02-24 18:34 ` Greg KH @ 2012-03-01 9:54 ` Thomas Gleixner 1 sibling, 0 replies; 68+ messages in thread From: Thomas Gleixner @ 2012-03-01 9:54 UTC (permalink / raw) To: Guenter Roeck; +Cc: Greg KH, Jidong Xiao, Kernel development list On Fri, 24 Feb 2012, Guenter Roeck wrote: > On Fri, 2012-02-24 at 12:17 -0500, Greg KH wrote: > > On Fri, Feb 24, 2012 at 09:07:09AM -0800, Guenter Roeck wrote: > > > How about dropping UIO support from the kernel ? That would make more > > > sense to me. > > > > Again, UIO solves a real need, are you to tell the users of that code > > that somehow we are now not going to support them anymore? > > > > UIO was created when Thomas and I sat in the back of a conference > > presentation and saw, for the umpteenth time, a presentation by someone > > who was trying to write userspace drivers, and obviously didn't know > > what they were doing. > > > > UIO provides a framework that actually works (unlike all of the previous > > research papers were trying to do), and is used in real systems (laser > > welding robots!) every day, manufacturing things that you use and rely > > on. > > > > You remove UIO at the risk of pissing off those robots, the choice is > > yours, I know I'm not going to do it... > > I understand the background and reasoning, but ... > > I have seen UIO used for networking drivers, hwmon drivers, I2C bus > master drivers (with matching I2C client drivers in user-space), mfd > devices, and so on. I have seen existing kernel drivers re-implemented > as UIO drivers. I have seen UIO drivers where the kernel part of the > driver is larger than the entire driver written as kernel driver. I have > seen UIO drivers using polling instead of interrupts "because it is > faster than interrupts". > > Often, those drivers are then re-written for the next board (to support > the same chip) because they were not written with HW-independence in > mind and don't support HW abstraction. > > Yes, there may be real need for UIO in some cases, but all I have seen > it used for so far is what I would call abuse, resulting in maintenance > nightmares. UIO always had a clear target: devices which are application bound and not related to any larger set of functonality (device class frameworks, communication stacks, ...) For that kind of devices in kernel drivers are overkill and performance bottlenecks. The drivers which are in tree come with open user space access libraries and are completely sane. We can't punish the sane users for the asinnity of other people. If people go there and abuse UIO for other purposes, we can't prevent it. We can't prevent either that people write totally crappy kernel drivers ... > Given the choice, I would be quite happy to piss off some robots. Call > it a prejudice if you like ;). You might become less happy when the robot welds your ear onto a car hood :) Thanks, tglx ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Can we move device drivers into user-space? 2012-02-24 15:19 ` Jidong Xiao 2012-02-24 15:38 ` Greg KH @ 2012-02-24 15:58 ` Valdis.Kletnieks 1 sibling, 0 replies; 68+ messages in thread From: Valdis.Kletnieks @ 2012-02-24 15:58 UTC (permalink / raw) To: Jidong Xiao; +Cc: Kernel development list, Greg KH [-- Attachment #1: Type: text/plain, Size: 1082 bytes --] On Fri, 24 Feb 2012 10:19:36 EST, Jidong Xiao said: > Actually, my major concern is, since UIO has been accepted, then why > don't we move all the rest device drivers into user space as well. As > I understand, currently, some of device drivers are running on user > space, while the other (or say the majority of) device drivers are > running on kernel space, so why don't we maintain a consistent device > drivers infrastructure, say, either all in user space, or all in > kernel space. (Sure some critical device drivers still need to be kept > in kernel space.) Sure we could do that, if you deal with all of the following issues: 1) All I/O would take a lot more CPU, due to the additional context switching that would be required to start an I/O and handle the completion interrupt. 2) It makes it a lot harder to do zero-copy I/O. 3) Once we push the API to userspace, we're stuck with it. See Documentation/stable_api_nonsense.txt for the details. And finally: 4) -ENOPATCH. You think it's a good idea, do up a patch that works at least as well as what we have now... [-- Attachment #2: Type: application/pgp-signature, Size: 865 bytes --] ^ permalink raw reply [flat|nested] 68+ messages in thread
end of thread, other threads:[~2012-03-01 9:54 UTC | newest] Thread overview: 68+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-02-23 4:56 Can we move device drivers into user-space? Jidong Xiao 2012-02-23 15:57 ` Cong Wang 2012-02-23 16:34 ` Jidong Xiao 2012-02-23 20:48 ` david 2012-02-23 21:01 ` Jidong Xiao 2012-02-24 18:21 ` Mauro Carvalho Chehab 2012-02-25 15:10 ` Eduard - Gabriel Munteanu 2012-02-26 0:06 ` Mauro Carvalho Chehab 2012-02-26 0:29 ` Richard Yao 2012-02-27 11:31 ` Mauro Carvalho Chehab 2012-02-26 1:58 ` Dr. David Alan Gilbert 2012-02-26 3:34 ` arts zhao 2012-02-27 11:29 ` Mauro Carvalho Chehab 2012-02-25 15:31 ` Richard Yao 2012-02-23 21:18 ` Roland Dreier 2012-02-24 15:19 ` Jidong Xiao 2012-02-24 15:38 ` Greg KH 2012-02-24 16:38 ` Jidong Xiao 2012-02-24 16:54 ` Greg KH 2012-02-24 17:06 ` Jidong Xiao 2012-02-24 17:13 ` Greg KH 2012-02-24 17:21 ` Jidong Xiao 2012-02-24 17:31 ` Greg KH 2012-02-25 2:33 ` Richard Yao 2012-02-25 4:28 ` Jidong Xiao 2012-02-24 17:10 ` Al Viro 2012-02-25 19:23 ` Jidong Xiao 2012-02-25 20:55 ` Greg KH 2012-02-25 23:43 ` Jidong Xiao 2012-02-26 17:40 ` Greg KH 2012-02-26 22:46 ` Greg KH 2012-02-27 11:17 ` Bernd Petrovitsch 2012-02-24 17:07 ` Guenter Roeck 2012-02-24 17:17 ` Greg KH 2012-02-24 17:47 ` Guenter Roeck 2012-02-24 18:34 ` Greg KH 2012-02-24 19:15 ` Henrik Rydberg 2012-02-24 19:26 ` Greg KH 2012-02-24 20:10 ` Henrik Rydberg 2012-02-24 20:16 ` Greg KH 2012-02-24 20:37 ` Henrik Rydberg 2012-02-24 20:56 ` Greg KH 2012-02-24 21:22 ` Henrik Rydberg 2012-02-24 21:30 ` Ted Ts'o 2012-02-24 22:14 ` Henrik Rydberg 2012-02-24 22:20 ` Greg KH 2012-02-24 22:49 ` Henrik Rydberg 2012-02-24 22:54 ` Greg KH 2012-02-24 23:14 ` Henrik Rydberg 2012-02-25 12:15 ` Theodore Tso 2012-02-26 9:54 ` Henrik Rydberg 2012-02-26 4:56 ` Bobby Powers 2012-02-26 10:47 ` Henrik Rydberg 2012-02-26 12:26 ` Richard Yao 2012-02-26 14:23 ` Bernd Petrovitsch 2012-02-26 15:29 ` Henrik Rydberg [not found] ` <365b85cee33d4f1aadc31336663de21c@HUBCAS2.cs.stonybrook.edu> 2012-02-26 15:05 ` Richard Yao 2012-02-26 20:30 ` Ted Ts'o [not found] ` <09a5cca9cffb4300843f682be529e8ca@HUBCAS2.cs.stonybrook.edu> 2012-02-26 21:25 ` Richard Yao 2012-02-26 21:35 ` Theodore Tso [not found] ` <10de0ef9fb5d44c08669191e12343a97@HUBCAS2.cs.stonybrook.edu> 2012-02-26 22:03 ` Richard Yao 2012-02-27 11:17 ` Bernd Petrovitsch 2012-02-26 23:08 ` david 2012-02-27 0:01 ` Henrik Rydberg 2012-02-27 0:53 ` david 2012-02-27 9:07 ` Henrik Rydberg 2012-03-01 9:54 ` Thomas Gleixner 2012-02-24 15:58 ` Valdis.Kletnieks
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.