From mboxrd@z Thu Jan 1 00:00:00 1970 From: Hajime Tazaki Subject: Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library Date: Tue, 31 Mar 2020 16:08:10 +0900 Message-ID: References: Mime-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII Return-path: Received: from mail-pl1-f193.google.com ([209.85.214.193]:34176 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726001AbgCaHIX (ORCPT ); Tue, 31 Mar 2020 03:08:23 -0400 Received: by mail-pl1-f193.google.com with SMTP id a23so7786880plm.1 for ; Tue, 31 Mar 2020 00:08:20 -0700 (PDT) In-Reply-To: Sender: linux-arch-owner@vger.kernel.org List-ID: To: richard.weinberger@gmail.com Cc: johannes@sipsolutions.net, linux-um@lists.infradead.org, linux-arch@vger.kernel.org, levex@linux.com, mattator@gmail.com, cem@freebsd.org, tavi.purdila@gmail.com, staal1978@gmail.com, motomuman@gmail.com, jiangshanlai@gmail.com, retrage01@gmail.com, petrosagg@gmail.com, edisonmcastro@hotmail.com, xiaoj@google.com, mark@stillwell.me, linux-kernel-library@freelists.org, pscollins@google.com, phh@phh.me, sigmaepsilon92@gmail.com, luca.dariz@gmail.com, liuyuan@google.com Hello Johannes, Richard On Tue, 31 Mar 2020 07:12:38 +0900, Richard Weinberger wrote: > > Johannes, Hajime, > > On Mon, Mar 30, 2020 at 11:53 PM Johannes Berg > wrote: > > > > On Mon, 2020-03-30 at 23:45 +0900, Hajime Tazaki wrote: > > > From: Octavian Purdila > > > > > > Adds the LKL Kconfig, vmlinux linker script, basic architecture > > > headers and miscellaneous basic functions or stubs such as > > > dump_stack(), show_regs() and cpuinfo proc ops. > > > > > > The headers we introduce in this patch are simple wrappers to the > > > asm-generic headers or stubs for things we don't support, such as > > > ptrace, DMA, signals, ELF handling and low level processor operations. > > > > > > The kernel configuration is automatically updated to reflect the > > > endianness of the host, 64bit support or the output format for > > > vmlinux's linker script. We do this by looking at the ld's default > > > output format. > > > > Can you elaborate what the plan is here? I always wrote down the current plan that I have in my mind in the cover letter (patch 00); please take a look at the "Milestone" block. Quoted: > Milestone > ========= > This patches is a first step toward upstreaming *library mode* of Linux kernel, > but we think we need to have several steps toward our goal, describing in the > below. > > 1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way > from UML. > 2. Use UML driver implementations in LKL as a minimum set of patches > - Only support x86 sub architecture (dependency to UML drivers) > 3. Support broader host supports > - add virtio device features > > For the step 1, we put LKL as one of UMMODE in order to make less effort to > integrate (make ARCH=um UMMODE=library). The modification to existing UML > code is trying to be minimized. > > I mean, you're not actually "unifying" anything with ARCH=um, you're > > just basically splitting ARCH=um into ARCH=um-um and ARCH=um-lkl or > > something. Is there much point? The current step (1 in the milestone) tries to cover this goal: splitting ARCH=um into UMMODE_KERNEL and UMMODE_LIB. > > Even the basic underlying building blocks are _very_ different, e.g. in > > UML the host interface is just a bunch of functions that must be > > implemented (os_*()), while you have a "struct lkl_host_operations". If > > we can't even unify at that trivial level, is there any point in it at > > all? I'm not even really sure what those ops are used for - are all of > > those things that the *application* using LKL necessarily must provide? > > > > Similarly with the IRQ routing mechanism - two completely different > > concepts. Where's the "unifying"? > > > > Ultimately, I can see two ways this goes: > > > > 1) We give up, and get ARCH=lkl, sharing just (some of) the drivers > > while moving them into the right drivers/somewhere/ place. Even that > > looks somewhat awkward looking at the later patches in this set, but > > seems like that at *least* should be done. > > Yeah, this would be a goal. > UML and LKL are quite different but they should share at least their userspace > drivers. > I also don't mind if we don't share every driver at the beginning but > it should be > a feasible goal for the future. Sharing drivers code is also included in this patchset, step 2 in the milestone. I was thinking that implementing os_*() functions with lkl_host_ops would be the further goal (e.g., step 3 or 4). > > 2) Ideally, instead, we actually unify: LKL grows support for userspace > > processes using UML infrastructure, the "in-kernel" IRQ mechanisms > > are unified, UML stuff moves into lkl-ops, and the UML binary > > basically becomes a user of the LKL library to start everything up. > > There may be some bits remaining that are just not interesting (e.g. > > some drivers you don't care about would continue to make direct calls > > to the user side instead of lkl-ops, and then they're just not > > compatible with lkl, only with the uml special case of lkl), but then > > things are clean. > > A few months ago I though this is doable but now I'm not so sure anymore. For the part of (2) which Johannes pointed out (I mean the part "UML stuff moves into lkl-ops"), I become to think that implementing os_*() functions using lkl_host_ops would be also interesting if those re-implementation makes the glue code eliminated. I'll work on that. > > Now, of course it seems like (2) would actually be better - LKL would > > actually get support for userspace processes using UML's tricks, most of > > the code is unified, and even LKL's users can take advantage of new > > things. At the same time, all of the duplication is avoided. > > > > However, I just don't know if it's actually _possible_ to do that > > though. Conceptually, it seems like it should be - why shouldn't a > > library be able to spawn other userspace processes - I mean, it's not > > really in the model that LKL really _wants_ since it's supposed to offer > > the syscall API, but you could make a "bool run_normal_init" or > > something in the lkl-ops for the user of the library to determine what > > should happen, right? > > > > However, there clearly are huge differences between LKL and UML in all > > respects, and I don't know if this wouldn't conflict with the library > > model, e.g. there may be linker issues etc. Or maybe the specific UML > > interrupt handling is required in UML and cannot be done in LKL (but > > then perhaps it could be put into the hypothetical UML-application vs. > > UML-the-LKL-library?) For the other part of (2), I agree that your definition of the unification will be the best and final goal of this integration. But, especially the support for UML userspace processes in LKL is not easy as far as I can see, at least the current LKL doesn't have, I treat those extensions (from the LKL pov) as further goals. Or the title of the cover letter is somehow overstatement: instead, "Minimum integration of UML with LKL" or something like this would be better ? > > Ultimately, personally I think it's going to have to be one or the other > > of those two options though, I don't really see much value in what > > you're doing here in the patchset now. This way just messes up > > everything, it's not clear what's UML and what's LKL, and they're > > intertwined with ifdefs everywhere; just look at where you have to add > > ifdefs in patch 23 - how would anyone later understand which part gets > > compiled for which of them? Since the patchset of LKL is relatively huge, I was trying to make a minimum patchset for the first step. Because having #ifdefs and glue code makes existing arch/um code untouched, I took this way to make the patchset as small as possible. But if this is not the case, I will look for the other way and touch arch/um code to fit the current LKL implementation. What do you think ? # For the particular ifdefs in patch 23, I will try to make it easy to understand. > > johannes > > > > PS: actually having something like lkl-ops in UML would've been nice > > also for my "time-travel" work, since it neatly abstracts the timers > > out. I do wonder a bit about the overhead of jumping through function > > pointers all the time though, it may be worth rethinking that overall > > anyway! > > Agreed. UML can also learn from LKL. :-) Ditto; this is also my wish. -- Hajime From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pj1-x1041.google.com ([2607:f8b0:4864:20::1041]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jJB0W-0005Jw-9t for linux-um@lists.infradead.org; Tue, 31 Mar 2020 07:08:22 +0000 Received: by mail-pj1-x1041.google.com with SMTP id o12so689563pjs.2 for ; Tue, 31 Mar 2020 00:08:20 -0700 (PDT) Date: Tue, 31 Mar 2020 16:08:10 +0900 Message-ID: From: Hajime Tazaki Subject: Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library In-Reply-To: References: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-um" Errors-To: linux-um-bounces+geert=linux-m68k.org@lists.infradead.org To: richard.weinberger@gmail.com Cc: linux-arch@vger.kernel.org, levex@linux.com, mattator@gmail.com, cem@freebsd.org, tavi.purdila@gmail.com, jiangshanlai@gmail.com, staal1978@gmail.com, motomuman@gmail.com, linux-um@lists.infradead.org, retrage01@gmail.com, petrosagg@gmail.com, liuyuan@google.com, phh@phh.me, xiaoj@google.com, mark@stillwell.me, pscollins@google.com, linux-kernel-library@freelists.org, johannes@sipsolutions.net, sigmaepsilon92@gmail.com, luca.dariz@gmail.com, edisonmcastro@hotmail.com Hello Johannes, Richard On Tue, 31 Mar 2020 07:12:38 +0900, Richard Weinberger wrote: > > Johannes, Hajime, > > On Mon, Mar 30, 2020 at 11:53 PM Johannes Berg > wrote: > > > > On Mon, 2020-03-30 at 23:45 +0900, Hajime Tazaki wrote: > > > From: Octavian Purdila > > > > > > Adds the LKL Kconfig, vmlinux linker script, basic architecture > > > headers and miscellaneous basic functions or stubs such as > > > dump_stack(), show_regs() and cpuinfo proc ops. > > > > > > The headers we introduce in this patch are simple wrappers to the > > > asm-generic headers or stubs for things we don't support, such as > > > ptrace, DMA, signals, ELF handling and low level processor operations. > > > > > > The kernel configuration is automatically updated to reflect the > > > endianness of the host, 64bit support or the output format for > > > vmlinux's linker script. We do this by looking at the ld's default > > > output format. > > > > Can you elaborate what the plan is here? I always wrote down the current plan that I have in my mind in the cover letter (patch 00); please take a look at the "Milestone" block. Quoted: > Milestone > ========= > This patches is a first step toward upstreaming *library mode* of Linux kernel, > but we think we need to have several steps toward our goal, describing in the > below. > > 1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way > from UML. > 2. Use UML driver implementations in LKL as a minimum set of patches > - Only support x86 sub architecture (dependency to UML drivers) > 3. Support broader host supports > - add virtio device features > > For the step 1, we put LKL as one of UMMODE in order to make less effort to > integrate (make ARCH=um UMMODE=library). The modification to existing UML > code is trying to be minimized. > > I mean, you're not actually "unifying" anything with ARCH=um, you're > > just basically splitting ARCH=um into ARCH=um-um and ARCH=um-lkl or > > something. Is there much point? The current step (1 in the milestone) tries to cover this goal: splitting ARCH=um into UMMODE_KERNEL and UMMODE_LIB. > > Even the basic underlying building blocks are _very_ different, e.g. in > > UML the host interface is just a bunch of functions that must be > > implemented (os_*()), while you have a "struct lkl_host_operations". If > > we can't even unify at that trivial level, is there any point in it at > > all? I'm not even really sure what those ops are used for - are all of > > those things that the *application* using LKL necessarily must provide? > > > > Similarly with the IRQ routing mechanism - two completely different > > concepts. Where's the "unifying"? > > > > Ultimately, I can see two ways this goes: > > > > 1) We give up, and get ARCH=lkl, sharing just (some of) the drivers > > while moving them into the right drivers/somewhere/ place. Even that > > looks somewhat awkward looking at the later patches in this set, but > > seems like that at *least* should be done. > > Yeah, this would be a goal. > UML and LKL are quite different but they should share at least their userspace > drivers. > I also don't mind if we don't share every driver at the beginning but > it should be > a feasible goal for the future. Sharing drivers code is also included in this patchset, step 2 in the milestone. I was thinking that implementing os_*() functions with lkl_host_ops would be the further goal (e.g., step 3 or 4). > > 2) Ideally, instead, we actually unify: LKL grows support for userspace > > processes using UML infrastructure, the "in-kernel" IRQ mechanisms > > are unified, UML stuff moves into lkl-ops, and the UML binary > > basically becomes a user of the LKL library to start everything up. > > There may be some bits remaining that are just not interesting (e.g. > > some drivers you don't care about would continue to make direct calls > > to the user side instead of lkl-ops, and then they're just not > > compatible with lkl, only with the uml special case of lkl), but then > > things are clean. > > A few months ago I though this is doable but now I'm not so sure anymore. For the part of (2) which Johannes pointed out (I mean the part "UML stuff moves into lkl-ops"), I become to think that implementing os_*() functions using lkl_host_ops would be also interesting if those re-implementation makes the glue code eliminated. I'll work on that. > > Now, of course it seems like (2) would actually be better - LKL would > > actually get support for userspace processes using UML's tricks, most of > > the code is unified, and even LKL's users can take advantage of new > > things. At the same time, all of the duplication is avoided. > > > > However, I just don't know if it's actually _possible_ to do that > > though. Conceptually, it seems like it should be - why shouldn't a > > library be able to spawn other userspace processes - I mean, it's not > > really in the model that LKL really _wants_ since it's supposed to offer > > the syscall API, but you could make a "bool run_normal_init" or > > something in the lkl-ops for the user of the library to determine what > > should happen, right? > > > > However, there clearly are huge differences between LKL and UML in all > > respects, and I don't know if this wouldn't conflict with the library > > model, e.g. there may be linker issues etc. Or maybe the specific UML > > interrupt handling is required in UML and cannot be done in LKL (but > > then perhaps it could be put into the hypothetical UML-application vs. > > UML-the-LKL-library?) For the other part of (2), I agree that your definition of the unification will be the best and final goal of this integration. But, especially the support for UML userspace processes in LKL is not easy as far as I can see, at least the current LKL doesn't have, I treat those extensions (from the LKL pov) as further goals. Or the title of the cover letter is somehow overstatement: instead, "Minimum integration of UML with LKL" or something like this would be better ? > > Ultimately, personally I think it's going to have to be one or the other > > of those two options though, I don't really see much value in what > > you're doing here in the patchset now. This way just messes up > > everything, it's not clear what's UML and what's LKL, and they're > > intertwined with ifdefs everywhere; just look at where you have to add > > ifdefs in patch 23 - how would anyone later understand which part gets > > compiled for which of them? Since the patchset of LKL is relatively huge, I was trying to make a minimum patchset for the first step. Because having #ifdefs and glue code makes existing arch/um code untouched, I took this way to make the patchset as small as possible. But if this is not the case, I will look for the other way and touch arch/um code to fit the current LKL implementation. What do you think ? # For the particular ifdefs in patch 23, I will try to make it easy to understand. > > johannes > > > > PS: actually having something like lkl-ops in UML would've been nice > > also for my "time-travel" work, since it neatly abstracts the timers > > out. I do wonder a bit about the overhead of jumping through function > > pointers all the time though, it may be worth rethinking that overall > > anyway! > > Agreed. UML can also learn from LKL. :-) Ditto; this is also my wish. -- Hajime _______________________________________________ linux-um mailing list linux-um@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-um