From: "Michael S. Tsirkin" <mst-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> To: Thiago Jung Bauermann <bauerman-tEXmvtCZX7AybS5Ee8rs3A@public.gmane.org> Cc: Mike Anderson <andmike-tEXmvtCZX7AybS5Ee8rs3A@public.gmane.org>, Michael Roth <mdroth-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>, Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>, Benjamin Herrenschmidt <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org>, Jason Wang <jasowang-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>, Alexey Kardashevskiy <aik-tEXmvtCZX7AybS5Ee8rs3A@public.gmane.org>, Ram Pai <linuxram-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org>, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, Paul Mackerras <paulus-mnsaURCQ41sdnm+yROfE0A@public.gmane.org>, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, linuxppc-dev-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org, Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>, David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> Subject: Re: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted Date: Wed, 24 Apr 2019 21:18:54 -0400 [thread overview] Message-ID: <20190424210813-mutt-send-email-mst@kernel.org> (raw) In-Reply-To: <875zr228zf.fsf-wxVGo8vDogbJvNEK5ZsId7p2dZbC/Bob@public.gmane.org> On Wed, Apr 24, 2019 at 10:01:56PM -0300, Thiago Jung Bauermann wrote: > > Michael S. Tsirkin <mst-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes: > > > On Wed, Apr 17, 2019 at 06:42:00PM -0300, Thiago Jung Bauermann wrote: > >> > >> Michael S. Tsirkin <mst-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes: > >> > >> > On Thu, Mar 21, 2019 at 09:05:04PM -0300, Thiago Jung Bauermann wrote: > >> >> > >> >> Michael S. Tsirkin <mst-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes: > >> >> > >> >> > On Wed, Mar 20, 2019 at 01:13:41PM -0300, Thiago Jung Bauermann wrote: > >> >> >> >From what I understand of the ACCESS_PLATFORM definition, the host will > >> >> >> only ever try to access memory addresses that are supplied to it by the > >> >> >> guest, so all of the secure guest memory that the host cares about is > >> >> >> accessible: > >> >> >> > >> >> >> If this feature bit is set to 0, then the device has same access to > >> >> >> memory addresses supplied to it as the driver has. In particular, > >> >> >> the device will always use physical addresses matching addresses > >> >> >> used by the driver (typically meaning physical addresses used by the > >> >> >> CPU) and not translated further, and can access any address supplied > >> >> >> to it by the driver. When clear, this overrides any > >> >> >> platform-specific description of whether device access is limited or > >> >> >> translated in any way, e.g. whether an IOMMU may be present. > >> >> >> > >> >> >> All of the above is true for POWER guests, whether they are secure > >> >> >> guests or not. > >> >> >> > >> >> >> Or are you saying that a virtio device may want to access memory > >> >> >> addresses that weren't supplied to it by the driver? > >> >> > > >> >> > Your logic would apply to IOMMUs as well. For your mode, there are > >> >> > specific encrypted memory regions that driver has access to but device > >> >> > does not. that seems to violate the constraint. > >> >> > >> >> Right, if there's a pre-configured 1:1 mapping in the IOMMU such that > >> >> the device can ignore the IOMMU for all practical purposes I would > >> >> indeed say that the logic would apply to IOMMUs as well. :-) > >> >> > >> >> I guess I'm still struggling with the purpose of signalling to the > >> >> driver that the host may not have access to memory addresses that it > >> >> will never try to access. > >> > > >> > For example, one of the benefits is to signal to host that driver does > >> > not expect ability to access all memory. If it does, host can > >> > fail initialization gracefully. > >> > >> But why would the ability to access all memory be necessary or even > >> useful? When would the host access memory that the driver didn't tell it > >> to access? > > > > When I say all memory I mean even memory not allowed by the IOMMU. > > Yes, but why? How is that memory relevant? It's relevant when driver is not trusted to only supply correct addresses. The feature was originally designed to support userspace drivers within guests. > >> >> >> >> > But the name "sev_active" makes me scared because at least AMD guys who > >> >> >> >> > were doing the sensible thing and setting ACCESS_PLATFORM > >> >> >> >> > >> >> >> >> My understanding is, AMD guest-platform knows in advance that their > >> >> >> >> guest will run in secure mode and hence sets the flag at the time of VM > >> >> >> >> instantiation. Unfortunately we dont have that luxury on our platforms. > >> >> >> > > >> >> >> > Well you do have that luxury. It looks like that there are existing > >> >> >> > guests that already acknowledge ACCESS_PLATFORM and you are not happy > >> >> >> > with how that path is slow. So you are trying to optimize for > >> >> >> > them by clearing ACCESS_PLATFORM and then you have lost ability > >> >> >> > to invoke DMA API. > >> >> >> > > >> >> >> > For example if there was another flag just like ACCESS_PLATFORM > >> >> >> > just not yet used by anyone, you would be all fine using that right? > >> >> >> > >> >> >> Yes, a new flag sounds like a great idea. What about the definition > >> >> >> below? > >> >> >> > >> >> >> VIRTIO_F_ACCESS_PLATFORM_NO_IOMMU This feature has the same meaning as > >> >> >> VIRTIO_F_ACCESS_PLATFORM both when set and when not set, with the > >> >> >> exception that the IOMMU is explicitly defined to be off or bypassed > >> >> >> when accessing memory addresses supplied to the device by the > >> >> >> driver. This flag should be set by the guest if offered, but to > >> >> >> allow for backward-compatibility device implementations allow for it > >> >> >> to be left unset by the guest. It is an error to set both this flag > >> >> >> and VIRTIO_F_ACCESS_PLATFORM. > >> >> > > >> >> > It looks kind of narrow but it's an option. > >> >> > >> >> Great! > >> >> > >> >> > I wonder how we'll define what's an iommu though. > >> >> > >> >> Hm, it didn't occur to me it could be an issue. I'll try. > >> > >> I rephrased it in terms of address translation. What do you think of > >> this version? The flag name is slightly different too: > >> > >> > >> VIRTIO_F_ACCESS_PLATFORM_NO_TRANSLATION This feature has the same > >> meaning as VIRTIO_F_ACCESS_PLATFORM both when set and when not set, > >> with the exception that address translation is guaranteed to be > >> unnecessary when accessing memory addresses supplied to the device > >> by the driver. Which is to say, the device will always use physical > >> addresses matching addresses used by the driver (typically meaning > >> physical addresses used by the CPU) and not translated further. This > >> flag should be set by the guest if offered, but to allow for > >> backward-compatibility device implementations allow for it to be > >> left unset by the guest. It is an error to set both this flag and > >> VIRTIO_F_ACCESS_PLATFORM. > > > > Thanks, I'll think about this approach. Will respond next week. > > Thanks! > > >> >> > Another idea is maybe something like virtio-iommu? > >> >> > >> >> You mean, have legacy guests use virtio-iommu to request an IOMMU > >> >> bypass? If so, it's an interesting idea for new guests but it doesn't > >> >> help with guests that are out today in the field, which don't have A > >> >> virtio-iommu driver. > >> > > >> > I presume legacy guests don't use encrypted memory so why do we > >> > worry about them at all? > >> > >> They don't use encrypted memory, but a host machine will run a mix of > >> secure and legacy guests. And since the hypervisor doesn't know whether > >> a guest will be secure or not at the time it is launched, legacy guests > >> will have to be launched with the same configuration as secure guests. > > > > OK and so I think the issue is that hosts generally fail if they set > > ACCESS_PLATFORM and guests do not negotiate it. > > So you can not just set ACCESS_PLATFORM for everyone. > > Is that the issue here? > > Yes, that is one half of the issue. The other is that even if hosts > didn't fail, existing legacy guests wouldn't "take the initiative" of > not negotiating ACCESS_PLATFORM to get the improved performance. They'd > have to be modified to do that. So there's a non-encrypted guest, hypervisor wants to set ACCESS_PLATFORM to allow encrypted guests but that will slow down legacy guests since their vIOMMU emulation is very slow. So enabling support for encryption slows down non-encrypted guests. Not great but not the end of the world, considering even older guests that don't support ACCESS_PLATFORM are completely broken and you do not seem to be too worried by that. For future non-encrypted guests, bypassing the emulated IOMMU for when that emulated IOMMU is very slow might be solvable in some other way, e.g. with virtio-iommu. Which reminds me, could you look at virtio-iommu as a solution for some of the issues? Review of that patchset from that POV would be appreciated. > -- > Thiago Jung Bauermann > IBM Linux Technology Center
WARNING: multiple messages have this Message-ID (diff)
From: "Michael S. Tsirkin" <mst@redhat.com> To: Thiago Jung Bauermann <bauerman@linux.ibm.com> Cc: Mike Anderson <andmike@linux.ibm.com>, Michael Roth <mdroth@linux.vnet.ibm.com>, Jean-Philippe Brucker <jean-philippe.brucker@arm.com>, Benjamin Herrenschmidt <benh@kernel.crashing.org>, Jason Wang <jasowang@redhat.com>, Alexey Kardashevskiy <aik@linux.ibm.com>, Ram Pai <linuxram@us.ibm.com>, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, Paul Mackerras <paulus@ozlabs.org>, iommu@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, Christoph Hellwig <hch@lst.de>, David Gibson <david@gibson.dropbear.id.au> Subject: Re: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted Date: Wed, 24 Apr 2019 21:18:54 -0400 [thread overview] Message-ID: <20190424210813-mutt-send-email-mst@kernel.org> (raw) Message-ID: <20190425011854.-JWkJjRWRStTvaKDv4n063kLAblUsr2VvJ5o-HfTXP4@z> (raw) In-Reply-To: <875zr228zf.fsf@morokweng.localdomain> On Wed, Apr 24, 2019 at 10:01:56PM -0300, Thiago Jung Bauermann wrote: > > Michael S. Tsirkin <mst@redhat.com> writes: > > > On Wed, Apr 17, 2019 at 06:42:00PM -0300, Thiago Jung Bauermann wrote: > >> > >> Michael S. Tsirkin <mst@redhat.com> writes: > >> > >> > On Thu, Mar 21, 2019 at 09:05:04PM -0300, Thiago Jung Bauermann wrote: > >> >> > >> >> Michael S. Tsirkin <mst@redhat.com> writes: > >> >> > >> >> > On Wed, Mar 20, 2019 at 01:13:41PM -0300, Thiago Jung Bauermann wrote: > >> >> >> >From what I understand of the ACCESS_PLATFORM definition, the host will > >> >> >> only ever try to access memory addresses that are supplied to it by the > >> >> >> guest, so all of the secure guest memory that the host cares about is > >> >> >> accessible: > >> >> >> > >> >> >> If this feature bit is set to 0, then the device has same access to > >> >> >> memory addresses supplied to it as the driver has. In particular, > >> >> >> the device will always use physical addresses matching addresses > >> >> >> used by the driver (typically meaning physical addresses used by the > >> >> >> CPU) and not translated further, and can access any address supplied > >> >> >> to it by the driver. When clear, this overrides any > >> >> >> platform-specific description of whether device access is limited or > >> >> >> translated in any way, e.g. whether an IOMMU may be present. > >> >> >> > >> >> >> All of the above is true for POWER guests, whether they are secure > >> >> >> guests or not. > >> >> >> > >> >> >> Or are you saying that a virtio device may want to access memory > >> >> >> addresses that weren't supplied to it by the driver? > >> >> > > >> >> > Your logic would apply to IOMMUs as well. For your mode, there are > >> >> > specific encrypted memory regions that driver has access to but device > >> >> > does not. that seems to violate the constraint. > >> >> > >> >> Right, if there's a pre-configured 1:1 mapping in the IOMMU such that > >> >> the device can ignore the IOMMU for all practical purposes I would > >> >> indeed say that the logic would apply to IOMMUs as well. :-) > >> >> > >> >> I guess I'm still struggling with the purpose of signalling to the > >> >> driver that the host may not have access to memory addresses that it > >> >> will never try to access. > >> > > >> > For example, one of the benefits is to signal to host that driver does > >> > not expect ability to access all memory. If it does, host can > >> > fail initialization gracefully. > >> > >> But why would the ability to access all memory be necessary or even > >> useful? When would the host access memory that the driver didn't tell it > >> to access? > > > > When I say all memory I mean even memory not allowed by the IOMMU. > > Yes, but why? How is that memory relevant? It's relevant when driver is not trusted to only supply correct addresses. The feature was originally designed to support userspace drivers within guests. > >> >> >> >> > But the name "sev_active" makes me scared because at least AMD guys who > >> >> >> >> > were doing the sensible thing and setting ACCESS_PLATFORM > >> >> >> >> > >> >> >> >> My understanding is, AMD guest-platform knows in advance that their > >> >> >> >> guest will run in secure mode and hence sets the flag at the time of VM > >> >> >> >> instantiation. Unfortunately we dont have that luxury on our platforms. > >> >> >> > > >> >> >> > Well you do have that luxury. It looks like that there are existing > >> >> >> > guests that already acknowledge ACCESS_PLATFORM and you are not happy > >> >> >> > with how that path is slow. So you are trying to optimize for > >> >> >> > them by clearing ACCESS_PLATFORM and then you have lost ability > >> >> >> > to invoke DMA API. > >> >> >> > > >> >> >> > For example if there was another flag just like ACCESS_PLATFORM > >> >> >> > just not yet used by anyone, you would be all fine using that right? > >> >> >> > >> >> >> Yes, a new flag sounds like a great idea. What about the definition > >> >> >> below? > >> >> >> > >> >> >> VIRTIO_F_ACCESS_PLATFORM_NO_IOMMU This feature has the same meaning as > >> >> >> VIRTIO_F_ACCESS_PLATFORM both when set and when not set, with the > >> >> >> exception that the IOMMU is explicitly defined to be off or bypassed > >> >> >> when accessing memory addresses supplied to the device by the > >> >> >> driver. This flag should be set by the guest if offered, but to > >> >> >> allow for backward-compatibility device implementations allow for it > >> >> >> to be left unset by the guest. It is an error to set both this flag > >> >> >> and VIRTIO_F_ACCESS_PLATFORM. > >> >> > > >> >> > It looks kind of narrow but it's an option. > >> >> > >> >> Great! > >> >> > >> >> > I wonder how we'll define what's an iommu though. > >> >> > >> >> Hm, it didn't occur to me it could be an issue. I'll try. > >> > >> I rephrased it in terms of address translation. What do you think of > >> this version? The flag name is slightly different too: > >> > >> > >> VIRTIO_F_ACCESS_PLATFORM_NO_TRANSLATION This feature has the same > >> meaning as VIRTIO_F_ACCESS_PLATFORM both when set and when not set, > >> with the exception that address translation is guaranteed to be > >> unnecessary when accessing memory addresses supplied to the device > >> by the driver. Which is to say, the device will always use physical > >> addresses matching addresses used by the driver (typically meaning > >> physical addresses used by the CPU) and not translated further. This > >> flag should be set by the guest if offered, but to allow for > >> backward-compatibility device implementations allow for it to be > >> left unset by the guest. It is an error to set both this flag and > >> VIRTIO_F_ACCESS_PLATFORM. > > > > Thanks, I'll think about this approach. Will respond next week. > > Thanks! > > >> >> > Another idea is maybe something like virtio-iommu? > >> >> > >> >> You mean, have legacy guests use virtio-iommu to request an IOMMU > >> >> bypass? If so, it's an interesting idea for new guests but it doesn't > >> >> help with guests that are out today in the field, which don't have A > >> >> virtio-iommu driver. > >> > > >> > I presume legacy guests don't use encrypted memory so why do we > >> > worry about them at all? > >> > >> They don't use encrypted memory, but a host machine will run a mix of > >> secure and legacy guests. And since the hypervisor doesn't know whether > >> a guest will be secure or not at the time it is launched, legacy guests > >> will have to be launched with the same configuration as secure guests. > > > > OK and so I think the issue is that hosts generally fail if they set > > ACCESS_PLATFORM and guests do not negotiate it. > > So you can not just set ACCESS_PLATFORM for everyone. > > Is that the issue here? > > Yes, that is one half of the issue. The other is that even if hosts > didn't fail, existing legacy guests wouldn't "take the initiative" of > not negotiating ACCESS_PLATFORM to get the improved performance. They'd > have to be modified to do that. So there's a non-encrypted guest, hypervisor wants to set ACCESS_PLATFORM to allow encrypted guests but that will slow down legacy guests since their vIOMMU emulation is very slow. So enabling support for encryption slows down non-encrypted guests. Not great but not the end of the world, considering even older guests that don't support ACCESS_PLATFORM are completely broken and you do not seem to be too worried by that. For future non-encrypted guests, bypassing the emulated IOMMU for when that emulated IOMMU is very slow might be solvable in some other way, e.g. with virtio-iommu. Which reminds me, could you look at virtio-iommu as a solution for some of the issues? Review of that patchset from that POV would be appreciated. > -- > Thiago Jung Bauermann > IBM Linux Technology Center _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2019-04-25 1:18 UTC|newest] Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-01-29 17:08 [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted Thiago Jung Bauermann 2019-01-29 17:42 ` Thiago Jung Bauermann 2019-01-29 19:02 ` Michael S. Tsirkin 2019-01-30 2:24 ` Jason Wang 2019-01-30 2:36 ` Michael S. Tsirkin 2019-01-30 3:05 ` Jason Wang 2019-01-30 3:26 ` Michael S. Tsirkin 2019-01-30 7:44 ` Christoph Hellwig 2019-02-04 18:15 ` Thiago Jung Bauermann 2019-02-04 21:38 ` Michael S. Tsirkin 2019-02-05 7:24 ` Christoph Hellwig [not found] ` <20190205072407.GA4311-jcswGhMUV9g@public.gmane.org> 2019-02-05 16:13 ` Michael S. Tsirkin 2019-03-26 16:53 ` Michael S. Tsirkin 2019-02-04 18:14 ` Thiago Jung Bauermann 2019-02-04 20:23 ` Michael S. Tsirkin 2019-03-20 16:13 ` Thiago Jung Bauermann 2019-03-20 21:17 ` Michael S. Tsirkin 2019-03-22 0:05 ` Thiago Jung Bauermann 2019-03-23 21:01 ` Michael S. Tsirkin 2019-03-25 0:57 ` David Gibson 2019-04-17 21:42 ` Thiago Jung Bauermann 2019-04-17 21:42 ` Thiago Jung Bauermann [not found] ` <20190323165456-mutt-send-email-mst-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> 2019-04-17 21:42 ` Thiago Jung Bauermann 2019-04-17 21:42 ` Thiago Jung Bauermann 2019-04-19 23:09 ` Michael S. Tsirkin 2019-04-19 23:09 ` Michael S. Tsirkin 2019-04-25 1:01 ` Thiago Jung Bauermann 2019-04-25 1:01 ` Thiago Jung Bauermann [not found] ` <875zr228zf.fsf-wxVGo8vDogbJvNEK5ZsId7p2dZbC/Bob@public.gmane.org> 2019-04-25 1:18 ` Michael S. Tsirkin [this message] 2019-04-25 1:18 ` Michael S. Tsirkin 2019-04-26 23:56 ` Thiago Jung Bauermann 2019-04-26 23:56 ` Thiago Jung Bauermann 2019-05-20 13:08 ` Michael S. Tsirkin 2019-05-20 13:16 ` Michael S. Tsirkin 2019-06-04 1:13 ` Thiago Jung Bauermann 2019-06-04 1:42 ` Michael S. Tsirkin 2019-06-28 1:58 ` Thiago Jung Bauermann 2019-07-01 14:17 ` Michael S. Tsirkin 2019-07-14 5:51 ` Thiago Jung Bauermann 2019-07-15 14:35 ` Michael S. Tsirkin 2019-07-15 20:29 ` Thiago Jung Bauermann 2019-07-15 20:36 ` Michael S. Tsirkin 2019-07-15 22:03 ` Thiago Jung Bauermann 2019-07-15 22:16 ` Michael S. Tsirkin 2019-07-15 23:05 ` Thiago Jung Bauermann 2019-07-15 23:24 ` Benjamin Herrenschmidt 2019-07-18 3:39 ` Thiago Jung Bauermann 2019-08-10 18:57 ` Michael S. Tsirkin 2019-08-10 22:07 ` Ram Pai 2019-08-11 5:56 ` Christoph Hellwig 2019-08-11 6:46 ` Ram Pai 2019-08-11 8:44 ` Michael S. Tsirkin 2019-08-12 12:13 ` Christoph Hellwig 2019-08-12 20:29 ` Ram Pai 2019-08-11 8:42 ` Michael S. Tsirkin 2019-08-11 8:55 ` Michael S. Tsirkin 2019-08-12 12:15 ` Christoph Hellwig 2019-09-06 5:07 ` Michael S. Tsirkin 2019-08-12 9:51 ` David Gibson 2019-08-13 13:26 ` Christoph Hellwig 2019-08-13 14:24 ` David Gibson 2019-08-13 15:45 ` Ram Pai 2019-08-26 17:48 ` Ram Pai 2019-08-11 8:12 ` Michael S. Tsirkin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190424210813-mutt-send-email-mst@kernel.org \ --to=mst-h+wxahxf7alqt0dzr+alfa@public.gmane.org \ --cc=aik-tEXmvtCZX7AybS5Ee8rs3A@public.gmane.org \ --cc=andmike-tEXmvtCZX7AybS5Ee8rs3A@public.gmane.org \ --cc=bauerman-tEXmvtCZX7AybS5Ee8rs3A@public.gmane.org \ --cc=benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org \ --cc=david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org \ --cc=hch-jcswGhMUV9g@public.gmane.org \ --cc=iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \ --cc=jasowang-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \ --cc=jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org \ --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=linuxppc-dev-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org \ --cc=linuxram-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org \ --cc=mdroth-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org \ --cc=paulus-mnsaURCQ41sdnm+yROfE0A@public.gmane.org \ --cc=virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).