From: Christian Schoenebeck <qemu_oss@crudebyte.com>
To: qemu-devel@nongnu.org
Cc: "Jiachen Zhang" <zhangjiachen.jaycee@bytedance.com>,
"Daniel P. Berrange" <berrange@redhat.com>,
slp@redhat.com, "Michael S . Tsirkin" <mst@redhat.com>,
"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
virtio-fs@redhat.com, "Xie Yongji" <xieyongji@bytedance.com>,
"Marc-André Lureau" <marcandre.lureau@gmail.com>,
"Stefan Hajnoczi" <stefanha@redhat.com>
Subject: Re: [External] Re: [RFC PATCH 0/9] Support for Virtio-fs daemon crash reconnection
Date: Thu, 18 Mar 2021 12:58:46 +0100 [thread overview]
Message-ID: <1711593.yAA9ihpmTb@silver> (raw)
In-Reply-To: <CAFQAk7jmvzL3tT=7ExcUKyo9SKNdhT29F5y3f4K8YigA08=BWQ@mail.gmail.com>
On Mittwoch, 17. März 2021 13:57:47 CET Jiachen Zhang wrote:
> On Wed, Mar 17, 2021 at 7:50 PM Christian Schoenebeck <
>
> qemu_oss@crudebyte.com> wrote:
> > On Mittwoch, 17. März 2021 11:05:32 CET Stefan Hajnoczi wrote:
> > > On Fri, Dec 18, 2020 at 05:39:34PM +0800, Jiachen Zhang wrote:
> > > > Thanks for the suggestions. Actually, we choose to save all state
> > > > information to QEMU because a virtiofsd has the same lifecycle as its
> > > > QEMU master. However, saving things to a file do avoid communication
> >
> > with
> >
> > > > QEMU, and we no longer need to increase the complexity of vhost-user
> > > > protocol. The suggestion to save fds to the systemd is also very
> > > > reasonable
> > > > if we don't consider the lifecycle issues, we will try it.
> > >
> > > Hi,
> > > We recently discussed crash recovery in the virtio-fs bi-weekly call and
> > > I read some of this email thread because it's a topic I'm interested in.
> >
> > I just had a quick fly over the patches so far. Shouldn't there be some
> > kind
> > of constraint for an automatic reconnection feature after a crash to
> > prevent
> > this being exploited by ROP brute force attacks?
> >
> > E.g. adding some (maybe continuously increasing) delay and/or limiting the
> > amount of reconnects within a certain time frame would come to my mind.
> >
> > Best regards,
> > Christian Schoenebeck
>
> Thanks, Christian. I am still trying to figure out the details of the ROP
> attacks.
>
> However, QEMU's vhost-user reconnection is based on chardev socket
> reconnection. The socket reconnection can be enabled by the "--chardev
> socket,...,reconnect=N" in QEMU command options, in which N means QEMU will
> try to connect the disconnected socket every N seconds. We can increase N
> to increase the reconnect delay. If we want to change the reconnect delay
> dynamically, I think we should change the chardev socket reconnection code.
> It is a more generic mechanism than vhost-user-fs and vhost-user backend.
>
> By the way, I also considered the socket reconnection delay time in the
> performance aspect. As the reconnection delay increase, if an application
> in the guest is doing I/Os, it will suffer larger tail latency. And for
> now, the smallest delay is 1 second, which is rather large for
> high-performance virtual I/O devices today. I think maybe a more performant
> and safer reconnect delay adjustment mechanism should be considered in the
> future. What are your thoughts?
So with N=1 an attacker could e.g. bypass a 16-bit PAC by brute-force in ~18
hours (e.g. on Arm if PAC + MTE was enabled). With 24-bit PAC (no MTE) it
would be ~194 days. Independent of what architecture and defend mechanism is
used, there is always the possibility though that some kind of side channel
attack exists that might require a much lower amount of attempts. So in an
untrusted environment I would personally limit the amount of automatic
reconnects and rather accept a down time for further investigation if a
suspicious high amount of crashes happened.
And yes, if a dynamic delay scheme was deployed in future then starting with a
value smaller than 1 second would make sense.
Best regards,
Christian Schoenebeck
next prev parent reply other threads:[~2021-03-18 12:01 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-15 16:21 [RFC PATCH 0/9] Support for Virtio-fs daemon crash reconnection Jiachen Zhang
2020-12-15 16:21 ` [RFC PATCH 1/9] vhost-user-fs: Add support for reconnection of vhost-user-fs backend Jiachen Zhang
2020-12-15 16:21 ` [RFC PATCH 2/9] vhost: Add vhost-user message types for sending shared memory and file fds Jiachen Zhang
2020-12-15 16:21 ` [RFC PATCH 3/9] vhost-user-fs: Support virtiofsd crash reconnection Jiachen Zhang
2020-12-15 16:21 ` [RFC PATCH 4/9] libvhost-user: Add vhost-user message types for sending shared memory and file fds Jiachen Zhang
2020-12-15 16:21 ` [RFC PATCH 5/9] virtiofsd: Convert the struct lo_map array to a more flatten layout Jiachen Zhang
2020-12-15 16:21 ` [RFC PATCH 6/9] virtiofsd: Add two new options for crash reconnection Jiachen Zhang
2021-02-04 12:08 ` Dr. David Alan Gilbert
2021-02-04 14:16 ` [External] " Jiachen Zhang
2020-12-15 16:21 ` [RFC PATCH 7/9] virtiofsd: Persist/restore lo_map and opened fds to/from QEMU Jiachen Zhang
2020-12-15 16:21 ` [RFC PATCH 8/9] virtiofsd: Ensure crash consistency after reconnection Jiachen Zhang
2020-12-15 16:21 ` [RFC PATCH 9/9] virtiofsd: (work around) Comment qsort in inflight I/O tracking Jiachen Zhang
2021-02-04 12:15 ` Dr. David Alan Gilbert
2021-02-04 14:20 ` [External] " Jiachen Zhang
2020-12-15 22:51 ` [RFC PATCH 0/9] Support for Virtio-fs daemon crash reconnection no-reply
2020-12-16 15:36 ` Marc-André Lureau
2020-12-18 9:39 ` [External] " Jiachen Zhang
2021-03-17 10:05 ` Stefan Hajnoczi
2021-03-17 11:49 ` Christian Schoenebeck
2021-03-17 12:57 ` Jiachen Zhang
2021-03-18 11:58 ` Christian Schoenebeck [this message]
2021-03-22 10:54 ` Stefan Hajnoczi
2021-03-23 12:54 ` Christian Schoenebeck
2021-03-23 14:25 ` Stefan Hajnoczi
2021-03-17 12:32 ` Jiachen Zhang
2021-03-22 11:00 ` Stefan Hajnoczi
2021-03-22 20:13 ` [Virtio-fs] " Vivek Goyal
2021-03-23 13:45 ` Stefan Hajnoczi
2021-05-10 14:38 ` Jiachen Zhang
2021-05-13 15:17 ` Stefan Hajnoczi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1711593.yAA9ihpmTb@silver \
--to=qemu_oss@crudebyte.com \
--cc=berrange@redhat.com \
--cc=dgilbert@redhat.com \
--cc=marcandre.lureau@gmail.com \
--cc=mst@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=slp@redhat.com \
--cc=stefanha@redhat.com \
--cc=virtio-fs@redhat.com \
--cc=xieyongji@bytedance.com \
--cc=zhangjiachen.jaycee@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).