From: Sage Weil <sweil@redhat.com>
To: ceph-devel@vger.kernel.org
Subject: msgr2 protocol
Date: Thu, 26 May 2016 14:17:37 -0400 (EDT) [thread overview]
Message-ID: <alpine.DEB.2.11.1605261358330.6221@cpach.fuggernut.com> (raw)
I wrote up a basic proposal for the new msgr2 protocol:
http://pad.ceph.com/p/msgr2
It is pretty similar to the current protocol, with a few key changes:
1. The initial banner has a version number for protocl features supported
and required. This will allow optional behavior later. The current
protocol doesn't allow this (the banner string is fixed and has to match
verbatim).
2. The auth handshake is a low-level msgr exchange now. This more or less
matches the MAuth and MAuthReply exchange with the mon. Also, the
authenticator/ticket presentation for established clients can be sent here
as part of this exchange, instead of as part of the msg_connect and
msg_connect_reply exchnage.
3. The identification of peers during connect is moved to the TAG_IDENT
stage. This way it could happen after authentication and/or encryption,
if we like. (Not sure it matters.)
4. Signatures are a separate message now that follows the previous
message. If a message doesn't have a signature that follows, it is
dropped. Once authenticated we can sign all the other handshake exchanges
(TAG_IDENT, etc.) as well as the messages themselves.
5. The reconnect behavior for stateful connections is a separate
exchange. This keeps the stateless connections free of clutter.
6. A few changes in the auth_none and cephx integratoin will be needed.
For example, all the current stubs assume that authentication happens over
MAuth message and authorization happens in an authorizer blob in
ceph_msg_connect. Now both are part of TAG_AUTH_REQUEST, so we'll need to
multiplex the cephx message blobs. Also, because the IDENT exchanges
happens later, we may need to pass additional info in the auth handshake
messages (like the peer type, or whatever else is needed).
7. Lots of messages can go either way, and I tried ot avoid a strict
request/response model so that things could be pipelined, and we'd spend a
minimal amount of time waiting for a response from the other end. For
example,
C:
initiates connection
S:
accepts connection
-> banner
-> TAG_AUTH_METHODS
C:
-> banner
-> TAG_AUTH_SET_METHOD
-> TAG_AUTH_AUTH_REQUEST
S:
-> TAG_AUTH_REPLY
C:
-> TAG_ENCRYPT_BEGIN
-> TAG_IDENT
-> TAG_SIGNATURE
S:
-> TAG_ENCRYPT_BEGIN
-> TAG_IDENT
-> TAG_SIGNATURE
C:
-> TAG_START
-> TAG_SIGNATURE
-> TAG_MSG
-> TAG_SIGNATURE
...
S:
-> TAG_MSG
-> TAG_SIGNATURE
...
Comments, please! The exhange is a bit less structured as far as who
sends what message, with the idea that we could pipeline a lot of it, but
it may end up being too ambiguous. Let me know what you think...
sage
next reply other threads:[~2016-05-26 18:17 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-05-26 18:17 Sage Weil [this message]
2016-05-27 4:41 ` msgr2 protocol Haomai Wang
2016-05-27 4:45 ` Haomai Wang
2016-05-27 8:28 ` Marcus Watts
2016-05-27 17:33 ` Sage Weil
2016-05-27 17:28 ` Sage Weil
2016-05-27 9:44 ` Yehuda Sadeh-Weinraub
2016-05-27 17:37 ` Sage Weil
2016-05-28 18:19 ` Yehuda Sadeh-Weinraub
2016-06-02 15:43 ` Sage Weil
2016-06-02 15:59 ` Haomai Wang
2016-06-02 16:35 ` Sage Weil
2016-06-02 18:11 ` Gregory Farnum
2016-06-02 18:24 ` Sage Weil
2016-06-02 18:34 ` Gregory Farnum
2016-06-03 13:11 ` Sage Weil
2016-06-03 13:24 ` Sage Weil
2016-06-03 16:47 ` Haomai Wang
2016-06-03 17:33 ` Sage Weil
2016-06-03 17:35 ` Haomai Wang
2016-06-06 8:23 ` Junwang Zhao
2016-06-10 8:31 ` Marcus Watts
2016-06-10 10:11 ` Sage Weil
2016-06-10 10:48 ` Sage Weil
2016-06-06 20:16 ` Gregory Farnum
2016-06-10 11:04 ` Sage Weil
2016-06-10 19:05 ` Marcus Watts
2016-06-10 21:15 ` Sage Weil
2016-06-10 21:22 ` Gregory Farnum
2016-06-11 23:05 ` Marcus Watts
2016-06-12 23:59 ` Sage Weil
[not found] ` <CACJqLyax_SXEZp3vA2_wR+CdwKOo2Re=SsK2xfXqmXjz9d8iNw@mail.gmail.com>
2016-09-09 21:14 ` Sage Weil
[not found] ` <CACJqLyYwKZ5_1OHR_5=+mr=1ED2Nt34x4TB29j5dE1D+NjzFpg@mail.gmail.com>
2016-09-10 14:43 ` Haomai Wang
2016-09-11 17:05 ` Sage Weil
2016-09-12 2:29 ` Haomai Wang
2016-09-12 13:21 ` Sage Weil
2016-09-13 0:03 ` Gregory Farnum
2016-09-13 1:35 ` Haomai Wang
2016-09-13 13:21 ` Sage Weil
2016-09-13 11:50 ` Jeff Layton
2016-09-13 11:18 ` Jeff Layton
2016-09-13 13:31 ` Sage Weil
2016-09-13 14:48 ` Jeff Layton
2016-09-13 15:10 ` Sage Weil
2016-09-13 20:07 ` Gregory Farnum
2016-06-02 18:16 ` Gregory Farnum
2016-06-29 11:59 Avner Ben Hanoch
2016-06-29 16:52 ` Yehuda Sadeh-Weinraub
2016-06-30 11:59 ` Avner Ben Hanoch
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.11.1605261358330.6221@cpach.fuggernut.com \
--to=sweil@redhat.com \
--cc=ceph-devel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.