Re: [PATCH v1 0/8] sysfs files for multipath transport control

[Chuck Lever <chuck.lever@oracle.com>]

> On Mar 4, 2021, at 6:58 AM, Dan Aloni <dan@kernelim.com> wrote:
> 
> On Mon, Mar 01, 2021 at 10:56:22PM -0500, Olga Kornievskaia wrote:
>> Hi Dan,
>> 
>> On Mon, Feb 15, 2021 at 12:43 PM Dan Aloni <dan@kernelim.com> wrote:
>>> 
>>> Hi Anna,
>>> 
>>> This patchset builds ontop v2 of your 'sysfs files for changing IP' changeset.
>>> 
>>> - The patchset adds two more sysfs objects, for one for transport and another
>>>  for multipath.
>>> - Also, `net` renamed to `client`, and `client` now has symlink to its principal
>>>  transport. A client also has a symlink to its `multipath` object.
>>> - The transport interface lets you change `dstaddr` of individual transports,
>>>  when `nconnect` is used (or if it wasn't used and these were added with the
>>>  new interface).
>>> - The interface to add a transport is using a single string written to 'add',
>>>  for example:
>>> 
>>>       echo 'dstaddr 192.168.40.8 kind rdma' \
>>>> /sys/kernel/sunrpc/client/0/multipath/add
>>> 
>>> These changes are independent of the method used to obtain a sunrpc ID for a
>>> mountpoint. For that I've sent a concept patch showing an fspick-based
>>> implementation: https://marc.info/?l=linux-nfs&m=161332454821849&w=4
>> 
>> I'm confused: does this allow adding arbitrary connections between a
>> client and some server IP to an existing RPC client? Given the above
>> description, that's how it reads to me, can you clarify please. I
>> thought it was something specifically for v3 (because it has no
>> concept of trunking). As for NFSv4 there is a notion of getting server
>> locations via FS_LOCATION and doing trunking (ie multipathing)? I
>> don't see how this code restricts the addition of transports to v3.
> 
> Indeed, there's no restriction to NFSv3.
> 
> There can be potential uses for this for NFSv4 too. FS_LOCATIONS serving
> as recommendation to which hosts the client can connect, while smart
> load-balancing logic in userspace can determine to which subset of these
> servers each client in a cluster should actually connect (a full mesh
> is not always desired).
> 
> At any case, if this restriction is desired, we can add a new sunrpc
> client flag for that and pass it only in NFSv3 client init.

IMO an NFSv3-only policy should not be built into this API.

This is a user-space / kernel API, not something that is an administrative
interface. The administrative interface, which is the place to apply an
NFSv3-only policy, would use this sysfs API. So would smart load-
balancing logic based on fs_locations.

If the API is under /sys/kernel/sunrpc/client, then including NFS-specific
controls is a layering violation. Consider that the kernel can send
multiple protocols over the same connection (NFSv3 and NFSACL, eg).


--
Chuck Lever