All of lore.kernel.org
 help / color / mirror / Atom feed
* Fwd: Testing Results - Add a tool for using the new sysfs files - rpcctl
       [not found] <CA+-cMpFmMr8FQKODmR5JAB8rZhzptZ_KPX5DasLM_sbvbko+GA@mail.gmail.com>
@ 2022-02-01 17:13 ` Rahul Rathore
  2022-02-07 16:03   ` Anna Schumaker
  0 siblings, 1 reply; 4+ messages in thread
From: Rahul Rathore @ 2022-02-01 17:13 UTC (permalink / raw)
  To: linux-nfs, Anna.Schumaker

[-- Attachment #1: Type: text/plain, Size: 6474 bytes --]

Hello Ana,

I have done some more testing.

Kindly look into it.

Setup

NFS Server IP:192.168.122.127
NFS Client IP:192.168.122.125


1- Transport Viewing

# ss
Netid     State      Recv-Q     Send-Q                             Local
Address:Port                 Peer Address:Port       Process
tcp       ESTAB        0          0
192.168.122.125:872                192.168.122.127:nfs


[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt
xprt 1: tcp, 192.168.122.127, port 0, state <CONNECTED,BOUND>, main
Source: (enoent), port 872, Requests: 2
Congestion: cur 0, win 256, Slots: min 2, max 65536
Queues: binding 0, sending 0, pending 0, backlog 0, tasks 0



Here port 0 is seen for Remote which is wrong. It should be nfs(2049).

And I guess the name is also wrong. it should not  be enoent. It should be
ens3.


2- I made the NIC down on the Server. And can see call traces as in the
attached image. (taken from console so can't paste here)

3- By client I understand RPC Client and if I tune the value of
tcp_slot_table_entries I should see an increase in number of RPC Client as
I would increase parallel connection of RPC Clients.

# ./tools/rpcctl/rpcctl.py client
client 0: switch 0, xprts 1, active 1, queue 0
xprt 1: tcp, 192.168.122.127 [main]
client 3: switch 0, xprts 1, active 1, queue 0
xprt 1: tcp, 192.168.122.127 [main]
# sysctl -a | grep tcp_slot_table_entries
sunrpc.tcp_slot_table_entries = 2

[root@rrathore-upstream-sysfs nfs-utils]# sysctl -w
sunrpc.tcp_slot_table_entries=8
sunrpc.tcp_slot_table_entries = 8
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# sysctl -a | grep
tcp_slot_table_entries
sunrpc.tcp_slot_table_entries = 8
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py client
<----------------------- Change in value of tcp_slot_table_entries doesn't
seem to have an effect
client 0: switch 0, xprts 1, active 1, queue 0
xprt 1: tcp, 192.168.122.127 [main]
client 3: switch 0, xprts 1, active 1, queue 0
xprt 1: tcp, 192.168.122.127 [main]

Later I started nfs and used this server as an NFS Server, then I could see
an increase in number.

# ./tools/rpcctl/rpcctl.py client
client 0: switch 0, xprts 1, active 1, queue 0
xprt 1: tcp, 192.168.122.127 [main]
client 1: switch 1, xprts 1, active 1, queue 0
xprt 0: local, /var/run/rpcbind.sock [main]
client 2: switch 1, xprts 1, active 1, queue 0
xprt 0: local, /var/run/rpcbind.sock [main]
client 3: switch 0, xprts 1, active 1, queue 0
xprt 1: tcp, 192.168.122.127 [main]
client 4: switch 2, xprts 1, active 1, queue 0
xprt 2: local, /var/run/gssproxy.sock [main]
client 5: switch 3, xprts 1, active 1, queue 0
xprt 3: tcp, 192.168.122.29 [main]


4- I am not sure if I am making a mistake or if it's the error due to which
value is not getting set.

[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt
xprt 0: local, /var/run/rpcbind.sock, port 0, state <CONNECTED,BOUND>, main
Source: (enoent), port 0, Requests: 8
Congestion: cur 0, win 256, Slots: min 8, max 65536
Queues: binding 0, sending 0, pending 0, backlog 0, tasks 0
xprt 1: tcp, 192.168.122.127, port 0, state <CONNECTED,BOUND>, main
Source: (enoent), port 813, Requests: 2
Congestion: cur 0, win 256, Slots: min 2, max 65536
Queues: binding 0, sending 0, pending 0, backlog 0, tasks 0
xprt 2: local, /var/run/gssproxy.sock, port 0, state <CONNECTED,BOUND>, main
Source: (enoent), port 0, Requests: 8
Congestion: cur 0, win 256, Slots: min 8, max 65536
Queues: binding 0, sending 0, pending 0, backlog 0, tasks 0
xprt 3: tcp, 192.168.122.29, port 0, state <CONNECTED,BOUND>, main
Source: (enoent), port 0, Requests: 8
Congestion: cur 0, win 256, Slots: min 8, max 8
Queues: binding 0, sending 0, pending 0, backlog 0, tasks 0

*None of the operation work*

[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
-h
usage: rpcctl.py xprt set [-h] --id ID [--dstaddr dstaddr] [--offline]
[--online] [--remove]

options:
  -h, --help         show this help message and exit
  --id ID            Id of a specific xprt to modify
  --dstaddr dstaddr  New dstaddr to set
  --offline          Set an xprt offline
  --online           Set an offline xprt back online
  --remove           Remove an xprt
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 3 --offline
[Errno 22] Invalid argument
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 3 192.168.122.29 --offline
usage: rpcctl.py [-h] {client,switch,xprt} ...
rpcctl.py: error: unrecognized arguments: 192.168.122.29
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 3 --dstaddr 192.168.122.29 --offline
[Errno 95] Operation not supported
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 3 --dstaddr 192.168.122.29 --online
[Errno 95] Operation not supported
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 3 --dstaddr 192.168.122.29 --remove
[Errno 95] Operation not supported
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 1 --dstaddr 192.168.122.127 --offline
[Errno 22] Invalid argument
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 1 --offline
[Errno 22] Invalid argument


5-* And it's similar If I do with switch*

[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py switch
switch 0: xprts 1, active 1, queue 0
xprt 1: tcp, 192.168.122.127 [main]
switch 1: xprts 1, active 1, queue 0
xprt 0: local, /var/run/rpcbind.sock [main]
switch 2: xprts 1, active 1, queue 0
xprt 2: local, /var/run/gssproxy.sock [main]
switch 3: xprts 1, active 1, queue 0
xprt 3: tcp, 192.168.122.29 [main]

[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py switch
set --id 3 --dstaddr 192.168.122.30
[Errno 95] Operation not supported
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py switch
set --id 3 --dstaddr 192.168.122.29
[Errno 95] Operation not supported

Regards,
Rahul

[-- Attachment #2: Screenshot from 2022-01-29 14-52-42.png --]
[-- Type: image/png, Size: 42307 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread
* Re: Testing Results - Add a tool for using the new sysfs files - rpcctl
  2022-02-01 17:13 ` Fwd: Testing Results - Add a tool for using the new sysfs files - rpcctl Rahul Rathore
@ 2022-02-07 16:03   ` Anna Schumaker
       [not found]     ` <CA+-cMpG8pbARdWSHyQG0mcg8ZJi6UntZSJk8555+OE5Ra5C2aw@mail.gmail.com>
  0 siblings, 1 reply; 4+ messages in thread
From: Anna Schumaker @ 2022-02-07 16:03 UTC (permalink / raw)
  To: Rahul Rathore; +Cc: Linux NFS Mailing List, Schumaker Anna

Hi Rahul,

Thanks for testing!

On Sat, Feb 5, 2022 at 7:19 AM Rahul Rathore <rrathore@redhat.com> wrote:
>
> Hello Ana,
>
> I have done some more testing.
>
> Kindly look into it.
>
> Setup
>
> NFS Server IP:192.168.122.127
> NFS Client IP:192.168.122.125
>
>
> 1- Transport Viewing
>
> # ss
> Netid     State      Recv-Q     Send-Q                             Local
> Address:Port                 Peer Address:Port       Process
> tcp       ESTAB        0          0
> 192.168.122.125:872                192.168.122.127:nfs
>
>
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt
> xprt 1: tcp, 192.168.122.127, port 0, state <CONNECTED,BOUND>, main
> Source: (enoent), port 872, Requests: 2
> Congestion: cur 0, win 256, Slots: min 2, max 65536
> Queues: binding 0, sending 0, pending 0, backlog 0, tasks 0
>
>
>
> Here port 0 is seen for Remote which is wrong. It should be nfs(2049).

Can I ask what kernel you are running? This was a kernel-side issue
that was fixed in v5.15 with this patch:

commit 5d46dd04cb68771f77ba66dbf6fd323a4a2ce00d
Author: Anna Schumaker <Anna.Schumaker@Netapp.com>
Date:   Tue Jul 20 16:04:42 2021 -0400

    sunrpc: Fix return value of get_srcport()

    Since bc1c56e9bbe9 transport->srcport may by unset, causing
    get_srcport() to return 0 when called. Fix this by querying the port
    from the underlying socket instead of the transport.

    Fixes: bc1c56e9bbe9 (SUNRPC: prevent port reuse on transports
which don't request it)
    Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>

>
> And I guess the name is also wrong. it should not  be enoent. It should be
> ens3.

enoent in this case means the file it was trying to read doesn't
exist. Probably you need this patch (also in v5.15):

commit e44773daf851dc2755144355723c1c305e7246a1
Author: Anna Schumaker <Anna.Schumaker@Netapp.com>
Date:   Thu Jul 29 16:45:23 2021 -0400

    SUNRPC: Add srcaddr as a file in sysfs

    I don't support changing it right now, but it could be useful
    information for clients with multiple network cards.

    Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>

>
>
> 2- I made the NIC down on the Server. And can see call traces as in the
> attached image. (taken from console so can't paste here)

I think this was the same bug that was fixed with this patch (which
went into v5.15-rc3):

commit 17f09d3f619a7ad2d2b021b4e5246f08225b1b0f
Author: Anna Schumaker <Anna.Schumaker@Netapp.com>
Date:   Thu Oct 28 15:17:41 2021 -0400

    SUNRPC: Check if the xprt is connected before handling sysfs reads

    xprts don't immediately reconnect when changing the "dstaddr" property,
    instead this gets handled the next time an operation uses the transport.
    This could lead to NULL pointer dereferences when trying to read sysfs
    files between the disconnect and reconnect operations. Fix this by
    returning an error if the xprt is not connected.

    Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
    Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>

>
> 3- By client I understand RPC Client and if I tune the value of
> tcp_slot_table_entries I should see an increase in number of RPC Client as
> I would increase parallel connection of RPC Clients.

I don't remember offhand how using the tcp_slot_table_entries works
for increasing RPC clients, so there might be some other trigger
you're missing. Try using the nconnect=N mount option instead, since
extra connections are set up at mount time.

>
> # ./tools/rpcctl/rpcctl.py client
> client 0: switch 0, xprts 1, active 1, queue 0
> xprt 1: tcp, 192.168.122.127 [main]
> client 3: switch 0, xprts 1, active 1, queue 0
> xprt 1: tcp, 192.168.122.127 [main]
> # sysctl -a | grep tcp_slot_table_entries
> sunrpc.tcp_slot_table_entries = 2
>
> [root@rrathore-upstream-sysfs nfs-utils]# sysctl -w
> sunrpc.tcp_slot_table_entries=8
> sunrpc.tcp_slot_table_entries = 8
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# sysctl -a | grep
> tcp_slot_table_entries
> sunrpc.tcp_slot_table_entries = 8
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py client
> <----------------------- Change in value of tcp_slot_table_entries doesn't
> seem to have an effect
> client 0: switch 0, xprts 1, active 1, queue 0
> xprt 1: tcp, 192.168.122.127 [main]
> client 3: switch 0, xprts 1, active 1, queue 0
> xprt 1: tcp, 192.168.122.127 [main]
>
> Later I started nfs and used this server as an NFS Server, then I could see
> an increase in number.
>
> # ./tools/rpcctl/rpcctl.py client
> client 0: switch 0, xprts 1, active 1, queue 0
> xprt 1: tcp, 192.168.122.127 [main]
> client 1: switch 1, xprts 1, active 1, queue 0
> xprt 0: local, /var/run/rpcbind.sock [main]
> client 2: switch 1, xprts 1, active 1, queue 0
> xprt 0: local, /var/run/rpcbind.sock [main]
> client 3: switch 0, xprts 1, active 1, queue 0
> xprt 1: tcp, 192.168.122.127 [main]
> client 4: switch 2, xprts 1, active 1, queue 0
> xprt 2: local, /var/run/gssproxy.sock [main]
> client 5: switch 3, xprts 1, active 1, queue 0
> xprt 3: tcp, 192.168.122.29 [main]
>
>
> 4- I am not sure if I am making a mistake or if it's the error due to which
> value is not getting set.
>
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt
> xprt 0: local, /var/run/rpcbind.sock, port 0, state <CONNECTED,BOUND>, main
> Source: (enoent), port 0, Requests: 8
> Congestion: cur 0, win 256, Slots: min 8, max 65536
> Queues: binding 0, sending 0, pending 0, backlog 0, tasks 0
> xprt 1: tcp, 192.168.122.127, port 0, state <CONNECTED,BOUND>, main
> Source: (enoent), port 813, Requests: 2
> Congestion: cur 0, win 256, Slots: min 2, max 65536
> Queues: binding 0, sending 0, pending 0, backlog 0, tasks 0
> xprt 2: local, /var/run/gssproxy.sock, port 0, state <CONNECTED,BOUND>, main
> Source: (enoent), port 0, Requests: 8
> Congestion: cur 0, win 256, Slots: min 8, max 65536
> Queues: binding 0, sending 0, pending 0, backlog 0, tasks 0
> xprt 3: tcp, 192.168.122.29, port 0, state <CONNECTED,BOUND>, main
> Source: (enoent), port 0, Requests: 8
> Congestion: cur 0, win 256, Slots: min 8, max 8
> Queues: binding 0, sending 0, pending 0, backlog 0, tasks 0
>
> *None of the operation work*
>
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> -h
> usage: rpcctl.py xprt set [-h] --id ID [--dstaddr dstaddr] [--offline]
> [--online] [--remove]
>
> options:
>   -h, --help         show this help message and exit
>   --id ID            Id of a specific xprt to modify
>   --dstaddr dstaddr  New dstaddr to set
>   --offline          Set an xprt offline
>   --online           Set an offline xprt back online
>   --remove           Remove an xprt
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 3 --offline
> [Errno 22] Invalid argument
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 3 192.168.122.29 --offline
> usage: rpcctl.py [-h] {client,switch,xprt} ...
> rpcctl.py: error: unrecognized arguments: 192.168.122.29
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 3 --dstaddr 192.168.122.29 --offline
> [Errno 95] Operation not supported
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 3 --dstaddr 192.168.122.29 --online
> [Errno 95] Operation not supported
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 3 --dstaddr 192.168.122.29 --remove
> [Errno 95] Operation not supported
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 1 --dstaddr 192.168.122.127 --offline
> [Errno 22] Invalid argument
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 1 --offline
> [Errno 22] Invalid argument

The commands themselves look okay. Can you update to a kernel that has
the other fixes and let me know if it's still a problem?

Thanks,
Anna

>
>
> 5-* And it's similar If I do with switch*
>
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py switch
> switch 0: xprts 1, active 1, queue 0
> xprt 1: tcp, 192.168.122.127 [main]
> switch 1: xprts 1, active 1, queue 0
> xprt 0: local, /var/run/rpcbind.sock [main]
> switch 2: xprts 1, active 1, queue 0
> xprt 2: local, /var/run/gssproxy.sock [main]
> switch 3: xprts 1, active 1, queue 0
> xprt 3: tcp, 192.168.122.29 [main]
>
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py switch
> set --id 3 --dstaddr 192.168.122.30
> [Errno 95] Operation not supported
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py switch
> set --id 3 --dstaddr 192.168.122.29
> [Errno 95] Operation not supported
>
> Regards,
> Rahul

^ permalink raw reply	[flat|nested] 4+ messages in thread
* Re: Testing Results - Add a tool for using the new sysfs files - rpcctl
       [not found]       ` <CA+-cMpGC5r7poAjQ65Tm97cEcyjMoZUzYCdfBnD-CziS-yKsOA@mail.gmail.com>
@ 2022-02-09 17:20         ` Rahul Rathore
  2022-02-10 17:06           ` Rahul Rathore
  0 siblings, 1 reply; 4+ messages in thread
From: Rahul Rathore @ 2022-02-09 17:20 UTC (permalink / raw)
  To: Anna Schumaker, Linux NFS Mailing List

Hello Anna,

I am yet to do more tests.

However, I will share what I have done till now.

I have upgraded to 5.16.5-200.fc35.x86_64 and some of the problems have
disappeared.

However, some still exist.

1- I am still unable to run xprt commands.

[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 3 --offline
[Errno 22] Invalid argument
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 3 192.168.122.29 --offline
usage: rpcctl.py [-h] {client,switch,xprt} ...
rpcctl.py: error: unrecognized arguments: 192.168.122.29
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 3 --dstaddr 192.168.122.29 --offline
[Errno 22] Invalid argument
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 3 192.168.122.127 --offline
usage: rpcctl.py [-h] {client,switch,xprt} ...
rpcctl.py: error: unrecognized arguments: 192.168.122.127
[root@rrathore-upstream-sysfs nfs-utils]#
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
--id 1 --dstaddr 192.168.122.29 --offline
[Errno 95] Operation not supported
[root@rrathore-upstream-sysfs nfs-utils]#

If I am doing something wrong, kindly provide some examples.

2- However, the switch command worked.

[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py switch
set --id 2 --dstaddr 192.168.122.30
switch 2: xprts 1, active 1, queue 1
xprt 3: tcp, 192.168.122.30
[root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py switch
switch 0: xprts 1, active 1, queue 0
xprt 0: local, /var/run/rpcbind.sock [main]
switch 1: xprts 1, active 1, queue 0
xprt 1: local, /var/run/gssproxy.sock [main]
switch 2: xprts 1, active 1, queue 1
xprt 3: tcp, 192.168.122.30

Now I see:-
[root@rrathore-upstream-sysfs nfs-utils]# ss | grep -i nfs
tcp   SYN-SENT 0      1                        192.168.122.125:883
192.168.122.30:nfs

This is picking correct info:-

cat /sys/kernel/debug/sunrpc/rpc_clnt/*/xprt/info
addr:  192.168.122.30
port:  2049
state: 0x15
netid: tcp
addr:  192.168.122.30
port:  2049
state: 0x15


I am not sure of the motive of this command. Is the motive to set IP of NFS
Server to set/change from Client.

Though the current O/P and all is correct. But in this manner NFS will
suffer. cd / ls or any other operation over nfs will hang like below:-

[root@rrathore-upstream-sysfs nfs-utils]# df -h

^C

If the motive was to set NFS Server IP, then it fails to do so as my NFS
Server IP is still an old one and not 192.168.122.30.


Just a suggestion, if you can post some examples in your man page it will
be great.


I am still performing some tests. Will keep you posted.

Regards,
Rahul


^ permalink raw reply	[flat|nested] 4+ messages in thread
* Re: Testing Results - Add a tool for using the new sysfs files - rpcctl
  2022-02-09 17:20         ` Rahul Rathore
@ 2022-02-10 17:06           ` Rahul Rathore
  0 siblings, 0 replies; 4+ messages in thread
From: Rahul Rathore @ 2022-02-10 17:06 UTC (permalink / raw)
  To: Anna Schumaker, Linux NFS Mailing List, Schumaker Anna
  Cc: David Wysochanski, Gonzalo Siero Humet

Hello Anna,

Hope you are well.

Yes , now making the NIC down doesn't show any call traces.

However, I am adding to Yesterday's mail.

When my NFS Server IP is 192.168.122.127, it allowed to change switch
to 192.168.122.30. However, nfs connection then was in syn_sent and
nfs was not responding.

Now, if I set my IP back to 192.168.122.127, the command simply hangs.
I am attaching the last 10 lines of strace.
# strace -T -tt -f -v -s 4096 -o /tmp/out.txt ./tools/rpcctl/rpcctl.py
switch set --id 2 --dstaddr 192.168.122.127


18937 11:28:58.225547 close(3)          = 0 <0.000010>
18937 11:28:58.225608 mprotect(0x7f7c83128000, 4096, PROT_READ) = 0 <0.000015>
18937 11:28:58.225993 brk(0x55d7df53d000) = 0x55d7df53d000 <0.000029>
18937 11:28:58.227163 openat(AT_FDCWD,
"/sys/kernel/sunrpc/xprt-switches/switch-2/xprt-3-tcp/dstaddr",
O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3 <0.000030>
18937 11:28:58.227241 newfstatat(3, "", {st_dev=makedev(0, 0x17),
st_ino=25021, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0,
st_blksize=4096, st_blocks=0, st_size=4096, st_atime=1644251948 /*
2022-02-07T11:39:08.446981173-0500 */, st_atime_nsec=446981173,
st_mtime=1644510538 /* 2022-02-10T11:28:58.225300241-0500 */,
st_mtime_nsec=225300241, st_ctime=1644510538 /*
2022-02-10T11:28:58.225300241-0500 */, st_ctime_nsec=225300241},
AT_EMPTY_PATH) = 0 <0.000013>
18937 11:28:58.227349 ioctl(3, TCGETS, 0x7ffe44427570) = -1 ENOTTY
(Inappropriate ioctl for device) <0.000016>
18937 11:28:58.227407 lseek(3, 0, SEEK_CUR) = 0 <0.000010>
18937 11:28:58.227449 ioctl(3, TCGETS, 0x7ffe44427390) = -1 ENOTTY
(Inappropriate ioctl for device) <0.000010>
18937 11:28:58.227503 lseek(3, 0, SEEK_CUR) = 0 <0.000010>
18937 11:28:58.227551 write(3, "192.168.122.127",
15[root@rrathore-upstream-sysfs ~]#
[root@rrathore-upstream-sysfs ~]#
[root@rrathore-upstream-sysfs ~]# tail /tmp/out.txt
18937 11:28:58.225547 close(3)          = 0 <0.000010>
18937 11:28:58.225608 mprotect(0x7f7c83128000, 4096, PROT_READ) = 0 <0.000015>
18937 11:28:58.225993 brk(0x55d7df53d000) = 0x55d7df53d000 <0.000029>
18937 11:28:58.227163 openat(AT_FDCWD,
"/sys/kernel/sunrpc/xprt-switches/switch-2/xprt-3-tcp/dstaddr",
O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3 <0.000030>
18937 11:28:58.227241 newfstatat(3, "", {st_dev=makedev(0, 0x17),
st_ino=25021, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0,
st_blksize=4096, st_blocks=0, st_size=4096, st_atime=1644251948 /*
2022-02-07T11:39:08.446981173-0500 */, st_atime_nsec=446981173,
st_mtime=1644510538 /* 2022-02-10T11:28:58.225300241-0500 */,
st_mtime_nsec=225300241, st_ctime=1644510538 /*
2022-02-10T11:28:58.225300241-0500 */, st_ctime_nsec=225300241},
AT_EMPTY_PATH) = 0 <0.000013>
18937 11:28:58.227349 ioctl(3, TCGETS, 0x7ffe44427570) = -1 ENOTTY
(Inappropriate ioctl for device) <0.000016>
18937 11:28:58.227407 lseek(3, 0, SEEK_CUR) = 0 <0.000010>
18937 11:28:58.227449 ioctl(3, TCGETS, 0x7ffe44427390) = -1 ENOTTY
(Inappropriate ioctl for device) <0.000010>
18937 11:28:58.227503 lseek(3, 0, SEEK_CUR) = 0 <0.000010>
18937 11:28:58.227551 write(3, "192.168.122.127", 15


So in the new Kernel I see issues with xprt set which I sent yesterday
and IP doesn't set to previous for switch which I explained above.

Regards,
Rahul


On Wed, Feb 9, 2022 at 10:50 PM Rahul Rathore <rrathore@redhat.com> wrote:
>
> Hello Anna,
>
> I am yet to do more tests.
>
> However, I will share what I have done till now.
>
> I have upgraded to 5.16.5-200.fc35.x86_64 and some of the problems have
> disappeared.
>
> However, some still exist.
>
> 1- I am still unable to run xprt commands.
>
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 3 --offline
> [Errno 22] Invalid argument
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 3 192.168.122.29 --offline
> usage: rpcctl.py [-h] {client,switch,xprt} ...
> rpcctl.py: error: unrecognized arguments: 192.168.122.29
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 3 --dstaddr 192.168.122.29 --offline
> [Errno 22] Invalid argument
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 3 192.168.122.127 --offline
> usage: rpcctl.py [-h] {client,switch,xprt} ...
> rpcctl.py: error: unrecognized arguments: 192.168.122.127
> [root@rrathore-upstream-sysfs nfs-utils]#
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py xprt set
> --id 1 --dstaddr 192.168.122.29 --offline
> [Errno 95] Operation not supported
> [root@rrathore-upstream-sysfs nfs-utils]#
>
> If I am doing something wrong, kindly provide some examples.
>
> 2- However, the switch command worked.
>
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py switch
> set --id 2 --dstaddr 192.168.122.30
> switch 2: xprts 1, active 1, queue 1
> xprt 3: tcp, 192.168.122.30
> [root@rrathore-upstream-sysfs nfs-utils]# ./tools/rpcctl/rpcctl.py switch
> switch 0: xprts 1, active 1, queue 0
> xprt 0: local, /var/run/rpcbind.sock [main]
> switch 1: xprts 1, active 1, queue 0
> xprt 1: local, /var/run/gssproxy.sock [main]
> switch 2: xprts 1, active 1, queue 1
> xprt 3: tcp, 192.168.122.30
>
> Now I see:-
> [root@rrathore-upstream-sysfs nfs-utils]# ss | grep -i nfs
> tcp   SYN-SENT 0      1                        192.168.122.125:883
> 192.168.122.30:nfs
>
> This is picking correct info:-
>
> cat /sys/kernel/debug/sunrpc/rpc_clnt/*/xprt/info
> addr:  192.168.122.30
> port:  2049
> state: 0x15
> netid: tcp
> addr:  192.168.122.30
> port:  2049
> state: 0x15
>
>
> I am not sure of the motive of this command. Is the motive to set IP of NFS
> Server to set/change from Client.
>
> Though the current O/P and all is correct. But in this manner NFS will
> suffer. cd / ls or any other operation over nfs will hang like below:-
>
> [root@rrathore-upstream-sysfs nfs-utils]# df -h
>
> ^C
>
> If the motive was to set NFS Server IP, then it fails to do so as my NFS
> Server IP is still an old one and not 192.168.122.30.
>
>
> Just a suggestion, if you can post some examples in your man page it will
> be great.
>
>
> I am still performing some tests. Will keep you posted.
>
> Regards,
> Rahul



-- 

Thanks & Regards,

Rahul Rathore

STSE| Global Support Services
Red Hat India Private Limited, Pune
Ext: 8367125| Mob-9651798912 |IRC: rrathore


^ permalink raw reply	[flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-02-10 17:06 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <CA+-cMpFmMr8FQKODmR5JAB8rZhzptZ_KPX5DasLM_sbvbko+GA@mail.gmail.com>
2022-02-01 17:13 ` Fwd: Testing Results - Add a tool for using the new sysfs files - rpcctl Rahul Rathore
2022-02-07 16:03   ` Anna Schumaker
     [not found]     ` <CA+-cMpG8pbARdWSHyQG0mcg8ZJi6UntZSJk8555+OE5Ra5C2aw@mail.gmail.com>
     [not found]       ` <CA+-cMpGC5r7poAjQ65Tm97cEcyjMoZUzYCdfBnD-CziS-yKsOA@mail.gmail.com>
2022-02-09 17:20         ` Rahul Rathore
2022-02-10 17:06           ` Rahul Rathore

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.