[SPDK] Re: Why two perf processes can't scale out?

[SPDK] Re: Why two perf processes can't scale out?

[645505358]

[-- Attachment #1: Type: text/plain, Size: 59 bytes --]

Thanks,  it is indeed because of the competition on core 0.

[SPDK] Re: Why two perf processes can't scale out?

[Harris, James R]

[-- Attachment #1: Type: text/plain, Size: 1030 bytes --]

Each perf process creates its own controller when connecting to the remote target.  Each controller has its own admin queue (qid 0).

-Jim


On 5/10/22, 11:52 PM, "645505358(a)qq.com" <645505358(a)qq.com> wrote:

    Hi,
    I have another question.
    I run two perf processes with the command line:
    "./build/examples/nvmf_perf -w write -q 1  -o 131072 -t 60 -r "trtype:RDMA adrfam:IPv4 traddr:192.168.2.86 trsvcid:5120 subnqn:nqn.2016-06.io.spdk:cnode2" and designate lcore with -c 0x5 in the second process.
    I found that both of them use the same qpair.
    Sepcifically, I print out the qpair ID in function nvme_ctrlr_create_io_qpair (i.e., qid = spdk_nvme_ctrlr_alloc_qid(ctrlr) ).
    Both of them use the qpair with ID equal to 0.
    So, why do they use the same qpair ? 
    Will this cause resource competition?

    Thanks.
    _______________________________________________
    SPDK mailing list -- spdk(a)lists.01.org
    To unsubscribe send an email to spdk-leave(a)lists.01.org

[SPDK] Re: Why two perf processes can't scale out?

[645505358]

[-- Attachment #1: Type: text/plain, Size: 119 bytes --]

Thank you so much.
It seems I mistakenly thought that the g_controllers variable was initialized only once for a node.

[SPDK] Re: Why two perf processes can't scale out?

[Cao, Gang]

[-- Attachment #1: Type: text/plain, Size: 1165 bytes --]

These are two separate SPDK processes (nvmf_perf) and running on different cores.

Each SPDK process will create its own qpair and have its own qpair ID. This resource is visible only for its own SPDK process.

-----Original Message-----
From: 645505358(a)qq.com <645505358(a)qq.com> 
Sent: Tuesday, May 10, 2022 23:44
To: spdk(a)lists.01.org
Subject: [SPDK] Re: Why two perf processes can't scale out?

Hi,
I have another question.
I run two perf processes with the command line:
"./build/examples/nvmf_perf -w write -q 1  -o 131072 -t 60 -r "trtype:RDMA adrfam:IPv4 traddr:192.168.2.86 trsvcid:5120 subnqn:nqn.2016-06.io.spdk:cnode2" and designate lcore with -c 0x5 in the second process.
I found that both of them use the same qpair.
Sepcifically, I print out the qpair ID in function nvme_ctrlr_create_io_qpair (i.e., qid = spdk_nvme_ctrlr_alloc_qid(ctrlr) ).
Both of them use the qpair with ID equal to 0.
So, why do they use the same qpair ? 
Will this cause resource competition?

Thanks.
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org

[SPDK] Re: Why two perf processes can't scale out?

[645505358]

[-- Attachment #1: Type: text/plain, Size: 601 bytes --]

Hi,
I have another question.
I run two perf processes with the command line:
"./build/examples/nvmf_perf -w write -q 1  -o 131072 -t 60 -r "trtype:RDMA adrfam:IPv4 traddr:192.168.2.86 trsvcid:5120 subnqn:nqn.2016-06.io.spdk:cnode2" and designate lcore with -c 0x5 in the second process.
I found that both of them use the same qpair.
Sepcifically, I print out the qpair ID in function nvme_ctrlr_create_io_qpair (i.e., qid = spdk_nvme_ctrlr_alloc_qid(ctrlr) ).
Both of them use the qpair with ID equal to 0.
So, why do they use the same qpair ? 
Will this cause resource competition?

Thanks.

[SPDK] Re: Why two perf processes can't scale out?

[Harris, James R]

[-- Attachment #1: Type: text/plain, Size: 1087 bytes --]

Hi,

Can you share exact details on how you are running the two perf processes?  Specifically are you using different core masks (-c option) for the two processes?

If not, this would be the competing resource.  By default, perf will run on core 0 only.

-Jim




On 5/9/22, 7:03 AM, "645505358(a)qq.com" <645505358(a)qq.com> wrote:

    Hi guys,
     The nvmf_tgt is running in one node and perf is running in another node.
    I first get the peak performance of nvmf_tgt by setting the io depth (-q) of perf to 64.
    And the peak performance is 4GB/s.
    Then I set the io depth of perf to 1 and found that the performance becomes 1GB/s.
    This is understandable to me.
    But when I run two perf with io depth equal to 1, the performance is shared by two perf processes (i.e., each has 500mb/s).
    so, what resources are they competing for so that performance cannot scale?
    Thanks.
    _______________________________________________
    SPDK mailing list -- spdk(a)lists.01.org
    To unsubscribe send an email to spdk-leave(a)lists.01.org