* Re: [SPDK] Multiple start/stop of reactor and add/remove of bdev
@ 2018-10-09 7:47 Shahar Salzman
0 siblings, 0 replies; 7+ messages in thread
From: Shahar Salzman @ 2018-10-09 7:47 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 5332 bytes --]
Hi,
My use case is a distributed highly available storage appliance with features such as dedup, snapshots etc.
We have both FC and iSCSI targets implemented in the kernel, and basically using the same cores as SPDK for receiving IO. Reactor polling comes at a cost which I pay happily as long as there is work for the NVMeF target. If the customer does not use the system with NVMeF, this is just taking away from FC/iSCSI performance.
With the patches bellow, I can start/stop without tearing down memory etc.
It should be noted that I am not using the spdk app, since I require this fine granularity, so I embedded the init/run/destroy flow in our app.
Shahar
________________________________
From: SPDK <spdk-bounces(a)lists.01.org> on behalf of Harris, James R <james.r.harris(a)intel.com>
Sent: Tuesday, October 9, 2018 3:28:13 AM
To: Storage Performance Development Kit
Subject: Re: [SPDK] Multiple start/stop of reactor and add/remove of bdev
Hi Michael,
The reactors definitely have a cost if they are polling with no real work to do.
Ben Walker has been working on a patch series [1] that would effectively decouple and “spdk_thread” from an “spdk_reactor”. Currently the relationship is always one-to-one. Vishal Verma pushed some patches earlier this year [2] that track how much real work a reactor is doing by tracking whether a poller did any real work. Combining these two would enable dynamic load balancing. It sounds like that might meet the vhost use case you’ve described.
This is why I’m interested in understanding more about Shahar’s request to start and stop reactors and reinitialize the bdev layer, and whether it is also based on a desire to do this kind of load balancing.
Thanks,
-Jim
[1] Patch series starts here: https://review.gerrithub.io/#/c/spdk/spdk/+/417784/
[2] Main patch is: https://review.gerrithub.io/#/c/spdk/spdk/+/412695/
On 10/8/18, 12:24 PM, "SPDK on behalf of Michael Haeuptle" <spdk-bounces(a)lists.01.org on behalf of michaelhaeuptle(a)gmail.com> wrote:
I can't comment on Shahar's use case but it would be helpful in cases where
I want to flex out reactors based on the # of devices which are not fixed.
In my use case, I don't really want to have reactors running if I don't
need them since they do incur some cost as far as I understand it (maybe
max_delay_us could be used).
For example, it may be good enough to have 1 reactor serving 2
vhost_scsi_controllers starting out. However, if I need to add more
controllers (since I want to add more devices), then I'd like to start
another reactor for the new vhost_scsi_controllers. Reducing controllers is
tricky but I'm not sure this happens very often in my use case.
-- Michael
On Mon, Oct 8, 2018 at 11:00 AM Harris, James R <james.r.harris(a)intel.com>
wrote:
> Hi Shahar,
>
> Can you describe why you require starting/stopping the reactor on demand?
>
> Thanks,
>
> -Jim
>
>
>
>
> On 10/8/18, 1:07 AM, "SPDK on behalf of Shahar Salzman" <
> spdk-bounces(a)lists.01.org on behalf of shahar.salzman(a)kaminario.com>
> wrote:
>
> Hi,
>
>
> I am now working with my passthrough bdev directly from my
> application, and require adding/removing them dynamically. In addition, I
> also require starting/stopping the reactor on demand without tearing down
> the entire system (e.g. dpdk stays initialized). This mostly works out of
> the box, but there are a few globals and bdev internal fields which require
> re-init or tear down.
>
> In addition, the copy engine did not io_unregister itself, so I also
> added this in the patchset, allowing me the following application life span:
>
>
> - env init (init config, spdk_env_init)
>
> - reactors init
>
> - while (application is alive)
>
> subsystem_init + reactor start
>
> ...
>
> Do some NVMeF stuff
>
> ...
>
> subsysem_fini + reactors stop
>
> - reactors_fini
>
> - rpc_finish
>
>
>
> I submitted the following patches for review:
>
> https://review.gerrithub.io/c/spdk/spdk/+/428305
>
> https://review.gerrithub.io/c/spdk/spdk/+/428304
>
> https://review.gerrithub.io/c/spdk/spdk/+/428303
>
> https://review.gerrithub.io/c/spdk/spdk/+/428302
>
>
> WDYT?
>
>
> Shahar
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SPDK] Multiple start/stop of reactor and add/remove of bdev
@ 2018-10-09 21:15 Harris, James R
0 siblings, 0 replies; 7+ messages in thread
From: Harris, James R @ 2018-10-09 21:15 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 6532 bytes --]
Hi Shahar,
Thanks for the clarifications – specifically that you are embedding the library init/fini calls directly in your app.
I looked at your patches – overall they seem reasonable. I’ve posted some comments on them in GerritHub – we should be able to get them merged pretty easily.
Thanks,
-Jim
On 10/9/18, 12:47 AM, "SPDK on behalf of Shahar Salzman" <spdk-bounces(a)lists.01.org on behalf of shahar.salzman(a)kaminario.com> wrote:
Hi,
My use case is a distributed highly available storage appliance with features such as dedup, snapshots etc.
We have both FC and iSCSI targets implemented in the kernel, and basically using the same cores as SPDK for receiving IO. Reactor polling comes at a cost which I pay happily as long as there is work for the NVMeF target. If the customer does not use the system with NVMeF, this is just taking away from FC/iSCSI performance.
With the patches bellow, I can start/stop without tearing down memory etc.
It should be noted that I am not using the spdk app, since I require this fine granularity, so I embedded the init/run/destroy flow in our app.
Shahar
________________________________
From: SPDK <spdk-bounces(a)lists.01.org> on behalf of Harris, James R <james.r.harris(a)intel.com>
Sent: Tuesday, October 9, 2018 3:28:13 AM
To: Storage Performance Development Kit
Subject: Re: [SPDK] Multiple start/stop of reactor and add/remove of bdev
Hi Michael,
The reactors definitely have a cost if they are polling with no real work to do.
Ben Walker has been working on a patch series [1] that would effectively decouple and “spdk_thread” from an “spdk_reactor”. Currently the relationship is always one-to-one. Vishal Verma pushed some patches earlier this year [2] that track how much real work a reactor is doing by tracking whether a poller did any real work. Combining these two would enable dynamic load balancing. It sounds like that might meet the vhost use case you’ve described.
This is why I’m interested in understanding more about Shahar’s request to start and stop reactors and reinitialize the bdev layer, and whether it is also based on a desire to do this kind of load balancing.
Thanks,
-Jim
[1] Patch series starts here: https://review.gerrithub.io/#/c/spdk/spdk/+/417784/
[2] Main patch is: https://review.gerrithub.io/#/c/spdk/spdk/+/412695/
On 10/8/18, 12:24 PM, "SPDK on behalf of Michael Haeuptle" <spdk-bounces(a)lists.01.org on behalf of michaelhaeuptle(a)gmail.com> wrote:
I can't comment on Shahar's use case but it would be helpful in cases where
I want to flex out reactors based on the # of devices which are not fixed.
In my use case, I don't really want to have reactors running if I don't
need them since they do incur some cost as far as I understand it (maybe
max_delay_us could be used).
For example, it may be good enough to have 1 reactor serving 2
vhost_scsi_controllers starting out. However, if I need to add more
controllers (since I want to add more devices), then I'd like to start
another reactor for the new vhost_scsi_controllers. Reducing controllers is
tricky but I'm not sure this happens very often in my use case.
-- Michael
On Mon, Oct 8, 2018 at 11:00 AM Harris, James R <james.r.harris(a)intel.com>
wrote:
> Hi Shahar,
>
> Can you describe why you require starting/stopping the reactor on demand?
>
> Thanks,
>
> -Jim
>
>
>
>
> On 10/8/18, 1:07 AM, "SPDK on behalf of Shahar Salzman" <
> spdk-bounces(a)lists.01.org on behalf of shahar.salzman(a)kaminario.com>
> wrote:
>
> Hi,
>
>
> I am now working with my passthrough bdev directly from my
> application, and require adding/removing them dynamically. In addition, I
> also require starting/stopping the reactor on demand without tearing down
> the entire system (e.g. dpdk stays initialized). This mostly works out of
> the box, but there are a few globals and bdev internal fields which require
> re-init or tear down.
>
> In addition, the copy engine did not io_unregister itself, so I also
> added this in the patchset, allowing me the following application life span:
>
>
> - env init (init config, spdk_env_init)
>
> - reactors init
>
> - while (application is alive)
>
> subsystem_init + reactor start
>
> ...
>
> Do some NVMeF stuff
>
> ...
>
> subsysem_fini + reactors stop
>
> - reactors_fini
>
> - rpc_finish
>
>
>
> I submitted the following patches for review:
>
> https://review.gerrithub.io/c/spdk/spdk/+/428305
>
> https://review.gerrithub.io/c/spdk/spdk/+/428304
>
> https://review.gerrithub.io/c/spdk/spdk/+/428303
>
> https://review.gerrithub.io/c/spdk/spdk/+/428302
>
>
> WDYT?
>
>
> Shahar
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SPDK] Multiple start/stop of reactor and add/remove of bdev
@ 2018-10-09 13:58 Michael Haeuptle
0 siblings, 0 replies; 7+ messages in thread
From: Michael Haeuptle @ 2018-10-09 13:58 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 5366 bytes --]
Hi Jim,
thanks for the reply! These patches actually solve another issue that we're
currently having. We have a cooperative threading environment and I had to
modify the current reactor loop to insert a yield to give other threads a
chance to run. Making the poller and message processing functions
accessible allows me to write our own reactor loop more easily.
As you stated correctly, my use case is to load balance based on the
current # of devices that are configured by a user, which changes over
time. Once the patches are merged, I need to take a closer look and do some
experiments.
-- Michael
On Mon, Oct 8, 2018 at 6:28 PM Harris, James R <james.r.harris(a)intel.com>
wrote:
> Hi Michael,
>
> The reactors definitely have a cost if they are polling with no real work
> to do.
>
> Ben Walker has been working on a patch series [1] that would effectively
> decouple and “spdk_thread” from an “spdk_reactor”. Currently the
> relationship is always one-to-one. Vishal Verma pushed some patches
> earlier this year [2] that track how much real work a reactor is doing by
> tracking whether a poller did any real work. Combining these two would
> enable dynamic load balancing. It sounds like that might meet the vhost
> use case you’ve described.
>
> This is why I’m interested in understanding more about Shahar’s request to
> start and stop reactors and reinitialize the bdev layer, and whether it is
> also based on a desire to do this kind of load balancing.
>
> Thanks,
>
> -Jim
>
> [1] Patch series starts here:
> https://review.gerrithub.io/#/c/spdk/spdk/+/417784/
> [2] Main patch is: https://review.gerrithub.io/#/c/spdk/spdk/+/412695/
>
> On 10/8/18, 12:24 PM, "SPDK on behalf of Michael Haeuptle" <
> spdk-bounces(a)lists.01.org on behalf of michaelhaeuptle(a)gmail.com> wrote:
>
> I can't comment on Shahar's use case but it would be helpful in cases
> where
> I want to flex out reactors based on the # of devices which are not
> fixed.
>
> In my use case, I don't really want to have reactors running if I don't
> need them since they do incur some cost as far as I understand it
> (maybe
> max_delay_us could be used).
>
> For example, it may be good enough to have 1 reactor serving 2
> vhost_scsi_controllers starting out. However, if I need to add more
> controllers (since I want to add more devices), then I'd like to start
> another reactor for the new vhost_scsi_controllers. Reducing
> controllers is
> tricky but I'm not sure this happens very often in my use case.
>
> -- Michael
>
>
>
> On Mon, Oct 8, 2018 at 11:00 AM Harris, James R <
> james.r.harris(a)intel.com>
> wrote:
>
> > Hi Shahar,
> >
> > Can you describe why you require starting/stopping the reactor on
> demand?
> >
> > Thanks,
> >
> > -Jim
> >
> >
> >
> >
> > On 10/8/18, 1:07 AM, "SPDK on behalf of Shahar Salzman" <
> > spdk-bounces(a)lists.01.org on behalf of shahar.salzman(a)kaminario.com>
> > wrote:
> >
> > Hi,
> >
> >
> > I am now working with my passthrough bdev directly from my
> > application, and require adding/removing them dynamically. In
> addition, I
> > also require starting/stopping the reactor on demand without tearing
> down
> > the entire system (e.g. dpdk stays initialized). This mostly works
> out of
> > the box, but there are a few globals and bdev internal fields which
> require
> > re-init or tear down.
> >
> > In addition, the copy engine did not io_unregister itself, so I
> also
> > added this in the patchset, allowing me the following application
> life span:
> >
> >
> > - env init (init config, spdk_env_init)
> >
> > - reactors init
> >
> > - while (application is alive)
> >
> > subsystem_init + reactor start
> >
> > ...
> >
> > Do some NVMeF stuff
> >
> > ...
> >
> > subsysem_fini + reactors stop
> >
> > - reactors_fini
> >
> > - rpc_finish
> >
> >
> >
> > I submitted the following patches for review:
> >
> > https://review.gerrithub.io/c/spdk/spdk/+/428305
> >
> > https://review.gerrithub.io/c/spdk/spdk/+/428304
> >
> > https://review.gerrithub.io/c/spdk/spdk/+/428303
> >
> > https://review.gerrithub.io/c/spdk/spdk/+/428302
> >
> >
> > WDYT?
> >
> >
> > Shahar
> > _______________________________________________
> > SPDK mailing list
> > SPDK(a)lists.01.org
> > https://lists.01.org/mailman/listinfo/spdk
> >
> >
> > _______________________________________________
> > SPDK mailing list
> > SPDK(a)lists.01.org
> > https://lists.01.org/mailman/listinfo/spdk
> >
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SPDK] Multiple start/stop of reactor and add/remove of bdev
@ 2018-10-09 0:28 Harris, James R
0 siblings, 0 replies; 7+ messages in thread
From: Harris, James R @ 2018-10-09 0:28 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 4251 bytes --]
Hi Michael,
The reactors definitely have a cost if they are polling with no real work to do.
Ben Walker has been working on a patch series [1] that would effectively decouple and “spdk_thread” from an “spdk_reactor”. Currently the relationship is always one-to-one. Vishal Verma pushed some patches earlier this year [2] that track how much real work a reactor is doing by tracking whether a poller did any real work. Combining these two would enable dynamic load balancing. It sounds like that might meet the vhost use case you’ve described.
This is why I’m interested in understanding more about Shahar’s request to start and stop reactors and reinitialize the bdev layer, and whether it is also based on a desire to do this kind of load balancing.
Thanks,
-Jim
[1] Patch series starts here: https://review.gerrithub.io/#/c/spdk/spdk/+/417784/
[2] Main patch is: https://review.gerrithub.io/#/c/spdk/spdk/+/412695/
On 10/8/18, 12:24 PM, "SPDK on behalf of Michael Haeuptle" <spdk-bounces(a)lists.01.org on behalf of michaelhaeuptle(a)gmail.com> wrote:
I can't comment on Shahar's use case but it would be helpful in cases where
I want to flex out reactors based on the # of devices which are not fixed.
In my use case, I don't really want to have reactors running if I don't
need them since they do incur some cost as far as I understand it (maybe
max_delay_us could be used).
For example, it may be good enough to have 1 reactor serving 2
vhost_scsi_controllers starting out. However, if I need to add more
controllers (since I want to add more devices), then I'd like to start
another reactor for the new vhost_scsi_controllers. Reducing controllers is
tricky but I'm not sure this happens very often in my use case.
-- Michael
On Mon, Oct 8, 2018 at 11:00 AM Harris, James R <james.r.harris(a)intel.com>
wrote:
> Hi Shahar,
>
> Can you describe why you require starting/stopping the reactor on demand?
>
> Thanks,
>
> -Jim
>
>
>
>
> On 10/8/18, 1:07 AM, "SPDK on behalf of Shahar Salzman" <
> spdk-bounces(a)lists.01.org on behalf of shahar.salzman(a)kaminario.com>
> wrote:
>
> Hi,
>
>
> I am now working with my passthrough bdev directly from my
> application, and require adding/removing them dynamically. In addition, I
> also require starting/stopping the reactor on demand without tearing down
> the entire system (e.g. dpdk stays initialized). This mostly works out of
> the box, but there are a few globals and bdev internal fields which require
> re-init or tear down.
>
> In addition, the copy engine did not io_unregister itself, so I also
> added this in the patchset, allowing me the following application life span:
>
>
> - env init (init config, spdk_env_init)
>
> - reactors init
>
> - while (application is alive)
>
> subsystem_init + reactor start
>
> ...
>
> Do some NVMeF stuff
>
> ...
>
> subsysem_fini + reactors stop
>
> - reactors_fini
>
> - rpc_finish
>
>
>
> I submitted the following patches for review:
>
> https://review.gerrithub.io/c/spdk/spdk/+/428305
>
> https://review.gerrithub.io/c/spdk/spdk/+/428304
>
> https://review.gerrithub.io/c/spdk/spdk/+/428303
>
> https://review.gerrithub.io/c/spdk/spdk/+/428302
>
>
> WDYT?
>
>
> Shahar
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SPDK] Multiple start/stop of reactor and add/remove of bdev
@ 2018-10-08 19:24 Michael Haeuptle
0 siblings, 0 replies; 7+ messages in thread
From: Michael Haeuptle @ 2018-10-08 19:24 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 2609 bytes --]
I can't comment on Shahar's use case but it would be helpful in cases where
I want to flex out reactors based on the # of devices which are not fixed.
In my use case, I don't really want to have reactors running if I don't
need them since they do incur some cost as far as I understand it (maybe
max_delay_us could be used).
For example, it may be good enough to have 1 reactor serving 2
vhost_scsi_controllers starting out. However, if I need to add more
controllers (since I want to add more devices), then I'd like to start
another reactor for the new vhost_scsi_controllers. Reducing controllers is
tricky but I'm not sure this happens very often in my use case.
-- Michael
On Mon, Oct 8, 2018 at 11:00 AM Harris, James R <james.r.harris(a)intel.com>
wrote:
> Hi Shahar,
>
> Can you describe why you require starting/stopping the reactor on demand?
>
> Thanks,
>
> -Jim
>
>
>
>
> On 10/8/18, 1:07 AM, "SPDK on behalf of Shahar Salzman" <
> spdk-bounces(a)lists.01.org on behalf of shahar.salzman(a)kaminario.com>
> wrote:
>
> Hi,
>
>
> I am now working with my passthrough bdev directly from my
> application, and require adding/removing them dynamically. In addition, I
> also require starting/stopping the reactor on demand without tearing down
> the entire system (e.g. dpdk stays initialized). This mostly works out of
> the box, but there are a few globals and bdev internal fields which require
> re-init or tear down.
>
> In addition, the copy engine did not io_unregister itself, so I also
> added this in the patchset, allowing me the following application life span:
>
>
> - env init (init config, spdk_env_init)
>
> - reactors init
>
> - while (application is alive)
>
> subsystem_init + reactor start
>
> ...
>
> Do some NVMeF stuff
>
> ...
>
> subsysem_fini + reactors stop
>
> - reactors_fini
>
> - rpc_finish
>
>
>
> I submitted the following patches for review:
>
> https://review.gerrithub.io/c/spdk/spdk/+/428305
>
> https://review.gerrithub.io/c/spdk/spdk/+/428304
>
> https://review.gerrithub.io/c/spdk/spdk/+/428303
>
> https://review.gerrithub.io/c/spdk/spdk/+/428302
>
>
> WDYT?
>
>
> Shahar
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SPDK] Multiple start/stop of reactor and add/remove of bdev
@ 2018-10-08 17:00 Harris, James R
0 siblings, 0 replies; 7+ messages in thread
From: Harris, James R @ 2018-10-08 17:00 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 1662 bytes --]
Hi Shahar,
Can you describe why you require starting/stopping the reactor on demand?
Thanks,
-Jim
On 10/8/18, 1:07 AM, "SPDK on behalf of Shahar Salzman" <spdk-bounces(a)lists.01.org on behalf of shahar.salzman(a)kaminario.com> wrote:
Hi,
I am now working with my passthrough bdev directly from my application, and require adding/removing them dynamically. In addition, I also require starting/stopping the reactor on demand without tearing down the entire system (e.g. dpdk stays initialized). This mostly works out of the box, but there are a few globals and bdev internal fields which require re-init or tear down.
In addition, the copy engine did not io_unregister itself, so I also added this in the patchset, allowing me the following application life span:
- env init (init config, spdk_env_init)
- reactors init
- while (application is alive)
subsystem_init + reactor start
...
Do some NVMeF stuff
...
subsysem_fini + reactors stop
- reactors_fini
- rpc_finish
I submitted the following patches for review:
https://review.gerrithub.io/c/spdk/spdk/+/428305
https://review.gerrithub.io/c/spdk/spdk/+/428304
https://review.gerrithub.io/c/spdk/spdk/+/428303
https://review.gerrithub.io/c/spdk/spdk/+/428302
WDYT?
Shahar
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
^ permalink raw reply [flat|nested] 7+ messages in thread
* [SPDK] Multiple start/stop of reactor and add/remove of bdev
@ 2018-10-08 8:07 Shahar Salzman
0 siblings, 0 replies; 7+ messages in thread
From: Shahar Salzman @ 2018-10-08 8:07 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 1069 bytes --]
Hi,
I am now working with my passthrough bdev directly from my application, and require adding/removing them dynamically. In addition, I also require starting/stopping the reactor on demand without tearing down the entire system (e.g. dpdk stays initialized). This mostly works out of the box, but there are a few globals and bdev internal fields which require re-init or tear down.
In addition, the copy engine did not io_unregister itself, so I also added this in the patchset, allowing me the following application life span:
- env init (init config, spdk_env_init)
- reactors init
- while (application is alive)
subsystem_init + reactor start
...
Do some NVMeF stuff
...
subsysem_fini + reactors stop
- reactors_fini
- rpc_finish
I submitted the following patches for review:
https://review.gerrithub.io/c/spdk/spdk/+/428305
https://review.gerrithub.io/c/spdk/spdk/+/428304
https://review.gerrithub.io/c/spdk/spdk/+/428303
https://review.gerrithub.io/c/spdk/spdk/+/428302
WDYT?
Shahar
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-10-09 21:15 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-10-09 7:47 [SPDK] Multiple start/stop of reactor and add/remove of bdev Shahar Salzman
-- strict thread matches above, loose matches on Subject: below --
2018-10-09 21:15 Harris, James R
2018-10-09 13:58 Michael Haeuptle
2018-10-09 0:28 Harris, James R
2018-10-08 19:24 Michael Haeuptle
2018-10-08 17:00 Harris, James R
2018-10-08 8:07 Shahar Salzman
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.