* QoS for iSCSI target?
@ 2016-03-10 22:24 Chris Friesen
2016-03-11 7:30 ` Christoph Hellwig
0 siblings, 1 reply; 12+ messages in thread
From: Chris Friesen @ 2016-03-10 22:24 UTC (permalink / raw)
To: Nicholas A. Bellinger, linux-scsi, target-devel
Hi,
I'm looking for information on whether the iSCSI target in the kernel offers any
way to do QoS between traffic driven by different initiators.
I'm trying to make sure that one initiator can't do a denial-of-service attack
against others.
Does the kernel target have this sort of thing built-in, or do I need to look at
network traffic-shaping to achieve this?
Thanks,
Chris
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: QoS for iSCSI target?
2016-03-10 22:24 QoS for iSCSI target? Chris Friesen
@ 2016-03-11 7:30 ` Christoph Hellwig
2016-03-11 7:45 ` Nicholas A. Bellinger
0 siblings, 1 reply; 12+ messages in thread
From: Christoph Hellwig @ 2016-03-11 7:30 UTC (permalink / raw)
To: Chris Friesen; +Cc: Nicholas A. Bellinger, linux-scsi, target-devel
On Thu, Mar 10, 2016 at 04:24:25PM -0600, Chris Friesen wrote:
> Hi,
>
> I'm looking for information on whether the iSCSI target in the kernel offers
> any way to do QoS between traffic driven by different initiators.
>
> I'm trying to make sure that one initiator can't do a denial-of-service
> attack against others.
>
> Does the kernel target have this sort of thing built-in, or do I need to
> look at network traffic-shaping to achieve this?
It doesn't right now, but it shouldn't be hard to integrate it with
blk cgroups.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: QoS for iSCSI target?
2016-03-11 7:30 ` Christoph Hellwig
@ 2016-03-11 7:45 ` Nicholas A. Bellinger
2016-03-16 16:48 ` Chris Friesen
0 siblings, 1 reply; 12+ messages in thread
From: Nicholas A. Bellinger @ 2016-03-11 7:45 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Chris Friesen, linux-scsi, target-devel
On Thu, 2016-03-10 at 23:30 -0800, Christoph Hellwig wrote:
> On Thu, Mar 10, 2016 at 04:24:25PM -0600, Chris Friesen wrote:
> > Hi,
> >
> > I'm looking for information on whether the iSCSI target in the kernel offers
> > any way to do QoS between traffic driven by different initiators.
> >
> > I'm trying to make sure that one initiator can't do a denial-of-service
> > attack against others.
> >
> > Does the kernel target have this sort of thing built-in, or do I need to
> > look at network traffic-shaping to achieve this?
>
> It doesn't right now, but it shouldn't be hard to integrate it with
> blk cgroups.
For iscsi-target application QoS, the per session command sequence
number window depth (CmdSN) exists to enforce per InitiatorName limits
via TPG default_cmdsn_depth + se_node_acl->queue_depth configfs
attributes.
Note these values can be changed on the fly for iscsi-target using
explicit se_node_acl->acl_group, but currently require a se_session
reinstatement event for updated ExpCmdSN + MaxCmdSN to take effect.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: QoS for iSCSI target?
2016-03-11 7:45 ` Nicholas A. Bellinger
@ 2016-03-16 16:48 ` Chris Friesen
2016-03-31 7:05 ` Nicholas A. Bellinger
0 siblings, 1 reply; 12+ messages in thread
From: Chris Friesen @ 2016-03-16 16:48 UTC (permalink / raw)
To: Nicholas A. Bellinger, Christoph Hellwig; +Cc: linux-scsi, target-devel
On 03/11/2016 01:45 AM, Nicholas A. Bellinger wrote:
> On Thu, 2016-03-10 at 23:30 -0800, Christoph Hellwig wrote:
>> On Thu, Mar 10, 2016 at 04:24:25PM -0600, Chris Friesen wrote:
>>> Hi,
>>>
>>> I'm looking for information on whether the iSCSI target in the kernel offers
>>> any way to do QoS between traffic driven by different initiators.
>>>
>>> I'm trying to make sure that one initiator can't do a denial-of-service
>>> attack against others.
>>>
>>> Does the kernel target have this sort of thing built-in, or do I need to
>>> look at network traffic-shaping to achieve this?
>>
>> It doesn't right now, but it shouldn't be hard to integrate it with
>> blk cgroups.
>
> For iscsi-target application QoS, the per session command sequence
> number window depth (CmdSN) exists to enforce per InitiatorName limits
> via TPG default_cmdsn_depth + se_node_acl->queue_depth configfs
> attributes.
>
> Note these values can be changed on the fly for iscsi-target using
> explicit se_node_acl->acl_group, but currently require a se_session
> reinstatement event for updated ExpCmdSN + MaxCmdSN to take effect.
On a slightly different note, is there any way to throttle or limit the overall
bandwidth consumed by the iSCSI target in the kernel? I'd like to ensure that
the iSCSI traffic doesn't completely swamp the host accesses to the same block
device.
I suppose I could do networking-based traffic shaping, but are there any
controls in the block IO subsystem?
Chris
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: QoS for iSCSI target?
2016-03-16 16:48 ` Chris Friesen
@ 2016-03-31 7:05 ` Nicholas A. Bellinger
2016-04-01 18:35 ` Chris Friesen
0 siblings, 1 reply; 12+ messages in thread
From: Nicholas A. Bellinger @ 2016-03-31 7:05 UTC (permalink / raw)
To: Chris Friesen; +Cc: Christoph Hellwig, linux-scsi, target-devel
On Wed, 2016-03-16 at 10:48 -0600, Chris Friesen wrote:
> On 03/11/2016 01:45 AM, Nicholas A. Bellinger wrote:
> > On Thu, 2016-03-10 at 23:30 -0800, Christoph Hellwig wrote:
> >> On Thu, Mar 10, 2016 at 04:24:25PM -0600, Chris Friesen wrote:
> >>> Hi,
> >>>
> >>> I'm looking for information on whether the iSCSI target in the kernel offers
> >>> any way to do QoS between traffic driven by different initiators.
> >>>
> >>> I'm trying to make sure that one initiator can't do a denial-of-service
> >>> attack against others.
> >>>
> >>> Does the kernel target have this sort of thing built-in, or do I need to
> >>> look at network traffic-shaping to achieve this?
> >>
> >> It doesn't right now, but it shouldn't be hard to integrate it with
> >> blk cgroups.
> >
> > For iscsi-target application QoS, the per session command sequence
> > number window depth (CmdSN) exists to enforce per InitiatorName limits
> > via TPG default_cmdsn_depth + se_node_acl->queue_depth configfs
> > attributes.
> >
> > Note these values can be changed on the fly for iscsi-target using
> > explicit se_node_acl->acl_group, but currently require a se_session
> > reinstatement event for updated ExpCmdSN + MaxCmdSN to take effect.
>
> On a slightly different note, is there any way to throttle or limit the overall
> bandwidth consumed by the iSCSI target in the kernel? I'd like to ensure that
> the iSCSI traffic doesn't completely swamp the host accesses to the same block
> device.
>
> I suppose I could do networking-based traffic shaping, but are there any
> controls in the block IO subsystem?
>
On a individual block_device backend basis, block cgroups is the
preferred method for doing this.
Note that any rate limiting imposed by block cgroups is subject to the
current se_node_acl->queue_depth enforced across all LUNs within a given
iscsi session.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: QoS for iSCSI target?
2016-03-31 7:05 ` Nicholas A. Bellinger
@ 2016-04-01 18:35 ` Chris Friesen
2016-04-03 1:15 ` Nicholas A. Bellinger
0 siblings, 1 reply; 12+ messages in thread
From: Chris Friesen @ 2016-04-01 18:35 UTC (permalink / raw)
To: Nicholas A. Bellinger; +Cc: Christoph Hellwig, linux-scsi, target-devel
On 03/31/2016 01:05 AM, Nicholas A. Bellinger wrote:
> On Wed, 2016-03-16 at 10:48 -0600, Chris Friesen wrote:
>> On 03/11/2016 01:45 AM, Nicholas A. Bellinger wrote:
>>> On Thu, 2016-03-10 at 23:30 -0800, Christoph Hellwig wrote:
>>>> On Thu, Mar 10, 2016 at 04:24:25PM -0600, Chris Friesen wrote:
>>>>> Hi,
>>>>>
>>>>> I'm looking for information on whether the iSCSI target in the kernel offers
>>>>> any way to do QoS between traffic driven by different initiators.
>>>>>
>>>>> I'm trying to make sure that one initiator can't do a denial-of-service
>>>>> attack against others.
>>>>>
>>>>> Does the kernel target have this sort of thing built-in, or do I need to
>>>>> look at network traffic-shaping to achieve this?
>>>>
>>>> It doesn't right now, but it shouldn't be hard to integrate it with
>>>> blk cgroups.
>>>
>>> For iscsi-target application QoS, the per session command sequence
>>> number window depth (CmdSN) exists to enforce per InitiatorName limits
>>> via TPG default_cmdsn_depth + se_node_acl->queue_depth configfs
>>> attributes.
>>>
>>> Note these values can be changed on the fly for iscsi-target using
>>> explicit se_node_acl->acl_group, but currently require a se_session
>>> reinstatement event for updated ExpCmdSN + MaxCmdSN to take effect.
>>
>> On a slightly different note, is there any way to throttle or limit the overall
>> bandwidth consumed by the iSCSI target in the kernel? I'd like to ensure that
>> the iSCSI traffic doesn't completely swamp the host accesses to the same block
>> device.
>>
>> I suppose I could do networking-based traffic shaping, but are there any
>> controls in the block IO subsystem?
>>
>
> On a individual block_device backend basis, block cgroups is the
> preferred method for doing this.
>
> Note that any rate limiting imposed by block cgroups is subject to the
> current se_node_acl->queue_depth enforced across all LUNs within a given
> iscsi session.
How would I use cgroups with the kernel iSCSI target? Is there a set of kernel
threads dedicated to the iSCSI target that I can assign to a particular group?
Chris
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: QoS for iSCSI target?
2016-04-01 18:35 ` Chris Friesen
@ 2016-04-03 1:15 ` Nicholas A. Bellinger
2016-04-04 15:20 ` Chris Friesen
0 siblings, 1 reply; 12+ messages in thread
From: Nicholas A. Bellinger @ 2016-04-03 1:15 UTC (permalink / raw)
To: Chris Friesen; +Cc: Christoph Hellwig, linux-scsi, target-devel
On Fri, 2016-04-01 at 12:35 -0600, Chris Friesen wrote:
> On 03/31/2016 01:05 AM, Nicholas A. Bellinger wrote:
> > On Wed, 2016-03-16 at 10:48 -0600, Chris Friesen wrote:
> >> On 03/11/2016 01:45 AM, Nicholas A. Bellinger wrote:
> >>> On Thu, 2016-03-10 at 23:30 -0800, Christoph Hellwig wrote:
> >>>> On Thu, Mar 10, 2016 at 04:24:25PM -0600, Chris Friesen wrote:
> >>>>> Hi,
> >>>>>
> >>>>> I'm looking for information on whether the iSCSI target in the kernel offers
> >>>>> any way to do QoS between traffic driven by different initiators.
> >>>>>
> >>>>> I'm trying to make sure that one initiator can't do a denial-of-service
> >>>>> attack against others.
> >>>>>
> >>>>> Does the kernel target have this sort of thing built-in, or do I need to
> >>>>> look at network traffic-shaping to achieve this?
> >>>>
> >>>> It doesn't right now, but it shouldn't be hard to integrate it with
> >>>> blk cgroups.
> >>>
> >>> For iscsi-target application QoS, the per session command sequence
> >>> number window depth (CmdSN) exists to enforce per InitiatorName limits
> >>> via TPG default_cmdsn_depth + se_node_acl->queue_depth configfs
> >>> attributes.
> >>>
> >>> Note these values can be changed on the fly for iscsi-target using
> >>> explicit se_node_acl->acl_group, but currently require a se_session
> >>> reinstatement event for updated ExpCmdSN + MaxCmdSN to take effect.
> >>
> >> On a slightly different note, is there any way to throttle or limit the overall
> >> bandwidth consumed by the iSCSI target in the kernel? I'd like to ensure that
> >> the iSCSI traffic doesn't completely swamp the host accesses to the same block
> >> device.
> >>
> >> I suppose I could do networking-based traffic shaping, but are there any
> >> controls in the block IO subsystem?
> >>
> >
> > On a individual block_device backend basis, block cgroups is the
> > preferred method for doing this.
> >
> > Note that any rate limiting imposed by block cgroups is subject to the
> > current se_node_acl->queue_depth enforced across all LUNs within a given
> > iscsi session.
>
>
> How would I use cgroups with the kernel iSCSI target? Is there a set of kernel
> threads dedicated to the iSCSI target that I can assign to a particular group?
>
block cgroups can set I/O throttling (bandwidth + IOPs) limits for any
normal struct block_device.
These values are configured via blkio.throttle.* resource class, and the
limits are imposed independently of the block_device's association with
target_core_mod backend driver export. Namely:
blkio.throttle.write_iops_device
blkio.throttle.read_iops_device
blkio.throttle.write_bps_device
blkio.throttle.read_bdp_device
Some examples using these values, plus more blkio.* info is here:
http://events.linuxfoundation.org/sites/events/files/slides/cgroups_0.pdf.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: QoS for iSCSI target?
2016-04-03 1:15 ` Nicholas A. Bellinger
@ 2016-04-04 15:20 ` Chris Friesen
2016-04-04 22:29 ` Nicholas A. Bellinger
0 siblings, 1 reply; 12+ messages in thread
From: Chris Friesen @ 2016-04-04 15:20 UTC (permalink / raw)
To: Nicholas A. Bellinger; +Cc: Christoph Hellwig, linux-scsi, target-devel
On 04/02/2016 07:15 PM, Nicholas A. Bellinger wrote:
> On Fri, 2016-04-01 at 12:35 -0600, Chris Friesen wrote:
>>>> On a slightly different note, is there any way to throttle or limit the overall
>>>> bandwidth consumed by the iSCSI target in the kernel? I'd like to ensure that
>>>> the iSCSI traffic doesn't completely swamp the host accesses to the same block
>>>> device.
>>>>
>>>> I suppose I could do networking-based traffic shaping, but are there any
>>>> controls in the block IO subsystem?
>>>>
>>>
>>> On a individual block_device backend basis, block cgroups is the
>>> preferred method for doing this.
>>>
>>> Note that any rate limiting imposed by block cgroups is subject to the
>>> current se_node_acl->queue_depth enforced across all LUNs within a given
>>> iscsi session.
>>
>>
>> How would I use cgroups with the kernel iSCSI target? Is there a set of kernel
>> threads dedicated to the iSCSI target that I can assign to a particular group?
>>
>
> block cgroups can set I/O throttling (bandwidth + IOPs) limits for any
> normal struct block_device.
>
> These values are configured via blkio.throttle.* resource class, and the
> limits are imposed independently of the block_device's association with
> target_core_mod backend driver export. Namely:
>
> blkio.throttle.write_iops_device
> blkio.throttle.read_iops_device
> blkio.throttle.write_bps_device
> blkio.throttle.read_bdp_device
>
> Some examples using these values, plus more blkio.* info is here:
> http://events.linuxfoundation.org/sites/events/files/slides/cgroups_0.pdf.
>
I understand how to set up the cgroups, but don't I need to ensure that the IO
operations happen in the context of a particular cgroup? If so, how do I ensure
that the in-kernel iSCSI target traffic happens in the context of the specified
group? Are there a set of kernel threads dedicated to processing iSCSI requests?
Chris
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: QoS for iSCSI target?
2016-04-04 15:20 ` Chris Friesen
@ 2016-04-04 22:29 ` Nicholas A. Bellinger
2016-04-04 23:01 ` Chris Friesen
0 siblings, 1 reply; 12+ messages in thread
From: Nicholas A. Bellinger @ 2016-04-04 22:29 UTC (permalink / raw)
To: Chris Friesen; +Cc: Christoph Hellwig, linux-scsi, target-devel
On Mon, 2016-04-04 at 09:20 -0600, Chris Friesen wrote:
> On 04/02/2016 07:15 PM, Nicholas A. Bellinger wrote:
> > On Fri, 2016-04-01 at 12:35 -0600, Chris Friesen wrote:
>
> >>>> On a slightly different note, is there any way to throttle or limit the overall
> >>>> bandwidth consumed by the iSCSI target in the kernel? I'd like to ensure that
> >>>> the iSCSI traffic doesn't completely swamp the host accesses to the same block
> >>>> device.
> >>>>
> >>>> I suppose I could do networking-based traffic shaping, but are there any
> >>>> controls in the block IO subsystem?
> >>>>
> >>>
> >>> On a individual block_device backend basis, block cgroups is the
> >>> preferred method for doing this.
> >>>
> >>> Note that any rate limiting imposed by block cgroups is subject to the
> >>> current se_node_acl->queue_depth enforced across all LUNs within a given
> >>> iscsi session.
> >>
> >>
> >> How would I use cgroups with the kernel iSCSI target? Is there a set of kernel
> >> threads dedicated to the iSCSI target that I can assign to a particular group?
> >>
> >
> > block cgroups can set I/O throttling (bandwidth + IOPs) limits for any
> > normal struct block_device.
> >
> > These values are configured via blkio.throttle.* resource class, and the
> > limits are imposed independently of the block_device's association with
> > target_core_mod backend driver export. Namely:
> >
> > blkio.throttle.write_iops_device
> > blkio.throttle.read_iops_device
> > blkio.throttle.write_bps_device
> > blkio.throttle.read_bdp_device
> >
> > Some examples using these values, plus more blkio.* info is here:
> > http://events.linuxfoundation.org/sites/events/files/slides/cgroups_0.pdf.
> >
>
> I understand how to set up the cgroups, but don't I need to ensure that the IO
> operations happen in the context of a particular cgroup? If so, how do I ensure
> that the in-kernel iSCSI target traffic happens in the context of the specified
> group? Are there a set of kernel threads dedicated to processing iSCSI requests?
>
block cgroups does I/O ratelimiting at struct block_device level.
Eg: The process cgroup (which AFAICT is what your thinking about) is
separate from block cgroups, and doesn't need to be explicitly enabled
in order for block cgroup to function.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: QoS for iSCSI target?
2016-04-04 22:29 ` Nicholas A. Bellinger
@ 2016-04-04 23:01 ` Chris Friesen
2016-04-05 0:25 ` Nicholas A. Bellinger
0 siblings, 1 reply; 12+ messages in thread
From: Chris Friesen @ 2016-04-04 23:01 UTC (permalink / raw)
To: Nicholas A. Bellinger; +Cc: Christoph Hellwig, linux-scsi, target-devel
On 04/04/2016 04:29 PM, Nicholas A. Bellinger wrote:
> On Mon, 2016-04-04 at 09:20 -0600, Chris Friesen wrote:
>> On 04/02/2016 07:15 PM, Nicholas A. Bellinger wrote:
>>> On Fri, 2016-04-01 at 12:35 -0600, Chris Friesen wrote:
>>
>>>>>> On a slightly different note, is there any way to throttle or limit the overall
>>>>>> bandwidth consumed by the iSCSI target in the kernel? I'd like to ensure that
>>>>>> the iSCSI traffic doesn't completely swamp the host accesses to the same block
>>>>>> device.
>>>>>>
>>>>>> I suppose I could do networking-based traffic shaping, but are there any
>>>>>> controls in the block IO subsystem?
>>>>>>
>>>>>
>>>>> On a individual block_device backend basis, block cgroups is the
>>>>> preferred method for doing this.
>>>>>
>>>>> Note that any rate limiting imposed by block cgroups is subject to the
>>>>> current se_node_acl->queue_depth enforced across all LUNs within a given
>>>>> iscsi session.
>>>>
>>>>
>>>> How would I use cgroups with the kernel iSCSI target? Is there a set of kernel
>>>> threads dedicated to the iSCSI target that I can assign to a particular group?
>>>>
>>>
>>> block cgroups can set I/O throttling (bandwidth + IOPs) limits for any
>>> normal struct block_device.
>>>
>>> These values are configured via blkio.throttle.* resource class, and the
>>> limits are imposed independently of the block_device's association with
>>> target_core_mod backend driver export. Namely:
>>>
>>> blkio.throttle.write_iops_device
>>> blkio.throttle.read_iops_device
>>> blkio.throttle.write_bps_device
>>> blkio.throttle.read_bdp_device
>>>
>>> Some examples using these values, plus more blkio.* info is here:
>>> http://events.linuxfoundation.org/sites/events/files/slides/cgroups_0.pdf.
>>>
>>
>> I understand how to set up the cgroups, but don't I need to ensure that the IO
>> operations happen in the context of a particular cgroup? If so, how do I ensure
>> that the in-kernel iSCSI target traffic happens in the context of the specified
>> group? Are there a set of kernel threads dedicated to processing iSCSI requests?
>>
>
> block cgroups does I/O ratelimiting at struct block_device level.
>
> Eg: The process cgroup (which AFAICT is what your thinking about) is
> separate from block cgroups, and doesn't need to be explicitly enabled
> in order for block cgroup to function.
>
I'm still confused.
I'm not trying to globally throttle IO on a particular block device. I'm trying
to control how much IO the iSCSI target in the kernel is allowed to drive on a
particular block device.
The goal is to ensure that the iSCSI target does not consume all of the
available bandwidth for a particular block device. I want to ensure that some
of the bandwidth for that device is available to other processes on the host
(for management purposes) rather than being consumed by a greedy iSCSI initiator.
In an ideal world I would like a set of rules that say things like:
1) if there is contention, ensure that the host is guaranteed X percent of the
available /dev/sdb IOPS
2) if there is contention, do not allow the iSCSI target traffic to consume more
than Y percent of /dev/sdb's write traffic
Chris
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: QoS for iSCSI target?
2016-04-04 23:01 ` Chris Friesen
@ 2016-04-05 0:25 ` Nicholas A. Bellinger
2016-04-05 14:22 ` Chris Friesen
0 siblings, 1 reply; 12+ messages in thread
From: Nicholas A. Bellinger @ 2016-04-05 0:25 UTC (permalink / raw)
To: Chris Friesen; +Cc: Christoph Hellwig, linux-scsi, target-devel
On Mon, 2016-04-04 at 17:01 -0600, Chris Friesen wrote:
> On 04/04/2016 04:29 PM, Nicholas A. Bellinger wrote:
> > On Mon, 2016-04-04 at 09:20 -0600, Chris Friesen wrote:
> >> On 04/02/2016 07:15 PM, Nicholas A. Bellinger wrote:
> >>> On Fri, 2016-04-01 at 12:35 -0600, Chris Friesen wrote:
> >>
> >>>>>> On a slightly different note, is there any way to throttle or limit the overall
> >>>>>> bandwidth consumed by the iSCSI target in the kernel? I'd like to ensure that
> >>>>>> the iSCSI traffic doesn't completely swamp the host accesses to the same block
> >>>>>> device.
> >>>>>>
> >>>>>> I suppose I could do networking-based traffic shaping, but are there any
> >>>>>> controls in the block IO subsystem?
> >>>>>>
> >>>>>
> >>>>> On a individual block_device backend basis, block cgroups is the
> >>>>> preferred method for doing this.
> >>>>>
> >>>>> Note that any rate limiting imposed by block cgroups is subject to the
> >>>>> current se_node_acl->queue_depth enforced across all LUNs within a given
> >>>>> iscsi session.
> >>>>
> >>>>
> >>>> How would I use cgroups with the kernel iSCSI target? Is there a set of kernel
> >>>> threads dedicated to the iSCSI target that I can assign to a particular group?
> >>>>
> >>>
> >>> block cgroups can set I/O throttling (bandwidth + IOPs) limits for any
> >>> normal struct block_device.
> >>>
> >>> These values are configured via blkio.throttle.* resource class, and the
> >>> limits are imposed independently of the block_device's association with
> >>> target_core_mod backend driver export. Namely:
> >>>
> >>> blkio.throttle.write_iops_device
> >>> blkio.throttle.read_iops_device
> >>> blkio.throttle.write_bps_device
> >>> blkio.throttle.read_bdp_device
> >>>
> >>> Some examples using these values, plus more blkio.* info is here:
> >>> http://events.linuxfoundation.org/sites/events/files/slides/cgroups_0.pdf.
> >>>
> >>
> >> I understand how to set up the cgroups, but don't I need to ensure that the IO
> >> operations happen in the context of a particular cgroup? If so, how do I ensure
> >> that the in-kernel iSCSI target traffic happens in the context of the specified
> >> group? Are there a set of kernel threads dedicated to processing iSCSI requests?
> >>
> >
> > block cgroups does I/O ratelimiting at struct block_device level.
> >
> > Eg: The process cgroup (which AFAICT is what your thinking about) is
> > separate from block cgroups, and doesn't need to be explicitly enabled
> > in order for block cgroup to function.
> >
>
> I'm still confused.
>
> I'm not trying to globally throttle IO on a particular block device. I'm trying
> to control how much IO the iSCSI target in the kernel is allowed to drive on a
> particular block device.
>
> The goal is to ensure that the iSCSI target does not consume all of the
> available bandwidth for a particular block device. I want to ensure that some
> of the bandwidth for that device is available to other processes on the host
> (for management purposes) rather than being consumed by a greedy iSCSI initiator.
>
AFAIK, different traffic classes for a single block device is not
supported by block cgroups.
Also given target_core_iblock claims a given block_device for exclusive
access, you can't actually use the same block device for a mounted
filesystem, LVM, MD, etc, once it's been registered for target usage.
> In an ideal world I would like a set of rules that say things like:
> 1) if there is contention, ensure that the host is guaranteed X percent of the
> available /dev/sdb IOPS
> 2) if there is contention, do not allow the iSCSI target traffic to consume more
> than Y percent of /dev/sdb's write traffic
>
Yeah, that doesn't exist in standlone block groups.
You could probably use network cgroups to limit bandwidth at the socket
level for iscsi_target_mod, but that's going to be across all LUNs in a
session, and not at individual LUN level.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: QoS for iSCSI target?
2016-04-05 0:25 ` Nicholas A. Bellinger
@ 2016-04-05 14:22 ` Chris Friesen
0 siblings, 0 replies; 12+ messages in thread
From: Chris Friesen @ 2016-04-05 14:22 UTC (permalink / raw)
To: Nicholas A. Bellinger; +Cc: Christoph Hellwig, linux-scsi, target-devel
On 04/04/2016 06:25 PM, Nicholas A. Bellinger wrote:
> On Mon, 2016-04-04 at 17:01 -0600, Chris Friesen wrote:
>> I'm not trying to globally throttle IO on a particular block device. I'm trying
>> to control how much IO the iSCSI target in the kernel is allowed to drive on a
>> particular block device.
>>
>> The goal is to ensure that the iSCSI target does not consume all of the
>> available bandwidth for a particular block device. I want to ensure that some
>> of the bandwidth for that device is available to other processes on the host
>> (for management purposes) rather than being consumed by a greedy iSCSI initiator.
>>
>
> AFAIK, different traffic classes for a single block device is not
> supported by block cgroups.
>
> Also given target_core_iblock claims a given block_device for exclusive
> access, you can't actually use the same block device for a mounted
> filesystem, LVM, MD, etc, once it's been registered for target usage.
The issue we're running into is in the context of OpenStack Cinder. It layers
LVM on top of the bare block device, then creates an LVM volume per cinder
volume, and exports the logical volumes via iSCSI.
The problem we're seeing is that if one or more iSCSI initiators drive heavy
traffic on a slow disk, it can become very slow to access anything at all on the
bare block device (/dev/sdb or equivalent). This can cause the server-side
management code to time out on operations like making a new volume, checking
free disk space, etc.
>> In an ideal world I would like a set of rules that say things like:
>> 1) if there is contention, ensure that the host is guaranteed X percent of the
>> available /dev/sdb IOPS
>> 2) if there is contention, do not allow the iSCSI target traffic to consume more
>> than Y percent of /dev/sdb's write traffic
>>
>
> Yeah, that doesn't exist in standlone block groups.
>
> You could probably use network cgroups to limit bandwidth at the socket
> level for iscsi_target_mod, but that's going to be across all LUNs in a
> session, and not at individual LUN level.
Are there a set of dedicated kernel threads being used for the iSCSI target? Or
is it handled by generic softirq mechanisms?
Chris
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2016-04-05 14:22 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-03-10 22:24 QoS for iSCSI target? Chris Friesen
2016-03-11 7:30 ` Christoph Hellwig
2016-03-11 7:45 ` Nicholas A. Bellinger
2016-03-16 16:48 ` Chris Friesen
2016-03-31 7:05 ` Nicholas A. Bellinger
2016-04-01 18:35 ` Chris Friesen
2016-04-03 1:15 ` Nicholas A. Bellinger
2016-04-04 15:20 ` Chris Friesen
2016-04-04 22:29 ` Nicholas A. Bellinger
2016-04-04 23:01 ` Chris Friesen
2016-04-05 0:25 ` Nicholas A. Bellinger
2016-04-05 14:22 ` Chris Friesen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.