[ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Sebastian Riemer]

Hello everyone,

I'm very proud to announce that we finally publish our SRP initiator
patches we've been working on for quite some time now.

In the first step we publish our way of failing IO fast as we've noticed
that the way Bart Van Assche does that in his GitHub repository doesn't
match our requirements completely.

His repo: https://github.com/bvanassche/ib_srp-backport
Our repo: https://github.com/sriemer/ib_srp

We want to fail IO fast in exactly the time we configure. With our
patches this works (or please tell us why not). We provide you with full
test descriptions and related shell scripts. Everything is done with as
little dependencies as possible.

The shell scripts can also be very useful to show how to configure and
use SRP with sysfs only. This is why I've added the scst-devel mailing
list here. We want to be as close as possible to the kernel.

We want to combine efforts here and to get valuable feedback from you
all. Evaluation, testing, criticism, comments, etc more than welcome!
Hopefully, we can get a really cool solution into the mainline together!
This would make my job as the maintainer for the ProfitBricks host
kernels a lot easier! ;-)

You'll notice: We've already adapted patches from Bart and parts of his
patches. So it is only fair to publish our patches as well. :-)

It is the same as with Bart's patches: This can't be used for
production, yet. Our patches don't have the reconnect for now. Ideas how
to implement that on top are welcome. Just send your patches directly to
me! :-)

Please further notice: There is a major bug in the upstream
multipath-tools. These read sysfs files cached which leads to IO on
offline devices. We've fixed it for us and publish the fix for you as
well. :-)

Git repo:
https://github.com/sriemer/multipath-tools

Thank you so much for your help in the past and in the future as well!
Thanks for the patience and reading this!

We'll continue publishing our SRP patches relevant for the mainline.
If you want to meet me or ProfitBricks in person, we'll have a booth on
LinuxTag in Berlin/Germany. I'll have a technical talk there about SRP:

http://www.linuxtag.org/2013/en/program/thursday-may-23-2013.html?eventid=208

Cheers,
Sebastian

-- 
Sebastian Riemer
Linux Kernel Developer - Storage

ProfitBricks GmbH • Greifswalder Str. 207 • 10405 Berlin, Germany
www.profitbricks.com • sebastian.riemer-EIkl63zCoXaH+58JC4qpiA@public.gmane.org

Sitz der Gesellschaft: Berlin
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B
Geschäftsführer: Andreas Gauger, Achim Weiss
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Vasiliy Tolstov]

Hello!
What is main difference between  bvanassche repo and sriemer ?

2013/4/12 Sebastian Riemer <sebastian.riemer-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>:
> Hello everyone,
>
> I'm very proud to announce that we finally publish our SRP initiator
> patches we've been working on for quite some time now.
>
> In the first step we publish our way of failing IO fast as we've noticed
> that the way Bart Van Assche does that in his GitHub repository doesn't
> match our requirements completely.
>
> His repo: https://github.com/bvanassche/ib_srp-backport
> Our repo: https://github.com/sriemer/ib_srp
>
> We want to fail IO fast in exactly the time we configure. With our
> patches this works (or please tell us why not). We provide you with full
> test descriptions and related shell scripts. Everything is done with as
> little dependencies as possible.
>
> The shell scripts can also be very useful to show how to configure and
> use SRP with sysfs only. This is why I've added the scst-devel mailing
> list here. We want to be as close as possible to the kernel.
>
> We want to combine efforts here and to get valuable feedback from you
> all. Evaluation, testing, criticism, comments, etc more than welcome!
> Hopefully, we can get a really cool solution into the mainline together!
> This would make my job as the maintainer for the ProfitBricks host
> kernels a lot easier! ;-)
>
> You'll notice: We've already adapted patches from Bart and parts of his
> patches. So it is only fair to publish our patches as well. :-)
>
> It is the same as with Bart's patches: This can't be used for
> production, yet. Our patches don't have the reconnect for now. Ideas how
> to implement that on top are welcome. Just send your patches directly to
> me! :-)
>
> Please further notice: There is a major bug in the upstream
> multipath-tools. These read sysfs files cached which leads to IO on
> offline devices. We've fixed it for us and publish the fix for you as
> well. :-)
>
> Git repo:
> https://github.com/sriemer/multipath-tools
>
> Thank you so much for your help in the past and in the future as well!
> Thanks for the patience and reading this!
>
> We'll continue publishing our SRP patches relevant for the mainline.
> If you want to meet me or ProfitBricks in person, we'll have a booth on
> LinuxTag in Berlin/Germany. I'll have a technical talk there about SRP:
>
> http://www.linuxtag.org/2013/en/program/thursday-may-23-2013.html?eventid=208
>
> Cheers,
> Sebastian
>
> --
> Sebastian Riemer
> Linux Kernel Developer - Storage
>
> ProfitBricks GmbH • Greifswalder Str. 207 • 10405 Berlin, Germany
> www.profitbricks.com • sebastian.riemer-EIkl63zCoXaH+58JC4qpiA@public.gmane.org
>
> Sitz der Gesellschaft: Berlin
> Registergericht: Amtsgericht Charlottenburg, HRB 125506 B
> Geschäftsführer: Andreas Gauger, Achim Weiss
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Vasiliy Tolstov,
e-mail: v.tolstov-+9FY0jupvH6HXe+LvDLADg@public.gmane.org
jabber: vase-+9FY0jupvH6HXe+LvDLADg@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Bart Van Assche]

On 04/30/13 09:34, Vasiliy Tolstov wrote:
> What is main difference between bvanassche repo and sriemer ?

Good question. As soon as I have the time I will try to find a single 
approach that works for everyone and post a new patch series for review 
on the linux-rdma mailing list such that these can be included in the 
mainline kernel.

Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Sebastian Riemer]

Hi Vasiliy,

sorry for the late reply! I was ill last week.

The main difference so far is that my patches are much easier to
understand as I don't provide back-porting and also don't put
performance improvements in between the stability fixes. I provide full
test cases + scripts which makes it simple to check if you've got the
expected/same behavior. Everything is straight forward and completely
described in the commit messages.

My work focuses on using ib_srp without the srp_tools. Just fast IO
failing + reconnect. No performance improvements. We gradually find
compromises to get a cool solution. We've e.g. agreed on detecting
connection loss upon the QP timeout only.

Today I'll release our own automatic reconnect.

Sorry Bart, but a reconnect with just the commit message
"IB/srp: Add kernel-level transport layer recovery" and no further
description isn't very trustworthy for me. I also wonder why you need so
much locking.

Cheers,
Sebastian


On 03.05.2013 14:27, Vasiliy Tolstov wrote:
> 
> 03.05.2013 15:45 пользователь "Bart Van Assche" <bvanassche-HInyCGIudOg@public.gmane.org
> <mailto:bvanassche-HInyCGIudOg@public.gmane.org>> написал:
>>
>> On 04/30/13 09:34, Vasiliy Tolstov wrote:
>>>
>>> What is main difference between bvanassche repo and sriemer ?
>>
>>
>> Good question. As soon as I have the time I will try to find a single
> approach that works for everyone and post a new patch series for review
> on the linux-rdma mailing list such that these can be included in the
> mainline kernel.
>>
>> Bart.
> 
> Thats very good! Thans! I'm already using 3.8 with you patches and all
> works very stable.
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Bart Van Assche]

On 05/06/13 10:44, Sebastian Riemer wrote:
> Sorry Bart, but a reconnect with just the commit message
> "IB/srp: Add kernel-level transport layer recovery" and no further
> description isn't very trustworthy for me. I also wonder why you need so
> much locking.

Hello Sebastian,

There is a very good reason for all that locking: during device removal 
the rport structure is freed before SCSI host removal finished. So when 
converting a SCSI host pointer into an rport pointer protection is 
needed against device removal. The current approach in the 
ib_srp-backport repository is to realize that protection via locking. 
I'm currently reworking that approach into protection via reference 
counting such that it is guaranteed that the rport remains to exist 
until scsi_remove_device() has finished. That will allow to eliminate 
several locking statements.

Sorry for the terse patch descriptions - I had not expected that anyone 
would try to understand the patches ib_srp-backport repository and use 
these as a basis for further work.

Bart.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Sebastian Riemer]

FYI: I've released version 0.6 of my SRP patches today.

The automatic reconnect is included now. The tests for that will follow
in the next version. But we already did quite intensive testing for that.

Hard reboot and also soft reboot of the target are possible with that
reconnect. It just reconnects and everything is fine again.

With soft reboot I mean: disabling the target, removing the exports,
rebooting, exporting the same LUNs, re-enabling the target.

It also has an automatic mechanism to reduce the possibility of a DDoS
attack reconnect. It automatically reconnects at different intervals.

Check it out:
https://github.com/sriemer/ib_srp

Cheers,
Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Vasiliy Tolstov]

if i need faster reconnects and ability to close session from
initiator side under qlogic hardware, does it possible? Or this
patches only covers mallanox cards?

2013/5/8 Sebastian Riemer <sebastian.riemer-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>:
> FYI: I've released version 0.6 of my SRP patches today.
>
> The automatic reconnect is included now. The tests for that will follow
> in the next version. But we already did quite intensive testing for that.
>
> Hard reboot and also soft reboot of the target are possible with that
> reconnect. It just reconnects and everything is fine again.
>
> With soft reboot I mean: disabling the target, removing the exports,
> rebooting, exporting the same LUNs, re-enabling the target.
>
> It also has an automatic mechanism to reduce the possibility of a DDoS
> attack reconnect. It automatically reconnects at different intervals.
>
> Check it out:
> https://github.com/sriemer/ib_srp
>
> Cheers,
> Sebastian



-- 
Vasiliy Tolstov,
e-mail: v.tolstov-+9FY0jupvH6HXe+LvDLADg@public.gmane.org
jabber: vase-+9FY0jupvH6HXe+LvDLADg@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Sebastian Riemer]

Of cause, also Qlogic HCAs can be used.
But please note that there is no back-port in my patches to make them
better readable. If you like my patches, then we can talk about how to
back-port them to a specific kernel version.

On 14.05.2013 12:00, Vasiliy Tolstov wrote:
> if i need faster reconnects and ability to close session from
> initiator side under qlogic hardware, does it possible? Or this
> patches only covers mallanox cards?
> 
> 2013/5/8 Sebastian Riemer <sebastian.riemer-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>:
>> FYI: I've released version 0.6 of my SRP patches today.
>>
>> The automatic reconnect is included now. The tests for that will follow
>> in the next version. But we already did quite intensive testing for that.
>>
>> Hard reboot and also soft reboot of the target are possible with that
>> reconnect. It just reconnects and everything is fine again.
>>
>> With soft reboot I mean: disabling the target, removing the exports,
>> rebooting, exporting the same LUNs, re-enabling the target.
>>
>> It also has an automatic mechanism to reduce the possibility of a DDoS
>> attack reconnect. It automatically reconnects at different intervals.
>>
>> Check it out:
>> https://github.com/sriemer/ib_srp
>>
>> Cheers,
>> Sebastian
> 
> 
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Bart Van Assche]

On 05/14/13 12:00, Vasiliy Tolstov wrote:
> if i need faster reconnects and ability to close session from
> initiator side under qlogic hardware, does it possible? Or this
> patches only covers mallanox cards?

The ability to close a session from the initiator side went upstream in 
kernel 3.8 (/sys/class/srp_remote_ports/port-<h>:<n>/delete). Regarding 
faster reconnects: please keep in mind that after a cable pull it can 
easily takes 20 seconds before link training and initialization by the 
subnet manager have finished. It's not possible to make an initiator 
reconnect in less time than what the hardware and subnet manager need to 
bring the link back.

Bart.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Vasiliy Tolstov]

2013/5/14 Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>:
> The ability to close a session from the initiator side went upstream in
> kernel 3.8 (/sys/class/srp_remote_ports/port-<h>:<n>/delete). Regarding
> faster reconnects: please keep in mind that after a cable pull it can easily
> takes 20 seconds before link training and initialization by the subnet
> manager have finished. It's not possible to make an initiator reconnect in
> less time than what the hardware and subnet manager need to bring the link
> back.


Thanks. What about close session from target side? For example i need
to close the srp session and block all access from specific initiator?

--
Vasiliy Tolstov,
e-mail: v.tolstov-+9FY0jupvH6HXe+LvDLADg@public.gmane.org
jabber: vase-+9FY0jupvH6HXe+LvDLADg@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Sebastian Riemer]

On 15.05.2013 07:12, Vasiliy Tolstov wrote:
> 2013/5/14 Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>:
>> The ability to close a session from the initiator side went upstream in
>> kernel 3.8 (/sys/class/srp_remote_ports/port-<h>:<n>/delete). Regarding
>> faster reconnects: please keep in mind that after a cable pull it can easily
>> takes 20 seconds before link training and initialization by the subnet
>> manager have finished. It's not possible to make an initiator reconnect in
>> less time than what the hardware and subnet manager need to bring the link
>> back.
> 
> 
> Thanks. What about close session from target side? For example i need
> to close the srp session and block all access from specific initiator?

AFAIK the session is blocked as long as an initiator is connected. The
only possibility besides disconnecting the initiators is to disable the
target completely. Then, it sends a DREQ (disconnection request) to the
initiators. These know then that the target is disconnected and send a
DREP (disconnection reply). In our patches we also activate the
reconnect in this situation as we do that to orderly reboot a storage
system (e.g. due to an issue). The storage system comes up again,
exports the same volumes and the initiators can reconnect.

Cheers,
Sebastian

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Bart Van Assche]

On 05/15/13 07:12, Vasiliy Tolstov wrote:
> Thanks. What about close session from target side? For example i need
> to close the srp session and block all access from specific initiator?

The traditional approach to block access from a specific initiator is to 
modify the LUN masking configuration at the target side dynamically. 
More information about LUN masking can be found in the scst.conf man 
page and in the srpt/README file. I will make a patch available for 
ib_srpt that allows to close a single session at a time.

Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] SRP: ProfitBricks publishes its SRP Initiator patches

[Vasiliy Tolstov]

2013/5/15 Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>:
> The traditional approach to block access from a specific initiator is to
> modify the LUN masking configuration at the target side dynamically. More
> information about LUN masking can be found in the scst.conf man page and in
> the srpt/README file. I will make a patch available for ib_srpt that allows
> to close a single session at a time.

Thanks for answers and Thanks for patch, i'm try it.


--
Vasiliy Tolstov,
e-mail: v.tolstov-+9FY0jupvH6HXe+LvDLADg@public.gmane.org
jabber: vase-+9FY0jupvH6HXe+LvDLADg@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html