[LSF/MM TOPIC] async buffered diskio read for userspace apps

[LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

I would like to talk about enhancing the user interfaces for doing
async buffered disk IO for userspace applications. There's a whole
class of distributed web applications (most new applications today)
that would benefit from such an API. Most of them today rely on
cobbling one together in user space using a threadpool.

The current in kernel AIO interfaces that only support DIRECTIO, they
were generally designed by and for big database vendors. The consensus
is that the current AIO interfaces usually lead to decreased
performance for those app.

I've been developing a new read syscall that allows non-blocking
diskio read (provided that data is in the page cache). It's analogous
to what exists today in the network world with recvmsg with MSG_NOWAIT
flag. The work has been previously described by LWN here:
https://lwn.net/Articles/612483/

Previous attempts (over the last 12+ years) at non-blocking buffered
diskio has stalled due to their complexity. I would like to talk about
the problem, my solution, and get feedback on the course of action.

Over the years I've been building the low level guys of various "web
applications". That usually involves async network based applications
(epoll based servers) and the biggest pain point for the last 8+ years
has been async disk IO.


-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeff Moyer]

Milosz Tanski <milosz@adfin.com> writes:

> I would like to talk about enhancing the user interfaces for doing
> async buffered disk IO for userspace applications. There's a whole
> class of distributed web applications (most new applications today)
> that would benefit from such an API. Most of them today rely on
> cobbling one together in user space using a threadpool.
>
> The current in kernel AIO interfaces that only support DIRECTIO, they
> were generally designed by and for big database vendors. The consensus
> is that the current AIO interfaces usually lead to decreased
> performance for those app.
>
> I've been developing a new read syscall that allows non-blocking
> diskio read (provided that data is in the page cache). It's analogous
> to what exists today in the network world with recvmsg with MSG_NOWAIT
> flag. The work has been previously described by LWN here:
> https://lwn.net/Articles/612483/
>
> Previous attempts (over the last 12+ years) at non-blocking buffered
> diskio has stalled due to their complexity. I would like to talk about
> the problem, my solution, and get feedback on the course of action.

This email seems to conflate async I/O and non-blocking I/O.  Could you
please be more specific about what you're proposing to talk about?  Is
it just the non-blocking read support?

Cheers,
Jeff

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeff Moyer]

Milosz Tanski <milosz@adfin.com> writes:

> I would like to talk about enhancing the user interfaces for doing
> async buffered disk IO for userspace applications. There's a whole
> class of distributed web applications (most new applications today)
> that would benefit from such an API. Most of them today rely on
> cobbling one together in user space using a threadpool.
>
> The current in kernel AIO interfaces that only support DIRECTIO, they
> were generally designed by and for big database vendors. The consensus
> is that the current AIO interfaces usually lead to decreased
> performance for those app.
>
> I've been developing a new read syscall that allows non-blocking
> diskio read (provided that data is in the page cache). It's analogous
> to what exists today in the network world with recvmsg with MSG_NOWAIT
> flag. The work has been previously described by LWN here:
> https://lwn.net/Articles/612483/
>
> Previous attempts (over the last 12+ years) at non-blocking buffered
> diskio has stalled due to their complexity. I would like to talk about
> the problem, my solution, and get feedback on the course of action.

This email seems to conflate async I/O and non-blocking I/O.  Could you
please be more specific about what you're proposing to talk about?  Is
it just the non-blocking read support?

Cheers,
Jeff

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jan Kara]

On Thu 15-01-15 12:43:23, Milosz Tanski wrote:
> I would like to talk about enhancing the user interfaces for doing
> async buffered disk IO for userspace applications. There's a whole
> class of distributed web applications (most new applications today)
> that would benefit from such an API. Most of them today rely on
> cobbling one together in user space using a threadpool.
> 
> The current in kernel AIO interfaces that only support DIRECTIO, they
> were generally designed by and for big database vendors. The consensus
> is that the current AIO interfaces usually lead to decreased
> performance for those app.
> 
> I've been developing a new read syscall that allows non-blocking
> diskio read (provided that data is in the page cache). It's analogous
> to what exists today in the network world with recvmsg with MSG_NOWAIT
> flag. The work has been previously described by LWN here:
> https://lwn.net/Articles/612483/
> 
> Previous attempts (over the last 12+ years) at non-blocking buffered
> diskio has stalled due to their complexity. I would like to talk about
> the problem, my solution, and get feedback on the course of action.
> 
> Over the years I've been building the low level guys of various "web
> applications". That usually involves async network based applications
> (epoll based servers) and the biggest pain point for the last 8+ years
> has been async disk IO.
  Maybe this topic will be sorted out before LSF/MM. I know Andrew had some
objections about doc and was suggesting a solution using fincore() (which
Christoph refuted as being racy). Also there was a pending question
regarding whether the async read in this form will be used by applications.
But if it doesn't get sorted out a short session on the pending issues
would be probably useful.

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

On Thu, Jan 15, 2015 at 1:30 PM, Jeff Moyer <jmoyer@redhat.com> wrote:
> Milosz Tanski <milosz@adfin.com> writes:
>
>> I would like to talk about enhancing the user interfaces for doing
>> async buffered disk IO for userspace applications. There's a whole
>> class of distributed web applications (most new applications today)
>> that would benefit from such an API. Most of them today rely on
>> cobbling one together in user space using a threadpool.
>>
>> The current in kernel AIO interfaces that only support DIRECTIO, they
>> were generally designed by and for big database vendors. The consensus
>> is that the current AIO interfaces usually lead to decreased
>> performance for those app.
>>
>> I've been developing a new read syscall that allows non-blocking
>> diskio read (provided that data is in the page cache). It's analogous
>> to what exists today in the network world with recvmsg with MSG_NOWAIT
>> flag. The work has been previously described by LWN here:
>> https://lwn.net/Articles/612483/
>>
>> Previous attempts (over the last 12+ years) at non-blocking buffered
>> diskio has stalled due to their complexity. I would like to talk about
>> the problem, my solution, and get feedback on the course of action.
>
> This email seems to conflate async I/O and non-blocking I/O.  Could you
> please be more specific about what you're proposing to talk about?  Is
> it just the non-blocking read support?
>
> Cheers,
> Jeff

Jeff, I'm sorry if I wasn't clear, let me restate why we should care
and why it matters.

The current applications that power the lower levels of the web stacks
as generally process streams of network data. Many of them (and the
frameworks for building them) are structured as a large async
processing loop. Disk IO a big pain point; the way are structured
(threadpools for diskio) introduces additional latency. sendfile() is
only helpful if you need to do additional processing (say SSL).

Non-blocking diskio can help us lower the response latency in those
webapps applications in the common cases (cached data, sequential
scan).

-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

On Thu, Jan 15, 2015 at 5:31 PM, Jan Kara <jack@suse.cz> wrote:
> On Thu 15-01-15 12:43:23, Milosz Tanski wrote:
>> I would like to talk about enhancing the user interfaces for doing
>> async buffered disk IO for userspace applications. There's a whole
>> class of distributed web applications (most new applications today)
>> that would benefit from such an API. Most of them today rely on
>> cobbling one together in user space using a threadpool.
>>
>> The current in kernel AIO interfaces that only support DIRECTIO, they
>> were generally designed by and for big database vendors. The consensus
>> is that the current AIO interfaces usually lead to decreased
>> performance for those app.
>>
>> I've been developing a new read syscall that allows non-blocking
>> diskio read (provided that data is in the page cache). It's analogous
>> to what exists today in the network world with recvmsg with MSG_NOWAIT
>> flag. The work has been previously described by LWN here:
>> https://lwn.net/Articles/612483/
>>
>> Previous attempts (over the last 12+ years) at non-blocking buffered
>> diskio has stalled due to their complexity. I would like to talk about
>> the problem, my solution, and get feedback on the course of action.
>>
>> Over the years I've been building the low level guys of various "web
>> applications". That usually involves async network based applications
>> (epoll based servers) and the biggest pain point for the last 8+ years
>> has been async disk IO.
>   Maybe this topic will be sorted out before LSF/MM. I know Andrew had some
> objections about doc and was suggesting a solution using fincore() (which
> Christoph refuted as being racy). Also there was a pending question
> regarding whether the async read in this form will be used by applications.
> But if it doesn't get sorted out a short session on the pending issues
> would be probably useful.
>
>                                                                 Honza
> --
> Jan Kara <jack@suse.cz>
> SUSE Labs, CR

I've spent the better part of yesterday wrapping up the first cut of
samba support to FIO so we can test a modified samba file server with
these changes in a few scenarios. Right now it's only sync but I hope
to have async in the future. I hope that by the time the summit rolls
around I'll have data to share from samba and maybe some other common
apps (node.js / twisted).

-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Fri, Jan 16, 2015 at 10:44:12AM -0500, Milosz Tanski wrote:
> On Thu, Jan 15, 2015 at 5:31 PM, Jan Kara <jack@suse.cz> wrote:
> > On Thu 15-01-15 12:43:23, Milosz Tanski wrote:
> >> I would like to talk about enhancing the user interfaces for doing
> >> async buffered disk IO for userspace applications. There's a whole
> >> class of distributed web applications (most new applications today)
> >> that would benefit from such an API. Most of them today rely on
> >> cobbling one together in user space using a threadpool.
> >>
> >> The current in kernel AIO interfaces that only support DIRECTIO, they
> >> were generally designed by and for big database vendors. The consensus
> >> is that the current AIO interfaces usually lead to decreased
> >> performance for those app.
> >>
> >> I've been developing a new read syscall that allows non-blocking
> >> diskio read (provided that data is in the page cache). It's analogous
> >> to what exists today in the network world with recvmsg with MSG_NOWAIT
> >> flag. The work has been previously described by LWN here:
> >> https://lwn.net/Articles/612483/
> >>
> >> Previous attempts (over the last 12+ years) at non-blocking buffered
> >> diskio has stalled due to their complexity. I would like to talk about
> >> the problem, my solution, and get feedback on the course of action.
> >>
> >> Over the years I've been building the low level guys of various "web
> >> applications". That usually involves async network based applications
> >> (epoll based servers) and the biggest pain point for the last 8+ years
> >> has been async disk IO.
> >   Maybe this topic will be sorted out before LSF/MM. I know Andrew had some
> > objections about doc and was suggesting a solution using fincore() (which
> > Christoph refuted as being racy). Also there was a pending question
> > regarding whether the async read in this form will be used by applications.
> > But if it doesn't get sorted out a short session on the pending issues
> > would be probably useful.
> >
> >                                                                 Honza
> > --
> > Jan Kara <jack@suse.cz>
> > SUSE Labs, CR
> 
> I've spent the better part of yesterday wrapping up the first cut of
> samba support to FIO so we can test a modified samba file server with
> these changes in a few scenarios. Right now it's only sync but I hope
> to have async in the future. I hope that by the time the summit rolls
> around I'll have data to share from samba and maybe some other common
> apps (node.js / twisted).

Don't forget to share the code changes :-). We @ Samba would
love to see them to keep track !

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Fri, Jan 16, 2015 at 10:44:12AM -0500, Milosz Tanski wrote:
> On Thu, Jan 15, 2015 at 5:31 PM, Jan Kara <jack@suse.cz> wrote:
> > On Thu 15-01-15 12:43:23, Milosz Tanski wrote:
> >> I would like to talk about enhancing the user interfaces for doing
> >> async buffered disk IO for userspace applications. There's a whole
> >> class of distributed web applications (most new applications today)
> >> that would benefit from such an API. Most of them today rely on
> >> cobbling one together in user space using a threadpool.
> >>
> >> The current in kernel AIO interfaces that only support DIRECTIO, they
> >> were generally designed by and for big database vendors. The consensus
> >> is that the current AIO interfaces usually lead to decreased
> >> performance for those app.
> >>
> >> I've been developing a new read syscall that allows non-blocking
> >> diskio read (provided that data is in the page cache). It's analogous
> >> to what exists today in the network world with recvmsg with MSG_NOWAIT
> >> flag. The work has been previously described by LWN here:
> >> https://lwn.net/Articles/612483/
> >>
> >> Previous attempts (over the last 12+ years) at non-blocking buffered
> >> diskio has stalled due to their complexity. I would like to talk about
> >> the problem, my solution, and get feedback on the course of action.
> >>
> >> Over the years I've been building the low level guys of various "web
> >> applications". That usually involves async network based applications
> >> (epoll based servers) and the biggest pain point for the last 8+ years
> >> has been async disk IO.
> >   Maybe this topic will be sorted out before LSF/MM. I know Andrew had some
> > objections about doc and was suggesting a solution using fincore() (which
> > Christoph refuted as being racy). Also there was a pending question
> > regarding whether the async read in this form will be used by applications.
> > But if it doesn't get sorted out a short session on the pending issues
> > would be probably useful.
> >
> >                                                                 Honza
> > --
> > Jan Kara <jack@suse.cz>
> > SUSE Labs, CR
> 
> I've spent the better part of yesterday wrapping up the first cut of
> samba support to FIO so we can test a modified samba file server with
> these changes in a few scenarios. Right now it's only sync but I hope
> to have async in the future. I hope that by the time the summit rolls
> around I'll have data to share from samba and maybe some other common
> apps (node.js / twisted).

Don't forget to share the code changes :-). We @ Samba would
love to see them to keep track !

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

On Fri, Jan 16, 2015 at 11:55 AM, Jeremy Allison <jra@samba.org> wrote:
> On Fri, Jan 16, 2015 at 10:44:12AM -0500, Milosz Tanski wrote:
>> On Thu, Jan 15, 2015 at 5:31 PM, Jan Kara <jack@suse.cz> wrote:
>> > On Thu 15-01-15 12:43:23, Milosz Tanski wrote:
>> >> I would like to talk about enhancing the user interfaces for doing
>> >> async buffered disk IO for userspace applications. There's a whole
>> >> class of distributed web applications (most new applications today)
>> >> that would benefit from such an API. Most of them today rely on
>> >> cobbling one together in user space using a threadpool.
>> >>
>> >> The current in kernel AIO interfaces that only support DIRECTIO, they
>> >> were generally designed by and for big database vendors. The consensus
>> >> is that the current AIO interfaces usually lead to decreased
>> >> performance for those app.
>> >>
>> >> I've been developing a new read syscall that allows non-blocking
>> >> diskio read (provided that data is in the page cache). It's analogous
>> >> to what exists today in the network world with recvmsg with MSG_NOWAIT
>> >> flag. The work has been previously described by LWN here:
>> >> https://lwn.net/Articles/612483/
>> >>
>> >> Previous attempts (over the last 12+ years) at non-blocking buffered
>> >> diskio has stalled due to their complexity. I would like to talk about
>> >> the problem, my solution, and get feedback on the course of action.
>> >>
>> >> Over the years I've been building the low level guys of various "web
>> >> applications". That usually involves async network based applications
>> >> (epoll based servers) and the biggest pain point for the last 8+ years
>> >> has been async disk IO.
>> >   Maybe this topic will be sorted out before LSF/MM. I know Andrew had some
>> > objections about doc and was suggesting a solution using fincore() (which
>> > Christoph refuted as being racy). Also there was a pending question
>> > regarding whether the async read in this form will be used by applications.
>> > But if it doesn't get sorted out a short session on the pending issues
>> > would be probably useful.
>> >
>> >                                                                 Honza
>> > --
>> > Jan Kara <jack@suse.cz>
>> > SUSE Labs, CR
>>
>> I've spent the better part of yesterday wrapping up the first cut of
>> samba support to FIO so we can test a modified samba file server with
>> these changes in a few scenarios. Right now it's only sync but I hope
>> to have async in the future. I hope that by the time the summit rolls
>> around I'll have data to share from samba and maybe some other common
>> apps (node.js / twisted).
>
> Don't forget to share the code changes :-). We @ Samba would
> love to see them to keep track !

I have the first version of the FIO cifs support via samba in my fork
of FIO here: https://github.com/mtanski/fio/tree/samba

Right now it only supports sync mode of FIO (eg. can't submit multiple
outstanding requests) but I'm looking into how to make it work with
smb2 read/write calls with the async flag.

Additionally, I'm sure I'm doing some things not quite right in terms
of smbcli usage as it was a decent amount of trial and error to get it
to connect (esp. the setup before smbcli_full_connection). Finally, it
looks like the more complex api I'm using (as opposed to smbclient,
because I want the async calls) doesn't quite fully export all calls I
need via headers / public dyn libs so it's a bit of a hack to get it
to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1

But it works for my randread tests with zipf and the great part is
that it should provide a flexible way to test samba with many fake
clients and access patterns. So... progress.

-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote:
> 
> I have the first version of the FIO cifs support via samba in my fork
> of FIO here: https://github.com/mtanski/fio/tree/samba
> 
> Right now it only supports sync mode of FIO (eg. can't submit multiple
> outstanding requests) but I'm looking into how to make it work with
> smb2 read/write calls with the async flag.
> 
> Additionally, I'm sure I'm doing some things not quite right in terms
> of smbcli usage as it was a decent amount of trial and error to get it
> to connect (esp. the setup before smbcli_full_connection). Finally, it
> looks like the more complex api I'm using (as opposed to smbclient,
> because I want the async calls) doesn't quite fully export all calls I
> need via headers / public dyn libs so it's a bit of a hack to get it
> to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1
> 
> But it works for my randread tests with zipf and the great part is
> that it should provide a flexible way to test samba with many fake
> clients and access patterns. So... progress.

One problem here. Looks like fio is under GPLv2-only,
is that correct ?

If so there's no way to combine the two codebases,
as Samba is under GPLv3-or-later with parts under LGPLv3-or-later.

fio needs to be GPLv2-or-later in order to be
able to use with libsmbclient.

Cheers,

	Jeremy.

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote:
> 
> I have the first version of the FIO cifs support via samba in my fork
> of FIO here: https://github.com/mtanski/fio/tree/samba
> 
> Right now it only supports sync mode of FIO (eg. can't submit multiple
> outstanding requests) but I'm looking into how to make it work with
> smb2 read/write calls with the async flag.
> 
> Additionally, I'm sure I'm doing some things not quite right in terms
> of smbcli usage as it was a decent amount of trial and error to get it
> to connect (esp. the setup before smbcli_full_connection). Finally, it
> looks like the more complex api I'm using (as opposed to smbclient,
> because I want the async calls) doesn't quite fully export all calls I
> need via headers / public dyn libs so it's a bit of a hack to get it
> to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1
> 
> But it works for my randread tests with zipf and the great part is
> that it should provide a flexible way to test samba with many fake
> clients and access patterns. So... progress.

One problem here. Looks like fio is under GPLv2-only,
is that correct ?

If so there's no way to combine the two codebases,
as Samba is under GPLv3-or-later with parts under LGPLv3-or-later.

fio needs to be GPLv2-or-later in order to be
able to use with libsmbclient.

Cheers,

	Jeremy.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[James Bottomley]

On Sun, 2015-01-18 at 23:12 -0800, Jeremy Allison wrote:
> On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote:
> > 
> > I have the first version of the FIO cifs support via samba in my fork
> > of FIO here: https://github.com/mtanski/fio/tree/samba
> > 
> > Right now it only supports sync mode of FIO (eg. can't submit multiple
> > outstanding requests) but I'm looking into how to make it work with
> > smb2 read/write calls with the async flag.
> > 
> > Additionally, I'm sure I'm doing some things not quite right in terms
> > of smbcli usage as it was a decent amount of trial and error to get it
> > to connect (esp. the setup before smbcli_full_connection). Finally, it
> > looks like the more complex api I'm using (as opposed to smbclient,
> > because I want the async calls) doesn't quite fully export all calls I
> > need via headers / public dyn libs so it's a bit of a hack to get it
> > to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1
> > 
> > But it works for my randread tests with zipf and the great part is
> > that it should provide a flexible way to test samba with many fake
> > clients and access patterns. So... progress.
> 
> One problem here. Looks like fio is under GPLv2-only,
> is that correct ?

Seems so from the LICENSE file.

> If so there's no way to combine the two codebases,
> as Samba is under GPLv3-or-later with parts under LGPLv3-or-later.
> 
> fio needs to be GPLv2-or-later in order to be
> able to use with libsmbclient.

That's one of these pointless licensing complexities that annoy
distributions so much ... they're both open source, so there's no real
problem except the licence incompatibility. The usual way out of it is
just to dual licence the incompatible component.

James


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[James Bottomley]

On Sun, 2015-01-18 at 23:12 -0800, Jeremy Allison wrote:
> On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote:
> > 
> > I have the first version of the FIO cifs support via samba in my fork
> > of FIO here: https://github.com/mtanski/fio/tree/samba
> > 
> > Right now it only supports sync mode of FIO (eg. can't submit multiple
> > outstanding requests) but I'm looking into how to make it work with
> > smb2 read/write calls with the async flag.
> > 
> > Additionally, I'm sure I'm doing some things not quite right in terms
> > of smbcli usage as it was a decent amount of trial and error to get it
> > to connect (esp. the setup before smbcli_full_connection). Finally, it
> > looks like the more complex api I'm using (as opposed to smbclient,
> > because I want the async calls) doesn't quite fully export all calls I
> > need via headers / public dyn libs so it's a bit of a hack to get it
> > to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1
> > 
> > But it works for my randread tests with zipf and the great part is
> > that it should provide a flexible way to test samba with many fake
> > clients and access patterns. So... progress.
> 
> One problem here. Looks like fio is under GPLv2-only,
> is that correct ?

Seems so from the LICENSE file.

> If so there's no way to combine the two codebases,
> as Samba is under GPLv3-or-later with parts under LGPLv3-or-later.
> 
> fio needs to be GPLv2-or-later in order to be
> able to use with libsmbclient.

That's one of these pointless licensing complexities that annoy
distributions so much ... they're both open source, so there's no real
problem except the licence incompatibility. The usual way out of it is
just to dual licence the incompatible component.

James


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Sun, 2015-01-18 at 23:12 -0800, Jeremy Allison wrote:
>> On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote:
>> >
>> > I have the first version of the FIO cifs support via samba in my fork
>> > of FIO here: https://github.com/mtanski/fio/tree/samba
>> >
>> > Right now it only supports sync mode of FIO (eg. can't submit multiple
>> > outstanding requests) but I'm looking into how to make it work with
>> > smb2 read/write calls with the async flag.
>> >
>> > Additionally, I'm sure I'm doing some things not quite right in terms
>> > of smbcli usage as it was a decent amount of trial and error to get it
>> > to connect (esp. the setup before smbcli_full_connection). Finally, it
>> > looks like the more complex api I'm using (as opposed to smbclient,
>> > because I want the async calls) doesn't quite fully export all calls I
>> > need via headers / public dyn libs so it's a bit of a hack to get it
>> > to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1
>> >
>> > But it works for my randread tests with zipf and the great part is
>> > that it should provide a flexible way to test samba with many fake
>> > clients and access patterns. So... progress.
>>
>> One problem here. Looks like fio is under GPLv2-only,
>> is that correct ?
>
> Seems so from the LICENSE file.
>
>> If so there's no way to combine the two codebases,
>> as Samba is under GPLv3-or-later with parts under LGPLv3-or-later.
>>
>> fio needs to be GPLv2-or-later in order to be
>> able to use with libsmbclient.
>
> That's one of these pointless licensing complexities that annoy
> distributions so much ... they're both open source, so there's no real
> problem except the licence incompatibility. The usual way out of it is
> just to dual licence the incompatible component.
>
> James
>
>

Sadly, in this case there's nothing I can do about the license; both
projects have a right to determine their own licensing. Hopefully, the
parties can come to some kind of agreement since it would be
beneficial to use fio to test samba.

This works well enough for me to test test preadv2 using samba and get
numbers. So I'll use this to do some preadv2 testing using samba for
different workloads.

-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/19/2015 07:18 AM, Milosz Tanski wrote:
> On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
>> On Sun, 2015-01-18 at 23:12 -0800, Jeremy Allison wrote:
>>> On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote:
>>>>
>>>> I have the first version of the FIO cifs support via samba in my fork
>>>> of FIO here: https://github.com/mtanski/fio/tree/samba
>>>>
>>>> Right now it only supports sync mode of FIO (eg. can't submit multiple
>>>> outstanding requests) but I'm looking into how to make it work with
>>>> smb2 read/write calls with the async flag.
>>>>
>>>> Additionally, I'm sure I'm doing some things not quite right in terms
>>>> of smbcli usage as it was a decent amount of trial and error to get it
>>>> to connect (esp. the setup before smbcli_full_connection). Finally, it
>>>> looks like the more complex api I'm using (as opposed to smbclient,
>>>> because I want the async calls) doesn't quite fully export all calls I
>>>> need via headers / public dyn libs so it's a bit of a hack to get it
>>>> to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1
>>>>
>>>> But it works for my randread tests with zipf and the great part is
>>>> that it should provide a flexible way to test samba with many fake
>>>> clients and access patterns. So... progress.
>>>
>>> One problem here. Looks like fio is under GPLv2-only,
>>> is that correct ?
>>
>> Seems so from the LICENSE file.
>>
>>> If so there's no way to combine the two codebases,
>>> as Samba is under GPLv3-or-later with parts under LGPLv3-or-later.
>>>
>>> fio needs to be GPLv2-or-later in order to be
>>> able to use with libsmbclient.
>>
>> That's one of these pointless licensing complexities that annoy
>> distributions so much ... they're both open source, so there's no real
>> problem except the licence incompatibility. The usual way out of it is
>> just to dual licence the incompatible component.
>>
>> James
>>
>>
>
> Sadly, in this case there's nothing I can do about the license; both
> projects have a right to determine their own licensing. Hopefully, the
> parties can come to some kind of agreement since it would be
> beneficial to use fio to test samba.
>
> This works well enough for me to test test preadv2 using samba and get
> numbers. So I'll use this to do some preadv2 testing using samba for
> different workloads.

I didn't look at your code yet, but I'm assuming it's a self contained 
IO engine. So we should be able to make that work, by only linking the 
engine itself against libsmbclient. But sheesh, what a pain in the butt, 
why can't we just all be friends.

So don't worry about licensing for now, just work on improving the 
engine and we'll sort the non-technical details out.

-- 
Jens Axboe

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/19/2015 07:18 AM, Milosz Tanski wrote:
> On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
>> On Sun, 2015-01-18 at 23:12 -0800, Jeremy Allison wrote:
>>> On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote:
>>>>
>>>> I have the first version of the FIO cifs support via samba in my fork
>>>> of FIO here: https://github.com/mtanski/fio/tree/samba
>>>>
>>>> Right now it only supports sync mode of FIO (eg. can't submit multiple
>>>> outstanding requests) but I'm looking into how to make it work with
>>>> smb2 read/write calls with the async flag.
>>>>
>>>> Additionally, I'm sure I'm doing some things not quite right in terms
>>>> of smbcli usage as it was a decent amount of trial and error to get it
>>>> to connect (esp. the setup before smbcli_full_connection). Finally, it
>>>> looks like the more complex api I'm using (as opposed to smbclient,
>>>> because I want the async calls) doesn't quite fully export all calls I
>>>> need via headers / public dyn libs so it's a bit of a hack to get it
>>>> to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1
>>>>
>>>> But it works for my randread tests with zipf and the great part is
>>>> that it should provide a flexible way to test samba with many fake
>>>> clients and access patterns. So... progress.
>>>
>>> One problem here. Looks like fio is under GPLv2-only,
>>> is that correct ?
>>
>> Seems so from the LICENSE file.
>>
>>> If so there's no way to combine the two codebases,
>>> as Samba is under GPLv3-or-later with parts under LGPLv3-or-later.
>>>
>>> fio needs to be GPLv2-or-later in order to be
>>> able to use with libsmbclient.
>>
>> That's one of these pointless licensing complexities that annoy
>> distributions so much ... they're both open source, so there's no real
>> problem except the licence incompatibility. The usual way out of it is
>> just to dual licence the incompatible component.
>>
>> James
>>
>>
>
> Sadly, in this case there's nothing I can do about the license; both
> projects have a right to determine their own licensing. Hopefully, the
> parties can come to some kind of agreement since it would be
> beneficial to use fio to test samba.
>
> This works well enough for me to test test preadv2 using samba and get
> numbers. So I'll use this to do some preadv2 testing using samba for
> different workloads.

I didn't look at your code yet, but I'm assuming it's a self contained 
IO engine. So we should be able to make that work, by only linking the 
engine itself against libsmbclient. But sheesh, what a pain in the butt, 
why can't we just all be friends.

So don't worry about licensing for now, just work on improving the 
engine and we'll sort the non-technical details out.

-- 
Jens Axboe

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[James Bottomley]

On Mon, 2015-01-19 at 08:31 -0700, Jens Axboe wrote:
> On 01/19/2015 07:18 AM, Milosz Tanski wrote:
> > On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley
> > <James.Bottomley@hansenpartnership.com> wrote:
> >> On Sun, 2015-01-18 at 23:12 -0800, Jeremy Allison wrote:
> >>> On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote:
> >>>>
> >>>> I have the first version of the FIO cifs support via samba in my fork
> >>>> of FIO here: https://github.com/mtanski/fio/tree/samba
> >>>>
> >>>> Right now it only supports sync mode of FIO (eg. can't submit multiple
> >>>> outstanding requests) but I'm looking into how to make it work with
> >>>> smb2 read/write calls with the async flag.
> >>>>
> >>>> Additionally, I'm sure I'm doing some things not quite right in terms
> >>>> of smbcli usage as it was a decent amount of trial and error to get it
> >>>> to connect (esp. the setup before smbcli_full_connection). Finally, it
> >>>> looks like the more complex api I'm using (as opposed to smbclient,
> >>>> because I want the async calls) doesn't quite fully export all calls I
> >>>> need via headers / public dyn libs so it's a bit of a hack to get it
> >>>> to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1
> >>>>
> >>>> But it works for my randread tests with zipf and the great part is
> >>>> that it should provide a flexible way to test samba with many fake
> >>>> clients and access patterns. So... progress.
> >>>
> >>> One problem here. Looks like fio is under GPLv2-only,
> >>> is that correct ?
> >>
> >> Seems so from the LICENSE file.
> >>
> >>> If so there's no way to combine the two codebases,
> >>> as Samba is under GPLv3-or-later with parts under LGPLv3-or-later.
> >>>
> >>> fio needs to be GPLv2-or-later in order to be
> >>> able to use with libsmbclient.
> >>
> >> That's one of these pointless licensing complexities that annoy
> >> distributions so much ... they're both open source, so there's no real
> >> problem except the licence incompatibility. The usual way out of it is
> >> just to dual licence the incompatible component.
> >>
> >> James
> >>
> >>
> >
> > Sadly, in this case there's nothing I can do about the license; both
> > projects have a right to determine their own licensing. Hopefully, the
> > parties can come to some kind of agreement since it would be
> > beneficial to use fio to test samba.
> >
> > This works well enough for me to test test preadv2 using samba and get
> > numbers. So I'll use this to do some preadv2 testing using samba for
> > different workloads.
> 
> I didn't look at your code yet, but I'm assuming it's a self contained 
> IO engine. So we should be able to make that work, by only linking the 
> engine itself against libsmbclient. But sheesh, what a pain in the butt, 
> why can't we just all be friends.
> 
> So don't worry about licensing for now, just work on improving the 
> engine and we'll sort the non-technical details out.

For fio, it likely doesn't matter.  Most people download the repository
and compile it themselves when building the tool. In that case, there's
no licence violation anyway (all GPL issues, including technical licence
incompatibility, manifest on distribution not on use).  It is a problem
for the distributors, but they're well used to these type of self
inflicted wounds.

James

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[James Bottomley]

On Mon, 2015-01-19 at 08:31 -0700, Jens Axboe wrote:
> On 01/19/2015 07:18 AM, Milosz Tanski wrote:
> > On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley
> > <James.Bottomley@hansenpartnership.com> wrote:
> >> On Sun, 2015-01-18 at 23:12 -0800, Jeremy Allison wrote:
> >>> On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote:
> >>>>
> >>>> I have the first version of the FIO cifs support via samba in my fork
> >>>> of FIO here: https://github.com/mtanski/fio/tree/samba
> >>>>
> >>>> Right now it only supports sync mode of FIO (eg. can't submit multiple
> >>>> outstanding requests) but I'm looking into how to make it work with
> >>>> smb2 read/write calls with the async flag.
> >>>>
> >>>> Additionally, I'm sure I'm doing some things not quite right in terms
> >>>> of smbcli usage as it was a decent amount of trial and error to get it
> >>>> to connect (esp. the setup before smbcli_full_connection). Finally, it
> >>>> looks like the more complex api I'm using (as opposed to smbclient,
> >>>> because I want the async calls) doesn't quite fully export all calls I
> >>>> need via headers / public dyn libs so it's a bit of a hack to get it
> >>>> to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1
> >>>>
> >>>> But it works for my randread tests with zipf and the great part is
> >>>> that it should provide a flexible way to test samba with many fake
> >>>> clients and access patterns. So... progress.
> >>>
> >>> One problem here. Looks like fio is under GPLv2-only,
> >>> is that correct ?
> >>
> >> Seems so from the LICENSE file.
> >>
> >>> If so there's no way to combine the two codebases,
> >>> as Samba is under GPLv3-or-later with parts under LGPLv3-or-later.
> >>>
> >>> fio needs to be GPLv2-or-later in order to be
> >>> able to use with libsmbclient.
> >>
> >> That's one of these pointless licensing complexities that annoy
> >> distributions so much ... they're both open source, so there's no real
> >> problem except the licence incompatibility. The usual way out of it is
> >> just to dual licence the incompatible component.
> >>
> >> James
> >>
> >>
> >
> > Sadly, in this case there's nothing I can do about the license; both
> > projects have a right to determine their own licensing. Hopefully, the
> > parties can come to some kind of agreement since it would be
> > beneficial to use fio to test samba.
> >
> > This works well enough for me to test test preadv2 using samba and get
> > numbers. So I'll use this to do some preadv2 testing using samba for
> > different workloads.
> 
> I didn't look at your code yet, but I'm assuming it's a self contained 
> IO engine. So we should be able to make that work, by only linking the 
> engine itself against libsmbclient. But sheesh, what a pain in the butt, 
> why can't we just all be friends.
> 
> So don't worry about licensing for now, just work on improving the 
> engine and we'll sort the non-technical details out.

For fio, it likely doesn't matter.  Most people download the repository
and compile it themselves when building the tool. In that case, there's
no licence violation anyway (all GPL issues, including technical licence
incompatibility, manifest on distribution not on use).  It is a problem
for the distributors, but they're well used to these type of self
inflicted wounds.

James


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Volker Lendecke]

On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
> I didn't look at your code yet, but I'm assuming it's a self
> contained IO engine. So we should be able to make that work, by only
> linking the engine itself against libsmbclient. But sheesh, what a
> pain in the butt, why can't we just all be friends.

The published libsmbclient API misses the async features
that are needed here. Milosz needs to go lower-level.

Volker

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:kontakt@sernet.de

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Volker Lendecke]

On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
> I didn't look at your code yet, but I'm assuming it's a self
> contained IO engine. So we should be able to make that work, by only
> linking the engine itself against libsmbclient. But sheesh, what a
> pain in the butt, why can't we just all be friends.

The published libsmbclient API misses the async features
that are needed here. Milosz needs to go lower-level.

Volker

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Gottingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Gottingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:kontakt@sernet.de

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

On Mon, Jan 19, 2015 at 11:10 AM, Volker Lendecke
<Volker.Lendecke@sernet.de> wrote:
> On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
>> I didn't look at your code yet, but I'm assuming it's a self
>> contained IO engine. So we should be able to make that work, by only
>> linking the engine itself against libsmbclient. But sheesh, what a
>> pain in the butt, why can't we just all be friends.
>
> The published libsmbclient API misses the async features
> that are needed here. Milosz needs to go lower-level.
>
> Volker

Volker, the sync code path works; in fact I pushed some minor
corrections to my branch this morning. And for now using FIO I can
generate multiple clients (threads / processes).

I started working on the async features (SMB2 async read/write) for
client library the samba repo. There's a patch there for the first
step of it it there; see the other email I sent to you and Jeremy. I
was going to make sure it licensed under whatever it needs to get into
the samba repo... and since this is done on my own time I personally
don't care what license it's under provided it's not a PITA.

-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

On Mon, Jan 19, 2015 at 11:10 AM, Volker Lendecke
<Volker.Lendecke@sernet.de> wrote:
> On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
>> I didn't look at your code yet, but I'm assuming it's a self
>> contained IO engine. So we should be able to make that work, by only
>> linking the engine itself against libsmbclient. But sheesh, what a
>> pain in the butt, why can't we just all be friends.
>
> The published libsmbclient API misses the async features
> that are needed here. Milosz needs to go lower-level.
>
> Volker

Volker, the sync code path works; in fact I pushed some minor
corrections to my branch this morning. And for now using FIO I can
generate multiple clients (threads / processes).

I started working on the async features (SMB2 async read/write) for
client library the samba repo. There's a patch there for the first
step of it it there; see the other email I sent to you and Jeremy. I
was going to make sure it licensed under whatever it needs to get into
the samba repo... and since this is done on my own time I personally
don't care what license it's under provided it's not a PITA.

-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Mon, Jan 19, 2015 at 07:49:41AM -0800, James Bottomley wrote:
> 
> For fio, it likely doesn't matter.  Most people download the repository
> and compile it themselves when building the tool. In that case, there's
> no licence violation anyway (all GPL issues, including technical licence
> incompatibility, manifest on distribution not on use).  It is a problem
> for the distributors, but they're well used to these type of self
> inflicted wounds.

That's true, but it is setting a bear-trap for distributors.

Might be better to keep the code repositories separate so at
lease people have a *chance* of noticing there's a problem
here.

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Mon, Jan 19, 2015 at 07:49:41AM -0800, James Bottomley wrote:
> 
> For fio, it likely doesn't matter.  Most people download the repository
> and compile it themselves when building the tool. In that case, there's
> no licence violation anyway (all GPL issues, including technical licence
> incompatibility, manifest on distribution not on use).  It is a problem
> for the distributors, but they're well used to these type of self
> inflicted wounds.

That's true, but it is setting a bear-trap for distributors.

Might be better to keep the code repositories separate so at
lease people have a *chance* of noticing there's a problem
here.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Mon, Jan 19, 2015 at 11:20:50AM -0500, Milosz Tanski wrote:
> 
> Volker, the sync code path works; in fact I pushed some minor
> corrections to my branch this morning. And for now using FIO I can
> generate multiple clients (threads / processes).
> 
> I started working on the async features (SMB2 async read/write) for
> client library the samba repo. There's a patch there for the first
> step of it it there; see the other email I sent to you and Jeremy. I
> was going to make sure it licensed under whatever it needs to get into
> the samba repo... and since this is done on my own time I personally
> don't care what license it's under provided it's not a PITA.

Anything going into Samba would need to be permissively licensed
(MIT/BSD) or GPLv3+ or LGPLv3+. We'd prefer the latter, but we're
happy with either.

The one thing it *can't* be though, is GPLv2-only.

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Mon, Jan 19, 2015 at 11:20:50AM -0500, Milosz Tanski wrote:
> 
> Volker, the sync code path works; in fact I pushed some minor
> corrections to my branch this morning. And for now using FIO I can
> generate multiple clients (threads / processes).
> 
> I started working on the async features (SMB2 async read/write) for
> client library the samba repo. There's a patch there for the first
> step of it it there; see the other email I sent to you and Jeremy. I
> was going to make sure it licensed under whatever it needs to get into
> the samba repo... and since this is done on my own time I personally
> don't care what license it's under provided it's not a PITA.

Anything going into Samba would need to be permissively licensed
(MIT/BSD) or GPLv3+ or LGPLv3+. We'd prefer the latter, but we're
happy with either.

The one thing it *can't* be though, is GPLv2-only.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
>
> That's one of these pointless licensing complexities that annoy
> distributions so much ... they're both open source, so there's no real
> problem except the licence incompatibility. The usual way out of it is
> just to dual licence the incompatible component.

Just one point here - we're not able to dual license
Samba to go back to GPLv2 anything. There are too many
contributors to this who have contributed under v3-or-later
licensing in order for this to be possible for us.

I'm hoping adding the 'or-later' clause to fio might
be easier.

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
>
> That's one of these pointless licensing complexities that annoy
> distributions so much ... they're both open source, so there's no real
> problem except the licence incompatibility. The usual way out of it is
> just to dual licence the incompatible component.

Just one point here - we're not able to dual license
Samba to go back to GPLv2 anything. There are too many
contributors to this who have contributed under v3-or-later
licensing in order for this to be possible for us.

I'm hoping adding the 'or-later' clause to fio might
be easier.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Richard Sharpe]

On Mon, Jan 19, 2015 at 9:04 AM, Jeremy Allison <jra@samba.org> wrote:
> On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
>>
>> That's one of these pointless licensing complexities that annoy
>> distributions so much ... they're both open source, so there's no real
>> problem except the licence incompatibility. The usual way out of it is
>> just to dual licence the incompatible component.
>
> Just one point here - we're not able to dual license
> Samba to go back to GPLv2 anything. There are too many
> contributors to this who have contributed under v3-or-later
> licensing in order for this to be possible for us.
>
> I'm hoping adding the 'or-later' clause to fio might
> be easier.

As someone who has worked for companies that distribute Samba for
quite a while I cannot see us distributing fio. Rather, we would use
it as a performance testing tool.

That being the case, the license differences are not a problem.

Am I missing something here?

-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Mon, Jan 19, 2015 at 09:18:14AM -0800, Richard Sharpe wrote:
> On Mon, Jan 19, 2015 at 9:04 AM, Jeremy Allison <jra@samba.org> wrote:
> > On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley
> > <James.Bottomley@hansenpartnership.com> wrote:
> >>
> >> That's one of these pointless licensing complexities that annoy
> >> distributions so much ... they're both open source, so there's no real
> >> problem except the licence incompatibility. The usual way out of it is
> >> just to dual licence the incompatible component.
> >
> > Just one point here - we're not able to dual license
> > Samba to go back to GPLv2 anything. There are too many
> > contributors to this who have contributed under v3-or-later
> > licensing in order for this to be possible for us.
> >
> > I'm hoping adding the 'or-later' clause to fio might
> > be easier.
> 
> As someone who has worked for companies that distribute Samba for
> quite a while I cannot see us distributing fio. Rather, we would use
> it as a performance testing tool.
> 
> That being the case, the license differences are not a problem.
> 
> Am I missing something here?

No, it's only a problem for distributors, so it's
much less of a problem than it might be.

But it's still a problem I'd rather not have to
think about :-).

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Mon, Jan 19, 2015 at 09:18:14AM -0800, Richard Sharpe wrote:
> On Mon, Jan 19, 2015 at 9:04 AM, Jeremy Allison <jra@samba.org> wrote:
> > On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley
> > <James.Bottomley@hansenpartnership.com> wrote:
> >>
> >> That's one of these pointless licensing complexities that annoy
> >> distributions so much ... they're both open source, so there's no real
> >> problem except the licence incompatibility. The usual way out of it is
> >> just to dual licence the incompatible component.
> >
> > Just one point here - we're not able to dual license
> > Samba to go back to GPLv2 anything. There are too many
> > contributors to this who have contributed under v3-or-later
> > licensing in order for this to be possible for us.
> >
> > I'm hoping adding the 'or-later' clause to fio might
> > be easier.
> 
> As someone who has worked for companies that distribute Samba for
> quite a while I cannot see us distributing fio. Rather, we would use
> it as a performance testing tool.
> 
> That being the case, the license differences are not a problem.
> 
> Am I missing something here?

No, it's only a problem for distributors, so it's
much less of a problem than it might be.

But it's still a problem I'd rather not have to
think about :-).

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[James Bottomley]

On Mon, 2015-01-19 at 08:48 -0800, Jeremy Allison wrote:
> On Mon, Jan 19, 2015 at 07:49:41AM -0800, James Bottomley wrote:
> > 
> > For fio, it likely doesn't matter.  Most people download the repository
> > and compile it themselves when building the tool. In that case, there's
> > no licence violation anyway (all GPL issues, including technical licence
> > incompatibility, manifest on distribution not on use).  It is a problem
> > for the distributors, but they're well used to these type of self
> > inflicted wounds.
> 
> That's true, but it is setting a bear-trap for distributors.

It's hardly a bear trap ... this type of annoyance is what they're used
to.  Some even just ignore it on the grounds of no harm no foul.  The
first thing they'll ask when they notice is for the protagonists to dual
licence.

> Might be better to keep the code repositories separate so at
> lease people have a *chance* of noticing there's a problem
> here.

Actually, it might be better to *resolve* the problem before people
notice ... if the combination is considered useful, of course.

James


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[James Bottomley]

On Mon, 2015-01-19 at 08:48 -0800, Jeremy Allison wrote:
> On Mon, Jan 19, 2015 at 07:49:41AM -0800, James Bottomley wrote:
> > 
> > For fio, it likely doesn't matter.  Most people download the repository
> > and compile it themselves when building the tool. In that case, there's
> > no licence violation anyway (all GPL issues, including technical licence
> > incompatibility, manifest on distribution not on use).  It is a problem
> > for the distributors, but they're well used to these type of self
> > inflicted wounds.
> 
> That's true, but it is setting a bear-trap for distributors.

It's hardly a bear trap ... this type of annoyance is what they're used
to.  Some even just ignore it on the grounds of no harm no foul.  The
first thing they'll ask when they notice is for the protagonists to dual
licence.

> Might be better to keep the code repositories separate so at
> lease people have a *chance* of noticing there's a problem
> here.

Actually, it might be better to *resolve* the problem before people
notice ... if the combination is considered useful, of course.

James


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Mon, Jan 19, 2015 at 09:26:54AM -0800, James Bottomley wrote:
> 
> Actually, it might be better to *resolve* the problem before people
> notice ... if the combination is considered useful, of course.

Oh sure - no arguments from me there !

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Richard Sharpe]

On Mon, Jan 19, 2015 at 9:26 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Mon, 2015-01-19 at 08:48 -0800, Jeremy Allison wrote:
>> On Mon, Jan 19, 2015 at 07:49:41AM -0800, James Bottomley wrote:
>> >
>> > For fio, it likely doesn't matter.  Most people download the repository
>> > and compile it themselves when building the tool. In that case, there's
>> > no licence violation anyway (all GPL issues, including technical licence
>> > incompatibility, manifest on distribution not on use).  It is a problem
>> > for the distributors, but they're well used to these type of self
>> > inflicted wounds.
>>
>> That's true, but it is setting a bear-trap for distributors.
>
> It's hardly a bear trap ... this type of annoyance is what they're used
> to.  Some even just ignore it on the grounds of no harm no foul.  The
> first thing they'll ask when they notice is for the protagonists to dual
> licence.
>
>> Might be better to keep the code repositories separate so at
>> lease people have a *chance* of noticing there's a problem
>> here.
>
> Actually, it might be better to *resolve* the problem before people
> notice ... if the combination is considered useful, of course.

Actually, I retract my earlier comment. The combination could be very
useful to some companies that ship Samba. I can see it being used by
field support to test whether there are performance problems either
from Windows (under cygwin) or from Linux.

That being the case, it would be useful to resolve the license issue
before it becomes an issue.

-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/19/2015 09:48 AM, Jeremy Allison wrote:
> On Mon, Jan 19, 2015 at 07:49:41AM -0800, James Bottomley wrote:
>>
>> For fio, it likely doesn't matter.  Most people download the repository
>> and compile it themselves when building the tool. In that case, there's
>> no licence violation anyway (all GPL issues, including technical licence
>> incompatibility, manifest on distribution not on use).  It is a problem
>> for the distributors, but they're well used to these type of self
>> inflicted wounds.
>
> That's true, but it is setting a bear-trap for distributors.

But not unique. Most distros are behind on fio anyway, so most people do 
end up compiling on their own.

> Might be better to keep the code repositories separate so at
> lease people have a *chance* of noticing there's a problem
> here.

But that's a pain for users, I'd much rather include it and let the 
distro sort it. They can just add --disable-smb or something to their 
configure scripts.

-- 
Jens Axboe

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/19/2015 09:48 AM, Jeremy Allison wrote:
> On Mon, Jan 19, 2015 at 07:49:41AM -0800, James Bottomley wrote:
>>
>> For fio, it likely doesn't matter.  Most people download the repository
>> and compile it themselves when building the tool. In that case, there's
>> no licence violation anyway (all GPL issues, including technical licence
>> incompatibility, manifest on distribution not on use).  It is a problem
>> for the distributors, but they're well used to these type of self
>> inflicted wounds.
>
> That's true, but it is setting a bear-trap for distributors.

But not unique. Most distros are behind on fio anyway, so most people do 
end up compiling on their own.

> Might be better to keep the code repositories separate so at
> lease people have a *chance* of noticing there's a problem
> here.

But that's a pain for users, I'd much rather include it and let the 
distro sort it. They can just add --disable-smb or something to their 
configure scripts.

-- 
Jens Axboe

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/19/2015 08:31 AM, Jens Axboe wrote:
> I didn't look at your code yet, but I'm assuming it's a self contained
> IO engine. So we should be able to make that work, by only linking the
> engine itself against libsmbclient. But sheesh, what a pain in the butt,
> why can't we just all be friends.

I pulled it in for testing, and came up with this patch [1]. If you 
don't do anything, it'll build cifs into fio as before. If you add 
--cifs-external to the configure arguments, it'll build cifs.so as an 
externally loadable module. You'd then use:

ioengine=/path/to/cifs.so

to use that module. I did not add an install target, I'll leave that to 
distros...

Let me know how that works for you. And let me know how far along you 
are with the cifs engine, I'd like to pull it in.

http://git.kernel.dk/?p=fio.git;a=shortlog;h=refs/heads/cifs

[1] 
http://git.kernel.dk/?p=fio.git;a=commit;h=c2c05e33b753ae686e24b43d1034d0c474203729

-- 
Jens Axboe

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/19/2015 08:31 AM, Jens Axboe wrote:
> I didn't look at your code yet, but I'm assuming it's a self contained
> IO engine. So we should be able to make that work, by only linking the
> engine itself against libsmbclient. But sheesh, what a pain in the butt,
> why can't we just all be friends.

I pulled it in for testing, and came up with this patch [1]. If you 
don't do anything, it'll build cifs into fio as before. If you add 
--cifs-external to the configure arguments, it'll build cifs.so as an 
externally loadable module. You'd then use:

ioengine=/path/to/cifs.so

to use that module. I did not add an install target, I'll leave that to 
distros...

Let me know how that works for you. And let me know how far along you 
are with the cifs engine, I'd like to pull it in.

http://git.kernel.dk/?p=fio.git;a=shortlog;h=refs/heads/cifs

[1] 
http://git.kernel.dk/?p=fio.git;a=commit;h=c2c05e33b753ae686e24b43d1034d0c474203729

-- 
Jens Axboe

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

Great, I'll pull into my branch. I'm already using FIO with the cifs
engine to test and help me debug preadv2 changes to smbd and it works
without issues for me today. I'm going to work on the async cifs
engine but that will take longer because I need to expose build the
async SMB2 support into libsmbclient. I'm going to leave that work
till after I get further with preadv2 in samba (hopefully this week).

I think the biggest issue with the changes is the configure part of my
changes (as seen here:
http://git.kernel.dk/?p=fio.git;a=blobdiff;f=configure;h=d4502095250cdf5187b24276c327b727d3d87288;hp=33d1327ebbba5b70a001e422bb5ad9b24d7c7b49;hb=7fd35359259b409ed023b924cb2758e9efb9950c;hpb=5fb4b36674b194ae6c6756314dc0c665fcaea06d
).

The way samba packages the client libraries beyond just smbclient-raw
requiring me to pull in other libraries and then mess with rpath to
guess the distro location is far for ideal. I haven't reported it yet
mostly because I was interested it making progress and making it work.
Ideally samba folks would fix the pkgconfig file for smbclient-raw to
set right LDPATH (including all the depending libraries and rpath) so
that stuff is not needed.

Additionally, there's a few things not exported in the header files
(but used) like:
http://git.kernel.dk/?p=fio.git;a=blob;f=engines/cifs.c;h=67c23fac0c70cfc75932c758f44dd377fc3f2608;hb=7fd35359259b409ed023b924cb2758e9efb9950c#l16
. It looks like lpcfg_resolve_context() is the only way to create a
struct resolve_context which is used in the cliraw hreads, but
lpcfg_resolve_context() is not exported via the header files.

Some of this might be not using the library correctly... there really
wasn't any documentation so I just guessed by looking the torture code
in samba and the smbclient and to see what order to punch the
lpcfg_stuff to make smbcli_full_connection() not fail.

On Tue, Jan 20, 2015 at 2:33 PM, Jens Axboe <axboe@kernel.dk> wrote:
> On 01/19/2015 08:31 AM, Jens Axboe wrote:
>>
>> I didn't look at your code yet, but I'm assuming it's a self contained
>> IO engine. So we should be able to make that work, by only linking the
>> engine itself against libsmbclient. But sheesh, what a pain in the butt,
>> why can't we just all be friends.
>
>
> I pulled it in for testing, and came up with this patch [1]. If you don't do
> anything, it'll build cifs into fio as before. If you add --cifs-external to
> the configure arguments, it'll build cifs.so as an externally loadable
> module. You'd then use:
>
> ioengine=/path/to/cifs.so
>
> to use that module. I did not add an install target, I'll leave that to
> distros...
>
> Let me know how that works for you. And let me know how far along you are
> with the cifs engine, I'd like to pull it in.
>
> http://git.kernel.dk/?p=fio.git;a=shortlog;h=refs/heads/cifs
>
> [1]
> http://git.kernel.dk/?p=fio.git;a=commit;h=c2c05e33b753ae686e24b43d1034d0c474203729
>
> --
> Jens Axboe
>



-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

Great, I'll pull into my branch. I'm already using FIO with the cifs
engine to test and help me debug preadv2 changes to smbd and it works
without issues for me today. I'm going to work on the async cifs
engine but that will take longer because I need to expose build the
async SMB2 support into libsmbclient. I'm going to leave that work
till after I get further with preadv2 in samba (hopefully this week).

I think the biggest issue with the changes is the configure part of my
changes (as seen here:
http://git.kernel.dk/?p=fio.git;a=blobdiff;f=configure;h=d4502095250cdf5187b24276c327b727d3d87288;hp=33d1327ebbba5b70a001e422bb5ad9b24d7c7b49;hb=7fd35359259b409ed023b924cb2758e9efb9950c;hpb=5fb4b36674b194ae6c6756314dc0c665fcaea06d
).

The way samba packages the client libraries beyond just smbclient-raw
requiring me to pull in other libraries and then mess with rpath to
guess the distro location is far for ideal. I haven't reported it yet
mostly because I was interested it making progress and making it work.
Ideally samba folks would fix the pkgconfig file for smbclient-raw to
set right LDPATH (including all the depending libraries and rpath) so
that stuff is not needed.

Additionally, there's a few things not exported in the header files
(but used) like:
http://git.kernel.dk/?p=fio.git;a=blob;f=engines/cifs.c;h=67c23fac0c70cfc75932c758f44dd377fc3f2608;hb=7fd35359259b409ed023b924cb2758e9efb9950c#l16
. It looks like lpcfg_resolve_context() is the only way to create a
struct resolve_context which is used in the cliraw hreads, but
lpcfg_resolve_context() is not exported via the header files.

Some of this might be not using the library correctly... there really
wasn't any documentation so I just guessed by looking the torture code
in samba and the smbclient and to see what order to punch the
lpcfg_stuff to make smbcli_full_connection() not fail.

On Tue, Jan 20, 2015 at 2:33 PM, Jens Axboe <axboe@kernel.dk> wrote:
> On 01/19/2015 08:31 AM, Jens Axboe wrote:
>>
>> I didn't look at your code yet, but I'm assuming it's a self contained
>> IO engine. So we should be able to make that work, by only linking the
>> engine itself against libsmbclient. But sheesh, what a pain in the butt,
>> why can't we just all be friends.
>
>
> I pulled it in for testing, and came up with this patch [1]. If you don't do
> anything, it'll build cifs into fio as before. If you add --cifs-external to
> the configure arguments, it'll build cifs.so as an externally loadable
> module. You'd then use:
>
> ioengine=/path/to/cifs.so
>
> to use that module. I did not add an install target, I'll leave that to
> distros...
>
> Let me know how that works for you. And let me know how far along you are
> with the cifs engine, I'd like to pull it in.
>
> http://git.kernel.dk/?p=fio.git;a=shortlog;h=refs/heads/cifs
>
> [1]
> http://git.kernel.dk/?p=fio.git;a=commit;h=c2c05e33b753ae686e24b43d1034d0c474203729
>
> --
> Jens Axboe
>



-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

Side note Jens.

Can you add a configure flag to disable use of SHM (like for ESX)? It
took me a while to figure out the proper define to manually stick in
the configure.

The motivation for this is using rr (mozila's replay debugger) to
debug fio. rr doesn't support SHM. http://rr-project.org/ gdb's
reversible debugging is too painfully slow.

Thanks,
- Milosz

On Tue, Jan 20, 2015 at 6:07 PM, Milosz Tanski <milosz@adfin.com> wrote:
> Great, I'll pull into my branch. I'm already using FIO with the cifs
> engine to test and help me debug preadv2 changes to smbd and it works
> without issues for me today. I'm going to work on the async cifs
> engine but that will take longer because I need to expose build the
> async SMB2 support into libsmbclient. I'm going to leave that work
> till after I get further with preadv2 in samba (hopefully this week).
>
> I think the biggest issue with the changes is the configure part of my
> changes (as seen here:
> http://git.kernel.dk/?p=fio.git;a=blobdiff;f=configure;h=d4502095250cdf5187b24276c327b727d3d87288;hp=33d1327ebbba5b70a001e422bb5ad9b24d7c7b49;hb=7fd35359259b409ed023b924cb2758e9efb9950c;hpb=5fb4b36674b194ae6c6756314dc0c665fcaea06d
> ).
>
> The way samba packages the client libraries beyond just smbclient-raw
> requiring me to pull in other libraries and then mess with rpath to
> guess the distro location is far for ideal. I haven't reported it yet
> mostly because I was interested it making progress and making it work.
> Ideally samba folks would fix the pkgconfig file for smbclient-raw to
> set right LDPATH (including all the depending libraries and rpath) so
> that stuff is not needed.
>
> Additionally, there's a few things not exported in the header files
> (but used) like:
> http://git.kernel.dk/?p=fio.git;a=blob;f=engines/cifs.c;h=67c23fac0c70cfc75932c758f44dd377fc3f2608;hb=7fd35359259b409ed023b924cb2758e9efb9950c#l16
> . It looks like lpcfg_resolve_context() is the only way to create a
> struct resolve_context which is used in the cliraw hreads, but
> lpcfg_resolve_context() is not exported via the header files.
>
> Some of this might be not using the library correctly... there really
> wasn't any documentation so I just guessed by looking the torture code
> in samba and the smbclient and to see what order to punch the
> lpcfg_stuff to make smbcli_full_connection() not fail.
>
> On Tue, Jan 20, 2015 at 2:33 PM, Jens Axboe <axboe@kernel.dk> wrote:
>> On 01/19/2015 08:31 AM, Jens Axboe wrote:
>>>
>>> I didn't look at your code yet, but I'm assuming it's a self contained
>>> IO engine. So we should be able to make that work, by only linking the
>>> engine itself against libsmbclient. But sheesh, what a pain in the butt,
>>> why can't we just all be friends.
>>
>>
>> I pulled it in for testing, and came up with this patch [1]. If you don't do
>> anything, it'll build cifs into fio as before. If you add --cifs-external to
>> the configure arguments, it'll build cifs.so as an externally loadable
>> module. You'd then use:
>>
>> ioengine=/path/to/cifs.so
>>
>> to use that module. I did not add an install target, I'll leave that to
>> distros...
>>
>> Let me know how that works for you. And let me know how far along you are
>> with the cifs engine, I'd like to pull it in.
>>
>> http://git.kernel.dk/?p=fio.git;a=shortlog;h=refs/heads/cifs
>>
>> [1]
>> http://git.kernel.dk/?p=fio.git;a=commit;h=c2c05e33b753ae686e24b43d1034d0c474203729
>>
>> --
>> Jens Axboe
>>
>
>
>
> --
> Milosz Tanski
> CTO
> 16 East 34th Street, 15th floor
> New York, NY 10016
>
> p: 646-253-9055
> e: milosz@adfin.com



-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/20/2015 04:22 PM, Milosz Tanski wrote:
> Side note Jens.
> 
> Can you add a configure flag to disable use of SHM (like for ESX)? It
> took me a while to figure out the proper define to manually stick in
> the configure.
> 
> The motivation for this is using rr (mozila's replay debugger) to
> debug fio. rr doesn't support SHM. http://rr-project.org/ gdb's
> reversible debugging is too painfully slow.

Yeah definitely, that's mean that thread=1 would be a requirement,
obviously. But I'd be fine with adding that flag.

-- 
Jens Axboe

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/20/2015 04:22 PM, Milosz Tanski wrote:
> Side note Jens.
> 
> Can you add a configure flag to disable use of SHM (like for ESX)? It
> took me a while to figure out the proper define to manually stick in
> the configure.
> 
> The motivation for this is using rr (mozila's replay debugger) to
> debug fio. rr doesn't support SHM. http://rr-project.org/ gdb's
> reversible debugging is too painfully slow.

Yeah definitely, that's mean that thread=1 would be a requirement,
obviously. But I'd be fine with adding that flag.

-- 
Jens Axboe

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/20/2015 04:26 PM, Jens Axboe wrote:
> On 01/20/2015 04:22 PM, Milosz Tanski wrote:
>> Side note Jens.
>>
>> Can you add a configure flag to disable use of SHM (like for ESX)? It
>> took me a while to figure out the proper define to manually stick in
>> the configure.
>>
>> The motivation for this is using rr (mozila's replay debugger) to
>> debug fio. rr doesn't support SHM. http://rr-project.org/ gdb's
>> reversible debugging is too painfully slow.
> 
> Yeah definitely, that's mean that thread=1 would be a requirement,
> obviously. But I'd be fine with adding that flag.

http://git.kernel.dk/?p=fio.git;a=commit;h=ba40757ed67c00b37dda3639e97c3ba0259840a4


-- 
Jens Axboe

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/20/2015 04:26 PM, Jens Axboe wrote:
> On 01/20/2015 04:22 PM, Milosz Tanski wrote:
>> Side note Jens.
>>
>> Can you add a configure flag to disable use of SHM (like for ESX)? It
>> took me a while to figure out the proper define to manually stick in
>> the configure.
>>
>> The motivation for this is using rr (mozila's replay debugger) to
>> debug fio. rr doesn't support SHM. http://rr-project.org/ gdb's
>> reversible debugging is too painfully slow.
> 
> Yeah definitely, that's mean that thread=1 would be a requirement,
> obviously. But I'd be fine with adding that flag.

http://git.kernel.dk/?p=fio.git;a=commit;h=ba40757ed67c00b37dda3639e97c3ba0259840a4


-- 
Jens Axboe

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

On Tue, Jan 20, 2015 at 6:30 PM, Jens Axboe <axboe@kernel.dk> wrote:
> On 01/20/2015 04:26 PM, Jens Axboe wrote:
>> On 01/20/2015 04:22 PM, Milosz Tanski wrote:
>>> Side note Jens.
>>>
>>> Can you add a configure flag to disable use of SHM (like for ESX)? It
>>> took me a while to figure out the proper define to manually stick in
>>> the configure.
>>>
>>> The motivation for this is using rr (mozila's replay debugger) to
>>> debug fio. rr doesn't support SHM. http://rr-project.org/ gdb's
>>> reversible debugging is too painfully slow.
>>
>> Yeah definitely, that's mean that thread=1 would be a requirement,
>> obviously. But I'd be fine with adding that flag.
>
> http://git.kernel.dk/?p=fio.git;a=commit;h=ba40757ed67c00b37dda3639e97c3ba0259840a4

Great, thanks for fixing it so quickly. Hopefully it'll be useful to
others as well.

- M

-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

On Tue, Jan 20, 2015 at 6:30 PM, Jens Axboe <axboe@kernel.dk> wrote:
> On 01/20/2015 04:26 PM, Jens Axboe wrote:
>> On 01/20/2015 04:22 PM, Milosz Tanski wrote:
>>> Side note Jens.
>>>
>>> Can you add a configure flag to disable use of SHM (like for ESX)? It
>>> took me a while to figure out the proper define to manually stick in
>>> the configure.
>>>
>>> The motivation for this is using rr (mozila's replay debugger) to
>>> debug fio. rr doesn't support SHM. http://rr-project.org/ gdb's
>>> reversible debugging is too painfully slow.
>>
>> Yeah definitely, that's mean that thread=1 would be a requirement,
>> obviously. But I'd be fine with adding that flag.
>
> http://git.kernel.dk/?p=fio.git;a=commit;h=ba40757ed67c00b37dda3639e97c3ba0259840a4

Great, thanks for fixing it so quickly. Hopefully it'll be useful to
others as well.

- M

-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/20/2015 04:53 PM, Milosz Tanski wrote:
> On Tue, Jan 20, 2015 at 6:30 PM, Jens Axboe <axboe@kernel.dk> wrote:
>> On 01/20/2015 04:26 PM, Jens Axboe wrote:
>>> On 01/20/2015 04:22 PM, Milosz Tanski wrote:
>>>> Side note Jens.
>>>>
>>>> Can you add a configure flag to disable use of SHM (like for ESX)? It
>>>> took me a while to figure out the proper define to manually stick in
>>>> the configure.
>>>>
>>>> The motivation for this is using rr (mozila's replay debugger) to
>>>> debug fio. rr doesn't support SHM. http://rr-project.org/ gdb's
>>>> reversible debugging is too painfully slow.
>>>
>>> Yeah definitely, that's mean that thread=1 would be a requirement,
>>> obviously. But I'd be fine with adding that flag.
>>
>> http://git.kernel.dk/?p=fio.git;a=commit;h=ba40757ed67c00b37dda3639e97c3ba0259840a4
>
> Great, thanks for fixing it so quickly. Hopefully it'll be useful to
> others as well.

No problem, it's in the 2.2.5 version as released. Let me know when you 
are comfortable with me pulling in the cifs engine.

-- 
Jens Axboe

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

[-- Attachment #1: Type: text/plain, Size: 9009 bytes --]

On Wed, Jan 21, 2015 at 4:17 PM, Jens Axboe <axboe@kernel.dk> wrote:

> On 01/20/2015 04:53 PM, Milosz Tanski wrote:
>
>> On Tue, Jan 20, 2015 at 6:30 PM, Jens Axboe <axboe@kernel.dk> wrote:
>>
>>> On 01/20/2015 04:26 PM, Jens Axboe wrote:
>>>
>>>> On 01/20/2015 04:22 PM, Milosz Tanski wrote:
>>>>
>>>>> Side note Jens.
>>>>>
>>>>> Can you add a configure flag to disable use of SHM (like for ESX)? It
>>>>> took me a while to figure out the proper define to manually stick in
>>>>> the configure.
>>>>>
>>>>> The motivation for this is using rr (mozila's replay debugger) to
>>>>> debug fio. rr doesn't support SHM. http://rr-project.org/ gdb's
>>>>> reversible debugging is too painfully slow.
>>>>>
>>>>
>>>> Yeah definitely, that's mean that thread=1 would be a requirement,
>>>> obviously. But I'd be fine with adding that flag.
>>>>
>>>
>>> http://git.kernel.dk/?p=fio.git;a=commit;h=
>>> ba40757ed67c00b37dda3639e97c3ba0259840a4
>>>
>>
>> Great, thanks for fixing it so quickly. Hopefully it'll be useful to
>> others as well.
>>
>
> No problem, it's in the 2.2.5 version as released. Let me know when you
> are comfortable with me pulling in the cifs engine.


Jermey, Volker,

Sorry for the spam to everybody in advance... this thread got away from me.


This is a general libsmbclient-raw question for you guys (even outside the
context of FIO). How should an external person consuming libsmbclient-raw
link to it?

What I mean by that is that that both linking to libsmbclient-raw and via
-llibsmbclient-raw or using pkgconfig doesn't really work do missing. Using
the current pkgconfig ends up with lot of missing symbols at link time. It
doesn't matter if I'm using samba built from source or samba built from my
distro package (Ubuntu or Debian). There's a couple things so let me try to
unpack them:

1. It doesn't seam like LDFLAGS pkgconfig setup in smbclient-raw.pc is
correct.

It doesn't include dependent libraries that are needed like libtalloc,
libdcerpc, libsamba-credentials.so... and many more private libraries.
Please see below errors.

2. There's an intention is to have private building blocks split been
public and private libraries and it doesn't make sense (to me).

Some of the libraries go into $PREFIX/lib/ and some go in to
$PREFIX/lib/private (seams that it's $PREFIX/lib/samba when it's packaged
by distros like Debian/Ubuntu). However, some very basic things (like
handling of NTSTATUS) end up going into private libraries like liberrors
(get_friendly_nt_error_msg, nt_errstr). It's hard to build error handling
that prints a useful message without them.

It gets even more difficult, lpcfg_resolve_context() lives in private
libcli-ldap functions live and doesn't get mentioned in any headers in
$PREFIX/include. To the best of my knowledge it's not even possible to make
a successful call to smbcli_full_connection with passing in a non-null
resolve_context struct. And it seams like the only way to do that is to
call lpcfg_resolve_context(). Every example of a utility in the samba tree
that does smbcli_full_connection(), uses a resolve_context created by
lpcfg_resolve_context(). Believe me, I tried a lot of different things and
without getting a NT_STATUS_INVALID_something.   smbcli_full_connection()
seams to a public function in a public library with a public header.


I can fix this and submit a patch / pull request to you guys; the first one
seams like an easy thing to tackle. The second one I need more guidance on
since I don't fully understand the intent / how did you guys design the
split.

This is what happens if I use pkgconfig:

gcc -rdynamic -std=gnu99 -Wwrite-strings -Wall
-Wdeclaration-after-statement -O3 -g -ffast-math  -D_GNU_SOURCE -include
config-host.h -DHAVE_IMMEDIATE_STRUCTURES=1 -I/usr/local/samba/include
-DBITS_PER_LONG=64 -DFIO_VERSION='"fio-2.1.11-23-g78d3d"' -o fio gettime.o
ioengines.o init.o stat.o log.o time.o filesetup.o eta.o verify.o memory.o
io_u.o parse.o mutex.o options.o lib/rbtree.o smalloc.o filehash.o
profile.o debug.o lib/rand.o lib/num2str.o lib/ieee754.o crc/crc16.o
crc/crc32.o crc/crc32c.o crc/crc32c-intel.o crc/crc64.o crc/crc7.o
crc/md5.o crc/sha1.o crc/sha256.o crc/sha512.o crc/test.o crc/xxhash.o
engines/cpu.o engines/mmap.o engines/sync.o engines/null.o engines/net.o
memalign.o server.o client.o iolog.o backend.o libfio.o flow.o cconv.o
lib/prio_tree.o json.o lib/zipf.o lib/axmap.o lib/lfsr.o gettime-thread.o
helpers.o lib/flist_sort.o lib/hweight.o lib/getrusage.o idletime.o
td_error.o profiles/tiobench.o profiles/act.o io_u_queue.o filelock.o
lib/tp.o engines/libaio.o engines/posixaio.o engines/falloc.o
engines/e4defrag.o engines/splice.o engines/cifs.o engines/cifs_sync.o
diskutil.o fifo.o blktrace.o cgroup.o trim.o engines/sg.o engines/binject.o
fio.o -lnuma -libverbs -lrt -laio -lz  -Wl,-rpath,/usr/local/samba/lib
-L/usr/local/samba/lib -lsmbclient-raw   -lm  -lpthread -ldl
engines/cifs_sync.o: In function `fio_cifs_queue':
/home/mtanski/src/fio/engines/cifs_sync.c:47: undefined reference to
`smbcli_write'
/home/mtanski/src/fio/engines/cifs_sync.c:43: undefined reference to
`smbcli_read'
engines/cifs.o: In function `fio_cifs_init':
/home/mtanski/src/fio/engines/cifs.c:64: undefined reference to
`talloc_named_const'
/home/mtanski/src/fio/engines/cifs.c:73: undefined reference to
`samba_tevent_context_init'
/home/mtanski/src/fio/engines/cifs.c:76: undefined reference to
`gensec_init'
/home/mtanski/src/fio/engines/cifs.c:78: undefined reference to
`loadparm_init_global'
/home/mtanski/src/fio/engines/cifs.c:79: undefined reference to
`lpcfg_load_default'
/home/mtanski/src/fio/engines/cifs.c:80: undefined reference to
`lpcfg_smbcli_options'
/home/mtanski/src/fio/engines/cifs.c:81: undefined reference to
`lpcfg_smbcli_session_options'
/home/mtanski/src/fio/engines/cifs.c:84: undefined reference to
`cli_credentials_init'
/home/mtanski/src/fio/engines/cifs.c:85: undefined reference to
`cli_credentials_set_anonymous'
/home/mtanski/src/fio/engines/cifs.c:88: undefined reference to
`cli_credentials_parse_string'
/home/mtanski/src/fio/engines/cifs.c:95: undefined reference to
`cli_credentials_set_password'
/home/mtanski/src/fio/engines/cifs.c:103: undefined reference to
`cli_credentials_guess'
/home/mtanski/src/fio/engines/cifs.c:105: undefined reference to
`lpcfg_gensec_settings'
/home/mtanski/src/fio/engines/cifs.c:105: undefined reference to
`lpcfg_resolve_context'
/home/mtanski/src/fio/engines/cifs.c:105: undefined reference to
`lpcfg_socket_options'
/home/mtanski/src/fio/engines/cifs.c:105: undefined reference to
`lpcfg_smb_ports'
/home/mtanski/src/fio/engines/cifs.c:105: undefined reference to
`smbcli_full_connection'
/home/mtanski/src/fio/engines/cifs.c:122: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:122: undefined reference to
`get_friendly_nt_error_msg'
/home/mtanski/src/fio/engines/cifs.c:134: undefined reference to
`_talloc_free'
engines/cifs.o: In function `fio_cifs_cleanup':
/home/mtanski/src/fio/engines/cifs.c:144: undefined reference to
`smbcli_tdis'
engines/cifs.o: In function `fio_cifs_open_file':
/home/mtanski/src/fio/engines/cifs.c:174: undefined reference to
`smbcli_open'
engines/cifs.o: In function `extend_file':
/home/mtanski/src/fio/engines/cifs.c:269: undefined reference to
`smbcli_getattrE'
/home/mtanski/src/fio/engines/cifs.c:318: undefined reference to
`smbcli_write'
/home/mtanski/src/fio/engines/cifs.c:284: undefined reference to
`smbcli_ftruncate'
/home/mtanski/src/fio/engines/cifs.c:288: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:288: undefined reference to
`get_friendly_nt_error_msg'
/home/mtanski/src/fio/engines/cifs.c:273: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:273: undefined reference to
`get_friendly_nt_error_msg'
engines/cifs.o: In function `fio_cifs_close_file':
/home/mtanski/src/fio/engines/cifs.c:192: undefined reference to
`smbcli_close'
/home/mtanski/src/fio/engines/cifs.c:195: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:195: undefined reference to
`get_friendly_nt_error_msg'
engines/cifs.o: In function `fio_cifs_unlink_file':
/home/mtanski/src/fio/engines/cifs.c:213: undefined reference to
`smbcli_unlink'
/home/mtanski/src/fio/engines/cifs.c:216: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:216: undefined reference to
`get_friendly_nt_error_msg'
engines/cifs.o: In function `fio_cifs_get_file_size':
/home/mtanski/src/fio/engines/cifs.c:238: undefined reference to
`smbcli_getattrE'
/home/mtanski/src/fio/engines/cifs.c:242: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:242: undefined reference to
`get_friendly_nt_error_msg'
engines/cifs.o: In function `fio_cifs_cleanup':
/home/mtanski/src/fio/engines/cifs.c:145: undefined reference to
`_talloc_free'
collect2: error: ld returned 1 exit status

-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

[-- Attachment #2: Type: text/html, Size: 11622 bytes --]

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Milosz Tanski]

On Wed, Jan 21, 2015 at 4:17 PM, Jens Axboe <axboe@kernel.dk> wrote:
> On 01/20/2015 04:53 PM, Milosz Tanski wrote:
>>
>> On Tue, Jan 20, 2015 at 6:30 PM, Jens Axboe <axboe@kernel.dk> wrote:
>>>
>>> On 01/20/2015 04:26 PM, Jens Axboe wrote:
>>>>
>>>> On 01/20/2015 04:22 PM, Milosz Tanski wrote:
>>>>>
>>>>> Side note Jens.
>>>>>
>>>>> Can you add a configure flag to disable use of SHM (like for ESX)? It
>>>>> took me a while to figure out the proper define to manually stick in
>>>>> the configure.
>>>>>
>>>>> The motivation for this is using rr (mozila's replay debugger) to
>>>>> debug fio. rr doesn't support SHM. http://rr-project.org/ gdb's
>>>>> reversible debugging is too painfully slow.
>>>>
>>>>
>>>> Yeah definitely, that's mean that thread=1 would be a requirement,
>>>> obviously. But I'd be fine with adding that flag.
>>>
>>>
>>>
>>> http://git.kernel.dk/?p=fio.git;a=commit;h=ba40757ed67c00b37dda3639e97c3ba0259840a4
>>
>>
>> Great, thanks for fixing it so quickly. Hopefully it'll be useful to
>> others as well.
>
>
> No problem, it's in the 2.2.5 version as released. Let me know when you are
> comfortable with me pulling in the cifs engine.
>
Jermey, Volker,

Sorry for the spam to everybody in advance... this thread got away
from me. Also, sorry for dup message and HTML. Gmail decided to
upgrade my message from text to HTML in the middle of the thread; this
is like the Nth time this has happened to me in a year.


This is a general libsmbclient-raw question for you guys (even outside
the context of FIO). How should an external person consuming
libsmbclient-raw link to it?

What I mean by that is that that both linking to libsmbclient-raw and
via -llibsmbclient-raw or using pkgconfig doesn't really work do
missing. Using the current pkgconfig ends up with lot of missing
symbols at link time. It doesn't matter if I'm using samba built from
source or samba built from my distro package (Ubuntu or Debian).
There's a couple things so let me try to unpack them:

1. It doesn't seam like LDFLAGS pkgconfig setup in smbclient-raw.pc is correct.

It doesn't include dependent libraries that are needed like libtalloc,
libdcerpc, libsamba-credentials.so... and many more private libraries.
Please see below errors.

2. There's an intention is to have private building blocks split been
public and private libraries and it doesn't make sense (to me).

Some of the libraries go into $PREFIX/lib/ and some go in to
$PREFIX/lib/private (seams that it's $PREFIX/lib/samba when it's
packaged by distros like Debian/Ubuntu). However, some very basic
things (like handling of NTSTATUS) end up going into private libraries
like liberrors (get_friendly_nt_error_msg, nt_errstr). It's hard to
build error handling that prints a useful message without them.

It gets even more difficult, lpcfg_resolve_context() lives in private
libcli-ldap functions live and doesn't get mentioned in any headers in
$PREFIX/include. To the best of my knowledge it's not even possible to
make a successful call to smbcli_full_connection with passing in a
non-null resolve_context struct. And it seams like the only way to do
that is to call lpcfg_resolve_context(). Every example of a utility in
the samba tree that does smbcli_full_connection(), uses a
resolve_context created by lpcfg_resolve_context(). Believe me, I
tried a lot of different things and without getting a
NT_STATUS_INVALID_something.   smbcli_full_connection() seams to a
public function in a public library with a public header.


I can fix this and submit a patch / pull request to you guys; the
first one seams like an easy thing to tackle. The second one I need
more guidance on since I don't fully understand the intent / how did
you guys design the split.

This is what happens if I use pkgconfig:

gcc -rdynamic -std=gnu99 -Wwrite-strings -Wall
-Wdeclaration-after-statement -O3 -g -ffast-math  -D_GNU_SOURCE
-include config-host.h -DHAVE_IMMEDIATE_STRUCTURES=1
-I/usr/local/samba/include   -DBITS_PER_LONG=64
-DFIO_VERSION='"fio-2.1.11-23-g78d3d"' -o fio gettime.o ioengines.o
init.o stat.o log.o time.o filesetup.o eta.o verify.o memory.o io_u.o
parse.o mutex.o options.o lib/rbtree.o smalloc.o filehash.o profile.o
debug.o lib/rand.o lib/num2str.o lib/ieee754.o crc/crc16.o crc/crc32.o
crc/crc32c.o crc/crc32c-intel.o crc/crc64.o crc/crc7.o crc/md5.o
crc/sha1.o crc/sha256.o crc/sha512.o crc/test.o crc/xxhash.o
engines/cpu.o engines/mmap.o engines/sync.o engines/null.o
engines/net.o memalign.o server.o client.o iolog.o backend.o libfio.o
flow.o cconv.o lib/prio_tree.o json.o lib/zipf.o lib/axmap.o
lib/lfsr.o gettime-thread.o helpers.o lib/flist_sort.o lib/hweight.o
lib/getrusage.o idletime.o td_error.o profiles/tiobench.o
profiles/act.o io_u_queue.o filelock.o lib/tp.o engines/libaio.o
engines/posixaio.o engines/falloc.o engines/e4defrag.o
engines/splice.o engines/cifs.o engines/cifs_sync.o diskutil.o fifo.o
blktrace.o cgroup.o trim.o engines/sg.o engines/binject.o fio.o -lnuma
-libverbs -lrt -laio -lz  -Wl,-rpath,/usr/local/samba/lib
-L/usr/local/samba/lib -lsmbclient-raw   -lm  -lpthread -ldl
engines/cifs_sync.o: In function `fio_cifs_queue':
/home/mtanski/src/fio/engines/cifs_sync.c:47: undefined reference to
`smbcli_write'
/home/mtanski/src/fio/engines/cifs_sync.c:43: undefined reference to
`smbcli_read'
engines/cifs.o: In function `fio_cifs_init':
/home/mtanski/src/fio/engines/cifs.c:64: undefined reference to
`talloc_named_const'
/home/mtanski/src/fio/engines/cifs.c:73: undefined reference to
`samba_tevent_context_init'
/home/mtanski/src/fio/engines/cifs.c:76: undefined reference to `gensec_init'
/home/mtanski/src/fio/engines/cifs.c:78: undefined reference to
`loadparm_init_global'
/home/mtanski/src/fio/engines/cifs.c:79: undefined reference to
`lpcfg_load_default'
/home/mtanski/src/fio/engines/cifs.c:80: undefined reference to
`lpcfg_smbcli_options'
/home/mtanski/src/fio/engines/cifs.c:81: undefined reference to
`lpcfg_smbcli_session_options'
/home/mtanski/src/fio/engines/cifs.c:84: undefined reference to
`cli_credentials_init'
/home/mtanski/src/fio/engines/cifs.c:85: undefined reference to
`cli_credentials_set_anonymous'
/home/mtanski/src/fio/engines/cifs.c:88: undefined reference to
`cli_credentials_parse_string'
/home/mtanski/src/fio/engines/cifs.c:95: undefined reference to
`cli_credentials_set_password'
/home/mtanski/src/fio/engines/cifs.c:103: undefined reference to
`cli_credentials_guess'
/home/mtanski/src/fio/engines/cifs.c:105: undefined reference to
`lpcfg_gensec_settings'
/home/mtanski/src/fio/engines/cifs.c:105: undefined reference to
`lpcfg_resolve_context'
/home/mtanski/src/fio/engines/cifs.c:105: undefined reference to
`lpcfg_socket_options'
/home/mtanski/src/fio/engines/cifs.c:105: undefined reference to
`lpcfg_smb_ports'
/home/mtanski/src/fio/engines/cifs.c:105: undefined reference to
`smbcli_full_connection'
/home/mtanski/src/fio/engines/cifs.c:122: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:122: undefined reference to
`get_friendly_nt_error_msg'
/home/mtanski/src/fio/engines/cifs.c:134: undefined reference to `_talloc_free'
engines/cifs.o: In function `fio_cifs_cleanup':
/home/mtanski/src/fio/engines/cifs.c:144: undefined reference to `smbcli_tdis'
engines/cifs.o: In function `fio_cifs_open_file':
/home/mtanski/src/fio/engines/cifs.c:174: undefined reference to `smbcli_open'
engines/cifs.o: In function `extend_file':
/home/mtanski/src/fio/engines/cifs.c:269: undefined reference to
`smbcli_getattrE'
/home/mtanski/src/fio/engines/cifs.c:318: undefined reference to `smbcli_write'
/home/mtanski/src/fio/engines/cifs.c:284: undefined reference to
`smbcli_ftruncate'
/home/mtanski/src/fio/engines/cifs.c:288: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:288: undefined reference to
`get_friendly_nt_error_msg'
/home/mtanski/src/fio/engines/cifs.c:273: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:273: undefined reference to
`get_friendly_nt_error_msg'
engines/cifs.o: In function `fio_cifs_close_file':
/home/mtanski/src/fio/engines/cifs.c:192: undefined reference to `smbcli_close'
/home/mtanski/src/fio/engines/cifs.c:195: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:195: undefined reference to
`get_friendly_nt_error_msg'
engines/cifs.o: In function `fio_cifs_unlink_file':
/home/mtanski/src/fio/engines/cifs.c:213: undefined reference to `smbcli_unlink'
/home/mtanski/src/fio/engines/cifs.c:216: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:216: undefined reference to
`get_friendly_nt_error_msg'
engines/cifs.o: In function `fio_cifs_get_file_size':
/home/mtanski/src/fio/engines/cifs.c:238: undefined reference to
`smbcli_getattrE'
/home/mtanski/src/fio/engines/cifs.c:242: undefined reference to `nt_errstr'
/home/mtanski/src/fio/engines/cifs.c:242: undefined reference to
`get_friendly_nt_error_msg'
engines/cifs.o: In function `fio_cifs_cleanup':
/home/mtanski/src/fio/engines/cifs.c:145: undefined reference to `_talloc_free'
collect2: error: ld returned 1 exit status

-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@adfin.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Steve French]

On Mon, Jan 19, 2015 at 10:20 AM, Milosz Tanski <milosz@adfin.com> wrote:
> On Mon, Jan 19, 2015 at 11:10 AM, Volker Lendecke
> <Volker.Lendecke@sernet.de> wrote:
>> On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
>>> I didn't look at your code yet, but I'm assuming it's a self
>>> contained IO engine. So we should be able to make that work, by only
>>> linking the engine itself against libsmbclient. But sheesh, what a
>>> pain in the butt, why can't we just all be friends.
>>
>> The published libsmbclient API misses the async features
>> that are needed here. Milosz needs to go lower-level.
>>
>> Volker
>
> Volker, the sync code path works; in fact I pushed some minor
> corrections to my branch this morning. And for now using FIO I can
> generate multiple clients (threads / processes).
>
> I started working on the async features (SMB2 async read/write) for
> client library the samba repo. There's a patch there for the first
> step of it it there; see the other email I sent to you and Jeremy. I
> was going to make sure it licensed under whatever it needs to get into
> the samba repo... and since this is done on my own time I personally
> don't care what license it's under provided it's not a PITA.

Why not do the async read/write via the kernel client if the license
is an issue?  It already has async SMB2/SMB3 operations
(with a synchronous send/receive-like wrapper).



-- 
Thanks,

Steve

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Steve French]

On Mon, Jan 19, 2015 at 10:20 AM, Milosz Tanski <milosz@adfin.com> wrote:
> On Mon, Jan 19, 2015 at 11:10 AM, Volker Lendecke
> <Volker.Lendecke@sernet.de> wrote:
>> On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
>>> I didn't look at your code yet, but I'm assuming it's a self
>>> contained IO engine. So we should be able to make that work, by only
>>> linking the engine itself against libsmbclient. But sheesh, what a
>>> pain in the butt, why can't we just all be friends.
>>
>> The published libsmbclient API misses the async features
>> that are needed here. Milosz needs to go lower-level.
>>
>> Volker
>
> Volker, the sync code path works; in fact I pushed some minor
> corrections to my branch this morning. And for now using FIO I can
> generate multiple clients (threads / processes).
>
> I started working on the async features (SMB2 async read/write) for
> client library the samba repo. There's a patch there for the first
> step of it it there; see the other email I sent to you and Jeremy. I
> was going to make sure it licensed under whatever it needs to get into
> the samba repo... and since this is done on my own time I personally
> don't care what license it's under provided it's not a PITA.

Why not do the async read/write via the kernel client if the license
is an issue?  It already has async SMB2/SMB3 operations
(with a synchronous send/receive-like wrapper).



-- 
Thanks,

Steve

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/23/2015 04:15 PM, Steve French wrote:
> On Mon, Jan 19, 2015 at 10:20 AM, Milosz Tanski <milosz@adfin.com> wrote:
>> On Mon, Jan 19, 2015 at 11:10 AM, Volker Lendecke
>> <Volker.Lendecke@sernet.de> wrote:
>>> On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
>>>> I didn't look at your code yet, but I'm assuming it's a self
>>>> contained IO engine. So we should be able to make that work, by only
>>>> linking the engine itself against libsmbclient. But sheesh, what a
>>>> pain in the butt, why can't we just all be friends.
>>>
>>> The published libsmbclient API misses the async features
>>> that are needed here. Milosz needs to go lower-level.
>>>
>>> Volker
>>
>> Volker, the sync code path works; in fact I pushed some minor
>> corrections to my branch this morning. And for now using FIO I can
>> generate multiple clients (threads / processes).
>>
>> I started working on the async features (SMB2 async read/write) for
>> client library the samba repo. There's a patch there for the first
>> step of it it there; see the other email I sent to you and Jeremy. I
>> was going to make sure it licensed under whatever it needs to get into
>> the samba repo... and since this is done on my own time I personally
>> don't care what license it's under provided it's not a PITA.
>
> Why not do the async read/write via the kernel client if the license
> is an issue?  It already has async SMB2/SMB3 operations
> (with a synchronous send/receive-like wrapper).

The license issue has been solved. Fio is cross platform, so would be 
preferable to have this work through libsmbclient, if at all possible.

-- 
Jens Axboe

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/23/2015 04:15 PM, Steve French wrote:
> On Mon, Jan 19, 2015 at 10:20 AM, Milosz Tanski <milosz@adfin.com> wrote:
>> On Mon, Jan 19, 2015 at 11:10 AM, Volker Lendecke
>> <Volker.Lendecke@sernet.de> wrote:
>>> On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
>>>> I didn't look at your code yet, but I'm assuming it's a self
>>>> contained IO engine. So we should be able to make that work, by only
>>>> linking the engine itself against libsmbclient. But sheesh, what a
>>>> pain in the butt, why can't we just all be friends.
>>>
>>> The published libsmbclient API misses the async features
>>> that are needed here. Milosz needs to go lower-level.
>>>
>>> Volker
>>
>> Volker, the sync code path works; in fact I pushed some minor
>> corrections to my branch this morning. And for now using FIO I can
>> generate multiple clients (threads / processes).
>>
>> I started working on the async features (SMB2 async read/write) for
>> client library the samba repo. There's a patch there for the first
>> step of it it there; see the other email I sent to you and Jeremy. I
>> was going to make sure it licensed under whatever it needs to get into
>> the samba repo... and since this is done on my own time I personally
>> don't care what license it's under provided it's not a PITA.
>
> Why not do the async read/write via the kernel client if the license
> is an issue?  It already has async SMB2/SMB3 operations
> (with a synchronous send/receive-like wrapper).

The license issue has been solved. Fio is cross platform, so would be 
preferable to have this work through libsmbclient, if at all possible.

-- 
Jens Axboe

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Fri, Jan 23, 2015 at 07:17:05PM -0700, Jens Axboe wrote:
> On 01/23/2015 04:15 PM, Steve French wrote:
> >On Mon, Jan 19, 2015 at 10:20 AM, Milosz Tanski <milosz@adfin.com> wrote:
> >>On Mon, Jan 19, 2015 at 11:10 AM, Volker Lendecke
> >><Volker.Lendecke@sernet.de> wrote:
> >>>On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
> >>>>I didn't look at your code yet, but I'm assuming it's a self
> >>>>contained IO engine. So we should be able to make that work, by only
> >>>>linking the engine itself against libsmbclient. But sheesh, what a
> >>>>pain in the butt, why can't we just all be friends.
> >>>
> >>>The published libsmbclient API misses the async features
> >>>that are needed here. Milosz needs to go lower-level.
> >>>
> >>>Volker
> >>
> >>Volker, the sync code path works; in fact I pushed some minor
> >>corrections to my branch this morning. And for now using FIO I can
> >>generate multiple clients (threads / processes).
> >>
> >>I started working on the async features (SMB2 async read/write) for
> >>client library the samba repo. There's a patch there for the first
> >>step of it it there; see the other email I sent to you and Jeremy. I
> >>was going to make sure it licensed under whatever it needs to get into
> >>the samba repo... and since this is done on my own time I personally
> >>don't care what license it's under provided it's not a PITA.
> >
> >Why not do the async read/write via the kernel client if the license
> >is an issue?  It already has async SMB2/SMB3 operations
> >(with a synchronous send/receive-like wrapper).
> 
> The license issue has been solved. Fio is cross platform, so would
> be preferable to have this work through libsmbclient, if at all
> possible.

How did the license issue get solved ? Did I miss some email
on that ?

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison]

On Fri, Jan 23, 2015 at 07:17:05PM -0700, Jens Axboe wrote:
> On 01/23/2015 04:15 PM, Steve French wrote:
> >On Mon, Jan 19, 2015 at 10:20 AM, Milosz Tanski <milosz@adfin.com> wrote:
> >>On Mon, Jan 19, 2015 at 11:10 AM, Volker Lendecke
> >><Volker.Lendecke@sernet.de> wrote:
> >>>On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
> >>>>I didn't look at your code yet, but I'm assuming it's a self
> >>>>contained IO engine. So we should be able to make that work, by only
> >>>>linking the engine itself against libsmbclient. But sheesh, what a
> >>>>pain in the butt, why can't we just all be friends.
> >>>
> >>>The published libsmbclient API misses the async features
> >>>that are needed here. Milosz needs to go lower-level.
> >>>
> >>>Volker
> >>
> >>Volker, the sync code path works; in fact I pushed some minor
> >>corrections to my branch this morning. And for now using FIO I can
> >>generate multiple clients (threads / processes).
> >>
> >>I started working on the async features (SMB2 async read/write) for
> >>client library the samba repo. There's a patch there for the first
> >>step of it it there; see the other email I sent to you and Jeremy. I
> >>was going to make sure it licensed under whatever it needs to get into
> >>the samba repo... and since this is done on my own time I personally
> >>don't care what license it's under provided it's not a PITA.
> >
> >Why not do the async read/write via the kernel client if the license
> >is an issue?  It already has async SMB2/SMB3 operations
> >(with a synchronous send/receive-like wrapper).
> 
> The license issue has been solved. Fio is cross platform, so would
> be preferable to have this work through libsmbclient, if at all
> possible.

How did the license issue get solved ? Did I miss some email
on that ?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/23/2015 10:53 PM, Jeremy Allison wrote:
> On Fri, Jan 23, 2015 at 07:17:05PM -0700, Jens Axboe wrote:
>> On 01/23/2015 04:15 PM, Steve French wrote:
>>> On Mon, Jan 19, 2015 at 10:20 AM, Milosz Tanski <milosz@adfin.com> wrote:
>>>> On Mon, Jan 19, 2015 at 11:10 AM, Volker Lendecke
>>>> <Volker.Lendecke@sernet.de> wrote:
>>>>> On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
>>>>>> I didn't look at your code yet, but I'm assuming it's a self
>>>>>> contained IO engine. So we should be able to make that work, by only
>>>>>> linking the engine itself against libsmbclient. But sheesh, what a
>>>>>> pain in the butt, why can't we just all be friends.
>>>>>
>>>>> The published libsmbclient API misses the async features
>>>>> that are needed here. Milosz needs to go lower-level.
>>>>>
>>>>> Volker
>>>>
>>>> Volker, the sync code path works; in fact I pushed some minor
>>>> corrections to my branch this morning. And for now using FIO I can
>>>> generate multiple clients (threads / processes).
>>>>
>>>> I started working on the async features (SMB2 async read/write) for
>>>> client library the samba repo. There's a patch there for the first
>>>> step of it it there; see the other email I sent to you and Jeremy. I
>>>> was going to make sure it licensed under whatever it needs to get into
>>>> the samba repo... and since this is done on my own time I personally
>>>> don't care what license it's under provided it's not a PITA.
>>>
>>> Why not do the async read/write via the kernel client if the license
>>> is an issue?  It already has async SMB2/SMB3 operations
>>> (with a synchronous send/receive-like wrapper).
>>
>> The license issue has been solved. Fio is cross platform, so would
>> be preferable to have this work through libsmbclient, if at all
>> possible.
>
> How did the license issue get solved ? Did I miss some email
> on that ?

Only the cifs engine is linked with libsmbclient, so that particular 
engine can be licensed as v2 or later (or v3, u to Milosz).

-- 
Jens Axboe

Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jens Axboe]

On 01/23/2015 10:53 PM, Jeremy Allison wrote:
> On Fri, Jan 23, 2015 at 07:17:05PM -0700, Jens Axboe wrote:
>> On 01/23/2015 04:15 PM, Steve French wrote:
>>> On Mon, Jan 19, 2015 at 10:20 AM, Milosz Tanski <milosz@adfin.com> wrote:
>>>> On Mon, Jan 19, 2015 at 11:10 AM, Volker Lendecke
>>>> <Volker.Lendecke@sernet.de> wrote:
>>>>> On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote:
>>>>>> I didn't look at your code yet, but I'm assuming it's a self
>>>>>> contained IO engine. So we should be able to make that work, by only
>>>>>> linking the engine itself against libsmbclient. But sheesh, what a
>>>>>> pain in the butt, why can't we just all be friends.
>>>>>
>>>>> The published libsmbclient API misses the async features
>>>>> that are needed here. Milosz needs to go lower-level.
>>>>>
>>>>> Volker
>>>>
>>>> Volker, the sync code path works; in fact I pushed some minor
>>>> corrections to my branch this morning. And for now using FIO I can
>>>> generate multiple clients (threads / processes).
>>>>
>>>> I started working on the async features (SMB2 async read/write) for
>>>> client library the samba repo. There's a patch there for the first
>>>> step of it it there; see the other email I sent to you and Jeremy. I
>>>> was going to make sure it licensed under whatever it needs to get into
>>>> the samba repo... and since this is done on my own time I personally
>>>> don't care what license it's under provided it's not a PITA.
>>>
>>> Why not do the async read/write via the kernel client if the license
>>> is an issue?  It already has async SMB2/SMB3 operations
>>> (with a synchronous send/receive-like wrapper).
>>
>> The license issue has been solved. Fio is cross platform, so would
>> be preferable to have this work through libsmbclient, if at all
>> possible.
>
> How did the license issue get solved ? Did I miss some email
> on that ?

Only the cifs engine is linked with libsmbclient, so that particular 
engine can be licensed as v2 or later (or v3, u to Milosz).

-- 
Jens Axboe

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>