[lustre-devel] MDT test in rel2.11

[lustre-devel] MDT test in rel2.11

[Abe Asraoui]

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe
 

[lustre-devel] MDT test in rel2.11

[Patrick Farrell]

Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you?re interested in or something you?re concerned about?

- Patrick

________________________________
From: lustre-devel <lustre-devel-bounces@lists.lustre.org> on behalf of Abe Asraoui <AbeA@supermicro.com>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180718/d25aae2b/attachment.html>

[lustre-devel] MDT test in rel2.11

[John Bent]

I'm curious about how DOM improves IO500 scores.  :)
Also LSOM but I don't know actually whether that's in 2.11 or where.

On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf@cray.com> wrote:

>
> Abe,
>
> Any benchmarking would be highly dependent on hardware, both client and
> server.  Is there a particular comparison (say, between versions) you?re
> interested in or something you?re concerned about?
>
> - Patrick
>
> ------------------------------
> *From:* lustre-devel <lustre-devel-bounces@lists.lustre.org> on behalf of
> Abe Asraoui <AbeA@supermicro.com>
> *Sent:* Tuesday, July 17, 2018 9:23:10 PM
> *To:* lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org; Abe
> Asraoui
> *Subject:* [lustre-devel] MDT test in rel2.11
>
> Hi All,
>
>
> Has anyone done any MDT testing under the latest rel2.11 and have
> benchmark data to share?
>
>
> Thanks,
> Abe
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180717/29edbd32/attachment.html>

[lustre-devel] MDT test in rel2.11

[Abe Asraoui]

Hi Patrick,

I?m more interested in what improvements have been in comparaison to rel2.10,
We have seen some performance degradation in rel2.10 with zfs and wondering if this is still the case in rel2.11 with the
Inclusion of DOM feature etc..


Thanks,
Abe


From: Patrick Farrell <paf@cray.com>
Date: Tuesday, July 17, 2018 at 8:32 PM
To: "Abe Asraoui (System)" <AbeA@supermicro.com>, "lustre-devel at lists.lustre.org" <lustre-devel@lists.lustre.org>, "lustre-discuss at lists.lustre.org" <lustre-discuss@lists.lustre.org>
Subject: Re: MDT test in rel2.11


Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you?re interested in or something you?re concerned about?

- Patrick
________________________________
From: lustre-devel <lustre-devel-bounces@lists.lustre.org> on behalf of Abe Asraoui <AbeA@supermicro.com>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddevel-2Dlustre.org&d=DwMF-g&c=4DxX-JX0i28X6V65hK0ft5M-1rZQeWgdMry9v8-eNr4&r=uOlOifyIDx4uh-m-EmK9TzkpR6RloKTYzjsbWwVKzmc&m=T3ZcoTwhQfWPNmVUj2vjMTfPAjwLEMX2xjMD-xadokU&s=dIbxsMKeXnEpMwOKA3OGr_0KQXbMONdnqxw9ZwTvUyY&e=>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180718/37224fa4/attachment-0001.html>

[lustre-devel] MDT test in rel2.11

[Patrick Farrell]

Lazy SoM is not landed yet, and it won?t be improving benchmark scores - it?s never ?known 100% correct?, so it can?t be used for actual POSIX ops - if a file size read out is used for a write offset, then you?ve got data corruption.

So for now it?s strictly limited to tools that know about it (accessed via an ioctl) and can accept information that may be stale.  The intended use case is scanning the FS for policy application.

________________________________
From: John Bent <johnbent@gmail.com>
Sent: Tuesday, July 17, 2018 10:55:24 PM
To: Patrick Farrell
Cc: Abe Asraoui; lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-devel] MDT test in rel2.11

I'm curious about how DOM improves IO500 scores.  :)
Also LSOM but I don't know actually whether that's in 2.11 or where.

On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf at cray.com<mailto:paf@cray.com>> wrote:

Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you?re interested in or something you?re concerned about?

- Patrick

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces@lists.lustre.org>> on behalf of Abe Asraoui <AbeA at supermicro.com<mailto:AbeA@supermicro.com>>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>; lustre-discuss at lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180718/21c12c32/attachment.html>

[lustre-devel] MDT test in rel2.11

[Patrick Farrell]

To be clear in case I sound too down on it - Lazy SoM is a very nice feature that will speed up important use cases.  It?s just not going to jazz up mdtest #s.


________________________________
From: Patrick Farrell
Sent: Tuesday, July 17, 2018 11:49:48 PM
To: John Bent
Cc: Abe Asraoui; lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-devel] MDT test in rel2.11

Lazy SoM is not landed yet, and it won?t be improving benchmark scores - it?s never ?known 100% correct?, so it can?t be used for actual POSIX ops - if a file size read out is used for a write offset, then you?ve got data corruption.

So for now it?s strictly limited to tools that know about it (accessed via an ioctl) and can accept information that may be stale.  The intended use case is scanning the FS for policy application.

________________________________
From: John Bent <johnbent@gmail.com>
Sent: Tuesday, July 17, 2018 10:55:24 PM
To: Patrick Farrell
Cc: Abe Asraoui; lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-devel] MDT test in rel2.11

I'm curious about how DOM improves IO500 scores.  :)
Also LSOM but I don't know actually whether that's in 2.11 or where.

On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf at cray.com<mailto:paf@cray.com>> wrote:

Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you?re interested in or something you?re concerned about?

- Patrick

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces@lists.lustre.org>> on behalf of Abe Asraoui <AbeA at supermicro.com<mailto:AbeA@supermicro.com>>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>; lustre-discuss at lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180718/b5109142/attachment.html>

[lustre-devel] MDT test in rel2.11

[John Bent]

Thanks Patrick.  That's interesting.  However, the exact motivation why
IO500 has the 'find' command is this same intended use case; stale results
therefore actually present an interesting dilemma to IO500.  They are not
POSIX compliant but that loss of compliance shouldn't necessarily
disqualify this result...

On Wed, Jul 18, 2018 at 12:49 AM, Patrick Farrell <paf@cray.com> wrote:

> Lazy SoM is not landed yet, and it won?t be improving benchmark scores -
> it?s never ?known 100% correct?, so it can?t be used for actual POSIX ops -
> if a file size read out is used for a write offset, then you?ve got data
> corruption.
>
> So for now it?s strictly limited to tools that know about it (accessed via
> an ioctl) and can accept information that may be stale.  The intended use
> case is scanning the FS for policy application.
>
> ------------------------------
> *From:* John Bent <johnbent@gmail.com>
> *Sent:* Tuesday, July 17, 2018 10:55:24 PM
> *To:* Patrick Farrell
> *Cc:* Abe Asraoui; lustre-devel at lists.lustre.org;
> lustre-discuss at lists.lustre.org
> *Subject:* Re: [lustre-devel] MDT test in rel2.11
>
> I'm curious about how DOM improves IO500 scores.  :)
> Also LSOM but I don't know actually whether that's in 2.11 or where.
>
> On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf@cray.com> wrote:
>
>
> Abe,
>
> Any benchmarking would be highly dependent on hardware, both client and
> server.  Is there a particular comparison (say, between versions) you?re
> interested in or something you?re concerned about?
>
> - Patrick
>
> ------------------------------
> *From:* lustre-devel <lustre-devel-bounces@lists.lustre.org> on behalf of
> Abe Asraoui <AbeA@supermicro.com>
> *Sent:* Tuesday, July 17, 2018 9:23:10 PM
> *To:* lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org; Abe
> Asraoui
> *Subject:* [lustre-devel] MDT test in rel2.11
>
> Hi All,
>
>
> Has anyone done any MDT testing under the latest rel2.11 and have
> benchmark data to share?
>
>
> Thanks,
> Abe
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180718/59074a8e/attachment-0001.html>

[lustre-devel] MDT test in rel2.11

[Patrick Farrell]

Yes, there is intention to add it to lfs find.  Whether or not it should disqualify results is up to you at I/O 500 - it seems like if most users would think it acceptable for find most of the time (and it should be), then it should probably be allowed.  But at the same time, its (theoretical - couldn?t today) use for mdtest would very much be ?writing to the benchmark? and defeating the intent.

________________________________
From: John Bent <johnbent@gmail.com>
Sent: Tuesday, July 17, 2018 11:54:32 PM
To: Patrick Farrell
Cc: Abe Asraoui; lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-devel] MDT test in rel2.11

Thanks Patrick.  That's interesting.  However, the exact motivation why IO500 has the 'find' command is this same intended use case; stale results therefore actually present an interesting dilemma to IO500.  They are not POSIX compliant but that loss of compliance shouldn't necessarily disqualify this result...

On Wed, Jul 18, 2018 at 12:49 AM, Patrick Farrell <paf at cray.com<mailto:paf@cray.com>> wrote:
Lazy SoM is not landed yet, and it won?t be improving benchmark scores - it?s never ?known 100% correct?, so it can?t be used for actual POSIX ops - if a file size read out is used for a write offset, then you?ve got data corruption.

So for now it?s strictly limited to tools that know about it (accessed via an ioctl) and can accept information that may be stale.  The intended use case is scanning the FS for policy application.

________________________________
From: John Bent <johnbent at gmail.com<mailto:johnbent@gmail.com>>
Sent: Tuesday, July 17, 2018 10:55:24 PM
To: Patrick Farrell
Cc: Abe Asraoui; lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>; lustre-discuss at lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>
Subject: Re: [lustre-devel] MDT test in rel2.11

I'm curious about how DOM improves IO500 scores.  :)
Also LSOM but I don't know actually whether that's in 2.11 or where.

On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf at cray.com<mailto:paf@cray.com>> wrote:

Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you?re interested in or something you?re concerned about?

- Patrick

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces@lists.lustre.org>> on behalf of Abe Asraoui <AbeA at supermicro.com<mailto:AbeA@supermicro.com>>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>; lustre-discuss at lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180718/b0c68d6a/attachment.html>

[lustre-devel] MDT test in rel2.11

[Cory Spitz]

Yes, if the IO500 is representing a use case where the file size or block count must be correct, then LSoM can?t be used.  However, the IO500 can be changed by consensus and perhaps there is a reason to include a use case which fits LSoM?

If so, the IO500 could be changed to allow `lfs find` and it in-turn could be used to get LSoM info as Andreas pointed out in his comment of LU-9538: https://jira.whamcloud.com/browse/LU-9538?focusedCommentId=230392&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-230392 .

-Cory


--


From: lustre-devel <lustre-devel-bounces@lists.lustre.org> on behalf of Patrick Farrell <paf@cray.com>
Date: Tuesday, July 17, 2018 at 11:50 PM
To: John Bent <johnbent@gmail.com>
Cc: "lustre-discuss at lists.lustre.org" <lustre-discuss@lists.lustre.org>, "lustre-devel at lists.lustre.org" <lustre-devel@lists.lustre.org>
Subject: Re: [lustre-devel] MDT test in rel2.11

Lazy SoM is not landed yet, and it won?t be improving benchmark scores - it?s never ?known 100% correct?, so it can?t be used for actual POSIX ops - if a file size read out is used for a write offset, then you?ve got data corruption.

So for now it?s strictly limited to tools that know about it (accessed via an ioctl) and can accept information that may be stale.  The intended use case is scanning the FS for policy application.
________________________________
From: John Bent <johnbent@gmail.com>
Sent: Tuesday, July 17, 2018 10:55:24 PM
To: Patrick Farrell
Cc: Abe Asraoui; lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-devel] MDT test in rel2.11

I'm curious about how DOM improves IO500 scores.  :)
Also LSOM but I don't know actually whether that's in 2.11 or where.

On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf at cray.com<mailto:paf@cray.com>> wrote:

Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you?re interested in or something you?re concerned about?

- Patrick
________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces@lists.lustre.org>> on behalf of Abe Asraoui <AbeA at supermicro.com<mailto:AbeA@supermicro.com>>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>; lustre-discuss at lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180718/123aac39/attachment-0001.html>

[lustre-devel] MDT test in rel2.11

[John Bent]

Certainly we can allow non-POSIX compliant "find" results in the list with
a column identifying them as being non-POSIX compliant.  The larger
question is whether one of these results can be considered a "winner".

On Wed, Jul 18, 2018 at 3:41 PM, Cory Spitz <spitzcor@cray.com> wrote:

> Yes, if the IO500 is representing a use case where the file size or block
> count must be correct, then LSoM can?t be used.  However, the IO500 can be
> changed by consensus and perhaps there is a reason to include a use case
> which fits LSoM?
>
>
>
> If so, the IO500 could be changed to allow `lfs find` and it in-turn could
> be used to get LSoM info as Andreas pointed out in his comment of LU-9538:
> https://jira.whamcloud.com/browse/LU-9538?focusedCommentId=230392&page=
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel#comment-230392 .
>
>
>
> -Cory
>
>
>
>
>
> --
>
>
>
>
>
> *From: *lustre-devel <lustre-devel-bounces@lists.lustre.org> on behalf of
> Patrick Farrell <paf@cray.com>
> *Date: *Tuesday, July 17, 2018 at 11:50 PM
> *To: *John Bent <johnbent@gmail.com>
> *Cc: *"lustre-discuss at lists.lustre.org" <lustre-discuss@lists.lustre.org>,
> "lustre-devel at lists.lustre.org" <lustre-devel@lists.lustre.org>
>
> *Subject: *Re: [lustre-devel] MDT test in rel2.11
>
>
>
> Lazy SoM is not landed yet, and it won?t be improving benchmark scores -
> it?s never ?known 100% correct?, so it can?t be used for actual POSIX ops -
> if a file size read out is used for a write offset, then you?ve got data
> corruption.
>
> So for now it?s strictly limited to tools that know about it (accessed via
> an ioctl) and can accept information that may be stale.  The intended use
> case is scanning the FS for policy application.
> ------------------------------
>
> *From:* John Bent <johnbent@gmail.com>
> *Sent:* Tuesday, July 17, 2018 10:55:24 PM
> *To:* Patrick Farrell
> *Cc:* Abe Asraoui; lustre-devel at lists.lustre.org;
> lustre-discuss at lists.lustre.org
> *Subject:* Re: [lustre-devel] MDT test in rel2.11
>
>
>
> I'm curious about how DOM improves IO500 scores.  :)
>
> Also LSOM but I don't know actually whether that's in 2.11 or where.
>
>
>
> On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf@cray.com> wrote:
>
>
> Abe,
>
> Any benchmarking would be highly dependent on hardware, both client and
> server.  Is there a particular comparison (say, between versions) you?re
> interested in or something you?re concerned about?
>
> - Patrick
> ------------------------------
>
> *From:* lustre-devel <lustre-devel-bounces@lists.lustre.org> on behalf of
> Abe Asraoui <AbeA@supermicro.com>
> *Sent:* Tuesday, July 17, 2018 9:23:10 PM
> *To:* lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org; Abe
> Asraoui
> *Subject:* [lustre-devel] MDT test in rel2.11
>
>
>
> Hi All,
>
>
> Has anyone done any MDT testing under the latest rel2.11 and have
> benchmark data to share?
>
>
> Thanks,
> Abe
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180718/7d793fc1/attachment.html>