RE: load-gen from an osd node

RE: load-gen from an osd node

[Deneau, Tom]

Oh, I just noticed that the client nodes I spoke of where load-gen actually worked were
running 0.94, not 9.0.1.  And when I upgrade them to 9.0.1, load-gen no longer works.

So more likely this is just a problem with newer rados load-gens

-- Tom



> -----Original Message-----
> From: Deneau, Tom
> Sent: Friday, June 26, 2015 7:48 PM
> To: ceph-devel
> Subject: load-gen from an osd node
> 
> I am running 9.0.1 and I noticed when I run rados load-gen from one of the
> osd nodes, it creates the objects but then always reports a throughput of 0
> MB/sec.
> 
> But if I run it from a separate client node, it works fine.
> Why would this be?
> 
> I'm not sure but I thought in earlier versions load-gen could be run from an
> osd node.
> 
> -- Tom Deneau

Re: load-gen from an osd node

[Gregory Farnum]

Hmm, the only changes I see between those two versions are some pretty
precise cleanups which shouldn't cause this. But it means that a
bisect or determined look should be easy. Can you create a ticket
which includes the exact output you're seeing and the exact versions
you're running?
-Greg

On Mon, Jun 29, 2015 at 11:27 PM, Deneau, Tom <tom.deneau@amd.com> wrote:
> Oh, I just noticed that the client nodes I spoke of where load-gen actually worked were
> running 0.94, not 9.0.1.  And when I upgrade them to 9.0.1, load-gen no longer works.
>
> So more likely this is just a problem with newer rados load-gens
>
> -- Tom
>
>
>
>> -----Original Message-----
>> From: Deneau, Tom
>> Sent: Friday, June 26, 2015 7:48 PM
>> To: ceph-devel
>> Subject: load-gen from an osd node
>>
>> I am running 9.0.1 and I noticed when I run rados load-gen from one of the
>> osd nodes, it creates the objects but then always reports a throughput of 0
>> MB/sec.
>>
>> But if I run it from a separate client node, it works fine.
>> Why would this be?
>>
>> I'm not sure but I thought in earlier versions load-gen could be run from an
>> osd node.
>>
>> -- Tom Deneau
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: load-gen from an osd node

[Deneau, Tom]

With the newer rados, just running  rados -p somepool load-gen, we get

preparing 200 objects
load-gen will run 60 seconds
    1: throughput=0MB/sec pending data=0
    2: throughput=0MB/sec pending data=0
    3: throughput=0MB/sec pending data=0
    4: throughput=0MB/sec pending data=0
    5: throughput=0MB/sec pending data=0
    6: throughput=0MB/sec pending data=0

But I see the change that caused this behavior was explicitly initializing lg.max_ops to 0.
Before this, max_ops was just whatever happened to be in memory.

So the newer load-gen works fine as long as you explicitly set --max-ops on the command line.

-- Tom Deneau


> -----Original Message-----
> From: Gregory Farnum [mailto:greg@gregs42.com]
> Sent: Wednesday, July 01, 2015 11:05 AM
> To: Deneau, Tom
> Cc: ceph-devel
> Subject: Re: load-gen from an osd node
> 
> Hmm, the only changes I see between those two versions are some pretty
> precise cleanups which shouldn't cause this. But it means that a bisect or
> determined look should be easy. Can you create a ticket which includes the
> exact output you're seeing and the exact versions you're running?
> -Greg
> 
> On Mon, Jun 29, 2015 at 11:27 PM, Deneau, Tom <tom.deneau@amd.com> wrote:
> > Oh, I just noticed that the client nodes I spoke of where load-gen
> > actually worked were running 0.94, not 9.0.1.  And when I upgrade them to
> 9.0.1, load-gen no longer works.
> >
> > So more likely this is just a problem with newer rados load-gens
> >
> > -- Tom
> >
> >
> >
> >> -----Original Message-----
> >> From: Deneau, Tom
> >> Sent: Friday, June 26, 2015 7:48 PM
> >> To: ceph-devel
> >> Subject: load-gen from an osd node
> >>
> >> I am running 9.0.1 and I noticed when I run rados load-gen from one
> >> of the osd nodes, it creates the objects but then always reports a
> >> throughput of 0 MB/sec.
> >>
> >> But if I run it from a separate client node, it works fine.
> >> Why would this be?
> >>
> >> I'm not sure but I thought in earlier versions load-gen could be run
> >> from an osd node.
> >>
> >> -- Tom Deneau
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> > in the body of a message to majordomo@vger.kernel.org More majordomo
> > info at  http://vger.kernel.org/majordomo-info.html

Re: load-gen from an osd node

[Gregory Farnum]

On Wed, Jul 8, 2015 at 5:17 PM, Deneau, Tom <tom.deneau@amd.com> wrote:
> With the newer rados, just running  rados -p somepool load-gen, we get
>
> preparing 200 objects
> load-gen will run 60 seconds
>     1: throughput=0MB/sec pending data=0
>     2: throughput=0MB/sec pending data=0
>     3: throughput=0MB/sec pending data=0
>     4: throughput=0MB/sec pending data=0
>     5: throughput=0MB/sec pending data=0
>     6: throughput=0MB/sec pending data=0
>
> But I see the change that caused this behavior was explicitly initializing lg.max_ops to 0.
> Before this, max_ops was just whatever happened to be in memory.
>
> So the newer load-gen works fine as long as you explicitly set --max-ops on the command line.

Can you create a ticket on the tracker describing this issue? That's
the kind of thing we'll want to fix. ;)
-Greg

>
> -- Tom Deneau
>
>
>> -----Original Message-----
>> From: Gregory Farnum [mailto:greg@gregs42.com]
>> Sent: Wednesday, July 01, 2015 11:05 AM
>> To: Deneau, Tom
>> Cc: ceph-devel
>> Subject: Re: load-gen from an osd node
>>
>> Hmm, the only changes I see between those two versions are some pretty
>> precise cleanups which shouldn't cause this. But it means that a bisect or
>> determined look should be easy. Can you create a ticket which includes the
>> exact output you're seeing and the exact versions you're running?
>> -Greg
>>
>> On Mon, Jun 29, 2015 at 11:27 PM, Deneau, Tom <tom.deneau@amd.com> wrote:
>> > Oh, I just noticed that the client nodes I spoke of where load-gen
>> > actually worked were running 0.94, not 9.0.1.  And when I upgrade them to
>> 9.0.1, load-gen no longer works.
>> >
>> > So more likely this is just a problem with newer rados load-gens
>> >
>> > -- Tom
>> >
>> >
>> >
>> >> -----Original Message-----
>> >> From: Deneau, Tom
>> >> Sent: Friday, June 26, 2015 7:48 PM
>> >> To: ceph-devel
>> >> Subject: load-gen from an osd node
>> >>
>> >> I am running 9.0.1 and I noticed when I run rados load-gen from one
>> >> of the osd nodes, it creates the objects but then always reports a
>> >> throughput of 0 MB/sec.
>> >>
>> >> But if I run it from a separate client node, it works fine.
>> >> Why would this be?
>> >>
>> >> I'm not sure but I thought in earlier versions load-gen could be run
>> >> from an osd node.
>> >>
>> >> -- Tom Deneau
>> > --
>> > To unsubscribe from this list: send the line "unsubscribe ceph-devel"
>> > in the body of a message to majordomo@vger.kernel.org More majordomo
>> > info at  http://vger.kernel.org/majordomo-info.html

load-gen from an osd node

[Deneau, Tom]

I am running 9.0.1 and I noticed when I run rados load-gen from one of the osd nodes,
it creates the objects but then always reports a throughput of 0 MB/sec.

But if I run it from a separate client node, it works fine.
Why would this be?

I'm not sure but I thought in earlier versions load-gen could be run from an osd node.

-- Tom Deneau