All of lore.kernel.org
 help / color / mirror / Atom feed
* Ceph 0.23.2 Consolidated questions
@ 2011-01-13  1:24 DongJin Lee
  2011-01-13 17:32 ` Gregory Farnum
  0 siblings, 1 reply; 8+ messages in thread
From: DongJin Lee @ 2011-01-13  1:24 UTC (permalink / raw)
  To: ceph-devel

Hi all:

After some time, I've got several consolidated (hopefully trivial) questions.
I've tried to cut them short, sorry for asking too many at once.

kernel 2.6.36-02063601-generic (23-Nov-2010)
ceph version 0.23.2 (commit:
5bdae2af8c53adb2e059022c58813e97e7a7ba5d)

1.
about the bug track: http://tracker.newdream.net/issues/479
when using Ext4, I now found that the hang happens when a client runs
on the same machine as the ceph-system.
But the hang does not happen when a client runs on different machine.
i.e.,
- machine1 (MDS,MON,OSD, plus Client mounts) = hangs
- machine1 (MDS,MON, OSD, plus separate (machine2) Client mounts) = ok.
I think this is because of the same IP address (loopback) with some
sync mismatch..?

For btrfs, the only no-hang way is to disable journaling (writeahead
and writeparallel).
DD copies keep crashing (mount hangs and makes it pretty much
unmountable again until hard reboot) at about 576MB.


2.
For benchmarking the entire ceph system in general, could you
recommend which tools to use, particularly best for Object based
storages?
Would the block based storage benchmark, e.g., Iometer, be a bad
choice?, and why and why not?
I'm running fio (iometer-alike) to do random/sequentail read/write.
I did rados test, too. 'rados -p metadata bench write' command should
be writing to mds node, but it was doing it on osd nodes (i.e,, same
as the data node)
also, 'seq' crashes with lots of errors.


3.
I've got four powerful machines, 2 for osd nodes, 1 for mds/mon
together and 1 for clients. But I was mainly testing for just 1 osd
node for a basic test.
The tests are independently ran for SSDs first and then again for HDDs.
(These disks are connected to HW RAID controllers (but all individual
configured for recommended optimal for servers,
e.g., 8k, directio, directcache, (writethru for ssd, and writeback for hdd)
The repl rules are all set to 1x (and again to default 2x). All osds
use 10G journal size (journal write ahead, parallel, and noJournal,
and ext4)
all links are 2Gb links and so I get near 230MB/s iperf.
Once Ceph started, I first do a large single-file dd copy (40GB for
SSDs, and 500GB for HDDs),
twiki.esc.auckland.ac.nz/twiki/bin/view/NDSG/Cephtest

the plot is here, for DD
twiki.esc.auckland.ac.nz/twiki/pub/NDSG/Cephtest/ddplot.pdf

After the files are created, I ran iometer-like (fio) for
random/sequential read/write (RR, SR, RW, and SW) to the file I just
created (i.e., 40GB for ssd and 500GB for hdd)
The results are here, (note, the plot is quite 'concised' and dense!)
twiki.esc.auckland.ac.nz/twiki/pub/NDSG/Cephtest/fioplot.pdf

The only understandable parts are the sequential 'read' parts, writes
are slow, and I'm not necessarily seeing any 'gaps' observed for any
osd change.

Questions.
- When journal size = 10000, why does the journal file in the /data/
shows as 1.8GB ?
- Also, journal write ahead or parallel? which mode do I need to set
for ssd or harddisk? (the plot answers, basically ahead/parallel okay)
- Increasing osds (1 to 3 to 6) doesnt seem to increase any
performance at all (if so, hardly by a few %).
- In some cases, 6osds is lower than 3osds (for hdds DD copy in
particular, 1osd to 3osd increase, but 6osds deteriorates badly)

- so I ended up using 2 nodes, with (1+1, 2+2, and 3+3 osd config),
and for hdds it seems to have increased linearly for DD.
- so why is having 6osds in 1 node, is slower than 3+3osds in 2 nodes?
at least some bottlenecks should occur in-between the nodes. Crushmap
appears to be the same.
I was really expecting to see some linear increase or even visible
increase as I add more disks, but appears not at all...

4.
For the PG setup, using a simple ceph.conf (just osd[123] to a single machine)
and setting the repl to x1, how would I expect the objects to get
mapped to disks?
How do I really configure and control how things go?
Currently,

rule data {
    ruleset 0
    type replicated
    min_size 1
    max_size 10
    step take root
    step choose firstn 0 type device
    step emit
}

Is it normal for it to be always 'type replicated' ?
Does it supposed to be different when repl set to x2, x3, etc?
How do I check and correctly make sure that the data are distrbuted
among all 3 (or 6) osds? (for both write and read, like the RAID0)

Using crushtool --num_osds 3 -o file --build host straw 0 root straw 0

# buckets
host host0 {
    id -1        # do not change unnecessarily
    alg straw
    hash 0    # rjenkins1
    item device0 weight 1.000
}
...(omitted for host1 and host2)
root root {
    id -4        # do not change unnecessarily
    alg straw
    hash 0    # rjenkins1
    item host0 weight 1.000
    item host1 weight 1.000
    item host2 weight 1.000
}
# rules
rule data {
    ruleset 1
    type replicated
    min_size 2
    max_size 2
    step take root
    step chooseleaf firstn 0 type host
    step emit
}

What exactly is the difference between the above and the existing map
(from default ceph.conf)?
Why is the ruleset starts from 1?
Does it have to be different for each rule?
where is this number min/max 2 came from?
why is it set to chooseleaf? (what mode do I set to 'choose'?)
Also, it doesn't list metadata rule, so I had to manually add that in.

I've added --crushmapsrc file.txt when mkcephfs, but when I read it
back from ceph, it seems to return the normal one (i.e., ceph.conf
regenerated)
So I end up using the online approach after the ceph started, and
mounted, but as I write files, it hangs.

5.
Other than the ceph, crush and rados papers, Has there been any ceph
work done on the applications/performances/benchmark comparisons using
others?


Thanks a lot in advance, and again sorry for asking too many.
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Ceph 0.23.2 Consolidated questions
  2011-01-13  1:24 Ceph 0.23.2 Consolidated questions DongJin Lee
@ 2011-01-13 17:32 ` Gregory Farnum
  2011-01-17  4:04   ` DongJin Lee
  0 siblings, 1 reply; 8+ messages in thread
From: Gregory Farnum @ 2011-01-13 17:32 UTC (permalink / raw)
  To: DongJin Lee; +Cc: ceph-devel

On Wed, Jan 12, 2011 at 5:24 PM, DongJin Lee <dongjin.lee@auckland.ac.nz> wrote:
> 1.
> about the bug track: http://tracker.newdream.net/issues/479
> when using Ext4, I now found that the hang happens when a client runs
> on the same machine as the ceph-system.
> But the hang does not happen when a client runs on different machine.
I haven't looked at any of this closely, but how's your memory use?
One reason we recommend not running clients on the same nodes as OSDs
is because if you run into memory pressure you can deadlock if trying
to write to the Ceph mount.

> 2.
> For benchmarking the entire ceph system in general, could you
> recommend which tools to use, particularly best for Object based
> storages?
Well, I think the tools should mimic whatever workload you expect to run on it!
> Would the block based storage benchmark, e.g., Iometer, be a bad
> choice?, and why and why not?
> I'm running fio (iometer-alike) to do random/sequentail read/write.
Tools like this are common choices, yes.
> I did rados test, too. 'rados -p metadata bench write' command should
> be writing to mds node, but it was doing it on osd nodes (i.e,, same
> as the data node)
I think you're misunderstanding how the MDS data storage works. They
don't store any data locally, all MDS data goes in the metadata pool,
which is stored on the OSDs. Pools are logical groupings of data
within RADOS.

> also, 'seq' crashes with lots of errors.
Hmmm. The rados bencher was always a pretty lightweight and fragile
tool, but you may have more luck if you build the tool in a newer
version of Ceph. Sam made a lot of changes to strengthen that stuff.

> Questions.
> - When journal size = 10000, why does the journal file in the /data/
> shows as 1.8GB ?
Is the journal a consistent 1.8GB, or was that just at one point (like
the end of a run)? IIRC, the journal size is the most space it will
use. A 10GB journal is very generous so probably the data is just
being flushed from the journal to the data store quickly enough that
the OSD doesn't need to keep around 10GB of data to guarantee
consistency in the event of a power failure.
> - Also, journal write ahead or parallel? which mode do I need to set
> for ssd or harddisk? (the plot answers, basically ahead/parallel okay)
In write ahead mode, all data goes to the journal and then to the data
store. In parallel, it gets sent to both at the same time.
Since you're running on ext4, you should be using writeahead as
parallel isn't actually safe. I believe that the OSD is actually
detecting that and switching you to writeahead, there should be a
warning somewhere. If it's not switching, that's a bug!

> - Increasing osds (1 to 3 to 6) doesnt seem to increase any
> performance at all (if so, hardly by a few %).
> - In some cases, 6osds is lower than 3osds (for hdds DD copy in
> particular, 1osd to 3osd increase, but 6osds deteriorates badly)
>
> - so I ended up using 2 nodes, with (1+1, 2+2, and 3+3 osd config),
> and for hdds it seems to have increased linearly for DD.
> - so why is having 6osds in 1 node, is slower than 3+3osds in 2 nodes?
You have some odd constraints here that are probably interacting to
produce strange results:
1) Your journal and your data store are on the same device. You seem
to have enough disks available that I'd recommend separating them.
2) You have multiple cosds sharing network connections. This will
produce some weird interactions in terms of bandwidth usage,
especially if you don't have a hierarchical crushmap which keeps
replicas on different physical hosts.
> I was really expecting to see some linear increase or even visible
> increase as I add more disks, but appears not at all...
In general, I'd recommend running tests with a few more
configurations. Try separating your journal and data store onto
separate disks, and see how your OSDs do then. Try combining all your
disks under your RAID card, running in no-journal mode (how well this
works will correlate directly with how good your RAID card is).

> 4.
> For the PG setup, using a simple ceph.conf (just osd[123] to a single machine)
> and setting the repl to x1, how would I expect the objects to get
> mapped to disks?
> How do I really configure and control how things go?
I'll leave your questions about the CRUSH map to others, except for one note:
The pseudo-random placement of data is one of Ceph's greatest
strengths. It provides you mechanisms for working around this in
certain cases, but if it's something you need to do often you might
want to look into another distributed FS which expects you to be
placing data manually.

> 5.
> Other than the ceph, crush and rados papers, Has there been any ceph
> work done on the applications/performances/benchmark comparisons using
> others?
We haven't done much in-house work benchmarking Ceph recently,
although we hope to start again soon (we're actively looking for QA
people, and this will be part of their job). I've seen a few papers
floating around that test Ceph in comparison to other distributed
FSes, but at this point they're pretty outdated. :)

> Thanks a lot in advance, and again sorry for asking too many.
No problem, we're happy to help!
-Greg

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Ceph 0.23.2 Consolidated questions
  2011-01-13 17:32 ` Gregory Farnum
@ 2011-01-17  4:04   ` DongJin Lee
  2011-01-18  6:53     ` Gregory Farnum
  0 siblings, 1 reply; 8+ messages in thread
From: DongJin Lee @ 2011-01-17  4:04 UTC (permalink / raw)
  To: Gregory Farnum, Sage Weil; +Cc: ceph-devel

>> 2.
>> For benchmarking the entire ceph system in general, could you
>> recommend which tools to use, particularly best for Object based
>> storages?
> Well, I think the tools should mimic whatever workload you expect to run on it!
>> Would the block based storage benchmark, e.g., Iometer, be a bad
>> choice?, and why and why not?
>> I'm running fio (iometer-alike) to do random/sequentail read/write.
> Tools like this are common choices, yes.

Great, so I'll use the opensource spc-1 benchmark, too.
I'm testing dbench also, will see how it goes..

>> Questions.
>> - When journal size = 10000, why does the journal file in the /data/
>> shows as 1.8GB ?
> Is the journal a consistent 1.8GB, or was that just at one point (like
> the end of a run)? IIRC, the journal size is the most space it will
> use. A 10GB journal is very generous so probably the data is just
> being flushed from the journal to the data store quickly enough that
> the OSD doesn't need to keep around 10GB of data to guarantee
> consistency in the event of a power failure.
I find that the journal sizes are consistent up to about 2040, after
things starts to get weird.
So this is what I get when I set journal size
size 1000 - 1 GB
size 2000 - 2 GB
size 2050 - 0 MB
size 2500 - 0 MB
size 3000 - 0 MB
size 4000 - 0 MB
size 5000 - 904 MB
size 10000 - 1.8 GB
when journal gets the 0 MB then I obviously get the 'unable to open
superblock' error.


>> - Increasing osds (1 to 3 to 6) doesnt seem to increase any
>> performance at all (if so, hardly by a few %).
>> - In some cases, 6osds is lower than 3osds (for hdds DD copy in
>> particular, 1osd to 3osd increase, but 6osds deteriorates badly)
>>
>> - so I ended up using 2 nodes, with (1+1, 2+2, and 3+3 osd config),
>> and for hdds it seems to have increased linearly for DD.
>> - so why is having 6osds in 1 node, is slower than 3+3osds in 2 nodes?
> You have some odd constraints here that are probably interacting to
> produce strange results:
> 1) Your journal and your data store are on the same device. You seem
> to have enough disks available that I'd recommend separating them.

If it was a journaling issue, hmm, the ddplot still shows .. (see hdd-no1)
no journal 1osd  - 68.5 MB/s
no journal 3osds - 68.4 MB/s
no journal 6osds - 67.7 MB/s

Even without the journal, there's no linear or alike improvements.
If possible, I'd like to keep the journal in the same disk.. Using the
'extra' disk for the journal (or perhaps even using tmpfs) doesn't
sound appealing as far as performance measurements are concerned, at
least in some fair-testing sense.
It's leveraging too much..?; then any other fs or disks can do this as
well to improve, correct me please?
I'm really mainly testing for a single, easy and reliable out-of-box
setup, and would like to add more osds to see the clear increments of
performance. But I haven't yet managed to get this.

> 2) You have multiple cosds sharing network connections. This will
> produce some weird interactions in terms of bandwidth usage,
> especially if you don't have a hierarchical crushmap which keeps
> replicas on different physical hosts.
The real major concern was the performance scalability, I was testing
all x1, no replications.
And as the ddplot and benchmark plot shows, there's just no evidence
of any scalability,
(and I did x2 as well)

>> I was really expecting to see some linear increase or even visible
>> increase as I add more disks, but appears not at all...
> In general, I'd recommend running tests with a few more
> configurations. Try separating your journal and data store onto
> separate disks, and see how your OSDs do then. Try combining all your
> disks under your RAID card, running in no-journal mode (how well this
> works will correlate directly with how good your RAID card is).

Ok, I will try this, :]
For the HDD osds, I should then use an extra dedicated SSD just for
journal purpose? If so, then wouldn't the benchmark results be just
the measurement of this SSD in the end?
The RAID card is quite a good one (512MB ddr2 ram with battery), I
wanted to stay away from any RAID0/1 configuration, basically leaving
up to ceph/osd to handle!.
The only reason why all of the disks (SSDs and HDDs) are connected to
RAID card is because of the typical server-grade environment (3U
supporting 16 disks), and the internal SATAs are only limited to 6.
I know there's a clear performance difference between the HW vs SW
controllers (mainly the ram). Currently all of the disks are already
advantaged by the RAID. so the previous plot results should already be
'better' than the internal SATA.


>
>> 4.
>> For the PG setup, using a simple ceph.conf (just osd[123] to a single machine)
>> and setting the repl to x1, how would I expect the objects to get
>> mapped to disks?
>> How do I really configure and control how things go?
> I'll leave your questions about the CRUSH map to others, except for one note:
> The pseudo-random placement of data is one of Ceph's greatest
> strengths. It provides you mechanisms for working around this in
> certain cases, but if it's something you need to do often you might
> want to look into another distributed FS which expects you to be
> placing data manually.

Right, say if I have say 10 objects, how would this be written/read to
the 2osds? for both 1x and 2x.
By default configuration, x2 with 2osds, will make the objects reside
on each osd (basically identical to the mirrored)
If not, how to manually configure (OSDs) to behave like RAID 0 and 1 ?
Can you give me an example please?
Seeing the crush example on the wiki 'A CRUSH map with 12 hosts, 4
hosts per rack and 3 racks',
It just shows the hierarchy and doesn't say which part will be used
for e.g., 1x, 2x or 3x replc, and so on..
and as I understand, 4 types of buckets are useful for adding/removing?
Again, the cpus and ram usages are not the bottleneck when running the
benchmark I see.
I think Sage's crush talked about multiple take and emit, so do I need
like take root1, and take root2 ?

Thanks a lot.
DJ

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Ceph 0.23.2 Consolidated questions
  2011-01-17  4:04   ` DongJin Lee
@ 2011-01-18  6:53     ` Gregory Farnum
  2011-01-18 19:54       ` Gregory Farnum
  0 siblings, 1 reply; 8+ messages in thread
From: Gregory Farnum @ 2011-01-18  6:53 UTC (permalink / raw)
  To: DongJin Lee; +Cc: Sage Weil, ceph-devel

On Sun, Jan 16, 2011 at 8:04 PM, DongJin Lee <dongjin.lee@auckland.ac.nz> wrote:
> I find that the journal sizes are consistent up to about 2040, after
> things starts to get weird.
> So this is what I get when I set journal size
> size 1000 - 1 GB
> size 2000 - 2 GB
> size 2050 - 0 MB
> size 2500 - 0 MB
> size 3000 - 0 MB
> size 4000 - 0 MB
> size 5000 - 904 MB
> size 10000 - 1.8 GB
> when journal gets the 0 MB then I obviously get the 'unable to open
> superblock' error.
Hmmm, I'll try and look into this a bit, or get somebody else to do so.

> Even without the journal, there's no linear or alike improvements.
> If possible, I'd like to keep the journal in the same disk.. Using the
> 'extra' disk for the journal (or perhaps even using tmpfs) doesn't
> sound appealing as far as performance measurements are concerned, at
> least in some fair-testing sense.
> It's leveraging too much..?; then any other fs or disks can do this as
> well to improve, correct me please?
> I'm really mainly testing for a single, easy and reliable out-of-box
> setup, and would like to add more osds to see the clear increments of
> performance. But I haven't yet managed to get this.
Well, if I were testing performance on distributed systems I'd want to
get the most I could out of them all during my testing.
If you're concerned about lack of scaling, I guess I'd just say that
it seems like you're stacking the deck against it a bit, since you do
have a number of oddly interacting constraints that I don't think
anybody knows how to optimize for yet. :)

> The real major concern was the performance scalability, I was testing
> all x1, no replications.
> And as the ddplot and benchmark plot shows, there's just no evidence
> of any scalability,
> (and I did x2 as well)
Oh, this reminds me. How were you setting your replication levels?

> For the HDD osds, I should then use an extra dedicated SSD just for
> journal purpose? If so, then wouldn't the benchmark results be just
> the measurement of this SSD in the end?
Probably you'd want to run it in a configuration you could expect to
use if you decided to use Ceph. I know some people have done this and
been pleased, although I doubt it's worth it with a high-end RAID
card. :)

> The RAID card is quite a good one (512MB ddr2 ram with battery), I
> wanted to stay away from any RAID0/1 configuration, basically leaving
> up to ceph/osd to handle!.
> The only reason why all of the disks (SSDs and HDDs) are connected to
> RAID card is because of the typical server-grade environment (3U
> supporting 16 disks), and the internal SATAs are only limited to 6.
> I know there's a clear performance difference between the HW vs SW
> controllers (mainly the ram). Currently all of the disks are already
> advantaged by the RAID. so the previous plot results should already be
> 'better' than the internal SATA.
Mmmm. I don't know that much about RAID cards, but I suspect you'll
get better performance out of a RAID underlying the OSD than out of 3
OSDs running through the RAID card because if you're actually RAIDing
the disks the card can be a bit smarter about handling flushes and
stuff. :)


Also, one possibility I forgot to consider/mention is that we've seen
a lot of clusters where performance seems abnormally low and it turns
out some of the disks are significantly slower than the others, and
this ends up having a large performance cluster-wide. Dealing with
this isn't something we've started work on yet, but you can at least
check for it, as described here:
http://ceph.newdream.net/wiki/Troubleshooting#OSD_performance

> Right, say if I have say 10 objects, how would this be written/read to
> the 2osds? for both 1x and 2x.
Well, CRUSH uses a pseudo-random mapping. Some of the objects will end
up on osd0 and some on osd1. In the large scale, it'll be 50% on each.
This is definitely not RAID0, though: it's possible (though unlikely)
that all 10 objects will go to osd0. With 2x replication....
> By default configuration, x2 with 2osds, will make the objects reside
> on each osd (basically identical to the mirrored)
Yep!
> If not, how to manually configure (OSDs) to behave like RAID 0 and 1 ?
You really can't get RAID0 out of RADOS using any reasonably means,
unless you want to take full responsibility for data placement in the
application layer. And at that point, you're better off with a
different storage solution.
> Can you give me an example please?
Maybe somebody else, but I'm just not that good with CRUSH maps. :/
> Seeing the crush example on the wiki 'A CRUSH map with 12 hosts, 4
> hosts per rack and 3 racks',
> It just shows the hierarchy and doesn't say which part will be used
> for e.g., 1x, 2x or 3x replc, and so on..
> and as I understand, 4 types of buckets are useful for adding/removing?
> Again, the cpus and ram usages are not the bottleneck when running the
> benchmark I see.
> I think Sage's crush talked about multiple take and emit, so do I need
> like take root1, and take root2 ?
Again, I'm not that good with CRUSH maps.
-Greg

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Ceph 0.23.2 Consolidated questions
  2011-01-18  6:53     ` Gregory Farnum
@ 2011-01-18 19:54       ` Gregory Farnum
  2011-01-20 17:13         ` tsuna
  0 siblings, 1 reply; 8+ messages in thread
From: Gregory Farnum @ 2011-01-18 19:54 UTC (permalink / raw)
  To: DongJin Lee; +Cc: Sage Weil, ceph-devel

On Mon, Jan 17, 2011 at 10:53 PM, Gregory Farnum <gregf@hq.newdream.net> wrote:
> On Sun, Jan 16, 2011 at 8:04 PM, DongJin Lee <dongjin.lee@auckland.ac.nz> wrote:
>> I find that the journal sizes are consistent up to about 2040, after
>> things starts to get weird.
>> So this is what I get when I set journal size
>> size 1000 - 1 GB
>> size 2000 - 2 GB
>> size 2050 - 0 MB
>> size 2500 - 0 MB
>> size 3000 - 0 MB
>> size 4000 - 0 MB
>> size 5000 - 904 MB
>> size 10000 - 1.8 GB
>> when journal gets the 0 MB then I obviously get the 'unable to open
>> superblock' error.
> Hmmm, I'll try and look into this a bit, or get somebody else to do so.
Are you running on 32-bit machines? It looks like we're accidentally
overflowing a 32-bit value, but I don't have any 32-bit machines handy
to test it on. Please apply the patch at http://pastebin.com/yahTxRzQ,
recompile, and let me know if you still have journal size issues. :)
-Greg

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Ceph 0.23.2 Consolidated questions
  2011-01-18 19:54       ` Gregory Farnum
@ 2011-01-20 17:13         ` tsuna
  2011-01-21 22:14           ` Gregory Farnum
  0 siblings, 1 reply; 8+ messages in thread
From: tsuna @ 2011-01-20 17:13 UTC (permalink / raw)
  To: Gregory Farnum; +Cc: DongJin Lee, Sage Weil, ceph-devel

On Tue, Jan 18, 2011 at 11:54 AM, Gregory Farnum <gregf@hq.newdream.net> wrote:
> Are you running on 32-bit machines? It looks like we're accidentally
> overflowing a 32-bit value, but I don't have any 32-bit machines handy
> to test it on. Please apply the patch at http://pastebin.com/yahTxRzQ,
> recompile, and let me know if you still have journal size issues. :)

There's a typo in your patch, "newsize << 20;" should be "newsize <<= 20;"

-- 
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Ceph 0.23.2 Consolidated questions
  2011-01-20 17:13         ` tsuna
@ 2011-01-21 22:14           ` Gregory Farnum
  2011-01-25 22:36             ` Gregory Farnum
  0 siblings, 1 reply; 8+ messages in thread
From: Gregory Farnum @ 2011-01-21 22:14 UTC (permalink / raw)
  To: ceph-devel; +Cc: DongJin Lee, tsuna

On Thu, Jan 20, 2011 at 9:13 AM, tsuna <tsunanet@gmail.com> wrote:
> On Tue, Jan 18, 2011 at 11:54 AM, Gregory Farnum <gregf@hq.newdream.net> wrote:
>> Are you running on 32-bit machines? It looks like we're accidentally
>> overflowing a 32-bit value, but I don't have any 32-bit machines handy
>> to test it on. Please apply the patch at http://pastebin.com/yahTxRzQ,
>> recompile, and let me know if you still have journal size issues. :)
>
> There's a typo in your patch, "newsize << 20;" should be "newsize <<= 20;"
Oh, yep. Thanks for that. It's fixed on the patch page and just got
pushed into the testing branch.

On Thu, Jan 20, 2011 at 8:40 PM, DongJin Lee <dongjin.lee@auckland.ac.nz> wrote:
> Greg: Thanks for the various supports (just emailing you since it's
> minor updates)
> and sorry for late reply, we were thinking to do your recommended
> updates, but we *always* used 64bit version..
> we followed the debian installation guide and apt-get all once as
> guided on the Wiki.
Hmmm. Well, your final two journal sizes are definitely overflowing
somehow, though I don't know why that would happen on a 64-bit
machine. I'm still not sure what's going on in the middle ones. Let us
know if they persist after .24.2 (which should be going out today). :)
-Greg

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Ceph 0.23.2 Consolidated questions
  2011-01-21 22:14           ` Gregory Farnum
@ 2011-01-25 22:36             ` Gregory Farnum
  0 siblings, 0 replies; 8+ messages in thread
From: Gregory Farnum @ 2011-01-25 22:36 UTC (permalink / raw)
  To: ceph-devel; +Cc: DongJin Lee, tsuna

On Fri, Jan 21, 2011 at 2:14 PM, Gregory Farnum <gregf@hq.newdream.net> wrote:
> On Thu, Jan 20, 2011 at 8:40 PM, DongJin Lee <dongjin.lee@auckland.ac.nz> wrote:
>> Greg: Thanks for the various supports (just emailing you since it's
>> minor updates)
>> and sorry for late reply, we were thinking to do your recommended
>> updates, but we *always* used 64bit version..
>> we followed the debian installation guide and apt-get all once as
>> guided on the Wiki.
> Hmmm. Well, your final two journal sizes are definitely overflowing
> somehow, though I don't know why that would happen on a 64-bit
> machine. I'm still not sure what's going on in the middle ones. Let us
> know if they persist after .24.2 (which should be going out today). :)
> -Greg

I just checked this briefly on version .24.2 and was unable to
reproduce any issues. Let me know if you encounter them again!
-Greg

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2011-01-25 22:36 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-01-13  1:24 Ceph 0.23.2 Consolidated questions DongJin Lee
2011-01-13 17:32 ` Gregory Farnum
2011-01-17  4:04   ` DongJin Lee
2011-01-18  6:53     ` Gregory Farnum
2011-01-18 19:54       ` Gregory Farnum
2011-01-20 17:13         ` tsuna
2011-01-21 22:14           ` Gregory Farnum
2011-01-25 22:36             ` Gregory Farnum

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.