minor file journal size bug

minor file journal size bug

[Phil Carns]

[-- Attachment #1: Type: text/plain, Size: 540 bytes --]

Hi,

I ran into a minor problem when configuring the ObjectStore to use a 
file journal that was 10 gigabytes in size (osd_journal_size = 10240).  
The osd_journal_size configuration parameter is an integer.  This would 
normally be fine since the units are in megabytes, but it can overflow 
when bit shifted in FileJournal.cc.  This causes the journal to fall 
back to trying to detect a raw block device journal.

I've attached a small patch that fixes the problem for me, but there is 
probably a cleaner way to do it.

thanks,
-Phil



[-- Attachment #2: ceph-0.20.2-journal-size-overflow.patch --]
[-- Type: text/x-patch, Size: 746 bytes --]

diff -Naupr ceph-0.20.2/src/os/FileJournal.cc ceph-0.20.2-patched/src/os/FileJournal.cc
--- ceph-0.20.2/src/os/FileJournal.cc	2010-05-26 18:52:59.000000000 -0500
+++ ceph-0.20.2-patched/src/os/FileJournal.cc	2010-06-14 08:54:22.416230668 -0500
@@ -58,8 +58,8 @@ int FileJournal::_open(bool forwrite, bo
   max_size = st.st_size;
   block_size = st.st_blksize;
 
-  if (create && max_size < (g_conf.osd_journal_size << 20)) {
-    uint64_t newsize = g_conf.osd_journal_size << 20;
+  if (create && max_size < (((uint64_t)g_conf.osd_journal_size) << 20)) {
+    uint64_t newsize = ((uint64_t)g_conf.osd_journal_size) << 20;
     dout(10) << "_open extending to " << newsize << " bytes" << dendl;
     r = ::ftruncate(fd, newsize);
     if (r == 0)

Re: minor file journal size bug

[Yehuda Sadeh Weinraub]

On Mon, Jun 14, 2010 at 7:19 AM, Phil Carns <carns@mcs.anl.gov> wrote:
> Hi,
>
> I ran into a minor problem when configuring the ObjectStore to use a file
> journal that was 10 gigabytes in size (osd_journal_size = 10240).  The
> osd_journal_size configuration parameter is an integer.  This would normally
> be fine since the units are in megabytes, but it can overflow when bit
> shifted in FileJournal.cc.  This causes the journal to fall back to trying
> to detect a raw block device journal.
>
> I've attached a small patch that fixes the problem for me, but there is
> probably a cleaner way to do it.
>

Thanks!

Just defining osd_journal_size as a long long would do the trick (as
follows), although I'm not sure whether we need to be able to define
journals that are bigger than 2^32 MB. So I guess that in this case we
should probably go with your solution, and also go over the code and
check to see whether we have some other places where we do the same.
We should really be using some macro that does this conversion with a
correct cast.

Yehuda

diff --git a/src/config.cc b/src/config.cc
index 9615186..bb4e2c9 100644
--- a/src/config.cc
+++ b/src/config.cc
@@ -431,7 +431,7 @@ static struct config_option config_optionsp[] = {
        OPTION(mds_kill_import_at, 0, OPT_INT, 0),
        OPTION(osd_data, 0, OPT_STR, ""),
        OPTION(osd_journal, 0, OPT_STR, ""),
-       OPTION(osd_journal_size, 0, OPT_INT, 0),         // in mb
+       OPTION(osd_journal_size, 0, OPT_LONGLONG, 0),         // in mb
        OPTION(osd_balance_reads, 0, OPT_BOOL, false),
        OPTION(osd_flash_crowd_iat_threshold, 0, OPT_INT, 0),
        OPTION(osd_flash_crowd_iat_alpha, 0, OPT_DOUBLE, 0.125),
diff --git a/src/config.h b/src/config.h
index 68dca6f..45206f6 100644
--- a/src/config.h
+++ b/src/config.h
@@ -290,7 +290,7 @@ struct md_config_t {
   // osd
   const char *osd_data;
   const char *osd_journal;
-  int osd_journal_size;  // in mb
+  long long osd_journal_size;  // in mb
   bool osd_balance_reads;
   int osd_flash_crowd_iat_threshold;  // flash crowd interarrival
time threshold in ms
   double osd_flash_crowd_iat_alpha;
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: minor file journal size bug

[Phil Carns]

On 06/14/2010 01:37 PM, Yehuda Sadeh Weinraub wrote:
> On Mon, Jun 14, 2010 at 7:19 AM, Phil Carns<carns@mcs.anl.gov>  wrote:
>    
>> Hi,
>>
>> I ran into a minor problem when configuring the ObjectStore to use a file
>> journal that was 10 gigabytes in size (osd_journal_size = 10240).  The
>> osd_journal_size configuration parameter is an integer.  This would normally
>> be fine since the units are in megabytes, but it can overflow when bit
>> shifted in FileJournal.cc.  This causes the journal to fall back to trying
>> to detect a raw block device journal.
>>
>> I've attached a small patch that fixes the problem for me, but there is
>> probably a cleaner way to do it.
>>
>>      
> Thanks!
>
> Just defining osd_journal_size as a long long would do the trick (as
> follows), although I'm not sure whether we need to be able to define
> journals that are bigger than 2^32 MB. So I guess that in this case we
> should probably go with your solution, and also go over the code and
> check to see whether we have some other places where we do the same.
> We should really be using some macro that does this conversion with a
> correct cast.
>
> Yehuda
>    

Thanks Yehuda.  I don't know what the right long term answer is, but 
either way works fine for my test case (I just tried your patch to confirm).

-Phil