Ceph status for Wheezy

Ceph status for Wheezy

[Laszlo Boszormenyi (GCS)]

Hi Sage,

As previously noted, using leveldb caused some trouble with Ceph could
be included in Wheezy or not.
I've proposed that the supported architectures should be limited in Ceph
and leveldb to the ones the latter supports. I got a release critical
bugreport[1] that it should be reversed. Alessio Treglia, the leveldb
maintainer in Debian wrote the following:
"I'm going to test leveldb on powerpc and then I'll report my results
to upstream.
Hence I cannot promise it will be ready in time for Wheezy, of course
I'll upload a patch as soon as a fix becomes available.".

Upstream was asked[2], noted that it fails on big-endian architectures,
but no answer until now.

Then, as it was previously announced, the freeze of Wheezy started some
hours ago[3].

Soon, after that Julien Cristau, one of our Release Assistant noted
about Ceph doesn't build on all architectures due to leveldb:
"Unless this gets fixed we'll have to remove ceph from wheezy.  Which
means qemu and qemu-kvm need to drop their build-deps on ceph packages.
They can always be re-added if/when leveldb/ceph are fixed.".

It seems everything is up to leveldb upstream/maintainer now. It is
fixed in time on big-endian machines or the package will be limited to
little-endian machines as I wanted to do with Ceph.

But please look into the reason why Ceph fails to build on ia64[4].
Also, how 0.48 goes? Will it contain big differences to 0.47.2? As you
can read, even 0.47.2 is in question for Wheezy. But would you propose
0.48 for inclusion when it's released?

Fingers crossed. Regards,
Laszlo/GCS
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=677626
[2] http://code.google.com/p/leveldb/issues/detail?id=84
[3] https://lists.debian.org/debian-devel-announce/2012/06/msg00009.html
[4] https://buildd.debian.org/status/fetch.php?pkg=ceph&arch=ia64&ver=0.47.2-1&stamp=1340802492

Re: Ceph status for Wheezy

[Sage Weil]

Hi Laszlo,

On Sun, 1 Jul 2012, Laszlo Boszormenyi (GCS) wrote:
> Hi Sage,
> 
> As previously noted, using leveldb caused some trouble with Ceph could
> be included in Wheezy or not.
> I've proposed that the supported architectures should be limited in Ceph
> and leveldb to the ones the latter supports. I got a release critical
> bugreport[1] that it should be reversed. Alessio Treglia, the leveldb
> maintainer in Debian wrote the following:
> "I'm going to test leveldb on powerpc and then I'll report my results
> to upstream.
> Hence I cannot promise it will be ready in time for Wheezy, of course
> I'll upload a patch as soon as a fix becomes available.".
> 
> Upstream was asked[2], noted that it fails on big-endian architectures,
> but no answer until now.
> 
> Then, as it was previously announced, the freeze of Wheezy started some
> hours ago[3].
> 
> Soon, after that Julien Cristau, one of our Release Assistant noted
> about Ceph doesn't build on all architectures due to leveldb:
> "Unless this gets fixed we'll have to remove ceph from wheezy.  Which
> means qemu and qemu-kvm need to drop their build-deps on ceph packages.
> They can always be re-added if/when leveldb/ceph are fixed.".
> 
> It seems everything is up to leveldb upstream/maintainer now. It is
> fixed in time on big-endian machines or the package will be limited to
> little-endian machines as I wanted to do with Ceph.

Fingers crossed!

> But please look into the reason why Ceph fails to build on ia64[4].

Argh, this looks like a (silly) problem with libatomic-ops-dev.  We should 
have a configure/build option to use the new c++11 atomic types instead; 
that may be less painful than dealing with it.

> Also, how 0.48 goes? Will it contain big differences to 0.47.2? As you
> can read, even 0.47.2 is in question for Wheezy. But would you propose
> 0.48 for inclusion when it's released?

Barring any last-minute catastrophies, 0.48 should be released Monday.  
Packaging-wise it will be pretty similar to 0.47.2, except for the stuff 
from you I've pulled upstream over the last couple of weeks (separate 
ceph-fs-common package, small fixups).  0.48 will also be our first 
longer-term stable release, so it is much better-suited than 0.4.2 for 
inclusion in wheezy.

sage

Re: Ceph status for Wheezy

[Gregory Farnum]

On Saturday, June 30, 2012 at 11:12 PM, Sage Weil wrote:
> Hi Laszlo,
> 
> On Sun, 1 Jul 2012, Laszlo Boszormenyi (GCS) wrote:
> > Hi Sage,
> > 
> > As previously noted, using leveldb caused some trouble with Ceph could
> > be included in Wheezy or not.
> > I've proposed that the supported architectures should be limited in Ceph
> > and leveldb to the ones the latter supports. I got a release critical
> > bugreport[1] that it should be reversed. Alessio Treglia, the leveldb
> > maintainer in Debian wrote the following:
> > "I'm going to test leveldb on powerpc and then I'll report my results
> > to upstream.
> > Hence I cannot promise it will be ready in time for Wheezy, of course
> > I'll upload a patch as soon as a fix becomes available.".
> > 
> > Upstream was asked[2], noted that it fails on big-endian architectures,
> > but no answer until now.
> > 
> > Then, as it was previously announced, the freeze of Wheezy started some
> > hours ago[3].
> > 
> > Soon, after that Julien Cristau, one of our Release Assistant noted
> > about Ceph doesn't build on all architectures due to leveldb:
> > "Unless this gets fixed we'll have to remove ceph from wheezy. Which
> > means qemu and qemu-kvm need to drop their build-deps on ceph packages.
> > They can always be re-added if/when leveldb/ceph are fixed.".
> > 
> > It seems everything is up to leveldb upstream/maintainer now. It is
> > fixed in time on big-endian machines or the package will be limited to
> > little-endian machines as I wanted to do with Ceph.
> 
> 
> 
> Fingers crossed!
> 
> > But please look into the reason why Ceph fails to build on ia64[4].
> 
> Argh, this looks like a (silly) problem with libatomic-ops-dev. We should 
> have a configure/build option to use the new c++11 atomic types instead; 
> that may be less painful than dealing with it.

This part at least shouldn't take long to get fixed for us; it's already in the Debian bug system and upstream accepted a patch:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=679680 
-Greg

Re: Ceph status for Wheezy

[Laszlo Boszormenyi (GCS)]

Hi Sage,

On Sat, 2012-06-30 at 23:12 -0700, Sage Weil wrote:
> Fingers crossed!
 It may go into Wheezy, but I need some information.

> Barring any last-minute catastrophies, 0.48 should be released Monday.
 It was released since then. But there are several files that I don't
know where to install. First are some headers under /usr/include/crush :
crush.h , hash.h , mapper.h and types.h .
I'm sure that librados-config and its manpage goes to librados-dev .
What about ceph-disk-activate , ceph-disk-prepare exists , boto_tool and
ceph-coverage ? They are python scripts except the latter being a shell
script.

Laszlo/GCS

Re: Ceph status for Wheezy

[Sage Weil]

On Sun, 8 Jul 2012, Laszlo Boszormenyi (GCS) wrote:
> Hi Sage,
> 
> On Sat, 2012-06-30 at 23:12 -0700, Sage Weil wrote:
> > Fingers crossed!
>  It may go into Wheezy, but I need some information.
> 
> > Barring any last-minute catastrophies, 0.48 should be released Monday.
>  It was released since then. But there are several files that I don't
> know where to install. First are some headers under /usr/include/crush :
> crush.h , hash.h , mapper.h and types.h .

I don't think these need to be installed anywhere; they're not currently 
packaged.  Why do you mention them?

> I'm sure that librados-config and its manpage goes to librados-dev .

Oh, right, we missed that; pushed a patch for that to wip-stable to make 
sure it builds on all distributions.

> What about ceph-disk-activate , ceph-disk-prepare exists , boto_tool and
> ceph-coverage ? They are python scripts except the latter being a shell
> script.

ceph-disk-* are already in ceph.  The other two shouldn't be packaged.

Is there anything missing on this end?

Thanks!
sage

Re: Ceph status for Wheezy

[Laszlo Boszormenyi (GCS)]

On Sun, 2012-07-08 at 21:39 -0700, Sage Weil wrote:
> On Sun, 8 Jul 2012, Laszlo Boszormenyi (GCS) wrote:
> >  It was released since then. But there are several files that I don't
> > know where to install. First are some headers under /usr/include/crush :
> > crush.h , hash.h , mapper.h and types.h .
> I don't think these need to be installed anywhere; they're not currently 
> packaged.  Why do you mention them?
 They are installed to /usr/include on 'make install'. Didn't know the
reason, but will ignore them then.

> > What about ceph-disk-activate , ceph-disk-prepare exists , boto_tool and
> > ceph-coverage ? They are python scripts except the latter being a shell
> > script.
> ceph-disk-* are already in ceph.  The other two shouldn't be packaged.
 OK, added them to my packaging as well. Adds a python dependency to the
base package.

> Is there anything missing on this end?
 Just checked, the python scripts are installed in your git tree.
Adjusted my packaging to match that. What do I miss? Manpages for
ceph-disk-activate , ceph-disk-prepare , rest-bench and maybe
librados-config .

Regards,
Laszlo/GCS

Re: Ceph status for Wheezy

[Sage Weil]

On Wed, 11 Jul 2012, Laszlo Boszormenyi (GCS) wrote:
> On Sun, 2012-07-08 at 21:39 -0700, Sage Weil wrote:
> > On Sun, 8 Jul 2012, Laszlo Boszormenyi (GCS) wrote:
> > >  It was released since then. But there are several files that I don't
> > > know where to install. First are some headers under /usr/include/crush :
> > > crush.h , hash.h , mapper.h and types.h .
> > I don't think these need to be installed anywhere; they're not currently 
> > packaged.  Why do you mention them?
>  They are installed to /usr/include on 'make install'. Didn't know the
> reason, but will ignore them then.

Oh, that old.. fixed it up.

> > > What about ceph-disk-activate , ceph-disk-prepare exists , boto_tool and
> > > ceph-coverage ? They are python scripts except the latter being a shell
> > > script.
> > ceph-disk-* are already in ceph.  The other two shouldn't be packaged.
>  OK, added them to my packaging as well. Adds a python dependency to the
> base package.
>
> > Is there anything missing on this end?
>  Just checked, the python scripts are installed in your git tree.
> Adjusted my packaging to match that. What do I miss? Manpages for
> ceph-disk-activate , ceph-disk-prepare , rest-bench and maybe
> librados-config .

Okay, those should be pretty simple.  There will be a point release before 
too long that includes all of this stuff.  If there is anything in your 
packaging that isn't fixed upstream let me know.

Thanks!
sage

Re: Ceph status for Wheezy

[Yehuda Sadeh]

On Sat, Jun 30, 2012 at 10:56 PM, Laszlo Boszormenyi (GCS)
<gcs@debian.hu> wrote:
> Hi Sage,
>
> As previously noted, using leveldb caused some trouble with Ceph could
> be included in Wheezy or not.
> I've proposed that the supported architectures should be limited in Ceph
> and leveldb to the ones the latter supports. I got a release critical
> bugreport[1] that it should be reversed. Alessio Treglia, the leveldb
> maintainer in Debian wrote the following:
> "I'm going to test leveldb on powerpc and then I'll report my results
> to upstream.
> Hence I cannot promise it will be ready in time for Wheezy, of course
> I'll upload a patch as soon as a fix becomes available.".
>
> Upstream was asked[2], noted that it fails on big-endian architectures,
> but no answer until now.
>
> Then, as it was previously announced, the freeze of Wheezy started some
> hours ago[3].
>
> Soon, after that Julien Cristau, one of our Release Assistant noted
> about Ceph doesn't build on all architectures due to leveldb:
> "Unless this gets fixed we'll have to remove ceph from wheezy.  Which
> means qemu and qemu-kvm need to drop their build-deps on ceph packages.
> They can always be re-added if/when leveldb/ceph are fixed.".
>
> It seems everything is up to leveldb upstream/maintainer now. It is
> fixed in time on big-endian machines or the package will be limited to
> little-endian machines as I wanted to do with Ceph.
>
> But please look into the reason why Ceph fails to build on ia64[4].
> Also, how 0.48 goes? Will it contain big differences to 0.47.2? As you
> can read, even 0.47.2 is in question for Wheezy. But would you propose
> 0.48 for inclusion when it's released?
>
> Fingers crossed. Regards,
> Laszlo/GCS
> [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=677626
> [2] http://code.google.com/p/leveldb/issues/detail?id=84
> [3] https://lists.debian.org/debian-devel-announce/2012/06/msg00009.html
> [4] https://buildd.debian.org/status/fetch.php?pkg=ceph&arch=ia64&ver=0.47.2-1&stamp=1340802492
>


I looked into why leveldb fails to build on big endian, and the reason
seems quite trivial. The bloom_test unitest passes different input due
to endianity, and the result is not as expected due to statistical
difference of the input; there are more false positive. I played with
it a bit, and when transforming the input to little endian it passed.
I also got the same failure on a little endian machine when I
transformed it to big endian. There are two solutions, one is to bump
up the expected false positives to a higher number (e.g., 3% instead
of 2%):

diff --git a/util/bloom_test.cc b/util/bloom_test.cc
index 4a6ea1b..61323af 100644
--- a/util/bloom_test.cc
+++ b/util/bloom_test.cc
@@ -139,7 +139,7 @@ TEST(BloomTest, VaryingLengths) {
       fprintf(stderr, "False positives: %5.2f%% @ length = %6d ;
bytes = %6d\n",
               rate*100.0, length, static_cast<int>(FilterSize()));
     }
-    ASSERT_LE(rate, 0.02);   // Must not be over 2%
+    ASSERT_LE(rate, 0.03);   // Must not be over 2%
     if (rate > 0.0125) mediocre_filters++;  // Allowed, but not too often
     else good_filters++;
   }



Another option is to transform data to little endian:


diff --git a/util/bloom_test.cc b/util/bloom_test.cc
index 4a6ea1b..6de7acc 100644
--- a/util/bloom_test.cc
+++ b/util/bloom_test.cc
@@ -5,6 +5,7 @@
 #include "leveldb/filter_policy.h"

 #include "util/logging.h"
+#include "util/coding.h"
 #include "util/testharness.h"
 #include "util/testutil.h"

@@ -12,8 +13,22 @@ namespace leveldb {

 static const int kVerbose = 1;

+static inline void EncodeFixed(char *buf, int32_t val)
+{
+    EncodeFixed32(buf, val);
+}
+
+static inline void EncodeFixed(char *buf, int64_t val)
+{
+    EncodeFixed64(buf, val);
+}
+
 static Slice Key(int i, char* buffer) {
-  memcpy(buffer, &i, sizeof(i));
+  if (port::kLittleEndian) {
+    memcpy(buffer, &i, sizeof(i));
+  } else {
+    EncodeFixed(buffer, i);
+  }
   return Slice(buffer, sizeof(i));
 }


I also had some issues with other tests, but all those problems
appeared to be due to the buggy std atomic implementation on older
gcc.

Yehuda

Re: Ceph status for Wheezy

[Yehuda Sadeh]

On Mon, Jul 2, 2012 at 2:13 PM, Yehuda Sadeh <yehuda@inktank.com> wrote:
> On Sat, Jun 30, 2012 at 10:56 PM, Laszlo Boszormenyi (GCS)
> <gcs@debian.hu> wrote:
>> Hi Sage,
>>
>> As previously noted, using leveldb caused some trouble with Ceph could
>> be included in Wheezy or not.
>> I've proposed that the supported architectures should be limited in Ceph
>> and leveldb to the ones the latter supports. I got a release critical
>> bugreport[1] that it should be reversed. Alessio Treglia, the leveldb
>> maintainer in Debian wrote the following:
>> "I'm going to test leveldb on powerpc and then I'll report my results
>> to upstream.
>> Hence I cannot promise it will be ready in time for Wheezy, of course
>> I'll upload a patch as soon as a fix becomes available.".
>>
>> Upstream was asked[2], noted that it fails on big-endian architectures,
>> but no answer until now.
>>
>> Then, as it was previously announced, the freeze of Wheezy started some
>> hours ago[3].
>>
>> Soon, after that Julien Cristau, one of our Release Assistant noted
>> about Ceph doesn't build on all architectures due to leveldb:
>> "Unless this gets fixed we'll have to remove ceph from wheezy.  Which
>> means qemu and qemu-kvm need to drop their build-deps on ceph packages.
>> They can always be re-added if/when leveldb/ceph are fixed.".
>>
>> It seems everything is up to leveldb upstream/maintainer now. It is
>> fixed in time on big-endian machines or the package will be limited to
>> little-endian machines as I wanted to do with Ceph.
>>
>> But please look into the reason why Ceph fails to build on ia64[4].
>> Also, how 0.48 goes? Will it contain big differences to 0.47.2? As you
>> can read, even 0.47.2 is in question for Wheezy. But would you propose
>> 0.48 for inclusion when it's released?
>>
>> Fingers crossed. Regards,
>> Laszlo/GCS
>> [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=677626
>> [2] http://code.google.com/p/leveldb/issues/detail?id=84
>> [3] https://lists.debian.org/debian-devel-announce/2012/06/msg00009.html
>> [4] https://buildd.debian.org/status/fetch.php?pkg=ceph&arch=ia64&ver=0.47.2-1&stamp=1340802492
>>
>
>
> I looked into why leveldb fails to build on big endian, and the reason
> seems quite trivial. The bloom_test unitest passes different input due
> to endianity, and the result is not as expected due to statistical
> difference of the input; there are more false positive. I played with
> it a bit, and when transforming the input to little endian it passed.
> I also got the same failure on a little endian machine when I
> transformed it to big endian. There are two solutions, one is to bump
> up the expected false positives to a higher number (e.g., 3% instead
> of 2%):
>
> diff --git a/util/bloom_test.cc b/util/bloom_test.cc
> index 4a6ea1b..61323af 100644
> --- a/util/bloom_test.cc
> +++ b/util/bloom_test.cc
> @@ -139,7 +139,7 @@ TEST(BloomTest, VaryingLengths) {
>        fprintf(stderr, "False positives: %5.2f%% @ length = %6d ;
> bytes = %6d\n",
>                rate*100.0, length, static_cast<int>(FilterSize()));
>      }
> -    ASSERT_LE(rate, 0.02);   // Must not be over 2%
> +    ASSERT_LE(rate, 0.03);   // Must not be over 2%
>      if (rate > 0.0125) mediocre_filters++;  // Allowed, but not too often
>      else good_filters++;
>    }
>
>

I sent a query to the leveldb mailing list (sorry for not cross
posting, was having trouble with google groups). I got a response from
Sanjay and he said that bumping up the check to 0.03 (as with the
above patch) is ok as a quick solution. He'll work on a better
solution along the lines of my other proposed change later. I made the
above patch available in the following git repository:

https://code.google.com/r/yehuda-leveldb/

Yehuda