Compiling for FreeBSD

Compiling for FreeBSD

[Willem Jan Withagen]

Hi,

Not unlike many others running FreeBSD I'd like to see if I/we can get
Ceph to build and run on FreeBSD. If not all components than at least
certain components.

With compilation I do get quite some way, even with the CLANG compiler.
But I run into obvious part where Linux goes a different direction from
what is available on FreeBSD.

If I google one of the reports I ran into, I get at:
http://lists.ceph.com/pipermail/ceph-commit-ceph.com/2013-November/005812.html

Which sort of suggests that some of the code for FreeBSD has again been
purged from the tree...

Is that to remove FreeBSD completely from the tree?
Or just because that code did not work?

Thanx,
--WjW

Re: Compiling for FreeBSD

[Haomai Wang]

On Mon, Nov 30, 2015 at 1:44 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> Hi,
>
> Not unlike many others running FreeBSD I'd like to see if I/we can get
> Ceph to build and run on FreeBSD. If not all components than at least
> certain components.
>
> With compilation I do get quite some way, even with the CLANG compiler.
> But I run into obvious part where Linux goes a different direction from
> what is available on FreeBSD.
>
> If I google one of the reports I ran into, I get at:
> http://lists.ceph.com/pipermail/ceph-commit-ceph.com/2013-November/005812.html
>
> Which sort of suggests that some of the code for FreeBSD has again been
> purged from the tree...
>
> Is that to remove FreeBSD completely from the tree?
> Or just because that code did not work?

I guess we still expect FreeBSD support, which version do you test to
compile? I'd like to help to make bsd work :-)

>
> Thanx,
> --WjW
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 29-11-2015 19:08, Haomai Wang wrote:

> I guess we still expect FreeBSD support, which version do you test to
> compile? I'd like to help to make bsd work :-)

I considered it is best to develop against HEAD aka:
11.0-CURRENT FreeBSD 11.0-CURRENT #0 r291381: Sat Nov 28 14:22:54 CET 2015
I'm also training configure the use as much of CLANG as possible.

I would guess that by the time we get anything worth mentioning 11.0
will be released of close to release.
Note that the release date for 11.0 is july 2016

--WjW

Re: Compiling for FreeBSD

[Yan, Zheng]

On Mon, Nov 30, 2015 at 2:57 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>
> On 29-11-2015 19:08, Haomai Wang wrote:
>
> > I guess we still expect FreeBSD support, which version do you test to
> > compile? I'd like to help to make bsd work :-)
>
> I considered it is best to develop against HEAD aka:
> 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r291381: Sat Nov 28 14:22:54 CET 2015
> I'm also training configure the use as much of CLANG as possible.
>
> I would guess that by the time we get anything worth mentioning 11.0
> will be released of close to release.
> Note that the release date for 11.0 is july 2016
>
> --WjW
>

I can compile infernalis on FreeBSD 10.1 with following commands

./autogen.sh
./configure CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib"
CXXFLAGS="-DGTEST_HAS_TR1_TUPLE=0" --without-tcmalloc --without-libaio
--without-libxfs
gmake

I don't know extact list of packages dependencies. But ./configure
should tell you what is missing.

Yan, Zheng
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Compiling for FreeBSD

[Mykola Golub]

On Mon, Nov 30, 2015 at 11:46:27AM +0800, Yan, Zheng wrote:
> On Mon, Nov 30, 2015 at 2:57 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> >
> > On 29-11-2015 19:08, Haomai Wang wrote:
> >
> > > I guess we still expect FreeBSD support, which version do you test to
> > > compile? I'd like to help to make bsd work :-)
> >
> > I considered it is best to develop against HEAD aka:
> > 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r291381: Sat Nov 28 14:22:54 CET 2015
> > I'm also training configure the use as much of CLANG as possible.
> >
> > I would guess that by the time we get anything worth mentioning 11.0
> > will be released of close to release.
> > Note that the release date for 11.0 is july 2016
> >
> > --WjW
> >
> 
> I can compile infernalis on FreeBSD 10.1 with following commands
> 
> ./autogen.sh
> ./configure CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib"
> CXXFLAGS="-DGTEST_HAS_TR1_TUPLE=0" --without-tcmalloc --without-libaio
> --without-libxfs
> gmake
> 
> I don't know extact list of packages dependencies. But ./configure
> should tell you what is missing.

I have a branch in my repo that contains patches to make compilation
on FreeBSD easier:

https://github.com/trociny/ceph/commits/wip-freebsd

The branch have not been rebased against master since March though,
still I hope to find time in future to update and pull some bits
upstream.

Particularly, this patch contains packages dependencies list:

https://github.com/trociny/ceph/commit/2307706461ae09922c087065a8b50a04a61159b1

And this one configuration options I used

https://github.com/trociny/ceph/commit/dcee0c0635d37f2b36257c55a3cc69d05b5afe5e

I have seen several commits since that time in master aimed to improve
FreeBSD compilation, so I suppose some of my patches are not needed
any more.

Still, if you experince issues building upstream Ceph on FreeBSD, and
just want to try it on FreeBSD, you could try my branch:

git clone --recursive -b wip-freebsd https://github.com/trociny/ceph.git
cd ceph
./install-deps.sh
./do_autogen.sh
gmake
cd src
./vstart.sh # start dev cluster
./ceph -s # check it works

-- 
Mykola Golub

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 30-11-2015 07:58, Mykola Golub wrote:
> On Mon, Nov 30, 2015 at 11:46:27AM +0800, Yan, Zheng wrote:
>> On Mon, Nov 30, 2015 at 2:57 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
...

Hi Mykola,

> I have a branch in my repo that contains patches to make compilation
> on FreeBSD easier:
> 
> https://github.com/trociny/ceph/commits/wip-freebsd

Right, I'll and see what you've all got worked out.
First get done with my daytime job that pays the rent.


> The branch have not been rebased against master since March though,
> still I hope to find time in future to update and pull some bits
> upstream.
> 
> Particularly, this patch contains packages dependencies list:
> 
> https://github.com/trociny/ceph/commit/2307706461ae09922c087065a8b50a04a61159b1

Right, I had most of those already figured out, but some much less obvious.

And the humor is on the first line... bash <> sh.
In just about all linux oriented software that is one of the things to
start with. FreePBX is also loaded with those.

I've sort of given up on this one, resistance is futile.
And I install bash in /bin.

> And this one configuration options I used
> 
> https://github.com/trociny/ceph/commit/dcee0c0635d37f2b36257c55a3cc69d05b5afe5e
>
> I have seen several commits since that time in master aimed to improve
> FreeBSD compilation, so I suppose some of my patches are not needed
> any more.
> 
> Still, if you experince issues building upstream Ceph on FreeBSD, and
> just want to try it on FreeBSD, you could try my branch:
> 
> git clone --recursive -b wip-freebsd https://github.com/trociny/ceph.git
> cd ceph
> ./install-deps.sh
> ./do_autogen.sh
> gmake
> cd src
> ./vstart.sh # start dev cluster
> ./ceph -s # check it works
> 

So what backend for the OSDs are you using here?

--WjW

Re: Compiling for FreeBSD

[Sage Weil]

The problem with all of the porting code in general is that it is doomed 
to break later on if we don't have (at least) ongoing build tests.  In 
order for a FreeBSD or OSX port to continue working we need VMs that run 
either gitbuilder or a jenkins job or similar so that we can tell when it 
breaks.

If someone is willing to run a VM somewhere to do this we can pretty 
easily stick it on the gitbuilder page at

	http://ceph.com/gitbuilder.cgi

sage

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 30-11-2015 14:21, Sage Weil wrote:
> The problem with all of the porting code in general is that it is doomed 
> to break later on if we don't have (at least) ongoing build tests.  In 
> order for a FreeBSD or OSX port to continue working we need VMs that run 
> either gitbuilder or a jenkins job or similar so that we can tell when it 
> breaks.
> 
> If someone is willing to run a VM somewhere to do this we can pretty 
> easily stick it on the gitbuilder page at
> 
> 	http://ceph.com/gitbuilder.cgi

Hi Sage,

Cool page that gitbuilder....
Would be nice to have a FreeBSD builder in there as well. :)

Once I get things up and running, I will be able to supply either VMs
and/or actual hardware to do building and testing.

My personal problem is that I'm a longtime FreeBSD user (since 1993) and
now I'm sort of forced into (the) other OS. Because we like Ceph. So we
are building currently on Linux, but would like to have a FreeBSD
possibility as well.

But first I need to get the code to compile, and then actually run.
Long term goal is to use ZFS as basis, since this is a rock solid FS on
FreeBSD. And then integrate it with another nice FreeBSD thingy: bhyve.

Now I'm taking small baby steps at the moment, and do have other chores
as well. So progress might be slow, but I think I found some of the
issues I did not get to yet in the git of Mykola.

Thanx for the feedback.

Regards,
--Willem Jan

Re: Compiling for FreeBSD

[Mykola Golub]

On Mon, Nov 30, 2015 at 12:53:54PM +0100, Willem Jan Withagen wrote:

> > git clone --recursive -b wip-freebsd https://github.com/trociny/ceph.git
> > cd ceph
> > ./install-deps.sh
> > ./do_autogen.sh
> > gmake
> > cd src
> > ./vstart.sh # start dev cluster
> > ./ceph -s # check it works
> > 
> 
> So what backend for the OSDs are you using here?

The default one, which is FileStore. I have not done much testing
though...

-- 
Mykola Golub

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 30-11-2015 15:13, Mykola Golub wrote:
> On Mon, Nov 30, 2015 at 12:53:54PM +0100, Willem Jan Withagen wrote:
> 
>>> git clone --recursive -b wip-freebsd https://github.com/trociny/ceph.git
>>> cd ceph
>>> ./install-deps.sh
>>> ./do_autogen.sh
>>> gmake
>>> cd src
>>> ./vstart.sh # start dev cluster
>>> ./ceph -s # check it works
>>>
>>
>> So what backend for the OSDs are you using here?
> 
> The default one, which is FileStore. I have not done much testing
> though...
> 

Well,
Cherry-picked a few of you diffs.
Compilation is now slugging thru the tests in my tree.
Running gmake in serial mode, so it is slow with clang.

--WjW

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 30-11-2015 15:13, Mykola Golub wrote:
> On Mon, Nov 30, 2015 at 12:53:54PM +0100, Willem Jan Withagen wrote:
> 
>>> git clone --recursive -b wip-freebsd https://github.com/trociny/ceph.git
>>> cd ceph
>>> ./install-deps.sh
>>> ./do_autogen.sh
>>> gmake
>>> cd src
>>> ./vstart.sh # start dev cluster
>>> ./ceph -s # check it works
>>>
>>
>> So what backend for the OSDs are you using here?
> 
> The default one, which is FileStore. I have not done much testing
> though...
> 

Seems we have passed the first hurdle...
It compiles...

Onto the next phase: Testing.
Lets see what the developers site has to tell about that.

--WjW

chmod +x ceph-coverage.tmp
chmod a-w ceph-coverage.tmp
mv ceph-coverage.tmp ceph-coverage
cd ./ceph-detect-init ; python setup.py build
running build
running build_py
gmake[3]: Leaving directory '/usr/srcs/Ceph/src/ceph/src'
gmake[2]: Leaving directory '/usr/srcs/Ceph/src/ceph/src'
gmake[1]: Leaving directory '/usr/srcs/Ceph/src/ceph/src'
Making all in man
gmake[1]: Entering directory '/usr/srcs/Ceph/src/ceph/man'
gmake[1]: Nothing to be done for 'all'.
gmake[1]: Leaving directory '/usr/srcs/Ceph/src/ceph/man'
Making all in doc
gmake[1]: Entering directory '/usr/srcs/Ceph/src/ceph/doc'
gmake[1]: Nothing to be done for 'all'.
gmake[1]: Leaving directory '/usr/srcs/Ceph/src/ceph/doc'
Making all in systemd
gmake[1]: Entering directory '/usr/srcs/Ceph/src/ceph/systemd'
gmake[1]: Nothing to be done for 'all'.
gmake[1]: Leaving directory '/usr/srcs/Ceph/src/ceph/systemd'
Making all in selinux
gmake[1]: Entering directory '/usr/srcs/Ceph/src/ceph/selinux'
gmake[1]: Nothing to be done for 'all'.
gmake[1]: Leaving directory '/usr/srcs/Ceph/src/ceph/selinux'

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 30-11-2015 15:40, Willem Jan Withagen wrote:
> On 30-11-2015 15:13, Mykola Golub wrote:
>> On Mon, Nov 30, 2015 at 12:53:54PM +0100, Willem Jan Withagen wrote:
>>
>>>> git clone --recursive -b wip-freebsd https://github.com/trociny/ceph.git
>>>> cd ceph
>>>> ./install-deps.sh
>>>> ./do_autogen.sh
>>>> gmake
>>>> cd src
>>>> ./vstart.sh # start dev cluster
>>>> ./ceph -s # check it works
>>>>
>>>
>>> So what backend for the OSDs are you using here?
>>
>> The default one, which is FileStore. I have not done much testing
>> though...
>>
> 
> Well,
> Cherry-picked a few of you diffs.
> Compilation is now slugging thru the tests in my tree.
> Running gmake in serial mode, so it is slow with clang.

Installed the results:
gmake install

And looked at what it delivered:
	zfs diff <fs-en>

And I have to be honest that I'm not really enjoying all the ceph-test
stuff in my /usr/local/bin....
Would prefer it to go into something like:
	/usr/local/libexec/ceph/tests
or
	/usr/local/share/ceph/tests

Not sure if this is possible without disrupting the automated testing.

--WjW

+       /sbin/mount.fuse.ceph
zfsroot/tmp
M       /tmp/
+       /tmp/fses
zfsroot/usr
M       /usr/local/sbin
M       /usr/local/bin
M       /usr/local/etc
M       /usr/local/include
M       /usr/local/lib
M       /usr/local/libexec
M       /usr/local/share
M       /usr/local/share/doc
M       /usr/local/lib/python2.7/site-packages
M       /usr/srcs/Ceph/src/ceph/src/.libs/libradosstriper.so.1
M       /usr/srcs/Ceph/src/ceph/src/.libs/librbd.so.1
+       /usr/local/share/ceph
+       /usr/local/share/ceph/known_hosts_drop.ceph.com
+       /usr/local/share/ceph/id_dsa_drop.ceph.com
+       /usr/local/share/ceph/id_dsa_drop.ceph.com.pub
+       /usr/local/lib/librados.so.2
+       /usr/local/lib/librados.so
+       /usr/local/lib/librados.la
+       /usr/srcs/Ceph/src/ceph/src/.libs/libradosstriper.so.1T
+       /usr/local/lib/libradosstriper.so.1
+       /usr/local/lib/libradosstriper.so
+       /usr/local/lib/libradosstriper.la
+       /usr/srcs/Ceph/src/ceph/src/.libs/librbd.so.1T
+       /usr/local/lib/librbd.so.1
+       /usr/local/lib/librbd.so
+       /usr/local/lib/librbd.la
+       /usr/local/lib/libcephfs.so.1
+       /usr/local/lib/libcephfs.so
+       /usr/local/lib/libcephfs.la
+       /usr/local/lib/librados.a
+       /usr/local/lib/libradosstriper.a
+       /usr/local/lib/librbd.a
+       /usr/local/lib/libcephfs.a
+       /usr/local/bin/ceph_test_ioctls
+       /usr/local/bin/ceph_erasure_code_benchmark
+       /usr/local/bin/ceph_erasure_code
+       /usr/local/bin/ceph_test_rados
+       /usr/local/bin/ceph_test_mutate
+       /usr/local/bin/ceph_smalliobench
+       /usr/local/bin/ceph_omapbench
+       /usr/local/bin/ceph_objectstore_bench
+       /usr/local/bin/ceph_multi_stress_watch
+       /usr/local/bin/ceph_test_cls_rbd
+       /usr/local/bin/ceph_test_cls_refcount
+       /usr/local/bin/ceph_test_cls_version
+       /usr/local/bin/ceph_test_cls_log
+       /usr/local/bin/ceph_test_cls_statelog
+       /usr/local/bin/ceph_test_cls_replica_log
+       /usr/local/bin/ceph_test_cls_lock
+       /usr/local/bin/ceph_test_cls_hello
+       /usr/local/bin/ceph_test_cls_numops
+       /usr/local/bin/ceph_test_cls_journal
+       /usr/local/bin/ceph_test_rados_api_cmd
+       /usr/local/bin/ceph_test_rados_api_io
+       /usr/local/bin/ceph_test_rados_api_c_write_operations
+       /usr/local/bin/ceph_test_rados_api_c_read_operations
+       /usr/local/bin/ceph_test_rados_api_aio
+       /usr/local/bin/ceph_test_rados_api_list
+       /usr/local/bin/ceph_test_rados_api_nlist
+       /usr/local/bin/ceph_test_rados_api_pool
+       /usr/local/bin/ceph_test_rados_api_stat
+       /usr/local/bin/ceph_test_rados_api_watch_notify
+       /usr/local/bin/ceph_test_rados_api_snapshots
+       /usr/local/bin/ceph_test_rados_api_cls
+       /usr/local/bin/ceph_test_rados_api_misc
+       /usr/local/bin/ceph_test_rados_api_tier
+       /usr/local/bin/ceph_test_rados_api_lock
+       /usr/local/bin/ceph_test_stress_watch
+       /usr/local/bin/ceph_smalliobenchrbd
+       /usr/local/bin/ceph_test_librbd
+       /usr/local/bin/ceph_test_librbd_api
+       /usr/local/bin/ceph_test_rados_striper_api_io
+       /usr/local/bin/ceph_test_rados_striper_api_aio
+       /usr/local/bin/ceph_test_rados_striper_api_striping
+       /usr/local/bin/ceph_test_libcephfs
+       /usr/local/bin/ceph_test_c_headers
+       /usr/local/bin/ceph_test_async_driver
+       /usr/local/bin/ceph_test_msgr
+       /usr/local/bin/ceph_streamtest
+       /usr/local/bin/ceph_test_trans
+       /usr/local/bin/ceph_test_mon_workloadgen
+       /usr/local/bin/ceph_test_mon_msg
+       /usr/local/bin/ceph_perf_objectstore
+       /usr/local/bin/ceph_perf_local
+       /usr/local/bin/ceph_perf_msgr_server
+       /usr/local/bin/ceph_perf_msgr_client
+       /usr/local/bin/ceph_test_objectstore_workloadgen
+       /usr/local/bin/ceph_test_filestore_idempotent
+       /usr/local/bin/ceph_test_filestore_idempotent_sequence
+       /usr/local/bin/ceph_xattr_bench
+       /usr/local/bin/ceph_test_filejournal
+       /usr/local/bin/ceph_test_object_map
+       /usr/local/bin/ceph_test_keyvaluedb_atomicity
+       /usr/local/bin/ceph_test_keyvaluedb_iterators
+       /usr/local/bin/ceph_smalliobenchfs
+       /usr/local/bin/ceph_smalliobenchdumb
+       /usr/local/bin/ceph_tpbench
+       /usr/local/bin/ceph_test_keys
+       /usr/local/bin/ceph_test_snap_mapper
+       /usr/local/bin/ceph_test_timers
+       /usr/local/bin/ceph_test_signal_handlers
+       /usr/local/bin/ceph_test_rewrite_latency
+       /usr/local/bin/ceph_test_crypto
+       /usr/local/bin/ceph_bench_log
+       /usr/local/bin/ceph_test_objectcacher_stress
+       /usr/local/bin/ceph_test_cfuse_cache_invalidate
+       /usr/local/bin/ceph_test_get_blkdev_size
+       /usr/local/bin/ceph_scratchtool
+       /usr/local/bin/ceph_scratchtoolpp
+       /usr/local/bin/ceph_radosacl
+       /usr/local/bin/ceph-client-debug
+       /usr/local/bin/ceph-osdomap-tool
+       /usr/local/bin/ceph-monstore-tool
+       /usr/local/bin/ceph-kvstore-tool
+       /usr/local/bin/ceph_psim
+       /usr/local/bin/ceph-dencoder
+       /usr/local/bin/rados
+       /usr/local/bin/ceph-objectstore-tool
+       /usr/local/bin/cephfs-journal-tool
+       /usr/local/bin/cephfs-table-tool
+       /usr/local/bin/cephfs-data-scan
+       /usr/local/bin/monmaptool
+       /usr/local/bin/crushtool
+       /usr/local/bin/osdmaptool
+       /usr/local/bin/ceph-conf
+       /usr/local/bin/ceph-authtool
+       /usr/local/bin/ceph-syn
+       /usr/local/bin/librados-config
+       /usr/local/bin/cephfs
+       /usr/local/bin/ceph-mon
+       /usr/local/bin/ceph-osd
+       /usr/local/bin/ceph-mds
+       /usr/local/bin/ceph-brag
+       /usr/local/bin/ceph
+       /usr/local/bin/ceph-post-file
+       /usr/local/bin/ceph-rbdnamer
+       /usr/local/bin/rbd-replay-many
+       /usr/local/bin/rbdmap
+       /usr/local/bin/ceph-run
+       /usr/local/bin/ceph-rest-api
+       /usr/local/bin/ceph-debugpack
+       /usr/local/bin/ceph-crush-location
+       /usr/local/bin/ceph-coverage
+       /usr/local/bin/ceph-clsinfo
+       /usr/local/libexec/ceph
+       /usr/local/libexec/ceph/ceph-osd-prestart.sh
+       /usr/local/etc/bash_completion.d/ceph
+       /usr/local/etc/bash_completion.d/rados
+       /usr/local/etc/bash_completion.d/radosgw-admin
+       /usr/local/etc/bash_completion.d/rbd
+       /usr/local/lib/ceph
+       /usr/local/lib/ceph/ceph-monstore-update-crush.sh
+       /usr/local/sbin/ceph-create-keys
+       /usr/local/sbin/ceph-disk
+       /usr/local/sbin/ceph-disk-udev
+
/usr/srcs/Ceph/src/ceph/src/ceph-detect-init/ceph_detect_init.egg-info
+
/usr/srcs/Ceph/src/ceph/src/ceph-detect-init/ceph_detect_init.egg-info/requires.txt
+
/usr/srcs/Ceph/src/ceph/src/ceph-detect-init/ceph_detect_init.egg-info/PKG-INFO
+
/usr/srcs/Ceph/src/ceph/src/ceph-detect-init/ceph_detect_init.egg-info/top_level.txt
+
/usr/srcs/Ceph/src/ceph/src/ceph-detect-init/ceph_detect_init.egg-info/dependency_links.txt
+
/usr/srcs/Ceph/src/ceph/src/ceph-detect-init/ceph_detect_init.egg-info/entry_points.txt
+
/usr/srcs/Ceph/src/ceph/src/ceph-detect-init/ceph_detect_init.egg-info/SOURCES.txt
+
/usr/srcs/Ceph/src/ceph/src/ceph-detect-init/build/bdist.freebsd-11.0-CURRENT-amd64
+       /usr/srcs/Ceph/src/ceph/src/ceph-detect-init/dist
+
/usr/srcs/Ceph/src/ceph/src/ceph-detect-init/dist/ceph_detect_init-1.0.1-py2.7.egg
+
/usr/local/lib/python2.7/site-packages/ceph_detect_init-1.0.1-py2.7.egg
+       /usr/local/bin/ceph-detect-init
+       /usr/local/lib/python2.7/site-packages/setuptools.pth
+       /usr/local/bin/easy_install
+       /usr/local/etc/ceph
+       /usr/local/var/log/ceph
+       /usr/local/var/lib/ceph
+       /usr/local/var/lib/ceph/tmp
+       /usr/local/share/doc/ceph
+       /usr/local/share/doc/ceph/sample.ceph.conf
+       /usr/local/share/doc/ceph/sample.fetch_config
+       /usr/local/lib/ceph/erasure-code
+       /usr/local/lib/ceph/erasure-code/libec_jerasure_generic.so
+       /usr/local/lib/ceph/erasure-code/libec_jerasure_generic.la
+       /usr/local/lib/ceph/erasure-code/libec_jerasure_sse3.so
+       /usr/local/lib/ceph/erasure-code/libec_jerasure_sse3.la
+       /usr/local/lib/ceph/erasure-code/libec_jerasure_sse4.so
+       /usr/local/lib/ceph/erasure-code/libec_jerasure_sse4.la
+       /usr/local/lib/ceph/erasure-code/libec_jerasure.so
+       /usr/local/lib/ceph/erasure-code/libec_jerasure.la
+       /usr/local/lib/ceph/erasure-code/libec_lrc.so
+       /usr/local/lib/ceph/erasure-code/libec_lrc.la
+       /usr/local/lib/ceph/erasure-code/libec_shec_generic.so
+       /usr/local/lib/ceph/erasure-code/libec_shec_generic.la
+       /usr/local/lib/ceph/erasure-code/libec_shec_sse3.so
+       /usr/local/lib/ceph/erasure-code/libec_shec_sse3.la
+       /usr/local/lib/ceph/erasure-code/libec_shec_sse4.so
+       /usr/local/lib/ceph/erasure-code/libec_shec_sse4.la
+       /usr/local/lib/ceph/erasure-code/libec_shec.so
+       /usr/local/lib/ceph/erasure-code/libec_shec.la
+       /usr/local/lib/ceph/erasure-code/libec_isa.so
+       /usr/local/lib/ceph/erasure-code/libec_isa.la
+       /usr/local/lib/ceph/erasure-code/libec_example.so.0
+       /usr/local/lib/ceph/erasure-code/libec_example.so
+       /usr/local/lib/ceph/erasure-code/libec_example.la
+       /usr/local/lib/ceph/erasure-code/libec_missing_entry_point.so.0
+       /usr/local/lib/ceph/erasure-code/libec_missing_entry_point.so
+       /usr/local/lib/ceph/erasure-code/libec_missing_entry_point.la
+       /usr/local/lib/ceph/erasure-code/libec_missing_version.so.0
+       /usr/local/lib/ceph/erasure-code/libec_missing_version.so
+       /usr/local/lib/ceph/erasure-code/libec_missing_version.la
+       /usr/local/lib/ceph/erasure-code/libec_hangs.so.0
+       /usr/local/lib/ceph/erasure-code/libec_hangs.so
+       /usr/local/lib/ceph/erasure-code/libec_hangs.la
+       /usr/local/lib/ceph/erasure-code/libec_fail_to_initialize.so.0
+       /usr/local/lib/ceph/erasure-code/libec_fail_to_initialize.so
+       /usr/local/lib/ceph/erasure-code/libec_fail_to_initialize.la
+       /usr/local/lib/ceph/erasure-code/libec_fail_to_register.so.0
+       /usr/local/lib/ceph/erasure-code/libec_fail_to_register.so
+       /usr/local/lib/ceph/erasure-code/libec_fail_to_register.la
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_neon.so.0
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_neon.so
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_neon.la
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_sse4.so.0
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_sse4.so
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_sse4.la
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_sse3.so.0
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_sse3.so
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_sse3.la
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_generic.so.0
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_generic.so
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_generic.la
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_neon.so.0
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_neon.so
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_neon.la
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_sse4.so.0
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_sse4.so
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_sse4.la
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_sse3.so.0
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_sse3.so
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_sse3.la
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_generic.so.0
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_generic.so
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_generic.la
+       /usr/local/lib/ceph/erasure-code/libec_example.a
+       /usr/local/lib/ceph/erasure-code/libec_missing_entry_point.a
+       /usr/local/lib/ceph/erasure-code/libec_missing_version.a
+       /usr/local/lib/ceph/erasure-code/libec_hangs.a
+       /usr/local/lib/ceph/erasure-code/libec_fail_to_initialize.a
+       /usr/local/lib/ceph/erasure-code/libec_fail_to_register.a
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_neon.a
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_sse4.a
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_sse3.a
+       /usr/local/lib/ceph/erasure-code/libec_test_jerasure_generic.a
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_neon.a
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_sse4.a
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_sse3.a
+       /usr/local/lib/ceph/erasure-code/libec_test_shec_generic.a
+       /usr/local/include/cephfs
+       /usr/local/include/cephfs/libcephfs.h
+       /usr/local/include/rbd
+       /usr/local/include/rbd/features.h
+       /usr/local/include/rbd/librbd.h
+       /usr/local/include/rbd/librbd.hpp
+       /usr/local/lib/python2.7/site-packages/ceph_argparse.py
+       /usr/local/lib/python2.7/site-packages/ceph_daemon.py
+       /usr/local/lib/python2.7/site-packages/rados.py
+       /usr/local/lib/python2.7/site-packages/rbd.py
+       /usr/local/lib/python2.7/site-packages/cephfs.py
+       /usr/local/lib/python2.7/site-packages/ceph_rest_api.py
+       /usr/local/lib/python2.7/site-packages/ceph_argparse.pyc
+       /usr/local/lib/python2.7/site-packages/ceph_daemon.pyc
+       /usr/local/lib/python2.7/site-packages/rados.pyc
+       /usr/local/lib/python2.7/site-packages/rbd.pyc
+       /usr/local/lib/python2.7/site-packages/cephfs.pyc
+       /usr/local/lib/python2.7/site-packages/ceph_rest_api.pyc
+       /usr/local/lib/python2.7/site-packages/ceph_argparse.pyo
+       /usr/local/lib/python2.7/site-packages/ceph_daemon.pyo
+       /usr/local/lib/python2.7/site-packages/rados.pyo
+       /usr/local/lib/python2.7/site-packages/rbd.pyo
+       /usr/local/lib/python2.7/site-packages/cephfs.pyo
+       /usr/local/lib/python2.7/site-packages/ceph_rest_api.pyo
+       /usr/local/include/rados
+       /usr/local/include/rados/librados.h
+       /usr/local/include/rados/rados_types.h
+       /usr/local/include/rados/rados_types.hpp
+       /usr/local/include/rados/librados.hpp
+       /usr/local/include/rados/buffer.h
+       /usr/local/include/rados/page.h
+       /usr/local/include/rados/crc32c.h
+       /usr/local/include/rados/memory.h
+       /usr/local/lib/rados-classes
+       /usr/local/lib/rados-classes/libcls_hello.so
+       /usr/local/lib/rados-classes/libcls_hello.la
+       /usr/local/lib/rados-classes/libcls_numops.so
+       /usr/local/lib/rados-classes/libcls_numops.la
+       /usr/local/lib/rados-classes/libcls_rbd.so
+       /usr/local/lib/rados-classes/libcls_rbd.la
+       /usr/local/lib/rados-classes/libcls_lock.so
+       /usr/local/lib/rados-classes/libcls_lock.la
+       /usr/local/lib/rados-classes/libcls_refcount.so
+       /usr/local/lib/rados-classes/libcls_refcount.la
+       /usr/local/lib/rados-classes/libcls_version.so
+       /usr/local/lib/rados-classes/libcls_version.la
+       /usr/local/lib/rados-classes/libcls_log.so
+       /usr/local/lib/rados-classes/libcls_log.la
+       /usr/local/lib/rados-classes/libcls_statelog.so
+       /usr/local/lib/rados-classes/libcls_statelog.la
+       /usr/local/lib/rados-classes/libcls_timeindex.so
+       /usr/local/lib/rados-classes/libcls_timeindex.la
+       /usr/local/lib/rados-classes/libcls_replica_log.so
+       /usr/local/lib/rados-classes/libcls_replica_log.la
+       /usr/local/lib/rados-classes/libcls_user.so
+       /usr/local/lib/rados-classes/libcls_user.la
+       /usr/local/lib/rados-classes/libcls_rgw.so
+       /usr/local/lib/rados-classes/libcls_rgw.la
+       /usr/local/lib/rados-classes/libcls_cephfs.so
+       /usr/local/lib/rados-classes/libcls_cephfs.la
+       /usr/local/lib/rados-classes/libcls_journal.so
+       /usr/local/lib/rados-classes/libcls_journal.la
+       /usr/local/include/radosstriper
+       /usr/local/include/radosstriper/libradosstriper.h
+       /usr/local/include/radosstriper/libradosstriper.hpp
+       /usr/local/lib/ceph/ceph_common.sh
M       /usr/local/etc/bash_completion.d
M       /usr/local/var/lib
M       /usr/local/var/log
M       /usr/srcs/Ceph/src/ceph/src/.libs
M       /usr/srcs/Ceph/src/ceph/src/ceph-detect-init
-       /usr/local/bin/easy_install
M       /usr/local/lib/python2.7/site-packages/easy-install.pth
M       /usr/srcs/Ceph/src/ceph/src/ceph-detect-init/build

Re: Compiling for FreeBSD

[Gregory Farnum]

On Mon, Nov 30, 2015 at 11:04 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 30-11-2015 15:40, Willem Jan Withagen wrote:
>> On 30-11-2015 15:13, Mykola Golub wrote:
>>> On Mon, Nov 30, 2015 at 12:53:54PM +0100, Willem Jan Withagen wrote:
>>>
>>>>> git clone --recursive -b wip-freebsd https://github.com/trociny/ceph.git
>>>>> cd ceph
>>>>> ./install-deps.sh
>>>>> ./do_autogen.sh
>>>>> gmake
>>>>> cd src
>>>>> ./vstart.sh # start dev cluster
>>>>> ./ceph -s # check it works
>>>>>
>>>>
>>>> So what backend for the OSDs are you using here?
>>>
>>> The default one, which is FileStore. I have not done much testing
>>> though...
>>>
>>
>> Well,
>> Cherry-picked a few of you diffs.
>> Compilation is now slugging thru the tests in my tree.
>> Running gmake in serial mode, so it is slow with clang.
>
> Installed the results:
> gmake install
>
> And looked at what it delivered:
>         zfs diff <fs-en>
>
> And I have to be honest that I'm not really enjoying all the ceph-test
> stuff in my /usr/local/bin....
> Would prefer it to go into something like:
>         /usr/local/libexec/ceph/tests
> or
>         /usr/local/share/ceph/tests
>
> Not sure if this is possible without disrupting the automated testing.

As far as I know, none of what we do upstream relies on "make install"
and I don't think it's very well-maintained. You could make some
changes and submit a PR — if it does break something it should show up
in the autobuilder bot.
-Greg
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 30-11-2015 17:20, Gregory Farnum wrote:
>> Installed the results:
>> gmake install
>>
>> And looked at what it delivered:
>>          zfs diff <fs-en>
>>
>> And I have to be honest that I'm not really enjoying all the ceph-test
>> stuff in my /usr/local/bin....
>> Would prefer it to go into something like:
>>          /usr/local/libexec/ceph/tests
>> or
>>          /usr/local/share/ceph/tests
>>
>> Not sure if this is possible without disrupting the automated testing.
>
> As far as I know, none of what we do upstream relies on "make install"
> and I don't think it's very well-maintained. You could make some
> changes and submit a PR — if it does break something it should show up
> in the autobuilder bot.

Well got that all compiled and installed.

Fixed a startup problem in /usr/local/bin/ceph's python with an unknown
error value. So that looks to run.

Created a config, and acompanying directories.

And now for the real work:
Run:
	ceph-mon -i freetest -d --debug_mon 10 --cluster digiceph -c 
/etc/ceph/ceph.conf

Which bombs out with:
2015-12-01 10:32:41.877243 804015000  0 ceph version 10.0.0-677-gd704c54 
(d704c54b7923ef7265fa27018e9411d8deb463b3), process (unknown), pid 93896
2015-12-01 10:32:41.879339 804015000 -1 load 
dlopen(/usr/local/lib/ceph/erasure-code/libec_jerasure.so): 
/usr/local/lib/ceph/erasure-code/libec_jerasure.so: Undefined symbol 
"ceph_arch_neon"

So of to find where ceph_arch_neon is, and why it seems not defined.
Perhaps as simple as loading the shared libs??

--WjW


--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 30-11-2015 14:21, Sage Weil wrote:
> The problem with all of the porting code in general is that it is doomed
> to break later on if we don't have (at least) ongoing build tests.  In
> order for a FreeBSD or OSX port to continue working we need VMs that run
> either gitbuilder or a jenkins job or similar so that we can tell when it
> breaks.
>
> If someone is willing to run a VM somewhere to do this we can pretty
> easily stick it on the gitbuilder page at
>
> 	http://ceph.com/gitbuilder.cgi


Hi Sage,

Could you give some pointers as to where to start running the tests.
I see a lot of "basic" tests to see if the platform is actually conformant.

So before plunging into running ceph-mon and stuff, it would perhaps be
better to actually run (parts of) the basic required tests..

Thanx,
--WjW

Re: Compiling for FreeBSD

[Mykola Golub]

On Tue, Dec 01, 2015 at 10:42:57AM +0100, Willem Jan Withagen wrote:
> On 30-11-2015 17:20, Gregory Farnum wrote:
> >>Installed the results:
> >>gmake install
> >>
> >>And looked at what it delivered:
> >>         zfs diff <fs-en>
> >>
> >>And I have to be honest that I'm not really enjoying all the ceph-test
> >>stuff in my /usr/local/bin....
> >>Would prefer it to go into something like:
> >>         /usr/local/libexec/ceph/tests
> >>or
> >>         /usr/local/share/ceph/tests
> >>
> >>Not sure if this is possible without disrupting the automated testing.
> >
> >As far as I know, none of what we do upstream relies on "make install"
> >and I don't think it's very well-maintained. You could make some
> >changes and submit a PR — if it does break something it should show up
> >in the autobuilder bot.
> 
> Well got that all compiled and installed.
> 
> Fixed a startup problem in /usr/local/bin/ceph's python with an unknown
> error value. So that looks to run.
> 
> Created a config, and acompanying directories.
> 
> And now for the real work:
> Run:
> 	ceph-mon -i freetest -d --debug_mon 10 --cluster digiceph -c
> /etc/ceph/ceph.conf
> 
> Which bombs out with:
> 2015-12-01 10:32:41.877243 804015000  0 ceph version 10.0.0-677-gd704c54
> (d704c54b7923ef7265fa27018e9411d8deb463b3), process (unknown), pid 93896
> 2015-12-01 10:32:41.879339 804015000 -1 load
> dlopen(/usr/local/lib/ceph/erasure-code/libec_jerasure.so):
> /usr/local/lib/ceph/erasure-code/libec_jerasure.so: Undefined symbol
> "ceph_arch_neon"
> 
> So of to find where ceph_arch_neon is, and why it seems not defined.
> Perhaps as simple as loading the shared libs??

You have to add -export-dynamic to LDFLAGS, something like in this
patch:

https://github.com/trociny/ceph/commit/dcee0c0635d37f2b36257c55a3cc69d05b5afe5e#diff-ef3c0ccbdde56cca822801c6ef1d289aR79

Also, you don't have to install binaries just to test if they work. As
I wrote previously:

cd src
./vstart.sh

It will start a dev cluster for you using binaries from the build
dir. You can check if it runs with:

./ceph -s

-- 
Mykola Golub
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 1-12-2015 13:22, Mykola Golub wrote:
> On Tue, Dec 01, 2015 at 10:42:57AM +0100, Willem Jan Withagen wrote:
>> On 30-11-2015 17:20, Gregory Farnum wrote:
>>>> Installed the results:
>>>> gmake install
>>>>
>>>> And looked at what it delivered:
>>>>          zfs diff <fs-en>
>>>>
>>>> And I have to be honest that I'm not really enjoying all the ceph-test
>>>> stuff in my /usr/local/bin....
>>>> Would prefer it to go into something like:
>>>>          /usr/local/libexec/ceph/tests
>>>> or
>>>>          /usr/local/share/ceph/tests
>>>>
>>>> Not sure if this is possible without disrupting the automated testing.
>>>
>>> As far as I know, none of what we do upstream relies on "make install"
>>> and I don't think it's very well-maintained. You could make some
>>> changes and submit a PR — if it does break something it should show up
>>> in the autobuilder bot.
>>
>> Well got that all compiled and installed.
>>
>> Fixed a startup problem in /usr/local/bin/ceph's python with an unknown
>> error value. So that looks to run.
>>
>> Created a config, and acompanying directories.
>>
>> And now for the real work:
>> Run:
>> 	ceph-mon -i freetest -d --debug_mon 10 --cluster digiceph -c
>> /etc/ceph/ceph.conf
>>
>> Which bombs out with:
>> 2015-12-01 10:32:41.877243 804015000  0 ceph version 10.0.0-677-gd704c54
>> (d704c54b7923ef7265fa27018e9411d8deb463b3), process (unknown), pid 93896
>> 2015-12-01 10:32:41.879339 804015000 -1 load
>> dlopen(/usr/local/lib/ceph/erasure-code/libec_jerasure.so):
>> /usr/local/lib/ceph/erasure-code/libec_jerasure.so: Undefined symbol
>> "ceph_arch_neon"
>>
>> So of to find where ceph_arch_neon is, and why it seems not defined.
>> Perhaps as simple as loading the shared libs??
>
> You have to add -export-dynamic to LDFLAGS, something like in this
> patch:

That one I missed...
I guess I need to read more carefully, since lots of the remainder did
not make sense. Mainly because I did not really want to start a dev cluster
right away.

So I'm  aiming for 'gmake check' atm.

--WjW

>
> https://github.com/trociny/ceph/commit/dcee0c0635d37f2b36257c55a3cc69d05b5afe5e#diff-ef3c0ccbdde56cca822801c6ef1d289aR79
>
> Also, you don't have to install binaries just to test if they work. As
> I wrote previously:
>
> cd src
> ./vstart.sh
>
> It will start a dev cluster for you using binaries from the build
> dir. You can check if it runs with:
>
> ./ceph -s
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Compiling for FreeBSD

[Sage Weil]

On Tue, 1 Dec 2015, Willem Jan Withagen wrote:
> On 30-11-2015 14:21, Sage Weil wrote:
> > The problem with all of the porting code in general is that it is doomed
> > to break later on if we don't have (at least) ongoing build tests.  In
> > order for a FreeBSD or OSX port to continue working we need VMs that run
> > either gitbuilder or a jenkins job or similar so that we can tell when it
> > breaks.
> > 
> > If someone is willing to run a VM somewhere to do this we can pretty
> > easily stick it on the gitbuilder page at
> > 
> > 	http://ceph.com/gitbuilder.cgi
> 
> 
> Hi Sage,
> 
> Could you give some pointers as to where to start running the tests.
> I see a lot of "basic" tests to see if the platform is actually conformant.
> 
> So before plunging into running ceph-mon and stuff, it would perhaps be
> better to actually run (parts of) the basic required tests..

I would start with 'make check' from src/... that's what we'd actually 
want the gitbuilder to do.

sage

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 1-12-2015 14:30, Sage Weil wrote:
> On Tue, 1 Dec 2015, Willem Jan Withagen wrote:
>> On 30-11-2015 14:21, Sage Weil wrote:
>>> The problem with all of the porting code in general is that it is doomed
>>> to break later on if we don't have (at least) ongoing build tests.  In
>>> order for a FreeBSD or OSX port to continue working we need VMs that run
>>> either gitbuilder or a jenkins job or similar so that we can tell when it
>>> breaks.
>>>
>>> If someone is willing to run a VM somewhere to do this we can pretty
>>> easily stick it on the gitbuilder page at
>>>
>>> 	http://ceph.com/gitbuilder.cgi
>>
>>
>> Hi Sage,
>>
>> Could you give some pointers as to where to start running the tests.
>> I see a lot of "basic" tests to see if the platform is actually conformant.
>>
>> So before plunging into running ceph-mon and stuff, it would perhaps be
>> better to actually run (parts of) the basic required tests..
>
> I would start with 'make check' from src/... that's what we'd actually
> want the gitbuilder to do.

I was running that at the moment....
Found the suggestion on the developers pages, in the manual section.
Sort of hidden at the bottom. :)

Did kill it in between, but now when I run it, it just only generates 
the report.
So I just went make clean, which is rather too much...
But could not really figure out the makefiles in test (yet)

How do I reset the test results?

--WjW

Re: Compiling for FreeBSD

[Sage Weil]

On Tue, 1 Dec 2015, Willem Jan Withagen wrote:
> On 1-12-2015 14:30, Sage Weil wrote:
> > On Tue, 1 Dec 2015, Willem Jan Withagen wrote:
> > > On 30-11-2015 14:21, Sage Weil wrote:
> > > > The problem with all of the porting code in general is that it is doomed
> > > > to break later on if we don't have (at least) ongoing build tests.  In
> > > > order for a FreeBSD or OSX port to continue working we need VMs that run
> > > > either gitbuilder or a jenkins job or similar so that we can tell when
> > > > it
> > > > breaks.
> > > > 
> > > > If someone is willing to run a VM somewhere to do this we can pretty
> > > > easily stick it on the gitbuilder page at
> > > > 
> > > > 	http://ceph.com/gitbuilder.cgi
> > > 
> > > 
> > > Hi Sage,
> > > 
> > > Could you give some pointers as to where to start running the tests.
> > > I see a lot of "basic" tests to see if the platform is actually
> > > conformant.
> > > 
> > > So before plunging into running ceph-mon and stuff, it would perhaps be
> > > better to actually run (parts of) the basic required tests..
> > 
> > I would start with 'make check' from src/... that's what we'd actually
> > want the gitbuilder to do.
> 
> I was running that at the moment....
> Found the suggestion on the developers pages, in the manual section.
> Sort of hidden at the bottom. :)
> 
> Did kill it in between, but now when I run it, it just only generates the
> report.
> So I just went make clean, which is rather too much...
> But could not really figure out the makefiles in test (yet)
> 
> How do I reset the test results?

I don't think there is anything to reset... just re-ru make check.  The 
exception is probably just if you hit control-c but it left running 
processes behind (./stop.sh should clean those up).

At least, that's the case on Linux.. maybe the (auto)tools are a bit 
different on *BSD?

sage

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 1-12-2015 15:35, Sage Weil wrote:
> On Tue, 1 Dec 2015, Willem Jan Withagen wrote:
>> On 1-12-2015 14:30, Sage Weil wrote:
>>> On Tue, 1 Dec 2015, Willem Jan Withagen wrote:
>>>> On 30-11-2015 14:21, Sage Weil wrote:
>>>>> The problem with all of the porting code in general is that it is doomed
>>>>> to break later on if we don't have (at least) ongoing build tests.  In
>>>>> order for a FreeBSD or OSX port to continue working we need VMs that run
>>>>> either gitbuilder or a jenkins job or similar so that we can tell when
>>>>> it
>>>>> breaks.
>>>>>
>>>>> If someone is willing to run a VM somewhere to do this we can pretty
>>>>> easily stick it on the gitbuilder page at
>>>>>
>>>>> 	http://ceph.com/gitbuilder.cgi
>>>>
>>>>
>>>> Hi Sage,
>>>>
>>>> Could you give some pointers as to where to start running the tests.
>>>> I see a lot of "basic" tests to see if the platform is actually
>>>> conformant.
>>>>
>>>> So before plunging into running ceph-mon and stuff, it would perhaps be
>>>> better to actually run (parts of) the basic required tests..
>>>
>>> I would start with 'make check' from src/... that's what we'd actually
>>> want the gitbuilder to do.
>>
>> I was running that at the moment....
>> Found the suggestion on the developers pages, in the manual section.
>> Sort of hidden at the bottom. :)
>>
>> Did kill it in between, but now when I run it, it just only generates the
>> report.
>> So I just went make clean, which is rather too much...
>> But could not really figure out the makefiles in test (yet)
>>
>> How do I reset the test results?
>
> I don't think there is anything to reset... just re-ru make check.  The
> exception is probably just if you hit control-c but it left running
> processes behind (./stop.sh should clean those up).
>

'mmm,
Strange I had it generate tests.... at one point.
And now just plain nothing....

Server has rebooted, so there should have nothing been left.

> At least, that's the case on Linux.. maybe the (auto)tools are a bit
> different on *BSD?

I think in the short run it will not be the code that is going to be a
major porting pain. But getting the run-time environment ironed out is
just plain (hard) work. Things where /bin/sh expects certain bash-isms.
Where paths have not been setup to the fullest all the way back into
./configure: like ${initrddir} => /etc/init.d versus /usr/local/etc/rc.d.
Probably plenty more like these.

I've also seen calls in the code to things like:
	arch
	hdparm
things that just are not there in (basic) FreeBSD....
But we'll get around al of that.

I survived porting Unix tools (including UUCP) to Win95 and OS/2. So
until we get to kernel things I just keep pushing along.

Just for clarity:
Gitbuilder just runs:
	make check
and collects the output?
So that would be the way to tackle this, get complete (successful) output
from:
	gmake clean && gmake test


--WjW

Re: Compiling for FreeBSD

[Alan Somers]

On Tue, Dec 1, 2015 at 9:24 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 1-12-2015 15:35, Sage Weil wrote:
>>
>> On Tue, 1 Dec 2015, Willem Jan Withagen wrote:
>>>
>>> On 1-12-2015 14:30, Sage Weil wrote:
>>>>
>>>> On Tue, 1 Dec 2015, Willem Jan Withagen wrote:
>>>>>
>>>>> On 30-11-2015 14:21, Sage Weil wrote:
>>>>>>
>>>>>> The problem with all of the porting code in general is that it is
>>>>>> doomed
>>>>>> to break later on if we don't have (at least) ongoing build tests.  In
>>>>>> order for a FreeBSD or OSX port to continue working we need VMs that
>>>>>> run
>>>>>> either gitbuilder or a jenkins job or similar so that we can tell when
>>>>>> it
>>>>>> breaks.
>>>>>>
>>>>>> If someone is willing to run a VM somewhere to do this we can pretty
>>>>>> easily stick it on the gitbuilder page at
>>>>>>
>>>>>>         http://ceph.com/gitbuilder.cgi
>>>>>
>>>>>
>>>>>
>>>>> Hi Sage,
>>>>>
>>>>> Could you give some pointers as to where to start running the tests.
>>>>> I see a lot of "basic" tests to see if the platform is actually
>>>>> conformant.
>>>>>
>>>>> So before plunging into running ceph-mon and stuff, it would perhaps be
>>>>> better to actually run (parts of) the basic required tests..
>>>>
>>>>
>>>> I would start with 'make check' from src/... that's what we'd actually
>>>> want the gitbuilder to do.
>>>
>>>
>>> I was running that at the moment....
>>> Found the suggestion on the developers pages, in the manual section.
>>> Sort of hidden at the bottom. :)
>>>
>>> Did kill it in between, but now when I run it, it just only generates the
>>> report.
>>> So I just went make clean, which is rather too much...
>>> But could not really figure out the makefiles in test (yet)
>>>
>>> How do I reset the test results?
>>
>>
>> I don't think there is anything to reset... just re-ru make check.  The
>> exception is probably just if you hit control-c but it left running
>> processes behind (./stop.sh should clean those up).
>>
>
> 'mmm,
> Strange I had it generate tests.... at one point.
> And now just plain nothing....
>
> Server has rebooted, so there should have nothing been left.
>
>> At least, that's the case on Linux.. maybe the (auto)tools are a bit
>> different on *BSD?
>
>
> I think in the short run it will not be the code that is going to be a
> major porting pain. But getting the run-time environment ironed out is
> just plain (hard) work. Things where /bin/sh expects certain bash-isms.
> Where paths have not been setup to the fullest all the way back into
> ./configure: like ${initrddir} => /etc/init.d versus /usr/local/etc/rc.d.
> Probably plenty more like these.
>
> I've also seen calls in the code to things like:
>         arch
>         hdparm
> things that just are not there in (basic) FreeBSD....
> But we'll get around al of that.
>
> I survived porting Unix tools (including UUCP) to Win95 and OS/2. So
> until we get to kernel things I just keep pushing along.
>
> Just for clarity:
> Gitbuilder just runs:
>         make check
> and collects the output?
> So that would be the way to tackle this, get complete (successful) output
> from:
>         gmake clean && gmake test
>
>
> --WjW

I did some work porting Ceph to FreeBSD, but got distracted and
stopped about two years ago.  You may find this port useful, though it
will probably need to be updated:

https://people.freebsd.org/~asomers/ports/net/ceph/

Also, there's one major outstanding issue that I know of.  It breaks
interoperability between FreeBSD and Linux Ceph nodes.  I posted a
patch to fix it, but it doesn't look like it's been merged yet.
http://tracker.ceph.com/issues/6636

-Alan

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 1-12-2015 18:22, Alan Somers wrote:
> I did some work porting Ceph to FreeBSD, but got distracted and
> stopped about two years ago.  You may find this port useful, though it
> will probably need to be updated:
>
> https://people.freebsd.org/~asomers/ports/net/ceph/

I'll chcek that one as well...

> Also, there's one major outstanding issue that I know of.  It breaks
> interoperability between FreeBSD and Linux Ceph nodes.  I posted a
> patch to fix it, but it doesn't look like it's been merged yet.
> http://tracker.ceph.com/issues/6636

In the issues I find:
====
Updated by Sage Weil almost 2 years ago

     Status changed from New to Verified
Updated by Sage Weil almost 2 years ago

     Assignee set to Noah Watkins
====

Probably left at that point because there was no presure to actually commit?

--WjW

Re: Compiling for FreeBSD

[Alan Somers]

On Tue, Dec 1, 2015 at 11:08 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 1-12-2015 18:22, Alan Somers wrote:
>>
>> I did some work porting Ceph to FreeBSD, but got distracted and
>> stopped about two years ago.  You may find this port useful, though it
>> will probably need to be updated:
>>
>> https://people.freebsd.org/~asomers/ports/net/ceph/
>
>
> I'll chcek that one as well...
>
>> Also, there's one major outstanding issue that I know of.  It breaks
>> interoperability between FreeBSD and Linux Ceph nodes.  I posted a
>> patch to fix it, but it doesn't look like it's been merged yet.
>> http://tracker.ceph.com/issues/6636
>
>
> In the issues I find:
> ====
> Updated by Sage Weil almost 2 years ago
>
>     Status changed from New to Verified
> Updated by Sage Weil almost 2 years ago
>
>     Assignee set to Noah Watkins
> ====
>
> Probably left at that point because there was no presure to actually commit?
>
> --WjW

It looks like Sage reviewed the change, but had some comments that
were mostly style-related.  Neither Noah nor I actually got around to
implementing Sage's suggestions.

https://github.com/ceph/ceph/pull/828

-Alan

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 1-12-2015 19:21, Alan Somers wrote:
> On Tue, Dec 1, 2015 at 11:08 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>> On 1-12-2015 18:22, Alan Somers wrote:
>>>
>>> I did some work porting Ceph to FreeBSD, but got distracted and
>>> stopped about two years ago.  You may find this port useful, though it
>>> will probably need to be updated:
>>>
>>> https://people.freebsd.org/~asomers/ports/net/ceph/
>>
>>
>> I'll chcek that one as well...
>>
>>> Also, there's one major outstanding issue that I know of.  It breaks
>>> interoperability between FreeBSD and Linux Ceph nodes.  I posted a
>>> patch to fix it, but it doesn't look like it's been merged yet.
>>> http://tracker.ceph.com/issues/6636
>>
>>
>> In the issues I find:
>> ====
>> Updated by Sage Weil almost 2 years ago
>>
>>      Status changed from New to Verified
>> Updated by Sage Weil almost 2 years ago
>>
>>      Assignee set to Noah Watkins
>> ====
>>
>> Probably left at that point because there was no presure to actually commit?
>>
>> --WjW
>
> It looks like Sage reviewed the change, but had some comments that
> were mostly style-related.  Neither Noah nor I actually got around to
> implementing Sage's suggestions.
>
> https://github.com/ceph/ceph/pull/828

Come to realise that this is about interoperability Linux <> FreeBSD.
Which is a problem when running a mix of nodes.

First I'm focussing on getting an all FreeBSD to
	configure
	compile
	test-run

Then see if we can actually build and run a dev-cluster up.

So your warning is very much appricated, but I'm not anywhere near that 
point. :(

--WjW

Re: Compiling for FreeBSD

[Sage Weil]

On Tue, 1 Dec 2015, Alan Somers wrote:
> On Tue, Dec 1, 2015 at 11:08 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> > On 1-12-2015 18:22, Alan Somers wrote:
> >>
> >> I did some work porting Ceph to FreeBSD, but got distracted and
> >> stopped about two years ago.  You may find this port useful, though it
> >> will probably need to be updated:
> >>
> >> https://people.freebsd.org/~asomers/ports/net/ceph/
> >
> >
> > I'll chcek that one as well...
> >
> >> Also, there's one major outstanding issue that I know of.  It breaks
> >> interoperability between FreeBSD and Linux Ceph nodes.  I posted a
> >> patch to fix it, but it doesn't look like it's been merged yet.
> >> http://tracker.ceph.com/issues/6636
> >
> >
> > In the issues I find:
> > ====
> > Updated by Sage Weil almost 2 years ago
> >
> >     Status changed from New to Verified
> > Updated by Sage Weil almost 2 years ago
> >
> >     Assignee set to Noah Watkins
> > ====
> >
> > Probably left at that point because there was no presure to actually commit?
> >
> > --WjW
> 
> It looks like Sage reviewed the change, but had some comments that
> were mostly style-related.  Neither Noah nor I actually got around to
> implementing Sage's suggestions.
> 
> https://github.com/ceph/ceph/pull/828

The uuid transition to boost::uuid has happened since then (a few months 
back) and I believe Rohan's AIX and Solaris ports for librados (that just 
merged) included a fix for the sockaddr_storage issue:

	https://github.com/ceph/ceph/blob/master/src/msg/msg_types.h#L180

and also

	https://github.com/ceph/ceph/blob/master/src/msg/msg_types.h#L160

?

sage

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 1-12-2015 19:36, Sage Weil wrote:
> On Tue, 1 Dec 2015, Alan Somers wrote:
>> On Tue, Dec 1, 2015 at 11:08 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>> On 1-12-2015 18:22, Alan Somers wrote:
>>>>
>>>> I did some work porting Ceph to FreeBSD, but got distracted and
>>>> stopped about two years ago.  You may find this port useful, though it
>>>> will probably need to be updated:
>>>>
>>>> https://people.freebsd.org/~asomers/ports/net/ceph/
>>>
>>>
>>> I'll chcek that one as well...
>>>
>>>> Also, there's one major outstanding issue that I know of.  It breaks
>>>> interoperability between FreeBSD and Linux Ceph nodes.  I posted a
>>>> patch to fix it, but it doesn't look like it's been merged yet.
>>>> http://tracker.ceph.com/issues/6636
>>>
>>>
>>> In the issues I find:
>>> ====
>>> Updated by Sage Weil almost 2 years ago
>>>
>>>      Status changed from New to Verified
>>> Updated by Sage Weil almost 2 years ago
>>>
>>>      Assignee set to Noah Watkins
>>> ====
>>>
>>> Probably left at that point because there was no presure to actually commit?
>>>
>>> --WjW
>>
>> It looks like Sage reviewed the change, but had some comments that
>> were mostly style-related.  Neither Noah nor I actually got around to
>> implementing Sage's suggestions.
>>
>> https://github.com/ceph/ceph/pull/828
>
> The uuid transition to boost::uuid has happened since then (a few months
> back) and I believe Rohan's AIX and Solaris ports for librados (that just
> merged) included a fix for the sockaddr_storage issue:
>
> 	https://github.com/ceph/ceph/blob/master/src/msg/msg_types.h#L180
>
> and also
>
> 	https://github.com/ceph/ceph/blob/master/src/msg/msg_types.h#L160


Would be nice to actually find that this works for FreeBSD as well.
But I'm putting this on the watch-list, once I get there.

--WjW

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 1-12-2015 17:24, Willem Jan Withagen wrote:
> I think in the short run it will not be the code that is going to be a
> major porting pain. But getting the run-time environment ironed out is
> just plain (hard) work. Things where /bin/sh expects certain bash-isms.
> Where paths have not been setup to the fullest all the way back into
> ./configure: like ${initrddir} => /etc/init.d versus /usr/local/etc/rc.d.
> Probably plenty more like these.

To answer myself:
	grep != grep

freetest# grep -P test *
grep: The -P option is not supported

:(

And only to match things like:
	ceph-authtool kring --list|grep -P '^\tcaps '

Start looking for a mode that works could work on both....

--WjW

Re: Compiling for FreeBSD

[Yan, Zheng]

see https://github.com/ceph/ceph/pull/6770. The code can be compiled
on FreeBSD/OSX, most client programs can connect to ceph servers on
Linux.

Regards
Yan. Zheng

On Wed, Dec 2, 2015 at 2:43 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 1-12-2015 19:36, Sage Weil wrote:
>>
>> On Tue, 1 Dec 2015, Alan Somers wrote:
>>>
>>> On Tue, Dec 1, 2015 at 11:08 AM, Willem Jan Withagen <wjw@digiware.nl>
>>> wrote:
>>>>
>>>> On 1-12-2015 18:22, Alan Somers wrote:
>>>>>
>>>>>
>>>>> I did some work porting Ceph to FreeBSD, but got distracted and
>>>>> stopped about two years ago.  You may find this port useful, though it
>>>>> will probably need to be updated:
>>>>>
>>>>> https://people.freebsd.org/~asomers/ports/net/ceph/
>>>>
>>>>
>>>>
>>>> I'll chcek that one as well...
>>>>
>>>>> Also, there's one major outstanding issue that I know of.  It breaks
>>>>> interoperability between FreeBSD and Linux Ceph nodes.  I posted a
>>>>> patch to fix it, but it doesn't look like it's been merged yet.
>>>>> http://tracker.ceph.com/issues/6636
>>>>
>>>>
>>>>
>>>> In the issues I find:
>>>> ====
>>>> Updated by Sage Weil almost 2 years ago
>>>>
>>>>      Status changed from New to Verified
>>>> Updated by Sage Weil almost 2 years ago
>>>>
>>>>      Assignee set to Noah Watkins
>>>> ====
>>>>
>>>> Probably left at that point because there was no presure to actually
>>>> commit?
>>>>
>>>> --WjW
>>>
>>>
>>> It looks like Sage reviewed the change, but had some comments that
>>> were mostly style-related.  Neither Noah nor I actually got around to
>>> implementing Sage's suggestions.
>>>
>>> https://github.com/ceph/ceph/pull/828
>>
>>
>> The uuid transition to boost::uuid has happened since then (a few months
>> back) and I believe Rohan's AIX and Solaris ports for librados (that just
>> merged) included a fix for the sockaddr_storage issue:
>>
>>         https://github.com/ceph/ceph/blob/master/src/msg/msg_types.h#L180
>>
>> and also
>>
>>         https://github.com/ceph/ceph/blob/master/src/msg/msg_types.h#L160
>
>
>
> Would be nice to actually find that this works for FreeBSD as well.
> But I'm putting this on the watch-list, once I get there.
>
> --WjW

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 2-12-2015 15:13, Yan, Zheng wrote:
> see https://github.com/ceph/ceph/pull/6770. The code can be compiled
> on FreeBSD/OSX, most client programs can connect to ceph servers on
> Linux.

Hi,

I do like some of the inline compiler tests.

I guess that the way the errno's are done like the other OS's have done
as well?
I'd normally solve this with a static array, and just index it.
But perhaps the compiler is smart enough to do the same.


I see that you have disabled uuid?
Might I ask why?

I Suggest you have a look at the issue Alan brought up.
Which is a possible fix for doing it the other way around:
	Linux clients on a FreeBSD "cluster"
But as Sage suggest: Could be very well solved by fixed brougt in for AIX.

--WjW

> Regards
> Yan. Zheng
> 
> On Wed, Dec 2, 2015 at 2:43 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>> On 1-12-2015 19:36, Sage Weil wrote:
>>>
>>> On Tue, 1 Dec 2015, Alan Somers wrote:
>>>>
>>>> On Tue, Dec 1, 2015 at 11:08 AM, Willem Jan Withagen <wjw@digiware.nl>
>>>> wrote:
>>>>>
>>>>> On 1-12-2015 18:22, Alan Somers wrote:
>>>>>>
>>>>>>
>>>>>> I did some work porting Ceph to FreeBSD, but got distracted and
>>>>>> stopped about two years ago.  You may find this port useful, though it
>>>>>> will probably need to be updated:
>>>>>>
>>>>>> https://people.freebsd.org/~asomers/ports/net/ceph/
>>>>>
>>>>>
>>>>>
>>>>> I'll chcek that one as well...
>>>>>
>>>>>> Also, there's one major outstanding issue that I know of.  It breaks
>>>>>> interoperability between FreeBSD and Linux Ceph nodes.  I posted a
>>>>>> patch to fix it, but it doesn't look like it's been merged yet.
>>>>>> http://tracker.ceph.com/issues/6636
>>>>>
>>>>>
>>>>>
>>>>> In the issues I find:
>>>>> ====
>>>>> Updated by Sage Weil almost 2 years ago
>>>>>
>>>>>      Status changed from New to Verified
>>>>> Updated by Sage Weil almost 2 years ago
>>>>>
>>>>>      Assignee set to Noah Watkins
>>>>> ====
>>>>>
>>>>> Probably left at that point because there was no presure to actually
>>>>> commit?
>>>>>
>>>>> --WjW
>>>>
>>>>
>>>> It looks like Sage reviewed the change, but had some comments that
>>>> were mostly style-related.  Neither Noah nor I actually got around to
>>>> implementing Sage's suggestions.
>>>>
>>>> https://github.com/ceph/ceph/pull/828
>>>
>>>
>>> The uuid transition to boost::uuid has happened since then (a few months
>>> back) and I believe Rohan's AIX and Solaris ports for librados (that just
>>> merged) included a fix for the sockaddr_storage issue:
>>>
>>>         https://github.com/ceph/ceph/blob/master/src/msg/msg_types.h#L180
>>>
>>> and also
>>>
>>>         https://github.com/ceph/ceph/blob/master/src/msg/msg_types.h#L160
>>
>>
>>
>> Would be nice to actually find that this works for FreeBSD as well.
>> But I'm putting this on the watch-list, once I get there.
>>
>> --WjW
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 1-12-2015 19:51, Willem Jan Withagen wrote:
> On 1-12-2015 17:24, Willem Jan Withagen wrote:
>> I think in the short run it will not be the code that is going to be a
>> major porting pain. But getting the run-time environment ironed out is
>> just plain (hard) work. Things where /bin/sh expects certain bash-isms.
>> Where paths have not been setup to the fullest all the way back into
>> ./configure: like ${initrddir} => /etc/init.d versus /usr/local/etc/rc.d.
>> Probably plenty more like these.
> 
> To answer myself:
>     grep != grep
> 
> freetest# grep -P test *
> grep: The -P option is not supported
> 
> :(
> 
> And only to match things like:
>     ceph-authtool kring --list|grep -P '^\tcaps '
> 
> Start looking for a mode that works could work on both....

Looks like that will be something along the lines of

	grep -E '^\Wtcaps'

And that gets a lot of tests accepted.

--WjW

Re: Compiling for FreeBSD, missing rbd

[Willem Jan Withagen]

Running gmake check is starting to work.
	reporting still thinks there are no successful tests
	but tests themselves report OKE

But where I really can not get it to work is with testing rbd.

Is starts with the simple:
	 /bin/sh: rbd: not found

And whatever I'm trying in configure, no way am I able to get either an
executable or a script that will do rbd....

So how do I get a rbd that can be used in the tests.

What I do have is rbdmap, but that is a totally different thingy.

--WjW

Re: Compiling for FreeBSD

[Yan, Zheng]

On Thu, Dec 3, 2015 at 4:52 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 2-12-2015 15:13, Yan, Zheng wrote:
>> see https://github.com/ceph/ceph/pull/6770. The code can be compiled
>> on FreeBSD/OSX, most client programs can connect to ceph servers on
>> Linux.
>
> Hi,
>
> I do like some of the inline compiler tests.
>
> I guess that the way the errno's are done like the other OS's have done
> as well?
> I'd normally solve this with a static array, and just index it.
> But perhaps the compiler is smart enough to do the same.
>
>
> I see that you have disabled uuid?
> Might I ask why?

not disable. Currently ceph uses boost uuid implementation. so no need
to link to libuuid.

>
> I Suggest you have a look at the issue Alan brought up.
> Which is a possible fix for doing it the other way around:
>         Linux clients on a FreeBSD "cluster"
> But as Sage suggest: Could be very well solved by fixed brougt in for AIX.
>
> --WjW
>
>> Regards
>> Yan. Zheng
>>
>> On Wed, Dec 2, 2015 at 2:43 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>> On 1-12-2015 19:36, Sage Weil wrote:
>>>>
>>>> On Tue, 1 Dec 2015, Alan Somers wrote:
>>>>>
>>>>> On Tue, Dec 1, 2015 at 11:08 AM, Willem Jan Withagen <wjw@digiware.nl>
>>>>> wrote:
>>>>>>
>>>>>> On 1-12-2015 18:22, Alan Somers wrote:
>>>>>>>
>>>>>>>
>>>>>>> I did some work porting Ceph to FreeBSD, but got distracted and
>>>>>>> stopped about two years ago.  You may find this port useful, though it
>>>>>>> will probably need to be updated:
>>>>>>>
>>>>>>> https://people.freebsd.org/~asomers/ports/net/ceph/
>>>>>>
>>>>>>
>>>>>>
>>>>>> I'll chcek that one as well...
>>>>>>
>>>>>>> Also, there's one major outstanding issue that I know of.  It breaks
>>>>>>> interoperability between FreeBSD and Linux Ceph nodes.  I posted a
>>>>>>> patch to fix it, but it doesn't look like it's been merged yet.
>>>>>>> http://tracker.ceph.com/issues/6636
>>>>>>
>>>>>>
>>>>>>
>>>>>> In the issues I find:
>>>>>> ====
>>>>>> Updated by Sage Weil almost 2 years ago
>>>>>>
>>>>>>      Status changed from New to Verified
>>>>>> Updated by Sage Weil almost 2 years ago
>>>>>>
>>>>>>      Assignee set to Noah Watkins
>>>>>> ====
>>>>>>
>>>>>> Probably left at that point because there was no presure to actually
>>>>>> commit?
>>>>>>
>>>>>> --WjW
>>>>>
>>>>>
>>>>> It looks like Sage reviewed the change, but had some comments that
>>>>> were mostly style-related.  Neither Noah nor I actually got around to
>>>>> implementing Sage's suggestions.
>>>>>
>>>>> https://github.com/ceph/ceph/pull/828
>>>>
>>>>
>>>> The uuid transition to boost::uuid has happened since then (a few months
>>>> back) and I believe Rohan's AIX and Solaris ports for librados (that just
>>>> merged) included a fix for the sockaddr_storage issue:
>>>>
>>>>         https://github.com/ceph/ceph/blob/master/src/msg/msg_types.h#L180
>>>>
>>>> and also
>>>>
>>>>         https://github.com/ceph/ceph/blob/master/src/msg/msg_types.h#L160
>>>
>>>
>>>
>>> Would be nice to actually find that this works for FreeBSD as well.
>>> But I'm putting this on the watch-list, once I get there.
>>>
>>> --WjW
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>

Re: Compiling for FreeBSD, runtimes for seperate tests.

[Willem Jan Withagen]

On 2-12-2015 22:10, Willem Jan Withagen wrote:

> Running gmake check

Now I start wondering how long certain tests are able to run:

I've killed:
	 unittest_chain_xattr
because it was running already voor 12 hours.

And unittest_lfnindex is also running for > 20 minutes....

Is there a way to specify something like a max runtime for these tests?
So that I am able to run the testset repeatedly, to monitor regression
once I start changing code.

I saw Loic talk about some of the expected test runtimes.
But having a better feeling per specific test would be useful in getting
the test to progress

Thanx,
--WjW

Re: Compiling for FreeBSD, missing rbd

[Mykola Golub]

On Wed, Dec 02, 2015 at 11:47:04PM +0100, Willem Jan Withagen wrote:
> 
> Running gmake check is starting to work.
> 	reporting still thinks there are no successful tests
> 	but tests themselves report OKE
> 
> But where I really can not get it to work is with testing rbd.
> 
> Is starts with the simple:
> 	 /bin/sh: rbd: not found
> 
> And whatever I'm trying in configure, no way am I able to get either an
> executable or a script that will do rbd....
> 
> So how do I get a rbd that can be used in the tests.

On the master rbd is built on Linux only, see src/tools/Makefile-client.am.

I have a patch in my branch that makes rbd build on all platforms
(excluding kernel bits):

https://github.com/trociny/ceph/commit/cd427e3ef2a5a307d6649e5b1dcfefdf67d60b29

But large rbd refactoring has happened in master since that time, and
my patch requires some work to apply on the current master.

-- 
Mykola Golub

Re: Compiling for FreeBSD, missing rbd

[Willem Jan Withagen]

On 3-12-2015 13:34, Mykola Golub wrote:
> On Wed, Dec 02, 2015 at 11:47:04PM +0100, Willem Jan Withagen wrote:
>>
>> Running gmake check is starting to work.
>> 	reporting still thinks there are no successful tests
>> 	but tests themselves report OKE
>>
>> But where I really can not get it to work is with testing rbd.
>>
>> Is starts with the simple:
>> 	 /bin/sh: rbd: not found
>>
>> And whatever I'm trying in configure, no way am I able to get either an
>> executable or a script that will do rbd....
>>
>> So how do I get a rbd that can be used in the tests.
> 
> On the master rbd is built on Linux only, see src/tools/Makefile-client.am.
> 
> I have a patch in my branch that makes rbd build on all platforms
> (excluding kernel bits):
> 
> https://github.com/trociny/ceph/commit/cd427e3ef2a5a307d6649e5b1dcfefdf67d60b29
> 
> But large rbd refactoring has happened in master since that time, and
> my patch requires some work to apply on the current master.
> 

Oke,

Is see the patch, which is simple enough.
Now when code has been moved around, I can imagine that there is going
to be some work required.

It's going on the list.

--WjW

Re: Compiling for FreeBSD, runtimes for seperate tests.

[Willem Jan Withagen]

On 3-12-2015 10:50, Willem Jan Withagen wrote:
> On 2-12-2015 22:10, Willem Jan Withagen wrote:
>
>> Running gmake check
>
> Now I start wondering how long certain tests are able to run:
>
> I've killed:
>       unittest_chain_xattr
> because it was running already voor 12 hours.
>
> And unittest_lfnindex is also running for > 20 minutes....
>
> Is there a way to specify something like a max runtime for these tests?
> So that I am able to run the testset repeatedly, to monitor regression
> once I start changing code.
>
> I saw Loic talk about some of the expected test runtimes.
> But having a better feeling per specific test would be useful in getting
> the test to progress

Excluded all rbd tests, and killed some of the processes that ran > 1200 
secs.
But below the results...

Need to figure the Makefiles out to exclude rbd testing.
And perhaps start a timelimit on running processes.

--WjW

============================================================================
Testsuite summary for ceph 10.0.0
============================================================================
# TOTAL: 119
# PASS:  87
# SKIP:  0
# XFAIL: 0
# FAIL:  32
# XPASS: 0
# ERROR: 0
============================================================================
See src/test-suite.log
Please report to ceph-devel@vger.kernel.org
============================================================================

Re: Compiling for FreeBSD, runtimes for seperate tests.

[Gregory Farnum]

On Thu, Dec 3, 2015 at 1:50 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 2-12-2015 22:10, Willem Jan Withagen wrote:
>
>> Running gmake check
>
>
> Now I start wondering how long certain tests are able to run:
>
> I've killed:
>          unittest_chain_xattr
> because it was running already voor 12 hours.
>
> And unittest_lfnindex is also running for > 20 minutes....

Well, 12 hours is ludicrous, but a few of the tests really do take 20
minutes — I think the shec EC one? I don't have any better info than
that for you, though. I mostly just wait on the gitbuilders.
-Greg


>
> Is there a way to specify something like a max runtime for these tests?
> So that I am able to run the testset repeatedly, to monitor regression
> once I start changing code.
>
> I saw Loic talk about some of the expected test runtimes.
> But having a better feeling per specific test would be useful in getting
> the test to progress
>
> Thanx,
> --WjW
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 3-12-2015 01:27, Yan, Zheng wrote:
> On Thu, Dec 3, 2015 at 4:52 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>> On 2-12-2015 15:13, Yan, Zheng wrote:

>> I see that you have disabled uuid?
>> Might I ask why?
> 
> not disable. Currently ceph uses boost uuid implementation. so no need
> to link to libuuid.

And

>>>>> The uuid transition to boost::uuid has happened since then (a few months
>>>>> back) and I believe Rohan's AIX and Solaris ports for librados (that just
>>>>> merged) included a fix for the sockaddr_storage issue:

I cannot seem to find the package or port that defines boost::uuid.
So how did you make it available to the build system?

Thanx,
--WjW

Re: Compiling for FreeBSD

[Gregory Farnum]

On Fri, Dec 4, 2015 at 10:30 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 3-12-2015 01:27, Yan, Zheng wrote:
>> On Thu, Dec 3, 2015 at 4:52 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>> On 2-12-2015 15:13, Yan, Zheng wrote:
>
>>> I see that you have disabled uuid?
>>> Might I ask why?
>>
>> not disable. Currently ceph uses boost uuid implementation. so no need
>> to link to libuuid.
>
> And
>
>>>>>> The uuid transition to boost::uuid has happened since then (a few months
>>>>>> back) and I believe Rohan's AIX and Solaris ports for librados (that just
>>>>>> merged) included a fix for the sockaddr_storage issue:
>
> I cannot seem to find the package or port that defines boost::uuid.
> So how did you make it available to the build system?

http://www.boost.org/doc/libs/1_59_0/libs/uuid/

It's part of the boost labyrinth. I think in Debian it's just part of
libboost-dev, but you might need to dig around in whatever packaging
you're using for FreeBSD.
-Greg

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 4-12-2015 19:44, Gregory Farnum wrote:
> On Fri, Dec 4, 2015 at 10:30 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>> On 3-12-2015 01:27, Yan, Zheng wrote:
>>> On Thu, Dec 3, 2015 at 4:52 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>> On 2-12-2015 15:13, Yan, Zheng wrote:
>>
>>>> I see that you have disabled uuid?
>>>> Might I ask why?
>>>
>>> not disable. Currently ceph uses boost uuid implementation. so no need
>>> to link to libuuid.
>>
>> And
>>
>>>>>>> The uuid transition to boost::uuid has happened since then (a few months
>>>>>>> back) and I believe Rohan's AIX and Solaris ports for librados (that just
>>>>>>> merged) included a fix for the sockaddr_storage issue:
>>
>> I cannot seem to find the package or port that defines boost::uuid.
>> So how did you make it available to the build system?
> 
> http://www.boost.org/doc/libs/1_59_0/libs/uuid/
> 
> It's part of the boost labyrinth. I think in Debian it's just part of
> libboost-dev, but you might need to dig around in whatever packaging
> you're using for FreeBSD.

I've dumped all of the labels in de boost libraries.
So it is not default available with the pre-build packages.
Which is understandable given the size of all that is available in
boost. But lets go and fetch/build some of that stuff.

--WjW

Re: Compiling for FreeBSD, Clang refuses to compile a test

[Willem Jan Withagen]

src/test/erasure-code/TestErasureCodeIsa.cc

contains snippets, function definition like:

buffer::ptr enc[k + m];
   // create buffers with a copy of the original data to be able to 
compare it after decoding
   {
     for (int i = 0; i < (k + m); i++) {

Clang refuses because the [k+m] size in not known at compiletime.
Suggesting to tempate this.

How would one normally handle this?

I've temporarily made it fixed size 1024*1024.
But I'm not sure if that is big enough

--WjW

Re: Compiling for FreeBSD, Clang refuses to compile a test

[Xinze Chi (信泽)]

I think "const int k = 12; const int m = 4" would pass the compile?

2015-12-05 20:56 GMT+08:00 Willem Jan Withagen <wjw@digiware.nl>:
> src/test/erasure-code/TestErasureCodeIsa.cc
>
> contains snippets, function definition like:
>
> buffer::ptr enc[k + m];
>   // create buffers with a copy of the original data to be able to compare
> it after decoding
>   {
>     for (int i = 0; i < (k + m); i++) {
>
> Clang refuses because the [k+m] size in not known at compiletime.
> Suggesting to tempate this.
>
> How would one normally handle this?
>
> I've temporarily made it fixed size 1024*1024.
> But I'm not sure if that is big enough
>
> --WjW
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Regards,
Xinze Chi

Re: Compiling for FreeBSD, Clang refuses to compile a test

[Willem Jan Withagen]

On 5-12-2015 14:02, Xinze Chi (信泽) wrote:
> I think "const int k = 12; const int m = 4" would pass the compile?


Are these sizes big enough??

--WjW

> 2015-12-05 20:56 GMT+08:00 Willem Jan Withagen <wjw@digiware.nl>:
>> src/test/erasure-code/TestErasureCodeIsa.cc
>>
>> contains snippets, function definition like:
>>
>> buffer::ptr enc[k + m];
>>    // create buffers with a copy of the original data to be able to compare
>> it after decoding
>>    {
>>      for (int i = 0; i < (k + m); i++) {
>>
>> Clang refuses because the [k+m] size in not known at compiletime.
>> Suggesting to tempate this.
>>
>> How would one normally handle this?
>>
>> I've temporarily made it fixed size 1024*1024.
>> But I'm not sure if that is big enough
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Compiling for FreeBSD, Clang refuses to compile a test

[Michal Jarzabek]

Hi Willem,

If you look at line 411 and 412 you will have variables k and m
defined. They are not changed anywhere(I think), so the sizes must be
big enough.
As Xinze mentioned just add const in front of it:
const int k = 12
const int m = 4
and it should fix the compile error.

buffer::ptr enc[k + m] works with gcc, because of the compiler
extension, but it's not standard
c++(https://gcc.gnu.org/onlinedocs/gcc/Variable-Length.html)

I will submit patch to  change it.

Thanks,
Michal

On Mon, Dec 7, 2015 at 9:44 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 5-12-2015 14:02, Xinze Chi (信泽) wrote:
>>
>> I think "const int k = 12; const int m = 4" would pass the compile?
>
>
>
> Are these sizes big enough??
>
> --WjW
>
>> 2015-12-05 20:56 GMT+08:00 Willem Jan Withagen <wjw@digiware.nl>:
>>>
>>> src/test/erasure-code/TestErasureCodeIsa.cc
>>>
>>> contains snippets, function definition like:
>>>
>>> buffer::ptr enc[k + m];
>>>    // create buffers with a copy of the original data to be able to
>>> compare
>>> it after decoding
>>>    {
>>>      for (int i = 0; i < (k + m); i++) {
>>>
>>> Clang refuses because the [k+m] size in not known at compiletime.
>>> Suggesting to tempate this.
>>>
>>> How would one normally handle this?
>>>
>>> I've temporarily made it fixed size 1024*1024.
>>> But I'm not sure if that is big enough
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Compiling for FreeBSD, Clang refuses to compile a test

[Willem Jan Withagen]

On 7-12-2015 23:19, Michal Jarzabek wrote:
> Hi Willem,
>
> If you look at line 411 and 412 you will have variables k and m
> defined. They are not changed anywhere(I think), so the sizes must be
> big enough.
> As Xinze mentioned just add const in front of it:
> const int k = 12
> const int m = 4
> and it should fix the compile error.
>
> buffer::ptr enc[k + m] works with gcc, because of the compiler
> extension, but it's not standard
> c++(https://gcc.gnu.org/onlinedocs/gcc/Variable-Length.html)
>
> I will submit patch to  change it.

That is exactly what I have done to get things compiling.
Have not yet gotten to the state that everything builds to start testing.

--WjW

Re: Compiling for FreeBSD, Clang refuses to compile a test

[Willem Jan Withagen]

On 8-12-2015 01:29, Willem Jan Withagen wrote:
> On 7-12-2015 23:19, Michal Jarzabek wrote:
>> Hi Willem,
>>
>> If you look at line 411 and 412 you will have variables k and m
>> defined. They are not changed anywhere(I think), so the sizes must
>> be big enough. As Xinze mentioned just add const in front of it:
>> const int k = 12 const int m = 4 and it should fix the compile
>> error.
>>
>> buffer::ptr enc[k + m] works with gcc, because of the compiler
>> extension, but it's not standard
>> c++(https://gcc.gnu.org/onlinedocs/gcc/Variable-Length.html)
>>
>> I will submit patch to  change it.
>
> That is exactly what I have done to get things compiling. Have not
> yet gotten to the state that everything builds to start testing.

Testing has started....

Not everything goes well, but it is getting there.
Had to disable ebd testing as that still does not get build.
But I think I saw a patch passing by that it only got build for Linux.

And the tests below tooks to run for atleast 7 hours on a
CPU: AMD Phenom(tm) II X6 1075T Processor (3013.83-MHz K8-class CPU)
So what I gather from that is that that is too long.

Some tests (like: unittest_erasure_code_shec_thread)  got killed because
they ran out of swap, others (unittest_on_exit) got a signal 6...
But then again that could be because the gtest ASSERT_DEATH stuff is
not suppoorted under FreeBSD (as by words of Google)

--WjW



PASS: unittest_erasure_code_plugin
PASS: unittest_erasure_code
PASS: unittest_erasure_code_jerasure
PASS: unittest_erasure_code_plugin_jerasure
PASS: unittest_erasure_code_isa
PASS: unittest_erasure_code_plugin_isa
PASS: unittest_erasure_code_lrc
PASS: unittest_erasure_code_plugin_lrc
PASS: unittest_erasure_code_shec
PASS: unittest_erasure_code_shec_all
PASS: unittest_erasure_code_shec_thread
Killed
FAIL: unittest_erasure_code_shec_arguments
PASS: unittest_erasure_code_plugin_shec
PASS: unittest_erasure_code_example
PASS: unittest_librados
PASS: unittest_librados_config
PASS: unittest_journal
PASS: unittest_rbd_replay
PASS: unittest_encoding
PASS: unittest_base64
PASS: unittest_run_cmd
PASS: unittest_simple_spin
PASS: unittest_libcephfs_config
PASS: unittest_mon_moncap
PASS: unittest_mon_pgmap
PASS: unittest_ecbackend
PASS: unittest_osdscrub
PASS: unittest_pglog
PASS: unittest_hitset
PASS: unittest_osd_osdcap
PASS: unittest_pageset
PASS: unittest_chain_xattr
PASS: unittest_lfnindex
PASS: unittest_mds_authcap
PASS: unittest_addrs
PASS: unittest_bloom_filter
PASS: unittest_histogram
PASS: unittest_prioritized_queue
PASS: unittest_str_map
PASS: unittest_sharedptr_registry
PASS: unittest_shared_cache
PASS: unittest_sloppy_crc_map
PASS: unittest_util
PASS: unittest_crush_wrapper
PASS: unittest_crush
PASS: unittest_osdmap
PASS: unittest_workqueue
PASS: unittest_striper
PASS: unittest_prebufferedstreambuf
PASS: unittest_str_list
PASS: unittest_log
PASS: unittest_throttle
PASS: unittest_ceph_argparse
PASS: unittest_ceph_compatset
PASS: unittest_mds_types
PASS: unittest_osd_types
PASS: unittest_lru
PASS: unittest_io_priority
PASS: unittest_gather
PASS: unittest_signals
PASS: unittest_bufferlist
PASS: unittest_xlist
PASS: unittest_crc32c
PASS: unittest_arch
PASS: unittest_crypto
PASS: unittest_crypto_init
PASS: unittest_perf_counters
PASS: unittest_admin_socket
PASS: unittest_ceph_crypto
PASS: unittest_utf8
PASS: unittest_mime
PASS: unittest_escape
PASS: unittest_strtol
PASS: unittest_confutils
PASS: unittest_config
PASS: unittest_context
PASS: unittest_safe_io
PASS: unittest_heartbeatmap
PASS: unittest_formatter
PASS: unittest_daemon_config
PASS: unittest_ipaddr
PASS: unittest_texttable
Abort trap (core dumped)
FAIL: unittest_on_exit
PASS: unittest_readahead
PASS: unittest_tableformatter
PASS: unittest_bit_vector
FAIL: ceph-detect-init/run-tox.sh
FAIL: test/erasure-code/test-erasure-code.sh
FAIL: test/erasure-code/test-erasure-eio.sh

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 4-12-2015 21:11, Willem Jan Withagen wrote:
> On 4-12-2015 19:44, Gregory Farnum wrote:
>> On Fri, Dec 4, 2015 at 10:30 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>> On 3-12-2015 01:27, Yan, Zheng wrote:
>>>> On Thu, Dec 3, 2015 at 4:52 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>> On 2-12-2015 15:13, Yan, Zheng wrote:
>>>
>>>>> I see that you have disabled uuid?
>>>>> Might I ask why?
>>>>
>>>> not disable. Currently ceph uses boost uuid implementation. so no need
>>>> to link to libuuid.
>>>
>>> And
>>>
>>>>>>>> The uuid transition to boost::uuid has happened since then (a few months
>>>>>>>> back) and I believe Rohan's AIX and Solaris ports for librados (that just
>>>>>>>> merged) included a fix for the sockaddr_storage issue:
>>>
>>> I cannot seem to find the package or port that defines boost::uuid.
>>> So how did you make it available to the build system?
>>
>> http://www.boost.org/doc/libs/1_59_0/libs/uuid/
>>
>> It's part of the boost labyrinth. I think in Debian it's just part of
>> libboost-dev, but you might need to dig around in whatever packaging
>> you're using for FreeBSD.
>
> I've dumped all of the labels in de boost libraries.
> So it is not default available with the pre-build packages.
> Which is understandable given the size of all that is available in
> boost. But lets go and fetch/build some of that stuff.

For the time being I've chosen to use the misc/e2fsprogs-libuuid package.
Reason for this is that is easily installed, instead of going thru the 
motions
of manually downloading/compiling a boost library.
Once boost-uuid is available reverting is easy.

--WjW

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 4-12-2015 21:11, Willem Jan Withagen wrote:

One larger problem with libraries I have is the stuff with dlopen.

In ./configure it seems that most of the code is short-circuited, and -ldl
gets appended in the Makefile.am's by default. The code there is rather 
hard
to parse.

FreeBSD has this in libc, so no specific includes needed.

Now the question is how do I cleanly fix this without breaking just
about every other platform?

Thanx,
--WjW

Re: Compiling for FreeBSD

[Mykola Golub]

On Tue, Dec 08, 2015 at 11:04:13AM +0100, Willem Jan Withagen wrote:
> On 4-12-2015 21:11, Willem Jan Withagen wrote:
> 
> One larger problem with libraries I have is the stuff with dlopen.
> 
> In ./configure it seems that most of the code is short-circuited, and -ldl
> gets appended in the Makefile.am's by default. The code there is rather hard
> to parse.
> 
> FreeBSD has this in libc, so no specific includes needed.
> 
> Now the question is how do I cleanly fix this without breaking just
> about every other platform?

Could you provide particular examples? Because what I see in the
master is usually like below:

if LINUX
xio_server_LDADD += -ldl
endif

If you see somewhere -ldl is added unconditionally it is likely to
have to be fixed the same way.

-- 
Mykola Golub

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 8-12-2015 13:36, Mykola Golub wrote:
> On Tue, Dec 08, 2015 at 11:04:13AM +0100, Willem Jan Withagen wrote:
>> On 4-12-2015 21:11, Willem Jan Withagen wrote:
>>
>> One larger problem with libraries I have is the stuff with dlopen.
>>
>> In ./configure it seems that most of the code is short-circuited, and -ldl
>> gets appended in the Makefile.am's by default. The code there is rather hard
>> to parse.
>>
>> FreeBSD has this in libc, so no specific includes needed.
>>
>> Now the question is how do I cleanly fix this without breaking just
>> about every other platform?
>
> Could you provide particular examples? Because what I see in the
> master is usually like below:
>
> if LINUX
> xio_server_LDADD += -ldl
> endif
>
> If you see somewhere -ldl is added unconditionally it is likely to
> have to be fixed the same way.

Several of the test makefiles did not have this.
For this I've patched at least:
src/tools/Makefile-server.am
src/tracing/Makefile.am
src/erasure-code/Makefile.am
src/rgw/Makefile.am

And this is indeed what I've added ATM to get my tests compiled.
But I really wonder if this is the way to go, instead of fixing it
in configure and be done with it. Just like wiht most of the other
libraries that are setup thru automake/configure. That would also
help other ports,  then for that port one does not have to augment
again all locations.

AND if new tests would be written, we do not have to check if all
exempts are correctly covered.

--WjW

Re: Compiling for FreeBSD, trouble in buffer.c

[Willem Jan Withagen]

I have a failure in:
  ./unittest_erasure_code_shec_arguments
All tests befor this PASS. (other than rbd which is disabled to
the time being)

Which I traceback to code in ErasureCodeShec.cc
Line 218:
	unsigned blocksize = (*chunks.begin()).second.length();
After a few iterations I get a "negative" blocksize, which causes
allocations further on to really thrash the system out of swap.

At first I expected it could be due to a Clang typecasting problem.
But after more debugging I found the following in
buffer.h
     unsigned length() const {
#if 0
       // DEBUG: verify _len
       unsigned len = 0;
       for (std::list<ptr>::const_iterator it = _buffers.begin();
            it != _buffers.end();
            it++) {
         len += (*it).length();
       }
       assert(len == _len);
#endif
       return _len;
     }

Which suggests that debugging was needed at this point earlier in life.
If I enable this debug block, I do get the assert affected.

Now the next question is why? Given the debug snippet it needed 
analyzing before.
And the derived question then is:
	What is the easiest path to find out what is actually wrong here.

All suggestions welcome.

--WjW

Re: Compiling for FreeBSD, trouble in buffer.c

[Willem Jan Withagen]

On 10-12-2015 16:03, Willem Jan Withagen wrote:
> I have a failure in:
>   ./unittest_erasure_code_shec_arguments
> All tests befor this PASS. (other than rbd which is disabled to
> the time being)
>
> Which I traceback to code in ErasureCodeShec.cc
> Line 218:
>      unsigned blocksize = (*chunks.begin()).second.length();
> After a few iterations I get a "negative" blocksize, which causes
> allocations further on to really thrash the system out of swap.
>
> At first I expected it could be due to a Clang typecasting problem.
> But after more debugging I found the following in
> buffer.h
>      unsigned length() const {
> #if 0
>        // DEBUG: verify _len
>        unsigned len = 0;
>        for (std::list<ptr>::const_iterator it = _buffers.begin();
>             it != _buffers.end();
>             it++) {
>          len += (*it).length();
>        }
>        assert(len == _len);
> #endif
>        return _len;
>      }
>
> Which suggests that debugging was needed at this point earlier in life.
> If I enable this debug block, I do get the assert affected.
>
> Now the next question is why? Given the debug snippet it needed
> analyzing before.
> And the derived question then is:
>      What is the easiest path to find out what is actually wrong here.


A further followup on this.

After some extensive debugging with gdb and watches, I've come to the 
conclusion
That the location of _len is used by more that one part of the code...
The location gets alternately written during:
TestErasureCodeShec_arguments.cc:136
     shec_table.insert(std::make_pair(table_key,table_value));

Old value = 63015016
New value = 4294954344
....
Old value = 4294954344
New value = 63015016
.....

To retain this value 4294954344, which is definitely not the length.
Because printing values on the Linux variant, it gives 32. Which sounds 
much more
sensible....

So there a few possibilities that I can think of:
  1) Clang gets it wrong
  2) There is a mixup of different type of libs that make for different 
offsets in
     the bufferlist structs
  3) the bufferlist code is has portability issues
  4) the bufferlist code has errors that do no show with gcc

Most likely it will be either 2) or 3) ....
But other suggestions are welcome...

And since bufferlists are at the center of Ceph, better get things right.
So I'm going to go over the test/bufferlist.cc code and see what is in 
there.
And/or extract a less convoluted example from 
TestErasureCodeShec_arguments.cc
and see if it is in there as well.

--WjW





	

Re: Compiling for FreeBSD, Bluestore requires AIO

[Willem Jan Withagen]

On 30-11-2015 14:21, Sage Weil wrote:
> The problem with all of the porting code in general is that it is doomed
> to break later on if we don't have (at least) ongoing build tests.  In
> order for a FreeBSD or OSX port to continue working we need VMs that run
> either gitbuilder or a jenkins job or similar so that we can tell when it
> breaks.
>
> If someone is willing to run a VM somewhere to do this we can pretty
> easily stick it on the gitbuilder page at
>
> 	http://ceph.com/gitbuilder.cgi

Well this is real nice case of such an incident.

I'm verifying my changes to the C/C++ code, after having forwarded to HEAD.
And now I get bluestore complaining that it requires AIO.
Something I've thusfar excluded in the build.

Does Bluestore really require AIO??
FreeBSD does have AIO, but to take simple steps one at the time, I 
excluded it
with configure. Mainly because it bombs at compile time. Probably due to
different include files, or function nameing.....

So, in order of preference:
- Can I disable AIO for Bluestore
- Can I disable bluestore being build?

Thanx,
--WjW

Re: Compiling for FreeBSD, Bluestore requires AIO

[Willem Jan Withagen]

On 15-1-2016 11:52, Willem Jan Withagen wrote:
> On 30-11-2015 14:21, Sage Weil wrote:
>> The problem with all of the porting code in general is that it is doomed
>> to break later on if we don't have (at least) ongoing build tests.  In
>> order for a FreeBSD or OSX port to continue working we need VMs that run
>> either gitbuilder or a jenkins job or similar so that we can tell when it
>> breaks.
>>
>> If someone is willing to run a VM somewhere to do this we can pretty
>> easily stick it on the gitbuilder page at
>>
>>     http://ceph.com/gitbuilder.cgi
>
> Well this is real nice case of such an incident.
>
> I'm verifying my changes to the C/C++ code, after having forwarded to HEAD.
> And now I get bluestore complaining that it requires AIO.
> Something I've thusfar excluded in the build.
>
> Does Bluestore really require AIO??
> FreeBSD does have AIO, but to take simple steps one at the time, I
> excluded it
> with configure. Mainly because it bombs at compile time. Probably due to
> different include files, or function nameing.....
>
> So, in order of preference:
> - Can I disable AIO for Bluestore
> - Can I disable bluestore being build?

Spoke a bit too soon.

FreeBSD has AIO, loadable as kernelmodule. with the following systemcalls.
aio_cancel(2), aio_error(2), aio_read(2), aio_return(2), aio_suspend(2),
      aio_waitcomplete(2), aio_write(2), lio_listio(2)

But libaio is based on:
/* Actual syscalls */
extern int io_setup(int maxevents, io_context_t *ctxp);
extern int io_destroy(io_context_t ctx);
extern int io_submit(io_context_t ctx, long nr, struct iocb *ios[]);
extern int io_cancel(io_context_t ctx, struct iocb *iocb, struct 
io_event *evt);
extern int io_getevents(io_context_t ctx_id, long min_nr, long nr, 
struct io_event *events, struct timespec *timeout);

So glueing that together is not just fixing some include files, and or 
fetching
some shim-library that is lurking out in /usr/ports...

And for the time being I would like to skip over the libaio hurdle.

Is that possible?

--WjW

Re: Compiling for FreeBSD, Bluestore requires AIO

[Sage Weil]

On Fri, 15 Jan 2016, Willem Jan Withagen wrote:
> On 30-11-2015 14:21, Sage Weil wrote:
> > The problem with all of the porting code in general is that it is doomed
> > to break later on if we don't have (at least) ongoing build tests.  In
> > order for a FreeBSD or OSX port to continue working we need VMs that run
> > either gitbuilder or a jenkins job or similar so that we can tell when it
> > breaks.
> > 
> > If someone is willing to run a VM somewhere to do this we can pretty
> > easily stick it on the gitbuilder page at
> > 
> > 	http://ceph.com/gitbuilder.cgi
> 
> Well this is real nice case of such an incident.
> 
> I'm verifying my changes to the C/C++ code, after having forwarded to HEAD.
> And now I get bluestore complaining that it requires AIO.
> Something I've thusfar excluded in the build.
> 
> Does Bluestore really require AIO??
> FreeBSD does have AIO, but to take simple steps one at the time, I excluded it
> with configure. Mainly because it bombs at compile time. Probably due to
> different include files, or function nameing.....
> 
> So, in order of preference:
> - Can I disable AIO for Bluestore
> - Can I disable bluestore being build?

AIO is currently required.  The good news is it is confined (mostly) to 
os/bluestore/BlockDevice.h, so providing an alternative (or non-aio) 
implementaiton shouldnt' be difficult.. in fact it is mostly there.

sage

Re: Compiling for FreeBSD, Bluestore requires AIO

[Willem Jan Withagen]

On 15-1-2016 18:30, Sage Weil wrote:
> On Fri, 15 Jan 2016, Willem Jan Withagen wrote:
>> On 30-11-2015 14:21, Sage Weil wrote:
>>> The problem with all of the porting code in general is that it is doomed
>>> to break later on if we don't have (at least) ongoing build tests.  In
>>> order for a FreeBSD or OSX port to continue working we need VMs that run
>>> either gitbuilder or a jenkins job or similar so that we can tell when it
>>> breaks.
>>>
>>> If someone is willing to run a VM somewhere to do this we can pretty
>>> easily stick it on the gitbuilder page at
>>>
>>> 	http://ceph.com/gitbuilder.cgi
>>
>> Well this is real nice case of such an incident.
>>
>> I'm verifying my changes to the C/C++ code, after having forwarded to HEAD.
>> And now I get bluestore complaining that it requires AIO.
>> Something I've thusfar excluded in the build.
>>
>> Does Bluestore really require AIO??
>> FreeBSD does have AIO, but to take simple steps one at the time, I excluded it
>> with configure. Mainly because it bombs at compile time. Probably due to
>> different include files, or function nameing.....
>>
>> So, in order of preference:
>> - Can I disable AIO for Bluestore
>> - Can I disable bluestore being build?
> 
> AIO is currently required.  The good news is it is confined (mostly) to 
> os/bluestore/BlockDevice.h, so providing an alternative (or non-aio) 
> implementaiton shouldnt' be difficult.. in fact it is mostly there.

Hi Sage,

ATM I've excluded the building of Bluestore, not really knowing how hard
that is going to bite me in run the tests.
But I guess we're going to find out the hard way. 8D

--WjW

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 1-12-2015 12:08, Willem Jan Withagen wrote:
> On 30-11-2015 14:21, Sage Weil wrote:
>> The problem with all of the porting code in general is that it is doomed
>> to break later on if we don't have (at least) ongoing build tests.  In
>> order for a FreeBSD or OSX port to continue working we need VMs that run
>> either gitbuilder or a jenkins job or similar so that we can tell when it
>> breaks.
>>
>> If someone is willing to run a VM somewhere to do this we can pretty
>> easily stick it on the gitbuilder page at
>>
>>     http://ceph.com/gitbuilder.cgi
> 

I have build a pull request that includes the changes to get the
automake tools do a decent job on the current master.

https://github.com/ceph/ceph/pull/7239

But then I tried building a second WIP with changes to the {cc,} code,
but then I end up with a WIP tree with a complete string of other
commits in between. Which is certainly not the intention.

https://github.com/wjwithagen/ceph/tree/wip-wjw-freebsd-compilechanges

The next one WIP would be wip-wjw-freebsd-testchanges which got me as
far as 110 tests successfully completed.

I see to be able to mess up the integration of the ceph/master into my
fork and my WIPs...

Is there a way to easily repair this? Or just
	save the diffs
	scrap the whole fork
	make a new fork and WIP
	push

Once I get this straight I promise to write this down as simple howto,
on then how to keep up in this fork and WIP with ceph/master

Thanx,
--WjW

Re: Compiling for FreeBSD, Bluestore requires AIO

[Mykola Golub]

On Fri, Jan 15, 2016 at 07:34:29PM +0100, Willem Jan Withagen wrote:

> ATM I've excluded the building of Bluestore, not really knowing how hard
> that is going to bite me in run the tests.

It looks like you have figured out how to exclude building Bluestore,
but just in case, I have the following PR open, that do the same:

https://github.com/ceph/ceph/pull/7169

-- 
Mykola Golub

Re: Compiling for FreeBSD, Bluestore requires AIO

[Willem Jan Withagen]

On 18-1-2016 10:54, Mykola Golub wrote:
> On Fri, Jan 15, 2016 at 07:34:29PM +0100, Willem Jan Withagen wrote:
> 
>> ATM I've excluded the building of Bluestore, not really knowing how hard
>> that is going to bite me in run the tests.
> 
> It looks like you have figured out how to exclude building Bluestore,
> but just in case, I have the following PR open, that do the same:
> 
> https://github.com/ceph/ceph/pull/7169
> 

I'd say: lgtm.... :)

The one thing to debate is to not make it depend on HAVE_LIBAIO, but
make it depend on a new configflag: WITH_BLUESTORE.

That even allows to include LIBAIO for the old stuff, and still run
without BlueStore.

And I patched configure.ac for that:
194,201d193
< # bluestore?
< AC_ARG_WITH([bluestore],
<       [AS_HELP_STRING([--with-bluestore], [build bluestore files])],
<       [],
<       [with_bluestore=yes])
< AM_CONDITIONAL(WITH_BLUESTORE, test "$with_bluestore" = "yes")
< #AS_IF([test "$with_bluestore" = "yes"],[AC_DEFINE([WITH_BLUESTORE])])
<


--WjW

Re: Compiling for FreeBSD

[Willem Jan Withagen]

On 30-11-2015 14:21, Sage Weil wrote:
> The problem with all of the porting code in general is that it is doomed 
> to break later on if we don't have (at least) ongoing build tests.  In 
> order for a FreeBSD or OSX port to continue working we need VMs that run 
> either gitbuilder or a jenkins job or similar so that we can tell when it 
> breaks.
> 
> If someone is willing to run a VM somewhere to do this we can pretty 
> easily stick it on the gitbuilder page at
> 
> 	http://ceph.com/gitbuilder.cgi

Hi Sage,

Just a bit of victory after 6 months.

I'm not yet there by far, but I've just passed the mark of only one test
not working (the osd-markdown.sh one) and all other tests that I think
that are passable have now passed.

And that all against a tree that is rather recent.

osd-markdown is a test that is rather hard to follow, analyse, debug...
Which starts with the fact that I cannot find any docs on the parameters
that are being manipulated.
Asked the author, but got no response.
So I'm very tempted to execlude that test for the time being.

And I'm hoping that you can merge the compiletime #9178 wip soon, and I
can begin with the next set of commit to merge the testing work.

Thanx,
--WjW