* [lustre-devel] Testing 2.14 on Debian + ZFS
@ 2021-02-12 23:07 Christian Kuntz
2021-02-18 11:09 ` Andreas Dilger
0 siblings, 1 reply; 9+ messages in thread
From: Christian Kuntz @ 2021-02-12 23:07 UTC (permalink / raw)
To: lustre-devel
[-- Attachment #1.1: Type: text/plain, Size: 1960 bytes --]
Hello,
I hope I'm communicating in the right place here, I'm currently working to
compile Lustre 2.14.0-RC2 on Debian 10.7 with ZFS 2.0.2 for OSDs. If
there's anything I can do to help with the testing effort or help Lustre's
Debian support be more robust, please let me know! I hope I'm not too late
in the release cycle to contribute.
On the topic of compiling, I'm currently working to get debian packages
made and running into a compilation issue that didn't seem to present on my
previous endeavors with 2.13 and 0.7. I've got the kernel and zfs locally
built and their packages made, and lustre's configure step successfully
completes, but when it comes to compiling it appears that the osd-zfs
portion has some broken include links:
In file included from /zfs-linux-2.0.2/include/sys/arc.h:32,
from /lustre-debian/lustre/osd-zfs/osd_internal.h:51,
from /lustre-debian/lustre/osd-zfs/osd_handler.c:52:
/zfs-linux-2.0.2/include/sys/zfs_context.h:45:10: fatal error: sys/types.h:
No such file or directory
#include <sys/types.h>
For reference, my configure and make looks like:
./configure --with-linux=/linux-4.19.171-2/ \
--with-linux-obj=linux-4.19.171-2/debian/build/build_amd64_none_amd64 \
--with-zfs=/zfs-linux-2.0.2 \
--enable-server --enable-modules --disable-ldiskfs
make -j"$(nproc)" debs
From the error and some probing, it looks like zfs_context.h expects some
header files to be in their libc standard `/usr/include/sys` (or somewhere
abouts), but lustre's make process is pulling them from the linux sources
in `SRC/include/linux/`. Predictably, I found this out the hard way by
adding `-I/usr/include` to the Makefile for the zfs osds and getting
battered with errors and warnings about duplicate definitions.
I'm more than happy to provide more detailed information on my setup, and
any and all suggestions/guidance is greatly appreciated!
Best regards,
Christian
[-- Attachment #1.2: Type: text/html, Size: 2179 bytes --]
[-- Attachment #2: Type: text/plain, Size: 165 bytes --]
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [lustre-devel] Testing 2.14 on Debian + ZFS 2021-02-12 23:07 [lustre-devel] Testing 2.14 on Debian + ZFS Christian Kuntz @ 2021-02-18 11:09 ` Andreas Dilger 2021-02-19 9:11 ` Christian Kuntz 0 siblings, 1 reply; 9+ messages in thread From: Andreas Dilger @ 2021-02-18 11:09 UTC (permalink / raw) To: Christian Kuntz; +Cc: Lustre-devel [-- Attachment #1.1: Type: text/plain, Size: 3223 bytes --] On Feb 12, 2021, at 16:07, Christian Kuntz <c.kuntz@opendrives.com<mailto:c.kuntz@opendrives.com>> wrote: Hello, I hope I'm communicating in the right place here, I'm currently working to compile Lustre 2.14.0-RC2 on Debian 10.7 with ZFS 2.0.2 for OSDs. If there's anything I can do to help with the testing effort or help Lustre's Debian support be more robust, please let me know! I hope I'm not too late in the release cycle to contribute. Thanks for the offer of testing. Unfortunately we are very late in the 2.14.0 release cycle (we are hoping to make the final release in the next week or so, barring any critical defects). However, even if any changes miss the final 2.14.0 release, having patches available to fix any issues you run into will make it easier for the next person to build on Debian. Ideally, any required fixes for Debian building should go into the Lustre tree directly rather than into a separate "downstream" package, so that "make debs" works out-of-the-box. We do build Ubuntu 18.04 and 20.04 clients regularly, and ZFS 2.0 servers on RHEL7/8, but I don't know if anyone is building the server code on Debian/Ubuntu on a regular basis. On the topic of compiling, I'm currently working to get debian packages made and running into a compilation issue that didn't seem to present on my previous endeavors with 2.13 and 0.7. I've got the kernel and zfs locally built and their packages made, and lustre's configure step successfully completes, but when it comes to compiling it appears that the osd-zfs portion has some broken include links: In file included from /zfs-linux-2.0.2/include/sys/arc.h:32, from /lustre-debian/lustre/osd-zfs/osd_internal.h:51, from /lustre-debian/lustre/osd-zfs/osd_handler.c:52: /zfs-linux-2.0.2/include/sys/zfs_context.h:45:10: fatal error: sys/types.h: No such file or directory #include <sys/types.h> For reference, my configure and make looks like: ./configure --with-linux=/linux-4.19.171-2/ \ --with-linux-obj=linux-4.19.171-2/debian/build/build_amd64_none_amd64 \ --with-zfs=/zfs-linux-2.0.2 \ --enable-server --enable-modules --disable-ldiskfs make -j"$(nproc)" debs From the error and some probing, it looks like zfs_context.h expects some header files to be in their libc standard `/usr/include/sys` (or somewhere abouts), but lustre's make process is pulling them from the linux sources in `SRC/include/linux/`. Predictably, I found this out the hard way by adding `-I/usr/include` to the Makefile for the zfs osds and getting battered with errors and warnings about duplicate definitions. It may be that you are looking at this from the wrong angle. Rather than trying to include more userspace headers, it might be worthwhile to see why <sys/arc.h> is being included by osd_internal.h and maybe there is a more appropriate header that could be used in the kernel? Failing that, it isn't clear wh zfs_context.h is including the <sys/type.h> header from userspace when building with __KERNEL__ set? Kernel code should be using <linux/types.h>. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud [-- Attachment #1.2: Type: text/html, Size: 6442 bytes --] [-- Attachment #2: Type: text/plain, Size: 165 bytes --] _______________________________________________ lustre-devel mailing list lustre-devel@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [lustre-devel] Testing 2.14 on Debian + ZFS 2021-02-18 11:09 ` Andreas Dilger @ 2021-02-19 9:11 ` Christian Kuntz 2021-02-19 14:17 ` Spitz, Cory James 2021-02-19 20:12 ` Andreas Dilger 0 siblings, 2 replies; 9+ messages in thread From: Christian Kuntz @ 2021-02-19 9:11 UTC (permalink / raw) To: Andreas Dilger; +Cc: Lustre-devel [-- Attachment #1.1: Type: text/plain, Size: 4009 bytes --] Thanks for your time and the thoughtful response. Looking at this a bit more, it seems to be caused by the recent ZFS 2.0 changes for linux and freebsd support, namely the `include/os/linux` paths that provide `<sys/types.h>` that ZFS is trying to use. Adding them into the includes for the osd-zfs directory has fixed the references being broken, and now I'm contending some redefinitions and argument type mismatches. I'll try and sort this out, in the meantime is there a better place for me to place messages about this? I know this mailing list isn't used very frequently and I'd hate to flood it with such off-the-beaten-path stuff. Best, Christian On Thu, Feb 18, 2021 at 3:09 AM Andreas Dilger <adilger@whamcloud.com> wrote: > On Feb 12, 2021, at 16:07, Christian Kuntz <c.kuntz@opendrives.com> wrote: > > > Hello, > > I hope I'm communicating in the right place here, I'm currently working to > compile Lustre 2.14.0-RC2 on Debian 10.7 with ZFS 2.0.2 for OSDs. If > there's anything I can do to help with the testing effort or help Lustre's > Debian support be more robust, please let me know! I hope I'm not too late > in the release cycle to contribute. > > > Thanks for the offer of testing. Unfortunately we are very late in the > 2.14.0 release cycle (we are hoping to make the final release in the next > week or so, barring any critical defects). However, even if any changes > miss the final 2.14.0 release, having patches available to fix any issues > you run into will make it easier for the next person to build on Debian. > > Ideally, any required fixes for Debian building should go into the Lustre > tree directly rather than into a separate "downstream" package, so that > "make debs" works out-of-the-box. We do build Ubuntu 18.04 and 20.04 > clients regularly, and ZFS 2.0 servers on RHEL7/8, but I don't know if > anyone is building the server code on Debian/Ubuntu on a regular basis. > > On the topic of compiling, I'm currently working to get debian packages > made and running into a compilation issue that didn't seem to present on my > previous endeavors with 2.13 and 0.7. I've got the kernel and zfs locally > built and their packages made, and lustre's configure step successfully > completes, but when it comes to compiling it appears that the osd-zfs > portion has some broken include links: > > In file included from /zfs-linux-2.0.2/include/sys/arc.h:32, > from /lustre-debian/lustre/osd-zfs/osd_internal.h:51, > from /lustre-debian/lustre/osd-zfs/osd_handler.c:52: > /zfs-linux-2.0.2/include/sys/zfs_context.h:45:10: fatal error: > sys/types.h: No such file or directory > #include <sys/types.h> > > For reference, my configure and make looks like: > ./configure --with-linux=/linux-4.19.171-2/ \ > > --with-linux-obj=linux-4.19.171-2/debian/build/build_amd64_none_amd64 \ > --with-zfs=/zfs-linux-2.0.2 \ > --enable-server --enable-modules --disable-ldiskfs > make -j"$(nproc)" debs > > > From the error and some probing, it looks like zfs_context.h expects some > header files to be in their libc standard `/usr/include/sys` (or somewhere > abouts), but lustre's make process is pulling them from the linux sources > in `SRC/include/linux/`. Predictably, I found this out the hard way by > adding `-I/usr/include` to the Makefile for the zfs osds and getting > battered with errors and warnings about duplicate definitions. > > > It may be that you are looking at this from the wrong angle. Rather than > trying to include more userspace headers, it might be worthwhile to see why > <sys/arc.h> is being included by osd_internal.h and maybe there is a more > appropriate header that could be used in the kernel? Failing that, it > isn't clear wh zfs_context.h is including the <sys/type.h> header from > userspace when building with __KERNEL__ set? Kernel code should be using > <linux/types.h>. > > Cheers, Andreas > -- > Andreas Dilger > Principal Lustre Architect > Whamcloud > > > > > > > [-- Attachment #1.2: Type: text/html, Size: 5911 bytes --] [-- Attachment #2: Type: text/plain, Size: 165 bytes --] _______________________________________________ lustre-devel mailing list lustre-devel@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [lustre-devel] Testing 2.14 on Debian + ZFS 2021-02-19 9:11 ` Christian Kuntz @ 2021-02-19 14:17 ` Spitz, Cory James 2021-02-19 20:12 ` Andreas Dilger 1 sibling, 0 replies; 9+ messages in thread From: Spitz, Cory James @ 2021-02-19 14:17 UTC (permalink / raw) To: Christian Kuntz, Andreas Dilger; +Cc: Lustre-devel [-- Attachment #1.1: Type: text/plain, Size: 4321 bytes --] Hello! > I know this mailing list isn't used very frequently and I'd hate to flood it with such off-the-beaten-path stuff. No worries, Christian. Any related devel topics are welcome here. You are in the right place! -Cory On 2/19/21, 4:11 AM, "lustre-devel" <lustre-devel-bounces@lists.lustre.org> wrote: Thanks for your time and the thoughtful response. Looking at this a bit more, it seems to be caused by the recent ZFS 2.0 changes for linux and freebsd support, namely the `include/os/linux` paths that provide `<sys/types.h>` that ZFS is trying to use. Adding them into the includes for the osd-zfs directory has fixed the references being broken, and now I'm contending some redefinitions and argument type mismatches. I'll try and sort this out, in the meantime is there a better place for me to place messages about this? I know this mailing list isn't used very frequently and I'd hate to flood it with such off-the-beaten-path stuff. Best, Christian On Thu, Feb 18, 2021 at 3:09 AM Andreas Dilger <adilger@whamcloud.com<mailto:adilger@whamcloud.com>> wrote: On Feb 12, 2021, at 16:07, Christian Kuntz <c.kuntz@opendrives.com<mailto:c.kuntz@opendrives.com>> wrote: Hello, I hope I'm communicating in the right place here, I'm currently working to compile Lustre 2.14.0-RC2 on Debian 10.7 with ZFS 2.0.2 for OSDs. If there's anything I can do to help with the testing effort or help Lustre's Debian support be more robust, please let me know! I hope I'm not too late in the release cycle to contribute. Thanks for the offer of testing. Unfortunately we are very late in the 2.14.0 release cycle (we are hoping to make the final release in the next week or so, barring any critical defects). However, even if any changes miss the final 2.14.0 release, having patches available to fix any issues you run into will make it easier for the next person to build on Debian. Ideally, any required fixes for Debian building should go into the Lustre tree directly rather than into a separate "downstream" package, so that "make debs" works out-of-the-box. We do build Ubuntu 18.04 and 20.04 clients regularly, and ZFS 2.0 servers on RHEL7/8, but I don't know if anyone is building the server code on Debian/Ubuntu on a regular basis. On the topic of compiling, I'm currently working to get debian packages made and running into a compilation issue that didn't seem to present on my previous endeavors with 2.13 and 0.7. I've got the kernel and zfs locally built and their packages made, and lustre's configure step successfully completes, but when it comes to compiling it appears that the osd-zfs portion has some broken include links: In file included from /zfs-linux-2.0.2/include/sys/arc.h:32, from /lustre-debian/lustre/osd-zfs/osd_internal.h:51, from /lustre-debian/lustre/osd-zfs/osd_handler.c:52: /zfs-linux-2.0.2/include/sys/zfs_context.h:45:10: fatal error: sys/types.h: No such file or directory #include <sys/types.h> For reference, my configure and make looks like: ./configure --with-linux=/linux-4.19.171-2/ \ --with-linux-obj=linux-4.19.171-2/debian/build/build_amd64_none_amd64 \ --with-zfs=/zfs-linux-2.0.2 \ --enable-server --enable-modules --disable-ldiskfs make -j"$(nproc)" debs From the error and some probing, it looks like zfs_context.h expects some header files to be in their libc standard `/usr/include/sys` (or somewhere abouts), but lustre's make process is pulling them from the linux sources in `SRC/include/linux/`. Predictably, I found this out the hard way by adding `-I/usr/include` to the Makefile for the zfs osds and getting battered with errors and warnings about duplicate definitions. It may be that you are looking at this from the wrong angle. Rather than trying to include more userspace headers, it might be worthwhile to see why <sys/arc.h> is being included by osd_internal.h and maybe there is a more appropriate header that could be used in the kernel? Failing that, it isn't clear wh zfs_context.h is including the <sys/type.h> header from userspace when building with __KERNEL__ set? Kernel code should be using <linux/types.h>. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud [-- Attachment #1.2: Type: text/html, Size: 9179 bytes --] [-- Attachment #2: Type: text/plain, Size: 165 bytes --] _______________________________________________ lustre-devel mailing list lustre-devel@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [lustre-devel] Testing 2.14 on Debian + ZFS 2021-02-19 9:11 ` Christian Kuntz 2021-02-19 14:17 ` Spitz, Cory James @ 2021-02-19 20:12 ` Andreas Dilger 2021-02-20 3:00 ` Christian Kuntz 1 sibling, 1 reply; 9+ messages in thread From: Andreas Dilger @ 2021-02-19 20:12 UTC (permalink / raw) To: Christian Kuntz; +Cc: Lustre-devel [-- Attachment #1.1: Type: text/plain, Size: 4280 bytes --] This is exactly the right place for such discussions. Once you have patches, please submit to Gerrit for review, testing, and landing: https://wiki.lustre.org/Using_Gerrit Cheers, Andreas On Feb 19, 2021, at 02:11, Christian Kuntz <c.kuntz@opendrives.com> wrote: Thanks for your time and the thoughtful response. Looking at this a bit more, it seems to be caused by the recent ZFS 2.0 changes for linux and freebsd support, namely the `include/os/linux` paths that provide `<sys/types.h>` that ZFS is trying to use. Adding them into the includes for the osd-zfs directory has fixed the references being broken, and now I'm contending some redefinitions and argument type mismatches. I'll try and sort this out, in the meantime is there a better place for me to place messages about this? I know this mailing list isn't used very frequently and I'd hate to flood it with such off-the-beaten-path stuff. Best, Christian On Thu, Feb 18, 2021 at 3:09 AM Andreas Dilger <adilger@whamcloud.com<mailto:adilger@whamcloud.com>> wrote: On Feb 12, 2021, at 16:07, Christian Kuntz <c.kuntz@opendrives.com<mailto:c.kuntz@opendrives.com>> wrote: Hello, I hope I'm communicating in the right place here, I'm currently working to compile Lustre 2.14.0-RC2 on Debian 10.7 with ZFS 2.0.2 for OSDs. If there's anything I can do to help with the testing effort or help Lustre's Debian support be more robust, please let me know! I hope I'm not too late in the release cycle to contribute. Thanks for the offer of testing. Unfortunately we are very late in the 2.14.0 release cycle (we are hoping to make the final release in the next week or so, barring any critical defects). However, even if any changes miss the final 2.14.0 release, having patches available to fix any issues you run into will make it easier for the next person to build on Debian. Ideally, any required fixes for Debian building should go into the Lustre tree directly rather than into a separate "downstream" package, so that "make debs" works out-of-the-box. We do build Ubuntu 18.04 and 20.04 clients regularly, and ZFS 2.0 servers on RHEL7/8, but I don't know if anyone is building the server code on Debian/Ubuntu on a regular basis. On the topic of compiling, I'm currently working to get debian packages made and running into a compilation issue that didn't seem to present on my previous endeavors with 2.13 and 0.7. I've got the kernel and zfs locally built and their packages made, and lustre's configure step successfully completes, but when it comes to compiling it appears that the osd-zfs portion has some broken include links: In file included from /zfs-linux-2.0.2/include/sys/arc.h:32, from /lustre-debian/lustre/osd-zfs/osd_internal.h:51, from /lustre-debian/lustre/osd-zfs/osd_handler.c:52: /zfs-linux-2.0.2/include/sys/zfs_context.h:45:10: fatal error: sys/types.h: No such file or directory #include <sys/types.h> For reference, my configure and make looks like: ./configure --with-linux=/linux-4.19.171-2/ \ --with-linux-obj=linux-4.19.171-2/debian/build/build_amd64_none_amd64 \ --with-zfs=/zfs-linux-2.0.2 \ --enable-server --enable-modules --disable-ldiskfs make -j"$(nproc)" debs From the error and some probing, it looks like zfs_context.h expects some header files to be in their libc standard `/usr/include/sys` (or somewhere abouts), but lustre's make process is pulling them from the linux sources in `SRC/include/linux/`. Predictably, I found this out the hard way by adding `-I/usr/include` to the Makefile for the zfs osds and getting battered with errors and warnings about duplicate definitions. It may be that you are looking at this from the wrong angle. Rather than trying to include more userspace headers, it might be worthwhile to see why <sys/arc.h> is being included by osd_internal.h and maybe there is a more appropriate header that could be used in the kernel? Failing that, it isn't clear wh zfs_context.h is including the <sys/type.h> header from userspace when building with __KERNEL__ set? Kernel code should be using <linux/types.h>. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud [-- Attachment #1.2: Type: text/html, Size: 6800 bytes --] [-- Attachment #2: Type: text/plain, Size: 165 bytes --] _______________________________________________ lustre-devel mailing list lustre-devel@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [lustre-devel] Testing 2.14 on Debian + ZFS 2021-02-19 20:12 ` Andreas Dilger @ 2021-02-20 3:00 ` Christian Kuntz 2021-02-22 19:58 ` Faaland, Olaf P. via lustre-devel 0 siblings, 1 reply; 9+ messages in thread From: Christian Kuntz @ 2021-02-20 3:00 UTC (permalink / raw) To: Andreas Dilger; +Cc: Lustre-devel [-- Attachment #1.1: Type: text/plain, Size: 5683 bytes --] Thanks for the confirmation! After going in a few circles, I think I've got it figured out and would appreciate some pointers on how to proceed: To preface, the way I'm building is from source-dirs after a package build (which is admittedly kind of janky as I write it out now). It would appear that the configure scripting will look in "zfs/include", and thus miss the new "zfs/include/os/..." directories that hold some important header files, causing configuration variables to miss-set in config.h thus creating the compile-time problems of type mismatches and redefinitions I was seeing. I was able to confirm manually setting some of the configuration defs and adding the new header dirs to the makefile fixed the problem, but I'm a little confused as to how to proceed in patching. Since the configuration scripting and the makefiles seem to expect only a single include-dir of header files for both zfs and spl respectively, I'm wondering what the best approach to patching this behavior would be. Thanks for your time and consideration, Christian On Fri, Feb 19, 2021 at 12:13 PM Andreas Dilger <adilger@whamcloud.com> wrote: > This is exactly the right place for such discussions. > > Once you have patches, please submit to Gerrit for review, testing, and > landing: > > https://wiki.lustre.org/Using_Gerrit > > Cheers, Andreas > > On Feb 19, 2021, at 02:11, Christian Kuntz <c.kuntz@opendrives.com> wrote: > > > Thanks for your time and the thoughtful response. > > Looking at this a bit more, it seems to be caused by the recent ZFS 2.0 > changes for linux and freebsd support, namely the `include/os/linux` paths > that provide `<sys/types.h>` that ZFS is trying to use. Adding them into > the includes for the osd-zfs directory has fixed the references being > broken, and now I'm contending some redefinitions and argument type > mismatches. I'll try and sort this out, in the meantime is there a better > place for me to place messages about this? I know this mailing list isn't > used very frequently and I'd hate to flood it with such off-the-beaten-path > stuff. > > Best, > Christian > > On Thu, Feb 18, 2021 at 3:09 AM Andreas Dilger <adilger@whamcloud.com> > wrote: > >> On Feb 12, 2021, at 16:07, Christian Kuntz <c.kuntz@opendrives.com> >> wrote: >> >> >> Hello, >> >> I hope I'm communicating in the right place here, I'm currently working >> to compile Lustre 2.14.0-RC2 on Debian 10.7 with ZFS 2.0.2 for OSDs. If >> there's anything I can do to help with the testing effort or help Lustre's >> Debian support be more robust, please let me know! I hope I'm not too late >> in the release cycle to contribute. >> >> >> Thanks for the offer of testing. Unfortunately we are very late in the >> 2.14.0 release cycle (we are hoping to make the final release in the next >> week or so, barring any critical defects). However, even if any changes >> miss the final 2.14.0 release, having patches available to fix any issues >> you run into will make it easier for the next person to build on Debian. >> >> Ideally, any required fixes for Debian building should go into the Lustre >> tree directly rather than into a separate "downstream" package, so that >> "make debs" works out-of-the-box. We do build Ubuntu 18.04 and 20.04 >> clients regularly, and ZFS 2.0 servers on RHEL7/8, but I don't know if >> anyone is building the server code on Debian/Ubuntu on a regular basis. >> >> On the topic of compiling, I'm currently working to get debian packages >> made and running into a compilation issue that didn't seem to present on my >> previous endeavors with 2.13 and 0.7. I've got the kernel and zfs locally >> built and their packages made, and lustre's configure step successfully >> completes, but when it comes to compiling it appears that the osd-zfs >> portion has some broken include links: >> >> In file included from /zfs-linux-2.0.2/include/sys/arc.h:32, >> from /lustre-debian/lustre/osd-zfs/osd_internal.h:51, >> from /lustre-debian/lustre/osd-zfs/osd_handler.c:52: >> /zfs-linux-2.0.2/include/sys/zfs_context.h:45:10: fatal error: >> sys/types.h: No such file or directory >> #include <sys/types.h> >> >> For reference, my configure and make looks like: >> ./configure --with-linux=/linux-4.19.171-2/ \ >> >> --with-linux-obj=linux-4.19.171-2/debian/build/build_amd64_none_amd64 \ >> --with-zfs=/zfs-linux-2.0.2 \ >> --enable-server --enable-modules --disable-ldiskfs >> make -j"$(nproc)" debs >> >> >> From the error and some probing, it looks like zfs_context.h expects some >> header files to be in their libc standard `/usr/include/sys` (or somewhere >> abouts), but lustre's make process is pulling them from the linux sources >> in `SRC/include/linux/`. Predictably, I found this out the hard way by >> adding `-I/usr/include` to the Makefile for the zfs osds and getting >> battered with errors and warnings about duplicate definitions. >> >> >> It may be that you are looking at this from the wrong angle. Rather than >> trying to include more userspace headers, it might be worthwhile to see why >> <sys/arc.h> is being included by osd_internal.h and maybe there is a more >> appropriate header that could be used in the kernel? Failing that, it >> isn't clear wh zfs_context.h is including the <sys/type.h> header from >> userspace when building with __KERNEL__ set? Kernel code should be using >> <linux/types.h>. >> >> Cheers, Andreas >> -- >> Andreas Dilger >> Principal Lustre Architect >> Whamcloud >> >> >> >> >> >> >> [-- Attachment #1.2: Type: text/html, Size: 8119 bytes --] [-- Attachment #2: Type: text/plain, Size: 165 bytes --] _______________________________________________ lustre-devel mailing list lustre-devel@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [lustre-devel] Testing 2.14 on Debian + ZFS 2021-02-20 3:00 ` Christian Kuntz @ 2021-02-22 19:58 ` Faaland, Olaf P. via lustre-devel 2021-02-23 3:41 ` Christian Kuntz via lustre-devel 0 siblings, 1 reply; 9+ messages in thread From: Faaland, Olaf P. via lustre-devel @ 2021-02-22 19:58 UTC (permalink / raw) To: Christian Kuntz, Andreas Dilger; +Cc: Lustre-devel Hi Christian, Can you explain in more detail what you mean by "building ... from source-dirs after a package build"? And can you identify the lines in config/lustre-build-zfs.m4 that you're referring to? thank you, -Olaf ________________________________________ From: Christian Kuntz <c.kuntz@opendrives.com> Sent: Friday, February 19, 2021 7:00 PM To: Andreas Dilger Cc: Lustre-devel; Behlendorf, Brian D.; Faaland, Olaf P. Subject: Re: [lustre-devel] Testing 2.14 on Debian + ZFS Thanks for the confirmation! After going in a few circles, I think I've got it figured out and would appreciate some pointers on how to proceed: To preface, the way I'm building is from source-dirs after a package build (which is admittedly kind of janky as I write it out now). It would appear that the configure scripting will look in "zfs/include", and thus miss the new "zfs/include/os/..." directories that hold some important header files, causing configuration variables to miss-set in config.h thus creating the compile-time problems of type mismatches and redefinitions I was seeing. I was able to confirm manually setting some of the configuration defs and adding the new header dirs to the makefile fixed the problem, but I'm a little confused as to how to proceed in patching. Since the configuration scripting and the makefiles seem to expect only a single include-dir of header files for both zfs and spl respectively, I'm wondering what the best approach to patching this behavior would be. Thanks for your time and consideration, Christian On Fri, Feb 19, 2021 at 12:13 PM Andreas Dilger <adilger@whamcloud.com<mailto:adilger@whamcloud.com>> wrote: This is exactly the right place for such discussions. Once you have patches, please submit to Gerrit for review, testing, and landing: https://wiki.lustre.org/Using_Gerrit<https://urldefense.us/v2/url?u=https-3A__wiki.lustre.org_Using-5FGerrit&d=DwMFaQ&c=pKoAVQro6qDbLoK0T8588B4mZJhJhC4e6QXJy0XnJec&r=2dxT0LLEUx0IbhFF8HRJVCTO_ihbEwKLn9gF7Kc8aG0&m=SU5sac04cekvaRvbUcheoU02W7EUu-F4sVkkrWHUnoE&s=gTcuGMUi5PycREbMB4La1s0dVNM8a5STb9pDVXZt-Lw&e=> Cheers, Andreas On Feb 19, 2021, at 02:11, Christian Kuntz <c.kuntz@opendrives.com<mailto:c.kuntz@opendrives.com>> wrote: Thanks for your time and the thoughtful response. Looking at this a bit more, it seems to be caused by the recent ZFS 2.0 changes for linux and freebsd support, namely the `include/os/linux` paths that provide `<sys/types.h>` that ZFS is trying to use. Adding them into the includes for the osd-zfs directory has fixed the references being broken, and now I'm contending some redefinitions and argument type mismatches. I'll try and sort this out, in the meantime is there a better place for me to place messages about this? I know this mailing list isn't used very frequently and I'd hate to flood it with such off-the-beaten-path stuff. Best, Christian On Thu, Feb 18, 2021 at 3:09 AM Andreas Dilger <adilger@whamcloud.com<mailto:adilger@whamcloud.com>> wrote: On Feb 12, 2021, at 16:07, Christian Kuntz <c.kuntz@opendrives.com<mailto:c.kuntz@opendrives.com>> wrote: Hello, I hope I'm communicating in the right place here, I'm currently working to compile Lustre 2.14.0-RC2 on Debian 10.7 with ZFS 2.0.2 for OSDs. If there's anything I can do to help with the testing effort or help Lustre's Debian support be more robust, please let me know! I hope I'm not too late in the release cycle to contribute. Thanks for the offer of testing. Unfortunately we are very late in the 2.14.0 release cycle (we are hoping to make the final release in the next week or so, barring any critical defects). However, even if any changes miss the final 2.14.0 release, having patches available to fix any issues you run into will make it easier for the next person to build on Debian. Ideally, any required fixes for Debian building should go into the Lustre tree directly rather than into a separate "downstream" package, so that "make debs" works out-of-the-box. We do build Ubuntu 18.04 and 20.04 clients regularly, and ZFS 2.0 servers on RHEL7/8, but I don't know if anyone is building the server code on Debian/Ubuntu on a regular basis. On the topic of compiling, I'm currently working to get debian packages made and running into a compilation issue that didn't seem to present on my previous endeavors with 2.13 and 0.7. I've got the kernel and zfs locally built and their packages made, and lustre's configure step successfully completes, but when it comes to compiling it appears that the osd-zfs portion has some broken include links: In file included from /zfs-linux-2.0.2/include/sys/arc.h:32, from /lustre-debian/lustre/osd-zfs/osd_internal.h:51, from /lustre-debian/lustre/osd-zfs/osd_handler.c:52: /zfs-linux-2.0.2/include/sys/zfs_context.h:45:10: fatal error: sys/types.h: No such file or directory #include <sys/types.h> For reference, my configure and make looks like: ./configure --with-linux=/linux-4.19.171-2/ \ --with-linux-obj=linux-4.19.171-2/debian/build/build_amd64_none_amd64 \ --with-zfs=/zfs-linux-2.0.2 \ --enable-server --enable-modules --disable-ldiskfs make -j"$(nproc)" debs From the error and some probing, it looks like zfs_context.h expects some header files to be in their libc standard `/usr/include/sys` (or somewhere abouts), but lustre's make process is pulling them from the linux sources in `SRC/include/linux/`. Predictably, I found this out the hard way by adding `-I/usr/include` to the Makefile for the zfs osds and getting battered with errors and warnings about duplicate definitions. It may be that you are looking at this from the wrong angle. Rather than trying to include more userspace headers, it might be worthwhile to see why <sys/arc.h> is being included by osd_internal.h and maybe there is a more appropriate header that could be used in the kernel? Failing that, it isn't clear wh zfs_context.h is including the <sys/type.h> header from userspace when building with __KERNEL__ set? Kernel code should be using <linux/types.h>. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud _______________________________________________ lustre-devel mailing list lustre-devel@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [lustre-devel] Testing 2.14 on Debian + ZFS 2021-02-22 19:58 ` Faaland, Olaf P. via lustre-devel @ 2021-02-23 3:41 ` Christian Kuntz via lustre-devel 2021-08-23 1:59 ` Christian Kuntz 0 siblings, 1 reply; 9+ messages in thread From: Christian Kuntz via lustre-devel @ 2021-02-23 3:41 UTC (permalink / raw) To: Faaland, Olaf P.; +Cc: Lustre-devel [-- Attachment #1.1: Type: text/plain, Size: 10909 bytes --] Hi Olaf, Sure! For reference this is how I build zfs: ./autogen.sh fakeroot dpkg-buildpackage -us -uc -b -j$cpus # Build the kernel modules, not just dkms KVERS=$KERNEL_ABI KSRC=$WITHLINUX_HDR KOBJ=$WITHLINUXOBJ fakeroot debian/rules override_dh_binary-modules Then, when I build lustre, I do like: sh autogen.sh ./configure --with-linux=$WITHLINUX_HDR --with-linux-obj=$WITHLINUXOBJ --with-zfs=$ZFS_SRC --enable-server --enable-modules --disable-ldiskfs make debs My zfs source is given as the root zfs source directory, not the contents of explicit package dirs in ZFS_SRC/debian/ which is technically where all the build artifacts live (for instance, the development headers that would end up on the machine would be in debian/libzfslinux-dev/usr/include) after a package build. After fiddling around with autoconf a bit (and learning it's just shell!), I landed on changing this bit of the autoconf: config/lustre-build-zfs.m4 AC_DEFUN([LB_ZFS_USER], [ dnl # dnl # Detect user space zfs development headers. dnl # AC_MSG_CHECKING([zfs devel headers]) AS_IF([test -z "${zfsinc}"], [ AS_IF([test -d "${zfssrc}/include/os/linux"], [ zfsinc="-I${zfssrc}/include/os/linux/kernel -I${zfssrc}/include/os/linux/spl -I${zfssrc}/include/os/linux/zfs -I${zfssrc}/include" zfslib="-L$zfssrc/lib/libzfs/.libs/ -L$zfssrc/lib/libnvpair/.libs" ], [test -e "${zfssrc}/include/libzfs.h" && test -e "${zfssrc}/lib/libspl/include"], [ zfsinc="-I $zfssrc/lib/libspl/include -I $zfssrc/include" zfslib="-L$zfssrc/lib/libzfs/.libs/ -L$zfssrc/lib/libnvpair/.libs" ], [test -d /usr/include/libzfs && test -d /usr/include/libspl], [ zfsinc="-I/usr/include/libspl -I /usr/include/libzfs" zfslib="" ], [ zfsinc="[Not Found]" zfslib="" enable_zfs=no ]) ]) AC_MSG_RESULT([$zfsinc]) I added a check around the new include/os style of the zfs header directories, which seems to be working OK, every config test is behaving as expected on my machine except for the multihost support one, which I'm chasing down a bit (sys/spa.h requires sys/sysmacros.h which requires... and so on). I also found that I had to modify the contents of config/lustre-build-linux.m4 to reflect the include dir change: AC_DEFUN([LB_LINUX_COMPILE_IFELSE], [m4_ifvaln([$1], [AC_LANG_CONFTEST([AC_LANG_SOURCE([$1])])])dnl rm -f build/conftest.o build/conftest.mod.c build/conftest.ko SUBARCH=$(echo $target_cpu | sed -e 's/powerpc.*/powerpc/' -e 's/ppc.*/powerpc/' -e 's/x86_64/x86/' -e 's/i.86/x86/' -e 's/k1om/x86/' -e 's/aarch64.*/arm64/' -e 's/armv7.*/arm/') AS_IF([AC_TRY_COMMAND(cp conftest.c build && make -d [$2] LDFLAGS= ${LD:+LD="$LD"} CC="$CC" -f $PWD/build/Makefile LUSTRE_LINUX_CONFIG=$LINUX_CONFIG LINUXINCLUDE="$EXTRA_CHECK_INCLUDE -I$LINUX/arch/$SUBARCH/include -Iinclude -Iarch/$SUBARCH/include/generated -I$LINUX/include -Iinclude2 -I$LINUX/include/uapi -Iinclude/generated -I$LINUX/arch/$SUBARCH/include/uapi -Iarch/$SUBARCH/include/generated/uapi -I$LINUX/include/uapi -Iinclude/generated/uapi ${SPL_OBJ:+-include $SPL_OBJ/spl_config.h} ${ZFS_OBJ:+-include $ZFS_OBJ/zfs_config.h} ${SPL:+-I$ZFS/lib/libspl/include/ } ${ZFS:+-I$ZFS/include/os/linux/ -I$ZFS/include/os/linux/kernel -I$ZFS/include/os/linux/spl -I$ZFS/include/os/linux/zfs -I$ZFS/include } -include $CONFIG_INCLUDE" KBUILD_EXTRA_SYMBOLS="${ZFS_OBJ:+$ZFS_OBJ/Module.symvers} $KBUILD_EXTRA_SYMBOLS" -o tmp_include_depends -o scripts -o include/config/MARKER -C $LINUX_OBJ EXTRA_CFLAGS="-Werror-implicit-function-declaration $EXTRA_KCFLAGS" $MODULE_TARGET=$PWD/build) >/dev/null && AC_TRY_COMMAND([$3])], I *think* I'll need to adjust the SPL variable in there, but not sure thus far. With the above settings I've gotten everything to succesfully compile and package up aside from a small modification I had to make to lustre/utils/libmount_utils_zfs.c to prevent a name collision with spa_get_hostid. I believe this occurs because the configuration doesn't set HAVE_ZFS_MULTIHOST appropriately. As you can see, I worked on it a little on the weekend :) Cheers, Christian On Mon, Feb 22, 2021 at 11:58 AM Faaland, Olaf P. <faaland1@llnl.gov> wrote: > Hi Christian, > > Can you explain in more detail what you mean by "building ... from > source-dirs after a package build"? And can you identify the lines in > config/lustre-build-zfs.m4 that you're referring to? > > thank you, > -Olaf > > ________________________________________ > From: Christian Kuntz <c.kuntz@opendrives.com> > Sent: Friday, February 19, 2021 7:00 PM > To: Andreas Dilger > Cc: Lustre-devel; Behlendorf, Brian D.; Faaland, Olaf P. > Subject: Re: [lustre-devel] Testing 2.14 on Debian + ZFS > > Thanks for the confirmation! > > After going in a few circles, I think I've got it figured out and would > appreciate some pointers on how to proceed: > > To preface, the way I'm building is from source-dirs after a package build > (which is admittedly kind of janky as I write it out now). It would appear > that the configure scripting will look in "zfs/include", and thus miss the > new "zfs/include/os/..." directories that hold some important header files, > causing configuration variables to miss-set in config.h thus creating the > compile-time problems of type mismatches and redefinitions I was seeing. > > I was able to confirm manually setting some of the configuration defs and > adding the new header dirs to the makefile fixed the problem, but I'm a > little confused as to how to proceed in patching. Since the configuration > scripting and the makefiles seem to expect only a single include-dir of > header files for both zfs and spl respectively, I'm wondering what the best > approach to patching this behavior would be. > > Thanks for your time and consideration, > Christian > > On Fri, Feb 19, 2021 at 12:13 PM Andreas Dilger <adilger@whamcloud.com > <mailto:adilger@whamcloud.com>> wrote: > This is exactly the right place for such discussions. > > Once you have patches, please submit to Gerrit for review, testing, and > landing: > > https://wiki.lustre.org/Using_Gerrit< > https://urldefense.us/v2/url?u=https-3A__wiki.lustre.org_Using-5FGerrit&d=DwMFaQ&c=pKoAVQro6qDbLoK0T8588B4mZJhJhC4e6QXJy0XnJec&r=2dxT0LLEUx0IbhFF8HRJVCTO_ihbEwKLn9gF7Kc8aG0&m=SU5sac04cekvaRvbUcheoU02W7EUu-F4sVkkrWHUnoE&s=gTcuGMUi5PycREbMB4La1s0dVNM8a5STb9pDVXZt-Lw&e= > > > > Cheers, Andreas > > On Feb 19, 2021, at 02:11, Christian Kuntz <c.kuntz@opendrives.com<mailto: > c.kuntz@opendrives.com>> wrote: > > > Thanks for your time and the thoughtful response. > > Looking at this a bit more, it seems to be caused by the recent ZFS 2.0 > changes for linux and freebsd support, namely the `include/os/linux` paths > that provide `<sys/types.h>` that ZFS is trying to use. Adding them into > the includes for the osd-zfs directory has fixed the references being > broken, and now I'm contending some redefinitions and argument type > mismatches. I'll try and sort this out, in the meantime is there a better > place for me to place messages about this? I know this mailing list isn't > used very frequently and I'd hate to flood it with such off-the-beaten-path > stuff. > > Best, > Christian > > On Thu, Feb 18, 2021 at 3:09 AM Andreas Dilger <adilger@whamcloud.com > <mailto:adilger@whamcloud.com>> wrote: > On Feb 12, 2021, at 16:07, Christian Kuntz <c.kuntz@opendrives.com<mailto: > c.kuntz@opendrives.com>> wrote: > > Hello, > > I hope I'm communicating in the right place here, I'm currently working to > compile Lustre 2.14.0-RC2 on Debian 10.7 with ZFS 2.0.2 for OSDs. If > there's anything I can do to help with the testing effort or help Lustre's > Debian support be more robust, please let me know! I hope I'm not too late > in the release cycle to contribute. > > Thanks for the offer of testing. Unfortunately we are very late in the > 2.14.0 release cycle (we are hoping to make the final release in the next > week or so, barring any critical defects). However, even if any changes > miss the final 2.14.0 release, having patches available to fix any issues > you run into will make it easier for the next person to build on Debian. > > Ideally, any required fixes for Debian building should go into the Lustre > tree directly rather than into a separate "downstream" package, so that > "make debs" works out-of-the-box. We do build Ubuntu 18.04 and 20.04 > clients regularly, and ZFS 2.0 servers on RHEL7/8, but I don't know if > anyone is building the server code on Debian/Ubuntu on a regular basis. > > On the topic of compiling, I'm currently working to get debian packages > made and running into a compilation issue that didn't seem to present on my > previous endeavors with 2.13 and 0.7. I've got the kernel and zfs locally > built and their packages made, and lustre's configure step successfully > completes, but when it comes to compiling it appears that the osd-zfs > portion has some broken include links: > > In file included from /zfs-linux-2.0.2/include/sys/arc.h:32, > from /lustre-debian/lustre/osd-zfs/osd_internal.h:51, > from /lustre-debian/lustre/osd-zfs/osd_handler.c:52: > /zfs-linux-2.0.2/include/sys/zfs_context.h:45:10: fatal error: > sys/types.h: No such file or directory > #include <sys/types.h> > > For reference, my configure and make looks like: > ./configure --with-linux=/linux-4.19.171-2/ \ > > --with-linux-obj=linux-4.19.171-2/debian/build/build_amd64_none_amd64 \ > --with-zfs=/zfs-linux-2.0.2 \ > --enable-server --enable-modules --disable-ldiskfs > make -j"$(nproc)" debs > > > From the error and some probing, it looks like zfs_context.h expects some > header files to be in their libc standard `/usr/include/sys` (or somewhere > abouts), but lustre's make process is pulling them from the linux sources > in `SRC/include/linux/`. Predictably, I found this out the hard way by > adding `-I/usr/include` to the Makefile for the zfs osds and getting > battered with errors and warnings about duplicate definitions. > > It may be that you are looking at this from the wrong angle. Rather than > trying to include more userspace headers, it might be worthwhile to see why > <sys/arc.h> is being included by osd_internal.h and maybe there is a more > appropriate header that could be used in the kernel? Failing that, it > isn't clear wh zfs_context.h is including the <sys/type.h> header from > userspace when building with __KERNEL__ set? Kernel code should be using > <linux/types.h>. > > Cheers, Andreas > -- > Andreas Dilger > Principal Lustre Architect > Whamcloud > > > > > > > [-- Attachment #1.2: Type: text/html, Size: 12954 bytes --] [-- Attachment #2: Type: text/plain, Size: 165 bytes --] _______________________________________________ lustre-devel mailing list lustre-devel@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [lustre-devel] Testing 2.14 on Debian + ZFS 2021-02-23 3:41 ` Christian Kuntz via lustre-devel @ 2021-08-23 1:59 ` Christian Kuntz 0 siblings, 0 replies; 9+ messages in thread From: Christian Kuntz @ 2021-08-23 1:59 UTC (permalink / raw) To: Faaland, Olaf P.; +Cc: Lustre-devel [-- Attachment #1.1: Type: text/plain, Size: 12221 bytes --] Hi all, I thought that perhaps I'd just run into a weird edge-case being on Debian and a little bleeding edge after seeing some other folks on the mailing list compile without issue, but it looks like after some experimental work on centos trying to get Lustre 2.14 and ZFS 2.0 compiled on the same box that similar weirdness exists in the configuration steps where the autoconf logic has a hard time finding function definitions thanks to the new BSD+Linux source tree of ZFS. I'm going to revisit this and see if I can't generate some patches that make the process a bit easier. For my own reference, are there any more recent sources-of-truth for compiling lustre from source than the one on the wiki? <https://wiki.lustre.org/Installing_the_Lustre_Software> There's every likelihood that I'm just missing some steps in the compilation process. Cheers, Christian On Mon, Feb 22, 2021 at 7:41 PM Christian Kuntz <c.kuntz@opendrives.com> wrote: > Hi Olaf, > > Sure! For reference this is how I build zfs: > > ./autogen.sh > fakeroot dpkg-buildpackage -us -uc -b -j$cpus > # Build the kernel modules, not just dkms > KVERS=$KERNEL_ABI KSRC=$WITHLINUX_HDR KOBJ=$WITHLINUXOBJ fakeroot > debian/rules override_dh_binary-modules > > Then, when I build lustre, I do like: > sh autogen.sh > ./configure --with-linux=$WITHLINUX_HDR --with-linux-obj=$WITHLINUXOBJ > --with-zfs=$ZFS_SRC --enable-server --enable-modules --disable-ldiskfs > make debs > > My zfs source is given as the root zfs source directory, not the contents > of explicit package dirs in ZFS_SRC/debian/ which is technically where all > the build artifacts live (for instance, the development headers that would > end up on the machine would be in debian/libzfslinux-dev/usr/include) after > a package build. After fiddling around with autoconf a bit (and learning > it's just shell!), I landed on changing this bit of the autoconf: > config/lustre-build-zfs.m4 > > AC_DEFUN([LB_ZFS_USER], [ > dnl # > dnl # Detect user space zfs development headers. > dnl # > AC_MSG_CHECKING([zfs devel headers]) > AS_IF([test -z "${zfsinc}"], [ > AS_IF([test -d "${zfssrc}/include/os/linux"], [ > zfsinc="-I${zfssrc}/include/os/linux/kernel > -I${zfssrc}/include/os/linux/spl -I${zfssrc}/include/os/linux/zfs > -I${zfssrc}/include" > zfslib="-L$zfssrc/lib/libzfs/.libs/ > -L$zfssrc/lib/libnvpair/.libs" > ], [test -e "${zfssrc}/include/libzfs.h" && test -e > "${zfssrc}/lib/libspl/include"], [ > zfsinc="-I $zfssrc/lib/libspl/include -I $zfssrc/include" > zfslib="-L$zfssrc/lib/libzfs/.libs/ > -L$zfssrc/lib/libnvpair/.libs" > ], [test -d /usr/include/libzfs && test -d /usr/include/libspl], > [ > zfsinc="-I/usr/include/libspl -I /usr/include/libzfs" > zfslib="" > ], [ > zfsinc="[Not Found]" > zfslib="" > enable_zfs=no > ]) > ]) > AC_MSG_RESULT([$zfsinc]) > > I added a check around the new include/os style of the zfs header > directories, which seems to be working OK, every config test is behaving as > expected on my machine except for the multihost support one, which I'm > chasing down a bit (sys/spa.h requires sys/sysmacros.h which requires... > and so on). I also found that I had to modify the contents of > config/lustre-build-linux.m4 to reflect the include dir change: > > AC_DEFUN([LB_LINUX_COMPILE_IFELSE], > [m4_ifvaln([$1], [AC_LANG_CONFTEST([AC_LANG_SOURCE([$1])])])dnl > rm -f build/conftest.o build/conftest.mod.c build/conftest.ko > SUBARCH=$(echo $target_cpu | sed -e 's/powerpc.*/powerpc/' -e > 's/ppc.*/powerpc/' -e 's/x86_64/x86/' -e 's/i.86/x86/' -e 's/k1om/x86/' -e > 's/aarch64.*/arm64/' -e 's/armv7.*/arm/') > AS_IF([AC_TRY_COMMAND(cp conftest.c build && make -d [$2] LDFLAGS= > ${LD:+LD="$LD"} CC="$CC" -f $PWD/build/Makefile > LUSTRE_LINUX_CONFIG=$LINUX_CONFIG LINUXINCLUDE="$EXTRA_CHECK_INCLUDE > -I$LINUX/arch/$SUBARCH/include -Iinclude -Iarch/$SUBARCH/include/generated > -I$LINUX/include -Iinclude2 -I$LINUX/include/uapi -Iinclude/generated > -I$LINUX/arch/$SUBARCH/include/uapi -Iarch/$SUBARCH/include/generated/uapi > -I$LINUX/include/uapi -Iinclude/generated/uapi ${SPL_OBJ:+-include > $SPL_OBJ/spl_config.h} ${ZFS_OBJ:+-include $ZFS_OBJ/zfs_config.h} > ${SPL:+-I$ZFS/lib/libspl/include/ } ${ZFS:+-I$ZFS/include/os/linux/ > -I$ZFS/include/os/linux/kernel -I$ZFS/include/os/linux/spl > -I$ZFS/include/os/linux/zfs -I$ZFS/include } -include $CONFIG_INCLUDE" > KBUILD_EXTRA_SYMBOLS="${ZFS_OBJ:+$ZFS_OBJ/Module.symvers} > $KBUILD_EXTRA_SYMBOLS" -o tmp_include_depends -o scripts -o > include/config/MARKER -C $LINUX_OBJ > EXTRA_CFLAGS="-Werror-implicit-function-declaration $EXTRA_KCFLAGS" > $MODULE_TARGET=$PWD/build) >/dev/null && AC_TRY_COMMAND([$3])], > > I *think* I'll need to adjust the SPL variable in there, but not sure > thus far. > > With the above settings I've gotten everything to succesfully compile and > package up aside from a small modification I had to make > to lustre/utils/libmount_utils_zfs.c to prevent a name collision with > spa_get_hostid. I believe this occurs because the configuration doesn't set > HAVE_ZFS_MULTIHOST appropriately. > > > As you can see, I worked on it a little on the weekend :) > > Cheers, > Christian > > On Mon, Feb 22, 2021 at 11:58 AM Faaland, Olaf P. <faaland1@llnl.gov> > wrote: > >> Hi Christian, >> >> Can you explain in more detail what you mean by "building ... from >> source-dirs after a package build"? And can you identify the lines in >> config/lustre-build-zfs.m4 that you're referring to? >> >> thank you, >> -Olaf >> >> ________________________________________ >> From: Christian Kuntz <c.kuntz@opendrives.com> >> Sent: Friday, February 19, 2021 7:00 PM >> To: Andreas Dilger >> Cc: Lustre-devel; Behlendorf, Brian D.; Faaland, Olaf P. >> Subject: Re: [lustre-devel] Testing 2.14 on Debian + ZFS >> >> Thanks for the confirmation! >> >> After going in a few circles, I think I've got it figured out and would >> appreciate some pointers on how to proceed: >> >> To preface, the way I'm building is from source-dirs after a package >> build (which is admittedly kind of janky as I write it out now). It would >> appear that the configure scripting will look in "zfs/include", and thus >> miss the new "zfs/include/os/..." directories that hold some important >> header files, causing configuration variables to miss-set in config.h thus >> creating the compile-time problems of type mismatches and redefinitions I >> was seeing. >> >> I was able to confirm manually setting some of the configuration defs >> and adding the new header dirs to the makefile fixed the problem, but I'm a >> little confused as to how to proceed in patching. Since the configuration >> scripting and the makefiles seem to expect only a single include-dir of >> header files for both zfs and spl respectively, I'm wondering what the best >> approach to patching this behavior would be. >> >> Thanks for your time and consideration, >> Christian >> >> On Fri, Feb 19, 2021 at 12:13 PM Andreas Dilger <adilger@whamcloud.com >> <mailto:adilger@whamcloud.com>> wrote: >> This is exactly the right place for such discussions. >> >> Once you have patches, please submit to Gerrit for review, testing, and >> landing: >> >> https://wiki.lustre.org/Using_Gerrit< >> https://urldefense.us/v2/url?u=https-3A__wiki.lustre.org_Using-5FGerrit&d=DwMFaQ&c=pKoAVQro6qDbLoK0T8588B4mZJhJhC4e6QXJy0XnJec&r=2dxT0LLEUx0IbhFF8HRJVCTO_ihbEwKLn9gF7Kc8aG0&m=SU5sac04cekvaRvbUcheoU02W7EUu-F4sVkkrWHUnoE&s=gTcuGMUi5PycREbMB4La1s0dVNM8a5STb9pDVXZt-Lw&e= >> > >> >> Cheers, Andreas >> >> On Feb 19, 2021, at 02:11, Christian Kuntz <c.kuntz@opendrives.com >> <mailto:c.kuntz@opendrives.com>> wrote: >> >> >> Thanks for your time and the thoughtful response. >> >> Looking at this a bit more, it seems to be caused by the recent ZFS 2.0 >> changes for linux and freebsd support, namely the `include/os/linux` paths >> that provide `<sys/types.h>` that ZFS is trying to use. Adding them into >> the includes for the osd-zfs directory has fixed the references being >> broken, and now I'm contending some redefinitions and argument type >> mismatches. I'll try and sort this out, in the meantime is there a better >> place for me to place messages about this? I know this mailing list isn't >> used very frequently and I'd hate to flood it with such off-the-beaten-path >> stuff. >> >> Best, >> Christian >> >> On Thu, Feb 18, 2021 at 3:09 AM Andreas Dilger <adilger@whamcloud.com >> <mailto:adilger@whamcloud.com>> wrote: >> On Feb 12, 2021, at 16:07, Christian Kuntz <c.kuntz@opendrives.com >> <mailto:c.kuntz@opendrives.com>> wrote: >> >> Hello, >> >> I hope I'm communicating in the right place here, I'm currently working >> to compile Lustre 2.14.0-RC2 on Debian 10.7 with ZFS 2.0.2 for OSDs. If >> there's anything I can do to help with the testing effort or help Lustre's >> Debian support be more robust, please let me know! I hope I'm not too late >> in the release cycle to contribute. >> >> Thanks for the offer of testing. Unfortunately we are very late in the >> 2.14.0 release cycle (we are hoping to make the final release in the next >> week or so, barring any critical defects). However, even if any changes >> miss the final 2.14.0 release, having patches available to fix any issues >> you run into will make it easier for the next person to build on Debian. >> >> Ideally, any required fixes for Debian building should go into the Lustre >> tree directly rather than into a separate "downstream" package, so that >> "make debs" works out-of-the-box. We do build Ubuntu 18.04 and 20.04 >> clients regularly, and ZFS 2.0 servers on RHEL7/8, but I don't know if >> anyone is building the server code on Debian/Ubuntu on a regular basis. >> >> On the topic of compiling, I'm currently working to get debian packages >> made and running into a compilation issue that didn't seem to present on my >> previous endeavors with 2.13 and 0.7. I've got the kernel and zfs locally >> built and their packages made, and lustre's configure step successfully >> completes, but when it comes to compiling it appears that the osd-zfs >> portion has some broken include links: >> >> In file included from /zfs-linux-2.0.2/include/sys/arc.h:32, >> from /lustre-debian/lustre/osd-zfs/osd_internal.h:51, >> from /lustre-debian/lustre/osd-zfs/osd_handler.c:52: >> /zfs-linux-2.0.2/include/sys/zfs_context.h:45:10: fatal error: >> sys/types.h: No such file or directory >> #include <sys/types.h> >> >> For reference, my configure and make looks like: >> ./configure --with-linux=/linux-4.19.171-2/ \ >> >> --with-linux-obj=linux-4.19.171-2/debian/build/build_amd64_none_amd64 \ >> --with-zfs=/zfs-linux-2.0.2 \ >> --enable-server --enable-modules --disable-ldiskfs >> make -j"$(nproc)" debs >> >> >> From the error and some probing, it looks like zfs_context.h expects some >> header files to be in their libc standard `/usr/include/sys` (or somewhere >> abouts), but lustre's make process is pulling them from the linux sources >> in `SRC/include/linux/`. Predictably, I found this out the hard way by >> adding `-I/usr/include` to the Makefile for the zfs osds and getting >> battered with errors and warnings about duplicate definitions. >> >> It may be that you are looking at this from the wrong angle. Rather than >> trying to include more userspace headers, it might be worthwhile to see why >> <sys/arc.h> is being included by osd_internal.h and maybe there is a more >> appropriate header that could be used in the kernel? Failing that, it >> isn't clear wh zfs_context.h is including the <sys/type.h> header from >> userspace when building with __KERNEL__ set? Kernel code should be using >> <linux/types.h>. >> >> Cheers, Andreas >> -- >> Andreas Dilger >> Principal Lustre Architect >> Whamcloud >> >> >> >> >> >> >> [-- Attachment #1.2: Type: text/html, Size: 14258 bytes --] [-- Attachment #2: Type: text/plain, Size: 165 bytes --] _______________________________________________ lustre-devel mailing list lustre-devel@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-08-23 1:59 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-02-12 23:07 [lustre-devel] Testing 2.14 on Debian + ZFS Christian Kuntz 2021-02-18 11:09 ` Andreas Dilger 2021-02-19 9:11 ` Christian Kuntz 2021-02-19 14:17 ` Spitz, Cory James 2021-02-19 20:12 ` Andreas Dilger 2021-02-20 3:00 ` Christian Kuntz 2021-02-22 19:58 ` Faaland, Olaf P. via lustre-devel 2021-02-23 3:41 ` Christian Kuntz via lustre-devel 2021-08-23 1:59 ` Christian Kuntz
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).