All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ruifeng Wang <Ruifeng.Wang@arm.com>
To: Aaron Conole <aconole@redhat.com>
Cc: "maicolgabriel@hotmail.com" <maicolgabriel@hotmail.com>,
	"thomas@monjalon.net" <thomas@monjalon.net>,
	"ferruh.yigit@intel.com" <ferruh.yigit@intel.com>,
	"arybchenko@solarflare.com" <arybchenko@solarflare.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	"david.marchand@redhat.com" <david.marchand@redhat.com>,
	Gavin Hu <Gavin.Hu@arm.com>,
	Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>,
	nd <nd@arm.com>, nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64
Date: Thu, 9 Jan 2020 07:00:37 +0000	[thread overview]
Message-ID: <AM0PR08MB39862ECFF0409EE1DD482AA09E390@AM0PR08MB3986.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <f7teewaylvi.fsf@dhcp-25.97.bos.redhat.com>


> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Thursday, January 9, 2020 00:05
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: maicolgabriel@hotmail.com; thomas@monjalon.net;
> ferruh.yigit@intel.com; arybchenko@solarflare.com; dev@dpdk.org;
> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v3 2/2] ci: add travis ci support for aarch64
> 
> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> 
> >> -----Original Message-----
> >> From: Aaron Conole <aconole@redhat.com>
> >> Sent: Monday, January 6, 2020 21:34
> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> Cc: maicolgabriel@hotmail.com; thomas@monjalon.net;
> >> ferruh.yigit@intel.com; arybchenko@solarflare.com; dev@dpdk.org;
> >> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>;
> Honnappa
> >> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> >> Subject: Re: [PATCH v3 2/2] ci: add travis ci support for aarch64
> >>
> >> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> >>
> >> > Add Travis compilation jobs for aarch64. gcc/clang compilations for
> >> > static/shared libraries are added.
> >> >
> >> > Some limitations for current aarch64 Travis support:
> >> > 1. Container is used. Huge page is not available due to security reason.
> >> > 2. Missing kernel header package in Xenial distribution.
> >> >
> >> > Solutions to address the limitations:
> >> > 1. Not to add unit test for now. And run tests with no-huge in future.
> >> > 2. Use Bionic distribution for all aarch64 jobs.
> >> >
> >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> > ---
> >>
> >> Can't we achieve the same thing by setting
> >>
> >> arch:
> >>   - amd64
> >>   - arm64
> >>
> >> in the build matrix?  Or will that also force the intel builds to use
> >> the container infrastructure (in which case the no-huge support needs to
> be fixed)?
> >
> > No, container infrastructure will not be imposed to intel builds.
> > AFAIN, Travis infrastructure for a specific CPU arch is provided as
> > is, and there is no config option to control.
> > The problem with just adding 'arch' in build matrix is that RUN_TESTS
> > on arm64 is not supported by now (Travis limitation). 'env' with RUN_TESTS
> will fail.
> 
> Okay I see.
> 
> >>
> >> One thing I wonder, isn't is possible to use qemu-user to do the
> >> amd64 unit tests?  Then do we really need some changes to do the native
> build?
> >
> > Do you mean to use qemu-user to do unit tests for non-x86 arch?
> 
> Yes.  This has the advantage of giving users a way to also do the multi-arch
> checks on their own systems (so a developer with just an x86 could at least
> do some testing on arm or ppc).
> 
Yes, users can do multi-arch checks *locally* by using qemu. 
This patch aims to enable *public* CI for aarch64. It has no sense to rely on specific arch while infrastructure
supports multi arch.

> > Changes will be needed as well to enable qemu-user to do unit test.
> > Since Travis support multi CPU arch, I think native build and test is simpler
> and more natural.
> 
> I agree, some script changes might be needed, but maybe not as many as
> you fear (can't we use binfmt_misc infrastructure to do this with qemu-user
> and then the actual 'execute' would work).
> 
It is more like a tool for local validation, and should be another story.

> >> Does it buy us anything *today* given the cost of the hugepage
> restriction?
> >> Will that ever be resolved (I didn't see so from the docs on travis)?
> >
> > The hugepage issue has been reported to Travis. I think it will be
> > resolved. But no set dates yet.
> 
> Is there a plan for them to address?  I guess probably not.  So we either need
> the ability for tests to run in the no-huge environment (and detect that no
> hugepages are available to run the tests that way), or we need the travis
> environment supporting hugepages.  Is there something I missed?
> 
Yes, over half of quick tests can run in no-huge environment.

> >>
> >> >  .ci/linux-setup.sh | 11 +++++++----
> >> >  .travis.yml        | 42
> +++++++++++++++++++++++++++++++++++++++++-
> >> >  2 files changed, 48 insertions(+), 5 deletions(-)
> >> >
> >> > diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh index
> >> > dfb9d4a20..a92978037 100755
> >> > --- a/.ci/linux-setup.sh
> >> > +++ b/.ci/linux-setup.sh
> >> > @@ -3,7 +3,10 @@
> >> >  # need to install as 'root' since some of the unit tests won't run
> >> > without it  sudo python3 -m pip install --upgrade meson
> >> >
> >> > -# setup hugepages
> >> > -cat /proc/meminfo
> >> > -sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> >> > -cat /proc/meminfo
> >> > +# hugepage settings are skipped on aarch64 due to environment
> >> > +limitation if [ "$TRAVIS_ARCH" != "aarch64" ]; then
> >> > +    # setup hugepages
> >> > +    cat /proc/meminfo
> >> > +    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> >> > +    cat /proc/meminfo
> >> > +fi
> >> > diff --git a/.travis.yml b/.travis.yml index 8f90d06f2..980c7605d
> >> > 100644
> >> > --- a/.travis.yml
> >> > +++ b/.travis.yml
> >> > @@ -115,6 +115,46 @@ matrix:
> >> >        apt:
> >> >          packages:
> >> >            - *extra_packages
> >> > -
> >> > +  - env: DEF_LIB="static"
> >> > +    arch: arm64
> >> > +    compiler: gcc
> >> > +    dist: bionic
> >> > +    addons:
> >> > +      apt:
> >> > +        packages:
> >> > +          - *required_packages
> >> > +  - env: DEF_LIB="shared"
> >> > +    arch: arm64
> >> > +    compiler: gcc
> >> > +    dist: bionic
> >> > +    addons:
> >> > +      apt:
> >> > +        packages:
> >> > +          - *required_packages
> >> > +  - env: DEF_LIB="static"
> >> > +    arch: arm64
> >> > +    dist: bionic
> >> > +    compiler: clang
> >> > +    addons:
> >> > +      apt:
> >> > +        packages:
> >> > +          - *required_packages
> >> > +  - env: DEF_LIB="shared"
> >> > +    arch: arm64
> >> > +    dist: bionic
> >> > +    compiler: clang
> >> > +    addons:
> >> > +      apt:
> >> > +        packages:
> >> > +          - *required_packages
> >> > +  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false"
> BUILD_DOCS=1
> >> > +    arch: arm64
> >> > +    compiler: gcc
> >> > +    dist: bionic
> >> > +    addons:
> >> > +      apt:
> >> > +        packages:
> >> > +          - *required_packages
> >> > +          - *doc_packages
> >> >
> >> >  script: ./.ci/${TRAVIS_OS_NAME}-build.sh


  parent reply	other threads:[~2020-01-09  7:00 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-18  5:39 [dpdk-dev] [PATCH 0/2] add travis ci support for aarch64 Ruifeng Wang
2019-12-18  5:39 ` [dpdk-dev] [PATCH 1/2] ci: " Ruifeng Wang
2019-12-18  5:39 ` [dpdk-dev] [PATCH 2/2] devtools: add path to additional shared object files Ruifeng Wang
2019-12-18  8:23   ` David Marchand
2019-12-18 13:43     ` Laatz, Kevin
2019-12-18 15:32       ` David Marchand
2019-12-18 16:00         ` Richardson, Bruce
2019-12-19  3:14         ` Ruifeng Wang
2019-12-20  9:37 ` [dpdk-dev] [PATCH v2 0/2] add travis ci support for aarch64 Ruifeng Wang
2019-12-20  9:37   ` [dpdk-dev] [PATCH v2 1/2] ci: " Ruifeng Wang
2020-01-06 20:17     ` dwilder
2020-01-07  6:42       ` Ruifeng Wang
2019-12-20  9:37   ` [dpdk-dev] [PATCH v2 2/2] devtools: add path to additional shared object files Ruifeng Wang
2019-12-20 13:57   ` [dpdk-dev] [PATCH v2 0/2] add travis ci support for aarch64 David Marchand
2019-12-23  7:08 ` [dpdk-dev] [PATCH v3 " Ruifeng Wang
2019-12-23  7:08   ` [dpdk-dev] [PATCH v3 1/2] devtools: add path to additional shared object files Ruifeng Wang
2019-12-23  7:08   ` [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64 Ruifeng Wang
2020-01-06 13:34     ` Aaron Conole
2020-01-07  6:24       ` Ruifeng Wang
2020-01-07 14:40         ` Honnappa Nagarahalli
2020-01-08 16:06           ` Aaron Conole
2020-01-08 16:05         ` Aaron Conole
2020-01-08 17:37           ` Bruce Richardson
2020-01-09  7:00           ` Ruifeng Wang [this message]
2020-01-09 15:50             ` Honnappa Nagarahalli
2020-01-09 17:45               ` Aaron Conole
2020-01-10  9:53 ` [dpdk-dev] [PATCH v4 0/2] " Ruifeng Wang
2020-01-10  9:53   ` [dpdk-dev] [PATCH v4 1/2] devtools: add path to additional shared object files Ruifeng Wang
2020-01-10 15:03     ` Aaron Conole
2020-01-10  9:53   ` [dpdk-dev] [PATCH v4 2/2] ci: add travis ci support for aarch64 Ruifeng Wang
2020-01-10 15:13     ` Aaron Conole
2020-01-13  6:09       ` Ruifeng Wang
2020-01-13  6:26 ` [dpdk-dev] [PATCH v5 0/2] add travis ci support for native aarch64 Ruifeng Wang
2020-01-13  6:26   ` [dpdk-dev] [PATCH v5 1/2] devtools: add path to additional shared object files Ruifeng Wang
2020-01-13  6:26   ` [dpdk-dev] [PATCH v5 2/2] ci: add travis ci support for native aarch64 Ruifeng Wang
2020-01-14 14:03   ` [dpdk-dev] [PATCH v5 0/2] " David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=AM0PR08MB39862ECFF0409EE1DD482AA09E390@AM0PR08MB3986.eurprd08.prod.outlook.com \
    --to=ruifeng.wang@arm.com \
    --cc=Gavin.Hu@arm.com \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=aconole@redhat.com \
    --cc=arybchenko@solarflare.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=maicolgabriel@hotmail.com \
    --cc=nd@arm.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.