Run unit tests with C++ too

Run unit tests with C++ too

[Mattias Rönnblom]

It would be great if the unit test suite (app/test/*) was compiled (and 
run) using a C++ (C++11) compiler as well. At least, if such is available.

With the current state of affairs, header file macros or functions are 
not verified to be functional (or even valid) C++.

"C is a subset of C++", which was never true, is becoming less and less so.

If all unit tests aren't valid C++, maybe one could start with an "opt 
in" model.

A drawback of this is that the unit tests need to be both valid C and 
valid C++.

Re: Run unit tests with C++ too

[Ferruh Yigit]

On 4/28/2024 8:46 AM, Mattias Rönnblom wrote:
> It would be great if the unit test suite (app/test/*) was compiled (and
> run) using a C++ (C++11) compiler as well. At least, if such is available.
> 
> With the current state of affairs, header file macros or functions are
> not verified to be functional (or even valid) C++.
> 
> "C is a subset of C++", which was never true, is becoming less and less so.
> 
> If all unit tests aren't valid C++, maybe one could start with an "opt
> in" model.
> 
> A drawback of this is that the unit tests need to be both valid C and
> valid C++.
>

+1

cc'ing CI mailing list and CI lab.

Re: Run unit tests with C++ too

[Tyler Retzlaff]

On Mon, Apr 29, 2024 at 09:01:08AM +0100, Ferruh Yigit wrote:
> On 4/28/2024 8:46 AM, Mattias Rönnblom wrote:
> > It would be great if the unit test suite (app/test/*) was compiled (and
> > run) using a C++ (C++11) compiler as well. At least, if such is available.
> > 
> > With the current state of affairs, header file macros or functions are
> > not verified to be functional (or even valid) C++.
> > 
> > "C is a subset of C++", which was never true, is becoming less and less so.
> > 
> > If all unit tests aren't valid C++, maybe one could start with an "opt
> > in" model.
> > 
> > A drawback of this is that the unit tests need to be both valid C and
> > valid C++.
> >
> 
> +1
> 
> cc'ing CI mailing list and CI lab.

+1 too

Re: Run unit tests with C++ too

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 940 bytes --]

On Sun, Apr 28, 2024 at 3:46 AM Mattias Rönnblom <hofors@lysator.liu.se>
wrote:

> It would be great if the unit test suite (app/test/*) was compiled (and
> run) using a C++ (C++11) compiler as well. At least, if such is available.
>

Sure, the UNH Lab can try this.


>
> With the current state of affairs, header file macros or functions are
> not verified to be functional (or even valid) C++.
>
> "C is a subset of C++", which was never true, is becoming less and less so.
>
> If all unit tests aren't valid C++, maybe one could start with an "opt
> in" model.
>

Okay, so basically run the fast-test suite, record all that don't pass,
submit a bugzilla ticket stating which unit tests are not valid on a
certain c++ compiler, then bring CI Testing online using the valid subset
of fast-tests. This should work.


>
> A drawback of this is that the unit tests need to be both valid C and
> valid C++.
>

[-- Attachment #2: Type: text/html, Size: 1628 bytes --]

Re: Run unit tests with C++ too

[Tyler Retzlaff]

On Tue, Apr 30, 2024 at 09:52:05AM -0400, Patrick Robb wrote:
> On Sun, Apr 28, 2024 at 3:46 AM Mattias Rönnblom <hofors@lysator.liu.se>
> wrote:
> 
> > It would be great if the unit test suite (app/test/*) was compiled (and
> > run) using a C++ (C++11) compiler as well. At least, if such is available.
> >
> 
> Sure, the UNH Lab can try this.
> 
> 
> >
> > With the current state of affairs, header file macros or functions are
> > not verified to be functional (or even valid) C++.
> >
> > "C is a subset of C++", which was never true, is becoming less and less so.
> >
> > If all unit tests aren't valid C++, maybe one could start with an "opt
> > in" model.
> >
> 
> Okay, so basically run the fast-test suite, record all that don't pass,
> submit a bugzilla ticket stating which unit tests are not valid on a
> certain c++ compiler, then bring CI Testing online using the valid subset
> of fast-tests. This should work.

this seems like a reasonable approach.

> 
> 
> >
> > A drawback of this is that the unit tests need to be both valid C and
> > valid C++.
> >

Re: Run unit tests with C++ too

[Mattias Rönnblom]

On 2024-04-30 15:52, Patrick Robb wrote:
> 
> 
> On Sun, Apr 28, 2024 at 3:46 AM Mattias Rönnblom <hofors@lysator.liu.se 
> <mailto:hofors@lysator.liu.se>> wrote:
> 
>     It would be great if the unit test suite (app/test/*) was compiled (and
>     run) using a C++ (C++11) compiler as well. At least, if such is
>     available.
> 
> 
> Sure, the UNH Lab can try this.
> 
> 
>     With the current state of affairs, header file macros or functions are
>     not verified to be functional (or even valid) C++.
> 
>     "C is a subset of C++", which was never true, is becoming less and
>     less so.
> 
>     If all unit tests aren't valid C++, maybe one could start with an "opt
>     in" model.
> 
> 
> Okay, so basically run the fast-test suite, record all that don't pass, 
> submit a bugzilla ticket stating which unit tests are not valid on a 
> certain c++ compiler, then bring CI Testing online using the valid 
> subset of fast-tests. This should work.
> 

Sounds good.

Just to be clear: the above includes extending the DPDK build system to 
build the app/test/dpdk-test binary in two versions: one C and one C++, 
so that anyone can run the C++ tests locally as well. Correct?


> 
>     A drawback of this is that the unit tests need to be both valid C and
>     valid C++.
> 

Re: Run unit tests with C++ too

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 2086 bytes --]

On Tue, Apr 30, 2024 at 4:13 PM Mattias Rönnblom <hofors@lysator.liu.se>
wrote:

> On 2024-04-30 15:52, Patrick Robb wrote:
> >
> >
> > On Sun, Apr 28, 2024 at 3:46 AM Mattias Rönnblom <hofors@lysator.liu.se
> > <mailto:hofors@lysator.liu.se>> wrote:
> >
> >     It would be great if the unit test suite (app/test/*) was compiled
> (and
> >     run) using a C++ (C++11) compiler as well. At least, if such is
> >     available.
> >
> >
> > Sure, the UNH Lab can try this.
> >
> >
> >     With the current state of affairs, header file macros or functions
> are
> >     not verified to be functional (or even valid) C++.
> >
> >     "C is a subset of C++", which was never true, is becoming less and
> >     less so.
> >
> >     If all unit tests aren't valid C++, maybe one could start with an
> "opt
> >     in" model.
> >
> >
> > Okay, so basically run the fast-test suite, record all that don't pass,
> > submit a bugzilla ticket stating which unit tests are not valid on a
> > certain c++ compiler, then bring CI Testing online using the valid
> > subset of fast-tests. This should work.
> >
>
> Sounds good.
>
> Just to be clear: the above includes extending the DPDK build system to
> build the app/test/dpdk-test binary in two versions: one C and one C++,
> so that anyone can run the C++ tests locally as well. Correct?
>

Okay, so now I am understanding this is not yet available. When I responded
this morning I was figuring that c++ compiler support was available and I
simply wasn't aware, and that we could quite easily set cc={some c++
compiler}, meson would pick it up, and we would be able to build DPDK and
then run unit tests in this manner in CI testing.

I didn't mean to suggest we would submit patches extending the build system
to this end. That's probably a little out of scope for what we try to
accomplish at the Community Lab.

But if the aforementioned build system support is added, of course we are
willing to add that as a build environment for unit tests and report those
respective results.

[-- Attachment #2: Type: text/html, Size: 2824 bytes --]

Re: Run unit tests with C++ too

[Ferruh Yigit]

On 4/30/2024 9:57 PM, Patrick Robb wrote:
> 
> 
> On Tue, Apr 30, 2024 at 4:13 PM Mattias Rönnblom <hofors@lysator.liu.se
> <mailto:hofors@lysator.liu.se>> wrote:
> 
>     On 2024-04-30 15:52, Patrick Robb wrote:
>     >
>     >
>     > On Sun, Apr 28, 2024 at 3:46 AM Mattias Rönnblom
>     <hofors@lysator.liu.se <mailto:hofors@lysator.liu.se>
>     > <mailto:hofors@lysator.liu.se <mailto:hofors@lysator.liu.se>>> wrote:
>     >
>     >     It would be great if the unit test suite (app/test/*) was
>     compiled (and
>     >     run) using a C++ (C++11) compiler as well. At least, if such is
>     >     available.
>     >
>     >
>     > Sure, the UNH Lab can try this.
>     >
>     >
>     >     With the current state of affairs, header file macros or
>     functions are
>     >     not verified to be functional (or even valid) C++.
>     >
>     >     "C is a subset of C++", which was never true, is becoming less and
>     >     less so.
>     >
>     >     If all unit tests aren't valid C++, maybe one could start with
>     an "opt
>     >     in" model.
>     >
>     >
>     > Okay, so basically run the fast-test suite, record all that don't
>     pass,
>     > submit a bugzilla ticket stating which unit tests are not valid on a
>     > certain c++ compiler, then bring CI Testing online using the valid
>     > subset of fast-tests. This should work.
>     >
> 
>     Sounds good.
> 
>     Just to be clear: the above includes extending the DPDK build system to
>     build the app/test/dpdk-test binary in two versions: one C and one C++,
>     so that anyone can run the C++ tests locally as well. Correct?
> 
> 
> Okay, so now I am understanding this is not yet available. When I
> responded this morning I was figuring that c++ compiler support was
> available and I simply wasn't aware, and that we could quite easily set
> cc={some c++ compiler}, meson would pick it up, and we would be able to
> build DPDK and then run unit tests in this manner in CI testing. 
> 
> I didn't mean to suggest we would submit patches extending the build
> system to this end. That's probably a little out of scope for what we
> try to accomplish at the Community Lab. 
> 
> But if the aforementioned build system support is added, of course we
> are willing to add that as a build environment for unit tests and report
> those respective results.
>  

Does it have to be 'app/test/dpdk-test', why not build examples with C++?

Examples source codes can be installed with existing build support.
Later we can build these examples with C++, this doesn't require any
update in build system.

Re: Run unit tests with C++ too

[Bruce Richardson]

On Wed, May 01, 2024 at 10:10:57AM +0100, Ferruh Yigit wrote:
> On 4/30/2024 9:57 PM, Patrick Robb wrote:
> > 
> > 
> > On Tue, Apr 30, 2024 at 4:13 PM Mattias Rönnblom <hofors@lysator.liu.se
> > <mailto:hofors@lysator.liu.se>> wrote:
> > 
> >     On 2024-04-30 15:52, Patrick Robb wrote:
> >     >
> >     >
> >     > On Sun, Apr 28, 2024 at 3:46 AM Mattias Rönnblom
> >     <hofors@lysator.liu.se <mailto:hofors@lysator.liu.se>
> >     > <mailto:hofors@lysator.liu.se <mailto:hofors@lysator.liu.se>>>
> >     > wrote:
> >     >
> >     >     It would be great if the unit test suite (app/test/*) was
> >     compiled (and
> >     >     run) using a C++ (C++11) compiler as well. At least, if such
> >     >is      available.
> >     >
> >     >
> >     > Sure, the UNH Lab can try this.
> >     >
> >     >
> >     >     With the current state of affairs, header file macros or
> >     functions are
> >     >     not verified to be functional (or even valid) C++.
> >     >
> >     >     "C is a subset of C++", which was never true, is becoming
> >     >less and      less so.
> >     >
> >     >     If all unit tests aren't valid C++, maybe one could start
> >     >with
> >     an "opt
> >     >     in" model.
> >     >
> >     >
> >     > Okay, so basically run the fast-test suite, record all that don't
> >     pass,
> >     > submit a bugzilla ticket stating which unit tests are not valid
> >     > on a certain c++ compiler, then bring CI Testing online using the
> >     > valid subset of fast-tests. This should work.
> >     >
> > 
> >     Sounds good.
> > 
> >     Just to be clear: the above includes extending the DPDK build
> >     system to build the app/test/dpdk-test binary in two versions: one
> >     C and one C++, so that anyone can run the C++ tests locally as
> >     well. Correct?
> > 
> > 
> > Okay, so now I am understanding this is not yet available. When I
> > responded this morning I was figuring that c++ compiler support was
> > available and I simply wasn't aware, and that we could quite easily set
> > cc={some c++ compiler}, meson would pick it up, and we would be able to
> > build DPDK and then run unit tests in this manner in CI testing. 
> > 
> > I didn't mean to suggest we would submit patches extending the build
> > system to this end. That's probably a little out of scope for what we
> > try to accomplish at the Community Lab. 
> > 
> > But if the aforementioned build system support is added, of course we
> > are willing to add that as a build environment for unit tests and
> > report those respective results.   
> 
> Does it have to be 'app/test/dpdk-test', why not build examples with C++?
> 
> Examples source codes can be installed with existing build support.
> Later we can build these examples with C++, this doesn't require any
> update in build system.
>
Reading through the history, I believe the ask here is to have the headers
validated for C++ compatibility. We previously added support to "chkincs"
for this - we can build chkincs-cpp with g++ as well as a regular
C-compiled chkincs binary.

/Bruce

Re: Run unit tests with C++ too

[Mattias Rönnblom]

On 2024-05-01 11:10, Ferruh Yigit wrote:
> On 4/30/2024 9:57 PM, Patrick Robb wrote:
>>
>>
>> On Tue, Apr 30, 2024 at 4:13 PM Mattias Rönnblom <hofors@lysator.liu.se
>> <mailto:hofors@lysator.liu.se>> wrote:
>>
>>      On 2024-04-30 15:52, Patrick Robb wrote:
>>      >
>>      >
>>      > On Sun, Apr 28, 2024 at 3:46 AM Mattias Rönnblom
>>      <hofors@lysator.liu.se <mailto:hofors@lysator.liu.se>
>>      > <mailto:hofors@lysator.liu.se <mailto:hofors@lysator.liu.se>>> wrote:
>>      >
>>      >     It would be great if the unit test suite (app/test/*) was
>>      compiled (and
>>      >     run) using a C++ (C++11) compiler as well. At least, if such is
>>      >     available.
>>      >
>>      >
>>      > Sure, the UNH Lab can try this.
>>      >
>>      >
>>      >     With the current state of affairs, header file macros or
>>      functions are
>>      >     not verified to be functional (or even valid) C++.
>>      >
>>      >     "C is a subset of C++", which was never true, is becoming less and
>>      >     less so.
>>      >
>>      >     If all unit tests aren't valid C++, maybe one could start with
>>      an "opt
>>      >     in" model.
>>      >
>>      >
>>      > Okay, so basically run the fast-test suite, record all that don't
>>      pass,
>>      > submit a bugzilla ticket stating which unit tests are not valid on a
>>      > certain c++ compiler, then bring CI Testing online using the valid
>>      > subset of fast-tests. This should work.
>>      >
>>
>>      Sounds good.
>>
>>      Just to be clear: the above includes extending the DPDK build system to
>>      build the app/test/dpdk-test binary in two versions: one C and one C++,
>>      so that anyone can run the C++ tests locally as well. Correct?
>>
>>
>> Okay, so now I am understanding this is not yet available. When I
>> responded this morning I was figuring that c++ compiler support was
>> available and I simply wasn't aware, and that we could quite easily set
>> cc={some c++ compiler}, meson would pick it up, and we would be able to
>> build DPDK and then run unit tests in this manner in CI testing.
>>
>> I didn't mean to suggest we would submit patches extending the build
>> system to this end. That's probably a little out of scope for what we
>> try to accomplish at the Community Lab.
>>
>> But if the aforementioned build system support is added, of course we
>> are willing to add that as a build environment for unit tests and report
>> those respective results.
>>   
> 
> Does it have to be 'app/test/dpdk-test', why not build examples with C++?
> 

The unit tests have the ability to test DPDK, which is exactly what we 
want to do here. Such testing isn't limited to "compiles yes/no", but to 
detect run-time (behavioral) issues, and properly report them.

This is especially important for cases where there is code only 
exercised in C++ translation units (i.e., in #ifdef __cplusplus).

> Examples source codes can be installed with existing build support.
> Later we can build these examples with C++, this doesn't require any
> update in build system.
> 

Re: Run unit tests with C++ too

[Ferruh Yigit]

On 5/1/2024 3:14 PM, Mattias Rönnblom wrote:
> On 2024-05-01 11:10, Ferruh Yigit wrote:
>> On 4/30/2024 9:57 PM, Patrick Robb wrote:
>>>
>>>
>>> On Tue, Apr 30, 2024 at 4:13 PM Mattias Rönnblom <hofors@lysator.liu.se
>>> <mailto:hofors@lysator.liu.se>> wrote:
>>>
>>>      On 2024-04-30 15:52, Patrick Robb wrote:
>>>      >
>>>      >
>>>      > On Sun, Apr 28, 2024 at 3:46 AM Mattias Rönnblom
>>>      <hofors@lysator.liu.se <mailto:hofors@lysator.liu.se>
>>>      > <mailto:hofors@lysator.liu.se <mailto:hofors@lysator.liu.se>>>
>>> wrote:
>>>      >
>>>      >     It would be great if the unit test suite (app/test/*) was
>>>      compiled (and
>>>      >     run) using a C++ (C++11) compiler as well. At least, if
>>> such is
>>>      >     available.
>>>      >
>>>      >
>>>      > Sure, the UNH Lab can try this.
>>>      >
>>>      >
>>>      >     With the current state of affairs, header file macros or
>>>      functions are
>>>      >     not verified to be functional (or even valid) C++.
>>>      >
>>>      >     "C is a subset of C++", which was never true, is becoming
>>> less and
>>>      >     less so.
>>>      >
>>>      >     If all unit tests aren't valid C++, maybe one could start
>>> with
>>>      an "opt
>>>      >     in" model.
>>>      >
>>>      >
>>>      > Okay, so basically run the fast-test suite, record all that don't
>>>      pass,
>>>      > submit a bugzilla ticket stating which unit tests are not
>>> valid on a
>>>      > certain c++ compiler, then bring CI Testing online using the
>>> valid
>>>      > subset of fast-tests. This should work.
>>>      >
>>>
>>>      Sounds good.
>>>
>>>      Just to be clear: the above includes extending the DPDK build
>>> system to
>>>      build the app/test/dpdk-test binary in two versions: one C and
>>> one C++,
>>>      so that anyone can run the C++ tests locally as well. Correct?
>>>
>>>
>>> Okay, so now I am understanding this is not yet available. When I
>>> responded this morning I was figuring that c++ compiler support was
>>> available and I simply wasn't aware, and that we could quite easily set
>>> cc={some c++ compiler}, meson would pick it up, and we would be able to
>>> build DPDK and then run unit tests in this manner in CI testing.
>>>
>>> I didn't mean to suggest we would submit patches extending the build
>>> system to this end. That's probably a little out of scope for what we
>>> try to accomplish at the Community Lab.
>>>
>>> But if the aforementioned build system support is added, of course we
>>> are willing to add that as a build environment for unit tests and report
>>> those respective results.
>>>   
>>
>> Does it have to be 'app/test/dpdk-test', why not build examples with C++?
>>
> 
> The unit tests have the ability to test DPDK, which is exactly what we
> want to do here. Such testing isn't limited to "compiles yes/no", but to
> detect run-time (behavioral) issues, and properly report them.
> 
> This is especially important for cases where there is code only
> exercised in C++ translation units (i.e., in #ifdef __cplusplus).
> 

And Bruce highlighted that compile check is already covered.

Than I guess this work needs to be done in two steps,
1. Enable building dpdk-test (or all applications) with C++ in build
system. And fix possible issues.

2. Enable in dpdk-test C++ build and run in CI.

We need a volunteer for 1. before asking CI lab for 2.

Re: Run unit tests with C++ too

[Bruce Richardson]

On Wed, May 01, 2024 at 03:38:10PM +0100, Ferruh Yigit wrote:
> On 5/1/2024 3:14 PM, Mattias Rönnblom wrote:
> > On 2024-05-01 11:10, Ferruh Yigit wrote:
> >> On 4/30/2024 9:57 PM, Patrick Robb wrote:
> >>>
> >>>
> >>> On Tue, Apr 30, 2024 at 4:13 PM Mattias Rönnblom <hofors@lysator.liu.se
> >>> <mailto:hofors@lysator.liu.se>> wrote:
> >>>
> >>>      On 2024-04-30 15:52, Patrick Robb wrote:
> >>>      >
> >>>      >
> >>>      > On Sun, Apr 28, 2024 at 3:46 AM Mattias Rönnblom
> >>>      <hofors@lysator.liu.se <mailto:hofors@lysator.liu.se>
> >>>      > <mailto:hofors@lysator.liu.se <mailto:hofors@lysator.liu.se>>>
> >>> wrote:
> >>>      >
> >>>      >     It would be great if the unit test suite (app/test/*) was
> >>>      compiled (and
> >>>      >     run) using a C++ (C++11) compiler as well. At least, if
> >>> such is
> >>>      >     available.
> >>>      >
> >>>      >
> >>>      > Sure, the UNH Lab can try this.
> >>>      >
> >>>      >
> >>>      >     With the current state of affairs, header file macros or
> >>>      functions are
> >>>      >     not verified to be functional (or even valid) C++.
> >>>      >
> >>>      >     "C is a subset of C++", which was never true, is becoming
> >>> less and
> >>>      >     less so.
> >>>      >
> >>>      >     If all unit tests aren't valid C++, maybe one could start
> >>> with
> >>>      an "opt
> >>>      >     in" model.
> >>>      >
> >>>      >
> >>>      > Okay, so basically run the fast-test suite, record all that don't
> >>>      pass,
> >>>      > submit a bugzilla ticket stating which unit tests are not
> >>> valid on a
> >>>      > certain c++ compiler, then bring CI Testing online using the
> >>> valid
> >>>      > subset of fast-tests. This should work.
> >>>      >
> >>>
> >>>      Sounds good.
> >>>
> >>>      Just to be clear: the above includes extending the DPDK build
> >>> system to
> >>>      build the app/test/dpdk-test binary in two versions: one C and
> >>> one C++,
> >>>      so that anyone can run the C++ tests locally as well. Correct?
> >>>
> >>>
> >>> Okay, so now I am understanding this is not yet available. When I
> >>> responded this morning I was figuring that c++ compiler support was
> >>> available and I simply wasn't aware, and that we could quite easily set
> >>> cc={some c++ compiler}, meson would pick it up, and we would be able to
> >>> build DPDK and then run unit tests in this manner in CI testing.
> >>>
> >>> I didn't mean to suggest we would submit patches extending the build
> >>> system to this end. That's probably a little out of scope for what we
> >>> try to accomplish at the Community Lab.
> >>>
> >>> But if the aforementioned build system support is added, of course we
> >>> are willing to add that as a build environment for unit tests and report
> >>> those respective results.
> >>>   
> >>
> >> Does it have to be 'app/test/dpdk-test', why not build examples with C++?
> >>
> > 
> > The unit tests have the ability to test DPDK, which is exactly what we
> > want to do here. Such testing isn't limited to "compiles yes/no", but to
> > detect run-time (behavioral) issues, and properly report them.
> > 
> > This is especially important for cases where there is code only
> > exercised in C++ translation units (i.e., in #ifdef __cplusplus).
> > 
> 
> And Bruce highlighted that compile check is already covered.
> 
> Than I guess this work needs to be done in two steps,
> 1. Enable building dpdk-test (or all applications) with C++ in build
> system. And fix possible issues.
> 
> 2. Enable in dpdk-test C++ build and run in CI.
> 
> We need a volunteer for 1. before asking CI lab for 2.
> 

For testing with C++ we only need to cover code contained in header files.
Code for functions built into the DPDK .so or .a libraries will behave the
same way when called from C, C++ or any other language. It's the inline
functions in headers that will be compiled differently in C++ so we should
only look to those tests rather than trying to make the whole unit test
suite buildable via C++ IMHO.

/Bruce