[PATCH macros] Add XORG_ENABLE_INTEGRATION_TESTS
chase.douglas at canonical.com
Mon Mar 12 12:29:15 PDT 2012
On 03/11/2012 06:25 AM, Gaetan Nadon wrote:
> On 12-03-10 09:02 PM, Chase Douglas wrote:
>> On 03/09/2012 05:33 PM, Gaetan Nadon wrote:
>>> On 12-03-09 05:49 PM, Chase Douglas wrote:
>>>> Copied from XORG_ENABLE_UNIT_TESTS.
>>>> Signed-off-by: Chase Douglas<chase.douglas at canonical.com>
>>>> I pasted in a minimum version of 1.17.0, but I don't know if that's
>>>> how we do
>>> yes, there is a bit of guess work here but it is the right number.
>>>> This will be needed for xorg-gtest and integration testing in other
>>>> xorg-macros.m4.in | 29 +++++++++++++++++++++++++++++
>>>> 1 files changed, 29 insertions(+), 0 deletions(-)
>>>> diff --git a/xorg-macros.m4.in b/xorg-macros.m4.in
>>>> index 4966df3..9cd73c0 100644
>>>> --- a/xorg-macros.m4.in
>>>> +++ b/xorg-macros.m4.in
>>>> @@ -1056,6 +1056,35 @@ AC_MSG_CHECKING([whether to build unit test
>>>> ]) # XORG_ENABLE_UNIT_TESTS
>>>> +# XORG_ENABLE_INTEGRATION_TESTS (enable_unit_tests=auto)
>>>> +# ------------------------------------------------------
>>>> +# Minimum version: 1.17.0
>>>> +# This macro enables a builder to enable/disable integration testing
>>>> +# It makes no assumption about the test cases' implementation
>>>> +# Test cases may or may not use Automake "Support for test suites"
>>>> +# Interface to module:
>>>> +# ENABLE_INTEGRATION_TESTS: used in makefiles to conditionally
>>>> build tests
>>>> +# enable_integration_tests: used in configure.ac for additional
>>>> +# --enable-integration-tests: 'yes' user instructs the module to
>>>> build tests
>>>> +# 'no' user instructs the module not to
>>>> build tests
>>>> +# parm1: specify the default value, yes or no.
>>>> +m4_define([_defopt], m4_default([$1], [auto]))
>>>> + [Enable building integration test cases (default: ]_defopt[)]),
>>>> + [enable_integration_tests=$enableval],
>>>> + [enable_integration_tests=]_defopt)
>>>> + [test "x$enable_integration_tests" != xno])
>>>> +AC_MSG_CHECKING([whether to build unit test cases])
>>>> +]) # XORG_ENABLE_INTEGRATION_TESTS
>>>> # XORG_WITH_GLIB([MIN-VERSION], [DEFAULT])
>>>> # ----------------------------------------
>>>> # Minimum version: 1.13.0
>>> You must have considered using ENABLE_UNIT_TESTS but came to the
>>> conclusion that it would not be suitable. On the other hand it seems to
>>> do exactly the same thing. Can you share your conclusions?
>>> The motivation behind ENABLE_UNIT_TEST is to solve the problem where a
>>> few modules started adding unit testing, each of them using a different
>>> configuration option, working a little differently. This makes it hard
>>> for builders to run unit testing on all modules as they must read and
>>> understand each module code and write their build script accordingly. No
>>> ones fault, it's the nature of X being distributed in over 240 separate
>>> The only mission of the macro is to provide a common configure option
>>> (--enable-unit-tests) with a matching ENABLE_UNIT_TESTS variable.
>>> Nothing else is planned for this macro. It does not require or depend on
>>> XORG_WITH_GLIB or XORG_LD_WRAP.
>>> I am not familiar with gtest, it's hard for me to say if
>>> --enable-unit-tests would be appropriate. You can look at xserver and
>>> libXt as examples of modules using it.
>> The issue is that integration testing usually has different
>> dependencies from unit testing. Unit testing usually has no
>> dependencies, or maybe a dependency on a test framework. However,
>> integration testing usually has dependencies of a working stack. For
>> example, input integration testing for the X server will depend on:
>> root privileges, or special udev rules
>> And development headers for it all. Integration testing for other
>> subsystems would have different stack requirements.
>> One example where builds will want unit tests but not integration
>> tests is package builds. Debian package builds build and run the unit
>> tests during the build and fails the build if the unit tests fail.
>> However, integration tests during debian package builds isn't really
>> feasible without potentially opening security holes.
>> I think unit and integration tests are sufficiently different that a
>> separate configuration option is required.
> This is what I was hoping for. I assume that this new macro can be used
> by many other packages, not just xorg-gtest.
> Would you mind adding the description in the commit text? I'd like to
> avoid others to randomly pick integration test macro over unit test
> macro or vice-versa. This is all the documentation we've got, aside from
> this list.
> Feel free to push. I'll make a release after I see a few runs on tinderbox.
> Reviewed-by: Gaetan Nadon<memsize at videotron.ca>
I've pushed it. Please review my last two patches on the list before
releasing 1.17.0. They are required for proper xorg-gtest setup in
More information about the xorg-devel