X.org testing with Google Test

[Ian Romanick]

On 12/21/2011 08:51 AM, Thomas Voß wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi list,
>
> my name is Thomas and I'm a software engineer in quality working for
> Canonical. Our quality team is currently extending the test coverage
> of Canonical's upstream projects. To this end, we are developing a
> number of components for providing a unified testing landscape to
> developers.
>
> One of our focus areas is the headless automated testing of
> applications and components that interact with the X display server.
> For many testing scenarios, only a dummy X server without any display
> capabilities is required and we abstracted/integrated start up and
> tear down of such a configuration within a Google Test environment
> that is readily usable by developers (see [1]).
>
> Google Test [2] is a unit testing framework tailored towards C and
> C++. It is lightweight and provides a decent set of functionality for
> writing unit tests:

I'm in favor of gtest.  I've just started using it with Mesa, and I have 
some (unposted) unit tests for the GLX_ARB_create_context patch series 
that I just posted to the list.

>    * Automatic test registration: Tests are automatically discovered and
>      recognized. That is, no manual maintenance for executing tests is
>      required.
>
>    * An extensive set of assertion macros is provided.
>
>    * The XML output is compatible with JUnit's XML dialect and can be
>      integrated easily with continuous integration systems (e.g. Jenkins
>      [3]).
>
>    * Support for custom test fixtures that unify test setup and cleanup
>      procedures, thus avoiding boiler plate code within the actual test.
>
>    * Support for custom test environments for bundling problem specific
>      setup and cleanup routines.

The big advantage of a test framework like gtest is that the 
infrastructure makes it easy to write a large number of small, easy to 
understand (WTF is this testing?), easy to debug (WTF is this failing?) 
tests.  For the work mentioned above, I wrote about 75 tests.  It wasn't 
until I was done that I even realized how many there were.  This seems 
like a good thing.

> Our extension to Google Test defines a custom test environment and a
> custom test fixture that take care of starting up and tearing down a
> dummy xserver environment (environment.h) as well as opening up a
> display connection (test.h). Moreover, a custom main is included
> within our extension that extends Google Test's main and makes sure
> that our dummy X server testing environment is known to the whole test
> suite. Please see [4] for a usage scenario within utouch-frame and [5]
> for xorg-gtest's doxygen documentation.
>
> As we think that this kind of unified testing architecture might be of
> interest to a wider developer audience, we would like to bring it to
> your attention by this mail. The project is currently hosted in
> cndougla's freedesktop.org place, but we feel a better home would be
> on freedesktop.org. It looks like a good landing spot would be in
> xorg/test on the git.freedesktop.org page. Is there a formal process
> for proposing projects, or should we just create a repository there?
>
> Of course, your feedback is highly appreciated.
>
> Kind regards,
>
>    Thomas
>
> [1] http://cgit.freedesktop.org/~cndougla/xorg-gtest/
> [2] http://code.google.com/p/googletest/
> [3] http://jenkins-ci.org/
> [4]
> http://bazaar.launchpad.net/~utouch-team/utouch-frame/trunk/view/head:/test/x11/recording.cpp
> [5] http://people.freedesktop.org/~cndougla/xorg-gtest
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQEcBAEBAgAGBQJO8g6vAAoJEClhAuflZ3WhCpAH/jIBuJqKQS9tSP8YBfvNDSS1
> JNQVeLD7KEZq5wD6la0qXWOHeGYQvW632aEU9ZPM3qLPq4zTPI7PvaRULSR/w1Dk
> 1XohweAM1iCOl5IYe7GAwUpoMHiRVTGW9hHaHxBz6NDMOd0OxymuSFvETG0v6lYU
> M9MyTGNvfXnRDq84GhRVXY+qZhXz0waEGXSwdvAg22LT+4mJdaqaN3SaQx9gPi7W
> tUr+3xOaj8n8GqI0jcTWls/+QplAd1gIpD9NJFGI7oISfqEZTDc0Rmot8Q6zK1sF
> ejTI/RobsIpI+NYwljmBuUSXX2S2Had36qJrQY1P4hqpVtpjEzn9PzaCs/heqUU=
> =P6J9
> -----END PGP SIGNATURE-----
> _______________________________________________
> xorg-devel at lists.x.org: X.Org development
> Archives: http://lists.x.org/archives/xorg-devel
> Info: http://lists.x.org/mailman/listinfo/xorg-devel