Modularization mailing list and initial strawman proposal

[Daniel Stone]

On Fri, Mar 18, 2005 at 09:39:32PM +0100, Roland Mainz wrote:
> Kevin E Martin wrote:
> >   Introduction
> [snip]
> >     The cons
> > 
> >      * The autotools are a new build technology, which throws away
> >        existing knowledge of the current imake build system. People that
> >        are not already familiar with the autotools will need to learn a
> >        new build system. That could slow some developers down for a
> >        while.
> 
> Additionally there is the question whether all platforms supported by
> Imake are supported or can be supported by design (OpenVMS and QNX come
> in mind where "autoconf" support can only be described as poor ...).
> What should we do here ? The worst case would be that X.org quickly
> becomes "upstream" for autoconf support on these platforms... =:-)

... which is no better than what we have today with imake.  As it is,
we're already upstream for every platform.  Surely if there are problems
and people genuinely need them resolved, then either a) they will let us
know, or b) they will let us know ... with patches!

I'm not convinced that our imake support for most platforms is very good
at all (it's really bad for Linux, actually), so I honestly don't see
this as a problem.

If the worst case is that we're 'upstream', then so be it -- it's not
any worse than what it is now.  The best case is that the autotools
guys, in conjunction with someone who knows the platforms, are upstream
for it.

> >      * To address this issue, we need people who understand both imake
> >        and autotools well to write a transition guide for the benefit of
> >        all.
> > 
> >    Some platforms may not "come along" if there are no maintainers.
> >      * This should be treated as an opportunity for us to contact the old
> >        maintainers and see if they are interested joining the community,
> >        or for us to encourage to new maintainers to step forward to
> >        support platforms if the old maintainer is no longer interested or
> >        available.
> 
> What happens if the maintainers of these platforms do not have the
> resources (e.g. time) to do the transition ? The Imake and Imakefiles
> exist since many many years and it is VERY unlikely that we can expect
> to pull over all changes to all platforms within just ... say... three
> months (assuming the X11R7 release should appear in summer 2005). That's
> pretty much impossible.

I honestly don't think we will have to.

One of the good things about autotools is that it's not a static list of
definitions.  imake says 'if we're using NetBSD, we'll use our own
malloc() implementation because someone sent in a patch to it in 1997
that was never merged, but broken'.  autotools goes and tests if malloc
is broken, and works around it if so.

I don't think we will have to do much porting at all.  But if there is,
I question that ports viability at all -- if the most popular build
system (at least for open source platforms) cannot be ported due to
lack of interest, surely we won't lose many users at all.

If the first release regresses a tiny bit of support, so be it.  We
can't expect everyone to be on the bleeding edge, and I don't think we
should stop moving and wait for the entire world to catch up.  If they
want to move with us, then they will do it eventually, regardless of
when we release.  If the port is dead enough that it won't get moved
over, then it won't get moved over and that port will never work.

But if that happens, then that port has far, far bigger problems than
not being able to run the latest X.Org.

> The alternative reaction by such maintainers may be to ignore the
> existance of the modular tree and move over to other development
> platforms which are still compatible with their OSes (such as Xfree86)
> ...

They can do so if they want to.  People can still move to XFree86 now.

We can't stop them; it's their choice.

> >    Autodetection can add dependencies that you didn't intend. For
> >    example, a developer may have a library on their build system that
> >    isn't available on the target system.
> >      * Dependencies will be explicitly listed and documented. Configure
> >        options will be used to control which dependencies are enabled or
> >        disabled.
> 
> How many "configure" options will that be ? 50... 100... 200 ? It may be
> better to look into a way of a config file a lá xc/config/cf/host.def to
> avoid having giant command lines which call "configure" (mozilla has a
> non-standard solution which may be adoptable in this case).

echo './configure --disable-xcomposite --disable-xdamage
--disable-xfixes --disable-xaw --enable-xorg-drivers=radeon,vesa' > foo
$(cat foo)

> >    With the imake build system, you can build and test in place with the
> >    link directory inside the tree. Building self-contained user
> >    environments are more difficult with autotools.
> >      * jhbuild solves this since it runs a self-contained environment.
> >        Other solutions using chroot, LD_LIBRARY_PATH, etc. are also in
> >        use today with autotools.
> 
> "chroot" is no option. It MUST be possible to build and test the tree
> without having "root" (or similar) priviledges. Otherwise it will be
> impossible to do testing on systems like "compile farms" (such as
> sourceforge's compile farm) as they (usually) NEVER grant "root" access.

'jhbuild solves this since it runs a self-contained environment.'

I have been installing into ~/x (or ~/install/x, or whatever) for around
a hojillion years now.  You just need to mangle your own LD_LIBRARY_PATH
if you're not root, but hey, you need to do that with imake anyway.

autotools can install into $PREFIX, and that works out of the box.  So
it's not a problem.  Next question.

> >    Note that some platforms only have 1 or 2 component revisions for
> >    their libraries. On these systems, either the second component (y)
> >    could be bumped, or symbol versioning could be used by vendors who
> >    package the libraries.
> > 
> >    By default, all libraries are built as shared objects; however, with
> >    other build options, static, profiled and debug versions can be built.
> 
> What about libraries which "unstable" APIs which are currently build as
> "static" versions for that reason (XprintUtils comes in mind from my
> side) ?

I'm going to go out on a limb here: that's stupid.

I don't think Xinerama should have ever been static.  Or XprintUtils.
Or Xxf86vm.

If you have a stable API, then show it to the world.  If you need to
change that API, then change it and adjust your soversion accordingly.

If you have an unstable API that is so undeveloped that you cannot even
ship a shared library, then I ask that you do not ship it with the
standard X distribution.  Either something is ready to ship, or it is
not.  Static libraries are a totally invalid argument because you will
need to change all the client callers if you change your API anyway,
so what's the point of keeping library portability?

I don't think such libraries belong in the tree.

> And what about helper libraries which are currently used to
> build the Xserver ?

Before you continue on any further, can I please ask you to have a look
at Debrix at some point?  Debrix builds static-only convenience
libraries as intermediate steps towards the final server binary.

Most of the things you are concerned about aren't problems, because they
have already been solved; so, to prevent anyone wasting any time, I
strongly suggest you check out the xlibs and debrix trees, and see how
they work.

> >      * As noted above, some DDXs require different built options, which
> >        implies that not all DDXs will be able to use the same dix, os,
> >        mi, etc. compile.
> 
> Why is this a problem for the modular build ? Right now all servers
> (Xorg, Xnest, Xdmx, Xprt, Xvfb) can be build in one "make World" step.
> The same should be possible in the "modular" world.

KDrive, as it stands today, cannot coexist with Xorg, et al.  I mean in
the xserver repository, not in Xorg.

> >        However, they can and should share the same
> >        source code. For example, the kdrive server does not support
> >        XINPUT whereas the Xorg server does, so dix (and other subdirs)
> >        will need to be built differently for each server. Note that these
> >        conflicts can be detected at configure time.
> 
> See above. There should be no such seperation - the "monolithic" Xorg
> tree didn't need it so why should the modular build system introduce
> such a pain (currently stub functions and *.a files are used to avoid
> this and the same could be done for the modular build) ?

The monolithic Xorg tree does not need it because it is incapable of
building all of today's X servers out of the same tree.

This problem is not solved in the monolithic tree, and it's not solved
in the modular tree, either.

> >      * The basic tree structure in the xserver module will look similar
> >        to the one used in the monolithic tree under xc/programs/Xserver.
> >        Notable exceptions include the drivers that will be moved to their
> >        own module (see below), and support programs that could be moved
> >        to the app module.
> > 
> >    The XFree86 drivers module: xfree86-driver
> 
> Why can't it be named "drivers" (yes, I know, "Xfree86" should be
> referenced but that can be done in the docs) ? I don't like to have
> giant-overlong-pathnames and the drivers/ subdir will likely host other
> kinds of drivers like the kdrive, Sun DDX or the Xprint ones...

Because 'drivers' is a really bad name.

Drivers for what?  Cars?  ALSA?  KDrive?  libX11's overengineered
localisation structure?

> >      * Vendors have expressed a strong desire to be able to ship updates
> >        to individual drivers as bugs are fixed, so they have been split
> >        out into their own module.
> >      * The drivers consist of both input and output (i.e., video)
> >        drivers.
> 
> And print drivers as the Xprint DDXs should be turned into loadable
> modules (in the long term the Xprt server is going away then[1],
> replaced by a Xorg server which can handle both video and print modules
> similar to what the Xserver in HP/UX can do today).

Yes, but no code has yet been shown to this end, that I know of.  It has
also shown to be a very difficult problem -- looking at any application
that uses Xprint, the fundamental differences between display and print
drivers are exposed to the application.  I don't see how this could
cleanly work in the server.

> >    The documentation module: doc
> >      * This module contains the documentation traditionally distributed
> >        with the monolithic releases that has not been moved to another
> >        module.
> >      * The majority of documentation in xc/doc will be relocated to the
> >        module that they document. For example, the protocol
> >        specifications will be moved to the proto module and the library
> >        docs will be moved to the lib module.
> 
> If this is done then we need a common build infrastructure to build the
> documentation in all places (this needs to be done in general a lá
> xc/config/cf/X11.tmp && xc/config/X11/Imake.* to have centralised place
> for all the build rules to avoid that they are replicated all over the
> source tree).

Why?  If I'm building only libX11 and below out of the libraries (e.g.
I'm not building libXt), why would I want the documentation for libXt
and libXaw?  If you're building every module, then you are building
every module nonetheless, and the documentation will just come with
that.

As for documentation rules, we don't really need to do anything here.
External projects like autotools, doxygen, whatever, have already
solved this problem, and we don't need to do any more than supply a
list of files.

Remember, we're not writing and maintaining our own build system here.
Someone else is doing that for us.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.x.org/archives/xorg-modular/attachments/20050320/c238d38a/attachment.pgp