Some thoughts on the modularization effort

[Kean Johnston]

> The automake argument I don't have as much experience with so I'd appreciate 
> some input.  The modularised bits we have now use Makefile.am's.  How do we 
> translate them to simple Makefile.in's?  Is there a positive benefit to using 
> automake?
Well, all automake really is is a front-end to generate your Makefile.in
files for you. Your configure.ac files lists the Makefile.in's and they
get substituted. What automake buys you is very simple creation of make
scripts that sorta work. They also give you consistency. Where they are
wrong, they are consistently wrong. Where they are right, they are
consistently right. Theres a lot to be said for consistency. I'm not
suggesting we summarily dismiss automake, not by any means. I just want
to point out the pitfalls I have personally encountered. The other good
thing about automake is its easier to maintain a Makefile.am than it is
to maintain a Makefile.in, there's no doubt about it. But hey, we've all
been used to maintain Imakefiles ... from that vantage point, working
with a Makefile.in is a walk in the park ;)

But the ease of use with automake comes at a price (the old truism
"anything thats too good to be true, isn't" applies here). You are
locked into a set of macro names that may not be to your chosing.
You have the added overhead of having to regenerate every Makefile.in
in teh system whenever you change tools, or discover bugs, or whatever.
Invariably, you end up with Makefiles that require GNU make. Thats not
a bad thing, and most if not all systems these days have it, but I
just want to make sure thats what you intended. The Makefiles that are
generated are very ... how shall I put this ... GNU centric. They
follow the GNU conventions of target names (again, not a bad thing,
just know what you're getting).

Mostly, though, its hard to debug. When I first started programming,
my mentor gave me a guiding maxim that has done me in very good
stead these last 20 years: "code for your debugger". While it may
impress the folks at the obfuscated C contest if you can write a
windowing system in 9 bytes of code (exageration intentional), its
not very usable. With a project teh size and scope of X.org, where
many maintainers come and go over the years, the simpler things are
the better off the project is. Now granted you dont use a debugger
on Makefiles, but if something goes wrong, and some target isn't
building correctly, or it doesn't pick up the right CFLAGS or
whatever, you have to look at the Makefile. The stuff thats generated
by automake is not *meant* to be human legible. The optimistic
assumption is that any problems will be obvious and simple to
edit in the Makefile.am. But in order for you to find the problem,
you now have to have intimate knowledge of how automake works, and
the multiple layers of transformation that you go through from
source to usable file.

Contrast that with using a Makefile.in. All a Makefile.in is is a
simple makefile that allows for variable substitution. You can
write very clear, very well commented Makefiles. The comments
survive (they may survive in automake too, but there is so much
other stuff thats injected that they are almost useless as the
comments lose their context). In fact, the way this could most
effectively be done is to not even use Makefile.in in each
subdirectory. Use a normal Makefile, and have it include a
Makefile.defs or some such that defines all of the standard targets,
sets up the defaults production rules, etc etc.

This last was the path I chose for a large project I maintain.
Its a collection of about 80 open source libraries, all built
together as one cohesive whole, with all the interdependencies
nicely worked out for you. It eliminates the problems that have
plagues Linux and BSD distros for years, where you have to
download a plethora of packages, all from different sources,
some of which rely on older versions, some which want newer
ones etc. Its a huge system (about 380MB of source in all), and
I tried the automake route. I *desparately* wanted it to work
becuase I thought it would make life easier. Turns out it did
not, and the approach I just mentioned made maintenance a whole
bunch easier. I still used Makefile.in's, but they all include
a glonal Makefile.top that has a lot of the common stuff in it.

One final "problem" with automake is its impact on source
control. Its a small nit, I grant you, but it has caused more
than one open source project I've worked on problems. You now
have to maintain two files in your SCM.  You have to remember
to regenerate them every time you check stuff in. People get
all inventive with SCM triggers to do it automatically, but
that sometimes fails too. You have to be extremely careful about
the machines that you use to generate source archives, and make
sure that they have the exact right version of automake. You
also have to make sure that every developer has the same version,
and thats problematic.

I totally buy the "its easy" argument. It is, but that ease
comes at a price. Lets just make sure we're prepared to pay
it before too much work gets done and people get automake happy :)

> I'll throw out some data points here: modern linux systems ship with only two 
> versions of autoconf, but as many as five versions of automake.  And by far 
> the most common build problems people have with the modular bits we have stem 
> from using the wrong version of automake.
I rest my case :)

> We can definitely lean on the autoconf people to get that merged.
Its attached. All it does is ensure that the sometimes very very
long list of substitution variables or config files get generated
one to a line, rather than as a single huge line, which overflows
some shells. The assumption is that the shell that you use to
generate the autoconf script on in the first place doesn't have
such limitations, which is almost always true as almost everyone
generates this stuff on Linux where bash is the shell of choice.
But it does generate more friendly configure scripts, so it makes
the result more usable than on the environment it was generated
on.

Kean
-------------- next part --------------
--- lib/autoconf/general.m4.osd	2004-02-17 06:40:41.000000000 -0800
+++ lib/autoconf/general.m4	2004-02-17 06:46:56.000000000 -0800
@@ -422,8 +422,10 @@
 
 m4_divert_pop([DEFAULTS])dnl
 m4_wrap([m4_divert_text([DEFAULTS],
-[ac_subst_vars='m4_ifdef([_AC_SUBST_VARS],  [m4_defn([_AC_SUBST_VARS])])'
-ac_subst_files='m4_ifdef([_AC_SUBST_FILES], [m4_defn([_AC_SUBST_FILES])])'])])dnl
+[m4_ifdef([_AC_SUBST_VARS], [m4_foreach([_AC_SPLIT_SUBST_VARS], m4_split(m4_normalize(_AC_SUBST_VARS)),[ac_subst_vars="$ac_subst_vars _AC_SPLIT_SUBST_VARS"
+])], ac_subst_vars='')
+m4_ifdef([_AC_SUBST_files], [m4_foreach([_AC_SPLIT_SUBST_FILES], m4_split(m4_normalize(_AC_SUBST_FILES)),[ac_subst_files="$ac_subst_files _AC_SPLIT_SUBST_FILES"
+])], ac_subst_files='')])])dnl
 ])# _AC_INIT_DEFAULTS
 
 
--- lib/autoconf/status.m4.osd	2004-02-17 06:40:47.000000000 -0800
+++ lib/autoconf/status.m4	2004-02-17 06:43:27.000000000 -0800
@@ -1324,19 +1324,27 @@
 
 # Files that config.status was made for.
 if test -n "$ac_config_files"; then
-  echo "config_files=\"$ac_config_files\"" >>$CONFIG_STATUS
+  for cfgfile in $ac_config_files; do
+    echo "config_files=\"\$config_files $cfgfile\"" >>$CONFIG_STATUS
+  done
 fi
 
 if test -n "$ac_config_headers"; then
-  echo "config_headers=\"$ac_config_headers\"" >>$CONFIG_STATUS
+  for hdrfile in $ac_config_headers; do
+    echo "config_headers=\"\$config_headers $hdrfile\"" >>$CONFIG_STATUS
+  done
 fi
 
 if test -n "$ac_config_links"; then
-  echo "config_links=\"$ac_config_links\"" >>$CONFIG_STATUS
+  for lnkfile in $ac_config_links; do
+    echo "config_links=\"\$config_links $lnkfile\"" >>$CONFIG_STATUS
+  done
 fi
 
 if test -n "$ac_config_commands"; then
-  echo "config_commands=\"$ac_config_commands\"" >>$CONFIG_STATUS
+  for ccmd in $ac_config_commands; do
+    echo "config_commands=\"\$config_commands $ccmd\"" >>$CONFIG_STATUS
+  done
 fi
 
 cat >>$CONFIG_STATUS <<\_ACEOF