BuildingX script from the Wiki

Michael Witten mfwitten at gmail.com
Fri May 28 20:23:01 PDT 2010


On Sat, May 22, 2010 at 05:24:59PM -0400, grarpamp wrote:

>> Guess I was looking for a general one-shot build
>> script... give it a work dir, get from repos,
>> configure and build, end up with some sort of
>> installable binaries/tarballs or a runnable
>> hierarchy already installed in a PREFIX. The old
>> build scripts I wrote for XFree86 don't exactly work
>> with xorg :-) Getting back into X and the modular
>> build has made more up front work as I'm totally in
>> the dark about build order and dependencies. But as
>> usual, once I iterate and make/find a build script
>> things will be cool.

On Sun, May 23, 2010 at 03:30, Dirk Wallenstein <halsmit at t-online.de> wrote:

> There's work in progress [1]. A new version of
> x-jhbuild is coming soon, with lots of improvements.
> I'm currently writing the announcment where I will
> try to convince how much fun auditing and testing
> will be, by means of writing small overriding
> modulesets. There are some patches to jhbuild which I
> consider crucial at [2].

I think that standardizing the build process is a very
good idea, and it would probably be best to get behind
a sophisticated, extensible system like the one you're
promoting.

However, I'll take this opportunity to submit a script
that I've been sitting on for a while and that might
make a decent interim replacement for the simple,
self-contained script provided by the wiki (a copy is
also inlined at the end of this email):

    http://pastie.org/private/kho0jo73kwxkmoxgcbbi2w

This script was inspired by the one on the wiki page,
but it has been written in a way that's meant to
promote generalization and necessary specialization,
and I think it's pretty useful for such a relatively
small script.

Though I wrote it many months ago (and hadn't used it
in earnest for as long), I was able to update it so that
it easily manages the latest commits for Xorg's module
repositories; by using this script, I just today cloned,
built, and installed into a custom prefix 52
seemingly-working Xorg modules.

If you run the script without parameters (or the wrong
parameters), you'll get a help message like the following,
which pretty much explains everything:

    Usage: bash '/path/to/script' <command> [<module0> <module1> ...]

    This program automates fetching, building, and
    installing Xorg module sources. Edit the configuration
    section of the source of this program before continuing.

    There are 2 organizational concepts:

        * topdir : The current working directory from which
                   this program is run.

        * module : A hierarchical Xorg module's name, such
                   as 'xorg/proto/fixesproto'; this name
                   is also used as a topdir-relative path
                   for the module source repository with
                   which this program deals.

    Essentially, choose a directory in which to work;
    this directory is 'topdir'; then run the following
    to copy the modules to topdir as well as configure,
    make, and install them:

        bash '/path/to/script' clone
        bash '/path/to/script' cmi

    To use the installed binaries, set LD_LIBRARY_PATH
    and PATH appropriately first; experiment with Xephyr.

        export LD_LIBRARY_PATH=$PREFIX/lib
        export PATH=$PREFIX/bin:$PATH
        Xephyr :1 &
        DISPLAY=:1 xeyes &

    The <command> only operates on those modules supplied
    as optional arguments on the command line. If a list of
    modules is not supplied, then it is as if all of the
    modules in the 'MODULES' array are supplied.

    For each module supplied, the <command> performs
    the task described:

        clone   : Copy the official source repository
                  to the local machine; the path of
                  this copy is the same as the module
                  name and is relative to topdir.

        pull    : fetch and merge the latest commits
                  and tags from the official source
                  repository.

        config  : Configure the module for building.

        make    : Build the source into binaries/products.

        install : Copy the products into "$PREFIX".

        cmi     : Run 'config', 'make', and 'install'.

        mi      : Run 'make' and 'install'.

        clean   : Remove intermediaries and products from
                  the build.

I hope somebody finds this script useful.

Sincerely,
Michael Witten

#!/usr/bin/env bash

# Copyright 2010 Michael Witten <mfwitten at gmail.com>
# 
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
# files (the "Software"), to deal in the Software without
# restriction, including without limitation the rights to use,
# copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following
# conditions:
# 
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
# 
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.

### Assumptions:
###   * Only relatively recent commits will be used (pull only gets HEAD);
###     build requirements and steps are different for older ones.
###   * The user has the necessary non-Xorg dependencies (python, libpthreads,
###     intltool, etc.; see comments by MODULES elements).
###   * The user has write permissions to "$PREFIX"
###   * The user can sudo (for chown root "$PREFIX/bin/Xorg")
###   * GNU coreutils (specifically 'install') is available
###   * GNU/Linux is the OS (though this script should be easy 'to port').

#######################
#### Configuration ####
#######################

    PREFIX=~/testroot/usr

    #FONT_DIR=$PREFIX/usr/share/fonts
    FONT_DIR=/usr/share/fonts            # Use existing fonts

    #DRM_OPTIONS=(intel radeon nouveau vmwgfx kms udev)
    DRM_OPTIONS=(radeon kms udev)

    MAKE=make

    export ACLOCAL="aclocal -I '$PREFIX/share/aclocal'"
    export PKG_CONFIG_PATH=$PREFIX/lib/pkgconfig:$PKG_CONFIG_PATH

    export CFLAGS=-I$PREFIX/include
    export CPPFLAGS=$CFLAGS

    export LD_LIBRARY_PATH=$PREFIX/lib

    export PYTHONPATH=$PREFIX/lib/python2.6/site-packages

# Select the repos (specify the input/video X drivers at the
# bottom of MODULES); the modules are built in the order given
# (naturally, build only what is necessary; the user can specify
# on the command line which modules to build):

    REPOS_BASE=git://git.freedesktop.org/git

    MODULES=(
        pixman
        mesa/drm                    # Requires [lib]pthread-stubs
        mesa/mesa
        mesa/demos
        xcb/proto
        xcb/pthread-stubs
        xcb/libxcb
        xkeyboard-config             # requires intltool
        xorg/util/macros
        xorg/proto/x11proto
        xorg/proto/damageproto
        xorg/proto/xextproto
        xorg/proto/fontsproto
        xorg/proto/videoproto
        xorg/proto/renderproto
        xorg/proto/inputproto
        xorg/proto/xf86vidmodeproto
        xorg/proto/xf86dgaproto
        xorg/proto/xf86driproto
        xorg/proto/xcmiscproto
        xorg/proto/scrnsaverproto
        xorg/proto/bigreqsproto
        xorg/proto/resourceproto
        xorg/proto/compositeproto
        xorg/proto/fixesproto
        xorg/proto/evieproto
        xorg/proto/kbproto
        xorg/proto/randrproto
        xorg/proto/dri2proto
        xorg/proto/glproto
        xorg/proto/xineramaproto
        xorg/lib/libpciaccess
        xorg/lib/libxtrans
        xorg/lib/libX11
        xorg/lib/libXrender
        xorg/lib/libXext
        xorg/lib/libXi
        xorg/lib/libxkbfile
        xorg/lib/libfontenc
        xorg/lib/libXfont
        xorg/lib/libXft
        xorg/lib/libXfixes
        xorg/lib/libXdamage
        xorg/lib/libXv
        xorg/lib/libXvMC
        xorg/lib/libXxf86vm
        xorg/lib/libXau
        xorg/lib/libXinerama
        xorg/xserver

        # Drivers/apps are necessarily built after xorg/xserver,
        # because they make use of the `SDK' headers:

        #xorg/driver/xf86-input-mouse
        #xorg/driver/xf86-input-keyboard
        #xorg/driver/xf86-video-intel
        xorg/driver/xf86-input-evdev
        xorg/driver/xf86-video-ati
        xorg/app/xkbcomp 
    )

# There are only a few modules that require special attention in
# various build phases. To identify these modules easily, each
# special module's name is used as a key in an associative
# array (the value is arbitrary, so long as it's non-empty).

    unset -v SPECIAL
    declare -A SPECIAL      # Create associative array

    SPECIAL=(

        [mesa_drm_config]=1

        [config_mesa_config]=1

        [xorg_xserver_config]=1
        [xorg_xserver_install]=1

    )

# Each module that requires special attention has its specialized
# handlers defined here with functions named similarly; they are
# run from within the top level of the module's cloned repo:

    #### mesa/drm ####

    mesa_drm_config()
    {
        print_heading Configuring

        declare -a OPTIONS

        in_array intel   "${DRM_OPTIONS[@]}"  &&
            OPTIONS+=(--enable-intel)         ||
            OPTIONS+=(--disable-intel)

        in_array radeon  "${DRM_OPTIONS[@]}"  &&
            OPTIONS+=(--enable-radeon)        ||
            OPTIONS+=(--disable-radeon)

        in_array vmwgfx  "${DRM_OPTIONS[@]}"  &&
            OPTIONS+=(--enable-vmwgfx-experimental-api)   ||
            OPTIONS+=(--disable-vmwgfx-experimental-api)

        in_array nouveau "${DRM_OPTIONS[@]}"  &&
            OPTIONS+=(--enable-nouveau-experimental-api)  ||
            OPTIONS+=(--disable-nouveau-experimental-api)

        in_array udev    "${DRM_OPTIONS[@]}"  &&
            OPTIONS+=(--enable-udev)          ||
            OPTIONS+=(--disable-udev)

        in_array kms     "${DRM_OPTIONS[@]}"  &&
            OPTIONS+=(--enable-libkms)        ||
            OPTIONS+=(--disable-libkms)

        ./autogen.sh                          \
            --prefix="$PREFIX"                \
            "${OPTIONS[@]}"                   || return 1
    }

    #### mesa/mesa ####

    mesa_mesa_config()
    {
        print_heading Configuring

        ./autogen.sh --prefix="$PREFIX"  \
                     --with-driver=dri   || return 1
    }

    #### xorg/xserver ####

    xorg_xserver_config()
    {
        print_heading Configuring

        ./autogen.sh --prefix="$PREFIX"         \
                     --with-fontdir="$FONT_DIR" \
                                                \
                     --enable-glx-tls           \
                     --enable-record            \
                     --enable-config-dbus       \
                                                \
                     --enable-xorg              \
                                                \
                     --enable-kdrive            \
                     --enable-xephyr            \
                     --disable-xsdl             \
                     --disable-xfake            \
                     --disable-xfbdev           || return 1
    }

    xorg_xserver_install()
    {
        print_heading Installing

        "$MAKE" install                     || return 1

        FINALIZATION_HANDLERS+=(xorg_xserver_install_finalize)
    }

    xorg_xserver_install_finalize()
    {
        echo -e "\nIt is necessary to set the setuid and ownership on the server;"
        echo -e "'sudo' will be used.\n"

        sudo chown root "$PREFIX/bin/Xorg"  || return 1
        sudo chmod +s   "$PREFIX/bin/Xorg"  || return 1 # This must come second
    }

# It may be useful to specify an array of handlers to be called
# after all is said and done, perhaps as a finalization or
# cleanup; for instance, the specialized build phase handlers
# previously defined can add handlers to this array to have
# certain operations take place as a final step.

unset -v FINALIZATION_HANDLERS
declare -a FINALIZATION_HANDLERS

################
#### Script ####
################

#### Helper functions ####

print_count() { echo -e "\n>>>>>>>>>>>>>>>>>>>> ($1/$2) $3\n"; }

print_failed() { echo "************** FAILED $1 **************"; }

print_failed_exit() { print_failed "$1"; exit "${2:-1}"; }

print_finished() { echo -e "\nFinished"; }

print_heading() { echo -e "\n//////// $1 ////////\n"; }

print_usage()
{
    cat >&2 <<END
Usage: bash '$0' <command> [<module0> <module1> ...]

This program automates fetching, building, and
installing Xorg module sources. Edit the configuration
section of the source of this program before continuing.

There are 2 organizational concepts:

    * topdir : The current working directory from which
               this program is run.

    * module : A hierarchical Xorg module's name, such
               as 'xorg/proto/fixesproto'; this name
               is also used as a topdir-relative path
               for the module source repository with
               which this program deals.

Essentially, choose a directory in which to work;
this directory is 'topdir'; then run the following
to copy the modules to topdir as well as configure,
make, and install them:

    bash '$0' clone
    bash '$0' cmi

To use the installed binaries, set LD_LIBRARY_PATH
and PATH appropriately first; experiment with Xephyr.

    export LD_LIBRARY_PATH=\$PREFIX/lib
    export PATH=\$PREFIX/bin:\$PATH
    Xephyr :1 &
    DISPLAY=:1 xeyes &

The <command> only operates on those modules supplied
as optional arguments on the command line. If a list of
modules is not supplied, then it is as if all of the
modules in the 'MODULES' array are supplied.

For each module supplied, the <command> performs
the task described:

    clone   : Copy the official source repository
              to the local machine; the path of
              this copy is the same as the module
              name and is relative to topdir.

    pull    : fetch and merge the latest commits
              and tags from the official source
              repository.

    config  : Configure the module for building.

    make    : Build the source into binaries/products.

    install : Copy the products into "\$PREFIX".

    cmi     : Run 'config', 'make', and 'install'.

    mi      : Run 'make' and 'install'.

    clean   : Remove intermediaries and products from
              the build.
END
    exit 3
}

in_array()
{
    local pattern=$1 element
    shift

    for element
    do
       [[ $element = $pattern ]] && return 0
    done

    return 1
}

default_config()
{
    print_heading Configuring
    ./autogen.sh --prefix="$PREFIX"
}

default_make()
{
    print_heading Compiling
    "$MAKE"
}

default_install()
{
    print_heading Installing
    "$MAKE" install
}

#### Main functions ####

run_build_phase_for_module_while_in_module()
{
    local phase=$1 module=$2
    local command=${module/\//_}_$phase

    [[ ${SPECIAL[$command]} ]]     &&
        "$command"       "$module" ||
        "default_$phase" "$module"
}

pull_module_while_in_module()
{
    git pull
}

config_module_while_in_module()
{
    local module=$1
    run_build_phase_for_module_while_in_module config "$module"
}

make_module_while_in_module() {
    local module=$1
    run_build_phase_for_module_while_in_module make "$module"
}

install_module_while_in_module()
{
    local module=$1
    run_build_phase_for_module_while_in_module install "$module"
}

mi_module_while_in_module()
{
    local module=$1

    make_module_while_in_module    "$module" &&
    install_module_while_in_module "$module"
}

cmi_module_while_in_module()
{
    local module=$1

    config_module_while_in_module  "$module" &&
    mi_module_while_in_module
}

clean_module_while_in_module()
{
    git clean -fdx
}

run_command_in_module_while_in_topdir()
{
    local command=$1_module_while_in_module module=$2 topdir=$3
    local no_error=1

    cd "$module"          || print_failed_exit "entering $module"
    "$command" "$module"  || no_error=0
    cd "$topdir"          || print_failed_exit "exiting $module"

    ((no_error))
}

clone_module_while_in_topdir()
{
    local module=$1
    [[ -e "$module" ]] || git clone "$REPOS_BASE/$module" "$module"
}

pull_module_while_in_topdir()
{
    local module=$1 topdir=$2
    run_command_in_module_while_in_topdir pull "$module" "$topdir"
}

config_module_while_in_topdir()
{
    local module=$1 topdir=$2
    run_command_in_module_while_in_topdir config "$module" "$topdir"
}

make_module_while_in_topdir()
{
    local module=$1 topdir=$2
    run_command_in_module_while_in_topdir make "$module" "$topdir"
}

install_module_while_in_topdir()
{
    local module=$1 topdir=$2
    run_command_in_module_while_in_topdir install "$module" "$topdir"
}

cmi_module_while_in_topdir()
{
    local module=$1 topdir=$2
    run_command_in_module_while_in_topdir cmi "$module" "$topdir"
}

mi_module_while_in_topdir()
{
    local module=$1 topdir=$2
    run_command_in_module_while_in_topdir mi "$module" "$topdir"
}

clean_module_while_in_topdir()
{
    local module=$1 topdir=$2
    run_command_in_module_while_in_topdir clean "$module" "$topdir"
}

run_command_in_modules_while_in_topdir()
{
    local i=1 command=$1_module_while_in_topdir fail_handler=$2 topdir=$PWD
    shift 2

    for module in "$@"; do

        print_count $i $# "$module"
        "$command" "$module" "$topdir" || "$fail_handler" "$module"
        ((i++))

    done

    print_finished
}

clone_modules_while_in_topdir()   { run_command_in_modules_while_in_topdir clone   print_failed      "$@"; }
pull_modules_while_in_topdir()    { run_command_in_modules_while_in_topdir pull    print_failed      "$@"; }
config_modules_while_in_topdir()  { run_command_in_modules_while_in_topdir config  print_failed      "$@"; }
make_modules_while_in_topdir()    { run_command_in_modules_while_in_topdir make    print_failed_exit "$@"; }
install_modules_while_in_topdir() { run_command_in_modules_while_in_topdir install print_failed      "$@"; }
cmi_modules_while_in_topdir()     { run_command_in_modules_while_in_topdir cmi     print_failed_exit "$@"; }
mi_modules_while_in_topdir()      { run_command_in_modules_while_in_topdir mi      print_failed_exit "$@"; }
clean_modules_while_in_topdir()   { run_command_in_modules_while_in_topdir clean   print_failed      "$@"; }

#### main ####

# Otherwise ACLOCAL complains
mkdir -p "$PREFIX/share/aclocal"

command_name=$1
[[ $command_name =~ clone|pull|config|make|install|cmi|mi|clean ]] || print_usage
shift

command=${command_name}_modules_while_in_topdir

(($#)) && "$command" "$@" || "$command" "${MODULES[@]}"

for handler in "${FINALIZATION_HANDLERS[@]}"; do
    "$handler" || print_failed "handler $handler"
done


More information about the xorg-devel mailing list