[PATCH build] Separate build and repository functions.

Trevor Woerner twoerner at gmail.com
Fri Sep 17 11:13:43 PDT 2010


From: Trevor Woerner <twoerner at gmail.com>

Pull out the repository (i.e. git) functionality from the build script.
Provide a separate script for running arbitrary commands in each of the
modules which are built.

Signed-off-by: Trevor Woerner <twoerner at gmail.com>
---

The build.sh script is used for building the modular tree. But mixed in with
the *build* script are a couple, arbitrary repository commands: "git clone"
and "git pull --rebase".

But what if someone doesn't like to do "git pull --rebase" and would rather
perform "git fetch" followed by a "git merge"? Or just a "git pull" by
itself? Or what if someone wants to run a "git status" on each module?
Should we keep adding more and more git commands to a script meant for
building?

I think there should be a clear separation between building and working with
the source code repository. Simply removing the git commands from the build
script wouldn't be sensible without providing some sort of replacement. So
I propose this patch which removes the git commands from the build.sh script
but also provides the "xcmd.sh" script.

To use the xcmd.sh script you provide a command (say, "git status") and the
script runs that command in every module. Originally I had named the script
"xgit.sh" but realised that it could be used for any arbitrary command
applied to all modules, including build commands, so I renamed it to be more
generic.

Currently on the wiki, the instructions for obtaining the sources are:
    $> git clone git://anongit.freedesktop.org/git/xorg/util/modular util/modular
    $> util/modular/build.sh --clone -a -n $PREFIX

These steps produce output that looks like:
    Fri Sep 17 13:35:17 EDT 2010
    Cloning into util/macros...
    remote: Counting objects: 421, done.
    remote: Compressing objects: 100% (260/260), done.
    remote: Total 421 (delta 257), reused 265 (delta 161)
    Receiving objects: 100% (421/421), 76.15 KiB | 105 KiB/s, done.
    Resolving deltas: 100% (257/257), done.
    Building util module component macros...
    make: *** No targets specified and no makefile found.  Stop.
    ***** make failed on util/macros
    make: *** No rule to make target `install'.  Stop.
    ***** install failed on util/macros
    Cloning into font/util...
    remote: Counting objects: 225, done.
    remote: Compressing objects: 100% (101/101), done.
    remote: Total 225 (delta 139), reused 200 (delta 123)
    Receiving objects: 100% (225/225), 70.04 KiB | 119 KiB/s, done.
    Resolving deltas: 100% (139/139), done.
    Building font module component util...
    make: *** No targets specified and no makefile found.  Stop.
    ***** make failed on font/util
    make: *** No rule to make target `install'.  Stop.
    ***** install failed on font/util
[...snip...]

To me, seeing these errors doesn't look good, and we need to provide the "-n"
option so the script will keep pushing through the errors.

With my new proposed changes the procedure would be:
    $> git clone git://anongit.freedesktop.org/git/xorg/util/modular util/modular
    $> util/modular/xcmd.sh clone

Which would produce output that looks like:
    --  util/macros  ---------------------------------------------------------
    Cloning into util/macros...
    remote: Counting objects: 421, done.
    remote: Compressing objects: 100% (260/260), done.
    remote: Total 421 (delta 257), reused 265 (delta 161)
    Receiving objects: 100% (421/421), 76.15 KiB | 88 KiB/s, done.
    Resolving deltas: 100% (257/257), done.

    --  font/util  -----------------------------------------------------------
    Cloning into font/util...
    remote: Counting objects: 225, done.
    remote: Compressing objects: 100% (101/101), done.
    remote: Total 225 (delta 139), reused 200 (delta 123)
    Receiving objects: 100% (225/225), 70.04 KiB | 43 KiB/s, done.
    Resolving deltas: 100% (139/139), done.

    --  proto/bigreqsproto  --------------------------------------------------
    Cloning into proto/bigreqsproto...
    remote: Counting objects: 113, done.
    Receiving objects: 100% (113/113), 19.83 KiB, done.  49% (56/113)   
    remote: Compressing objects: 100% (67/67), done.
    remote: Total 113 (delta 50), reused 97 (delta 43)
    Resolving deltas: 100% (50/50), done.
[...snip...]

Then to build (same as before):
    $> util/modular/build.sh --autoresume built.modules $PREFIX

If you wanted to do repository stuff you could do something like:

    $> util/modular/xcmd.sh "git checkout master"
    --  util/macros  ---------------------------------------------------------
    ~/tmp/xorg-build/util/macros ~/tmp/xorg-build
    Already on 'master'
    ~/tmp/xorg-build

    --  font/util  -----------------------------------------------------------
    ~/tmp/xorg-build/font/util ~/tmp/xorg-build
    Already on 'master'
    ~/tmp/xorg-build
[...snip...]

    $> util/modular/xcmd.sh "git pull"
    --  util/macros  ---------------------------------------------------------
    ~/tmp/xorg-build/util/macros ~/tmp/xorg-build
    Already up-to-date.
    ~/tmp/xorg-build

    --  font/util  -----------------------------------------------------------
    ~/tmp/xorg-build/font/util ~/tmp/xorg-build
    Already up-to-date.
    ~/tmp/xorg-build
[...snip...]

    $> util/modular/xcmd.sh "git status"
    --  util/macros  ---------------------------------------------------------
    ~/tmp/xorg-build/util/macros ~/tmp/xorg-build
    # On branch master
    nothing to commit (working directory clean)
    ~/tmp/xorg-build

    --  font/util  -----------------------------------------------------------
    ~/tmp/xorg-build/font/util ~/tmp/xorg-build
    # On branch master
    nothing to commit (working directory clean)
    ~/tmp/xorg-build
[...snip...]

If this patch is accepted I'll modify the wiki to reflect these new changes.

 build.sh |   72 +-------------------------------
 xcmd.sh  |  141 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 142 insertions(+), 71 deletions(-)
 create mode 100755 xcmd.sh

diff --git a/build.sh b/build.sh
index 10f716b..6c3b5c7 100755
--- a/build.sh
+++ b/build.sh
@@ -12,8 +12,6 @@ global environment variables you may set to replace default functionality:
   MAKE:     program to use instead of 'make' (default: make)
   FONTPATH: font path to use (defaults under: \$PREFIX/\$LIBDIR...)
   LIBDIR:   path under \$PREFIX for libraries (e.g., lib64) (default: lib)
-  GITROOT:  path to freedesktop.org git root, only needed for --clone
-            (default: git://anongit.freedesktop.org/git)
 
 global environment variables you may set to augment functionality:
   CONFFLAGS:  additional flags to pass to all configure scripts
@@ -63,7 +61,6 @@ setup_buildenv() {
 
 failed_components=""
 nonexistent_components=""
-clonefailed_components=""
 
 failed() {
     if [ -n "${NOQUIT}" ]; then
@@ -139,41 +136,6 @@ checkfortars() {
     done
 }
 
-clone() {
-    case $1 in
-        "pixman")
-        BASEDIR=""
-        ;;
-        "xcb")
-        BASEDIR=""
-        ;;
-        "mesa")
-        BASEDIR=""
-        ;;
-        "xkeyboard-config")
-        BASEDIR=""
-        ;;
-        *)
-        BASEDIR="xorg/"
-        ;;
-    esac
-
-    DIR="$1/$2"
-    GITROOT=${GITROOT:="git://anongit.freedesktop.org/git"}
-
-    if [ ! -d "$DIR" ]; then
-        git clone "$GITROOT/$BASEDIR$DIR" "$DIR"
-        if [ $? -ne 0 ] && [ ! -d "$DIR" ]; then
-            return 1
-        fi
-    else
-        # git cannot clone into an existing directory
-	return 1
-    fi
-
-    return 0
-}
-
 build() {
     if [ -n "$LISTONLY" ]; then
 	echo "$1/$2"
@@ -195,18 +157,6 @@ build() {
     if [ -f $1/$2/autogen.sh ]; then
         SRCDIR="$1/$2"
         CONFCMD="autogen.sh"
-    elif [ -n "$CLONE" ]; then
-        clone $1 $2
-        if [ $? -ne 0 ]; then
-            echo "Failed to clone $1 module component $2. Ignoring."
-            clonefailed_components="$clonefailed_components $1/$2"
-            if [ -n "$BUILD_ONE" ]; then
-                exit 1
-            fi
-            return
-        fi
-        SRCDIR="$1/$2"
-        CONFCMD="autogen.sh"
     else
         checkfortars $1 $2
         CONFCMD="configure"
@@ -227,10 +177,6 @@ build() {
     old_pwd=`pwd`
     cd $SRCDIR || failed cd1 $1 $2
 
-    if [ -n "$PULL" ]; then
-	git pull --rebase || failed "git pull" $1 $2
-    fi
-
     # Build outside source directory
     if [ -n "$DIR_ARCH" ] ; then
 	mkdir -p "$DIR_ARCH" || failed mkdir $1 $2
@@ -731,10 +677,8 @@ usage() {
     echo "  -l : build libraries only (i.e. no drivers, no docs, etc.)"
     echo "  -n : do not quit after error; just print error message"
     echo "  -o module/component : build just this component"
-    echo "  -p : run git pull on each component"
     echo "  -r module/component : resume building with this component"
     echo "  -s sudo-command : sudo command to use"
-    echo "  --clone : clone non-existing repositories (uses \$GITROOT if set)"
     echo "  --autoresume file : autoresume from file"
     echo "  --check : run make check in addition to others"
     echo ""
@@ -750,7 +694,7 @@ DIR_CONFIG="."
 LIB_ONLY=0
 
 # Process command line args
-CMDLINE=`getopt -o abcdDf:ghlno:pr:s:L --long check,clone,help,autoresume: -n $0 -- "$@"`
+CMDLINE=`getopt -o abcdDf:ghlno:r:s:L --long check,help,autoresume: -n $0 -- "$@"`
 if [ $? != 0 ]; then
     echo "getopt(1) invocation error"
     exit 1
@@ -771,9 +715,6 @@ while [ 1 ]; do
     --check)
 	CHECK=1
 	;;
-    --clone)
-	CLONE=1
-	;;
     -d)
 	DISTCHECK=1
 	;;
@@ -804,9 +745,6 @@ while [ 1 ]; do
 	RESUME=$1
 	BUILD_ONE=1
 	;;
-    -p)
-	PULL=1
-	;;
     -r)
 	shift
 	RESUME=$1
@@ -889,11 +827,3 @@ if [ -n "$failed_components" ]; then
     echo "$failed_components"
     echo ""
 fi
-
-if [ -n "$CLONE" ] && [ -n "$clonefailed_components" ];  then
-    echo ""
-    echo "***** Components failed to clone *****"
-    echo "$clonefailed_components"
-    echo ""
-fi
-
diff --git a/xcmd.sh b/xcmd.sh
new file mode 100755
index 0000000..6d97e1d
--- /dev/null
+++ b/xcmd.sh
@@ -0,0 +1,141 @@
+#!/bin/bash
+
+_GITROOT_DEFAULT="git://anongit.freedesktop.org/git"
+
+usage() {
+    echo "Usage: `basename $0` [options] <cmd>"
+    echo "  where:"
+    echo "    <cmd>             command to execute"
+    echo "                      - if <cmd> is \"clone\" (by itself) perform"
+    echo "                        a git clone of all module sources"
+    echo "  options:"
+    echo "    -h|--help         print this help and exit successfully"
+    echo "    -k|--keep-going   continue with next module despite failure"
+    echo "                      similar to \"make\"'s \"-k\" option"
+    echo ""
+    echo "Environment variables which can affect this script"
+    echo "  GITROOT: specify alternate git root used with \"clone\" cmd"
+    echo "           (default: '$_GITROOT_DEFAULT')"
+    echo ""
+    location
+}
+
+location() {
+    echo "this script expects to be run in a location"
+    echo "such that build.sh is located at ./util/modular"
+}
+
+# this function prints a title line with the given string
+# $1 contains string to format into a title
+module_title() {
+    local _line="--------------------------------------------------------------------------------"
+
+    if [ -z "$1" ]; then
+	return
+    fi
+
+    echo "--  $1  ${_line:${#1}}"
+}
+
+# this function uses 'git clone' to retrieve the repository
+# $1 - module to clone in "dirx/diry" format
+# return:
+#   1 - failure
+#   0 - success
+clone() {
+    local _gitroot=${GITROOT:-"$_GITROOT_DEFAULT"}
+    local _basedir
+    local _dir1=`echo $1 | cut -d'/' -f1`
+
+    if [ -z "$1" ]; then
+	return 1
+    fi
+
+    # add basedir for most modules
+    case "$_dir1" in
+	"pixman"|"xcb"|"mesa"|"xkeyboard-config")
+	    _basedir=""
+	    ;;
+	*)
+	    _basedir="xorg/"
+	    ;;
+    esac
+
+    if [ -d "$1" ]; then
+	echo "git cannot clone into existing repository"
+	return 1
+    fi
+
+    git clone --progress "$_gitroot/${_basedir}${1}" "$1"
+    return $?
+}
+
+KEEPGOING=0
+
+CMDLINE=`getopt -o hk --long help,keep-going -n $0 -- "$@"`
+if [ $? -ne 0 ]; then
+    echo "getopt(1) invocation error"
+    exit 1
+fi
+eval set -- "$CMDLINE"
+while [ 1 ]; do
+    case "$1" in
+	-k|--keep-going)
+	    KEEPGOING=1
+	    shift
+	    ;;
+	-h|--help)
+	    usage
+	    exit 0
+	    ;;
+	--)
+	    shift
+	    break
+	    ;;
+	*)
+	    echo "internal getopt(1) error"
+	    exit 1
+	    ;;
+    esac
+done
+
+if [ $# -ne 1 ]; then
+    echo "wrong number of cmdline arguments"
+    echo ""
+    usage
+    exit 1
+fi
+CMD="$*"
+
+if [ ! -d util/modular ]; then
+    echo "incorrect location"
+    echo ""
+    location
+    exit 1
+fi
+if [ ! -x util/modular/build.sh ]; then
+    echo "can't find util/modular/build.sh executable"
+    exit 1
+fi
+
+for mod in `util/modular/build.sh -L`; do
+    module_title $mod
+
+    if [ "$CMD" = "clone" ]; then
+	clone $mod
+	RTN=$?
+    else
+	pushd $mod
+	$CMD
+	RTN=$?
+	popd
+    fi
+
+    if [ $RTN -ne 0 ] && [ $KEEPGOING -eq 0 ]; then
+	exit $RTN
+    fi
+
+    echo ""
+done
+
+exit 0
-- 
1.7.3.rc2



More information about the xorg-devel mailing list