X-JHBuild: A thin JHBuild wrapper with plug-ins.

Mon May 24 11:01:27 PDT 2010

X-JHBuild-0.1.3
===============

Notable new features:
---------------------
- Diverse moduleset specification forms. Add some convenient forms and
  aliases, but also re-enable the previously prevented arbitrary
  specification forms. There is more about that in the default
  configuration file and the corresponding commit message.
- Let git-config track the cwd-module-id. Manage the setting with
  internal commands.

Commit Selection and Workflows:
-------------------------------
In an effort to understand the additional (yet uncommented) git branch
attributes of jhbuild, I cleaned up the update operation somewhat [1]. I
don't know if the behavior is what most people want (or why it was
undocumented), and so I wait a bit before attempting to document it
online. However, this is what is currently available regarding commit
selection:

Branch switches only take place in an update operation.
Attributes in the moduleset:
  'tag' : check this out unconditionally, no matter what branch you are
        on. It is passed directly to 'git checkout'. (might detach the
        head; might start tracking a remote branch when used with newer
        git versions). 
  'revision' : switch to a local branch or start tracking a remote branch.
        Switch only if the current branch tracks a remote branch, to not
        disturb your own work (The _update_ command is 'git pull
        --rebase', so you can base your own work on a remote branch, and
        select that branch in the configuration file.).
In the configuration file:
  'branches' : Override the revision from the moduleset. Otherwise, it
        has the same behavior as the 'revision' attribute. This is also
        necessary to override the default branch, which is 'master'.

Now, by means of moduleset inclusion, it would be easy to advertise
testing modulesets or create a bug-squashing branch with a corresponding
moduleset.

For example, let's assume there is a bug in xwud and the imaginary user
'johndoe' creates an 'xwud' repo with an 'audit' branch, and passes the
URL of the following moduleset to a user willing to try it (that's
basically just copy-pasting and setting the revision, module and
checkoutdir attributes):

  <moduleset>
    <include href="http://cgit.freedesktop.org/xorg/util/modular/blob/xorg-common-test-base.modules" />
    <repository type="git" name="git.freedesktop.org" 
                href="git://anongit.freedesktop.org/git/"/>

    <autotools id="xwud">
      <branch repo="git.freedesktop.org" 
                revision="audit" 
                module="~johndoe/xwud" 
                checkoutdir="people/johndoe/xwud" />
      <dependencies>
        <dep package="libX11"/>
      </dependencies>
    </autotools>
  </moduleset>

The autotools tag overrides the general one from the included modules
file completely.
The user would just pass the URL of this moduleset to [x-]jhbuild and
could easily test versions the developer provides, even across different
modules.

In a same manner tests could be advertised. 

Someone who wants to regularly test different parts, could set up a
special configuration for it, and then simply select a moduleset to
test. A good thing here is build_policy=updated-deps, which builds only
the modules that changed, and the modules that rely on a changed module.
There is a patch at [1] that triggers a build for dirty repos with every
build-policy. Using this for building reverse dependencies is also much
better than using the strange revdep-rebuild plug-in -- which is just an
example.

Plans:
------
- repo-group local config: Similar to git, the configuration could be
  kept inside the checkoutroot directory of the repo-group. 
  I'd like to minimize code in the configfile, and also would like to
  avoid forging a configfile for jhbuild. Although jhbuild provides a
  '__file__' attribute inside of the config file, it's probably not so
  good to rely on that. However, a repo-group local config would be a
  nice feature, and so I guess I'll use an additional internal-command
  'init' that fills in the checkoutroot in that configfile and later
  ensure that it matches in each run.
  Additionally this 'init' command could set 'prefix' to a directory
  below the checkoutroot, maybe 'INSTALL' or similar.
  A user would just cd to a directory and execute 'x-jhbuild init' and
  'x-jhbuild build xorg'.
- x-jhbuild configuration file: For settings, like moduleset location
  aliases.

Any feedback is welcome.

[1] https://bugzilla.gnome.org/show_bug.cgi?id=618974

-- 
Greetings,
Dirk
-------------- next part --------------
commit 5578172d6a4f2bcda02824e3740c6520a5c1285b
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Mon May 24 09:12:42 2010

    config: Use the online moduleset in the default config

    Don't distribute a possibly stale modules file, but use the online
    version in the default configuration file.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit d8735ad8bf83fb1161d06b99e91c33bfda776678
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sun May 23 20:17:09 2010

    plugins: Remove the plug-in 'addthis'

    That plug-in is no longer necessary, as the internal cwd-module adaption
    does that now.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 090df60b5b1971d789e35f4151ac4bdd2b916995
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sun May 23 11:25:08 2010

    Rephrase error message regarding wrong checkoutroot

    Not every caller wants to _add_ the cwd module.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 31ad552c79035585d49b81f3741d108747e372ba
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sat May 22 14:21:46 2010

    Rename the env-var that holds the moduleset location

    As the environment variable may now contain http URLs, the old name
    would be misleading. Therefore, this renames it from
    XJHBUILD_MODULESET_PATH to XJHBUILD_MODULESET_URI.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit afc4bd0601f9bec111ae4b7c7377e03622c3ae8e
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sat May 22 13:39:06 2010

    Allow to unset overriding ids with set-cwd-module

    It is now possible to clear a cwd-module setting by passing an empty
    string to set-cwd-module.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit f5411d22f54609ad97a4e53eff21e5f2b34646f0
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sat May 22 09:36:03 2010

    config: Update the comment explaining moduleset selection

    Give examples for the different specification forms.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 3d2c2bd4b099efc1397294520e8d0b255e5d6585
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sat May 22 11:57:51 2010

    Support aliases for moduleset locations

    All moduleset specification forms are recognized, when used as alias
    expansion.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 6fb085e9d6f4a32116ad0a2921723e6e1af0c130
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sat May 22 11:56:08 2010

    Extract general moduleset spec validity check into a function

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 3ff0d9c9ab610cacb8a6e82d9306c8ece138cadd
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sat May 22 11:11:31 2010

    Allow a relative moduleset path on the command line

    It is now possible to specify a relative path to a moduleset on the
    commandline. The path will be made absolute in the first invocation and
    so it won't be a problem if plug-ins change the working directory.

    There is no whitespace allowed in the absolute path leading to the
    moduleset file, because the absolute path will be a callback argument. I
    still think this is an acceptable limitation which will make callbacks
    easier, in its current form. If there goes more work into the tools
    available to plug-ins and how they execute the callback, this can be
    reconsidered.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 9b27b3554ad5966956b45485eca49b18c32ae96e
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sat May 22 09:27:18 2010

    config: Update explanation about branch switches

    State the requirements that must be met to trigger a branch switch.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit c7a6dd19cc196e090f9e997009a104aba66550ad
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Wed May 19 10:50:21 2010

    Support multiple specification forms to select modulesets

    The 'moduleset repo' is the repository that holds general modulesets
    (util/modular). The user will be prompted for confirmation if an initial
    clone operation is necessary. Otherwise, that repo is left alone. It is
    possible to integrate it into a moduleset and consequently update it
    alongside other modules.

    There is no whitespace allowed in any specification form. That is only
    an issue for absolute paths, and there are alternatives. You can use
    repository-relative paths, symbolic links, or copy the modulesets into
    the configuration directory.

    There are two main ways of specifying moduleset locations.
    - Convenient, token based. Without suffix. Without any path separator:
        xorg : in the configuration directory
        o:xorg : moduleset repo accessed through http.
        r:xorg : moduleset repo accessed locally. (cloned if necessary)
    - Arbitrary locations. With suffix. With at least one path separator
        /tmp/my.modules : absolute paths. User expansion is supported.
        http://where.ever.com/more.modules : a http-URL
        r:people/xyz/audit.modules : repo relative paths.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 3ac8871ddcecfb215e633e96308e4d86882dca48
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Thu May 20 14:41:32 2010

    Extract the path separator test into a function

    Give the operation a meaningful name.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit c593abe24e9abbdd0af14c7fd495c002cc72dcb3
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Wed May 19 17:41:30 2010

    Relocate the cwd-module adaption call in expand-modules

    Inside of the jhbuild command line dissector has never been the right
    location. This also simplifies the dissector call.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 61360cc3516b90d77cb0643d5305076ce5fa937f
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Wed May 19 17:34:37 2010

    Let the cwd-module adaption catch its own exception

    This simplifies the call site. To differentiate the origin, there is now
    an additional message argument.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 4f13b8387bfc557787ed0b3a8d7af9852eee0f19
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Wed May 19 16:00:13 2010

    Use the term 'sec_argv' consistently throughout the script

    No behavioral changes.
    This renames all affected variables, and fixes line wrappings where
    necessary. The name makes it clear that the sub-command is included at
    index 0.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit b154beeade85f02009675a0bf5358cd1a7a7eef3
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Wed May 19 15:56:48 2010

    Pass a whole secondary command line when testing for cwd-module

    This is part of the preparations to switch to the name sec_argv for all
    secondary command lines, which clarifies what that variable contains.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit b8219e92fcd21ce2fdc4f53c810bb6bb25889964
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Wed May 19 15:20:58 2010

    Pass a whole secondary command line to internal commands

    This prepares the step to consistently use the term sec_argv throughout
    the script. I've been wondering regularly how to name the different
    parts of the secondary command line and mixed different terms. To bring
    an end to this I will use the term 'sec_argv' now, which will make it
    clear that the secondary command is included at index 0.

    This commit prepares that step by passing the whole secondary command
    line to the internal commands which haven't gotten it prior to this.

    This also introduces two new helper functions which will help with
    fencepost calculations.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 755658f72116dc301aaa876f36ddf81d02f27e8a
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Wed May 19 13:40:35 2010

    Extract main command line parsing into a function

    This simplifies main a bit.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 2e637025734ea397621601c3361930c15d80a701
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Wed May 19 13:20:23 2010

    Stylistic improvements in configuration related functions

    No behavioral change. Just sync to the improved style started in the
    previous commit.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 9a8e7b4266327c1df9f9e335f7e07d7c85b1e840
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Wed May 19 13:15:29 2010

    Rearrange configuration file restriction checks

    Not all checks were implemented in the right function. Move them to the
    right place, and merge the restriction check call into the configuration
    getter.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit cb86855611db2bf3714615a38dab57dc93e3a5b6
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Wed May 19 10:03:47 2010

    Wait _after_ reading output in the shell wrapper

    Avoid filled buffer issues.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 763c902364854de7eedd25bf6b2c76f22596145c
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Tue May 18 20:25:40 2010

    Add cwd-module command line switches to syntax check.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 637b609d8cce43a67fdc98389b43623813f43830
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sun May 16 17:16:16 2010

    plugins: revdep-rebuild now sorts the modules for buildone

    The jhbuild command buildone does not sort its input after their
    dependencies. Therefore, this will use the 'sort-u' plugin to do that
    for buildone.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 11fca3b87d9d6072e107ed59d1e9a9faaa3d3f60
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sun May 16 17:10:50 2010

    plugins: Allow to sort modules after their dependencies

    Add a plug-in that takes multiple modules as argument and outputs them
    sorted after their dependencies. Multiple occurrences of the same module
    are eliminated.

    This is useful for the jhbuild command 'buildone' which does not sort
    its input.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit bb05daf22875b2243ba832630130223f10d6c81a
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sat May 15 10:23:22 2010

    Add internal command 'set-cwd-module'

    Set the cwd module id with this new command.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 2ec65acdc42d1a6b1ec4b77cb88b389145c4d8a1
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sat May 15 09:48:47 2010

    Rename internal command print-cwd-module to get-cwd-module

    This aligns the name with the coming 'set-cwd-module' function.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 4c051f160e842864da5a5bab7f5cab5b1c945e4e
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Sat May 15 09:40:37 2010

    Obtain a custom cwd-module id per git-config

    It is now possible to specify the module id for a repository with a
    git-config setting. Currently, the full key is 'x-jhbuild.moduleid'.
    $> git config x-jhbuild.moduleid libGL

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 8fe3d6cb06f4ede36a80aca06498163c3202d553
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Fri May 21 20:50:44 2010

    Rename args to secondary_commandline

    That does say more about the content.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>

commit 615f8cddabad5418fdd4a898760e615a80b4c701
Author: Dirk Wallenstein <halsmit at t-online.de>
Date:   Fri May 21 20:45:05 2010

    Pass config_settings into get_cwd_module

    Reduces the indexing operation (and the knowledge about that) to one.

    Signed-off-by: Dirk Wallenstein <halsmit at t-online.de>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patches-against-the-main-script.log.bz2
Type: application/x-bzip2
Size: 13158 bytes
Desc: not available
URL: <http://lists.x.org/archives/xorg-devel/attachments/20100524/af10d4e2/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: x-jhbuild-0.1.3.tar.bz2
Type: application/x-bzip2
Size: 27560 bytes
Desc: not available
URL: <http://lists.x.org/archives/xorg-devel/attachments/20100524/af10d4e2/attachment-0003.bin>