[ANNOUNCE] x-jhbuild 0.2

[Dirk Wallenstein]

X-JHBuild-0.2
#############

X-JHBuild is a framework for building and working with X.org modules.
It makes use of JHBuild and is specialized for working with modules that
use Git as VCS.

http://sourceforge.net/projects/x-jhbuild/


Install/Uninstall easier than ever before:
==========================================
It is now possible install X-JHBuild with just a few commands.
    $> git clone git://x-jhbuild.git.sourceforge.net/gitroot/x-jhbuild/x-jhbuild
    $> cd x-jhbuild
    $> make install
Private versions of JHBuild and Git-Python will be installed behind the
scenes.  There is an uninstall target which will leave nothing but the
configuration in ~/.x-jhbuild and possibly the two standard directories
~/.local/bin and ~/.local/share behind.
More info about installing and using X-JHBuild can be found at in the wiki at:
http://www.x.org/wiki/XjhBuild

Early and premium adopters need to migrate the settings to the new
configuration mechanism, and modify/restore existing repository groups.
See below for more.


Maybe Start Over For Exploration?
=================================
There are diverse changes which might be best explored with a new
installation and a new repository group.  To do that just rename
~/.x-jhbuild, install anew, and create a new repository group with the
'init' plug-in.
The main changes are the new configuration mechanism and persistent
configuration/moduleset selection, both of which are described below and
in help messages of the corresponding plug-ins.
To use an existing repository group with this new version, rename or
remove the old configuration directory (.xjh/config) and execute the
following command within that repository group:
    $> xjh init --restore
The module pthread-stubs has been renamed and so its location moved from
xcb/pthread-stubs to xcb/libpthread-stubs.  In the future I intend to
provide a plug-in and a special file in the project's moduleset
repository to facilitate syncing with such repository moves.  Currently
you would need to execute the following command from the repository root
directory:
    $> mv -i xcb/pthread-stubs xcb/libpthread-stubs.  


Features new in this version:
=============================

Persistent Config-Token/Moduleset Selection
-------------------------------------------
The '-f' and '-m' main command line options are no more.  The new
plug-in 'checkout' can be used to permanently select a config-token
and/or moduleset to be used.  To have greater control over the state of
repositories, the subcommand 'update' does no longer switch to the
states requested in modulesets and buildconfig.branches for checked out
Git repositories.  That can be done with the new plug-in 'spool', which
in the future might also be able to backup and restore repository
states.  'update' does still switch revisions when cloning a repository.
Nothing changes for repositories that don't use Git.

This might complicate the one-time test execution, but I think the cases
where one does not want to execute a follow-up command with the same
selection are practically nonexistent.  There can be a three-liner
plug-in that does checkout/spool/build, to accomplish the same as before.

The prompt sign has been extended to optionally display the currently
selected configuration and/or moduleset.  This can result in a rather
lengthy sign, but note that the repogroup-name, config-token, and
moduleset-spec (moduleset-aliases) are under your control and can be
represented with short mnemonics.

The New Configuration Mechanism:
--------------------------------
The settings in the global configuration file ~/.x-jhbuild/xjhrc will be
overridden by the ones in a repository group at .xjh/xjhrc.  The result
can be further tweaked by selecting a file in .xjh/xjhrc.token.  These
files use an ini style format that is described in the example file at
~/.x-jhbuild/xjhrc.example.  The new plug-in 'config' can be used to
display whole configuration sections with accompanying help text.  Such
output can be used as template content for a configuration file.  The
previously used configuration files are no longer supported and the
settings have to be migrated.  The 'config' plug-in can be of great aid
here.  All JHBuild settings can be found in the configuration section
'buildconfig'.  The following command generates a template section for
the build configuration:
    $> xjh config --verbose buildconfig

Moduleset Comparison:
---------------------
The plug-in 'diff-ms' can compare two modulesets and, in a special mode,
display settings of one moduleset.  The focus lies on location, version,
and build settings.  It does not compare dependencies, currently.  I am
confident about Autotools and Git, but comparison of other types might
not be completely accurate.  Patches are welcome -- it's in the
moduletraits.py library module.

The New Status Plug-In:
-----------------------
The new plug-in 'status' is able to search selected or all Git
repositories for interesting characteristics and will display status
labels and a few log lines, in an attempt to remind about deviations
from a fresh, unmodified checkout.  It is possible to search for
specific characteristics like a configured octopus, or commits that are
nowhere upstream.  There is an alternative output format that can be
used to further process the information with another script.  See 'xjh
status -h' for more.

Environment, Callback and Execution
-----------------------------------
The callback and runtime-switch environment variables have been
deprecated.  Callbacks can now be done with another main command line
option '--callback' and the environment switch is demonstrated in the
plug-in 'show-env' -- copy-paste two instead of one line.
It is possible to run xjh from source by setting the environment
variable XJHBUILD_RUN_FROM_SOURCE_WITH_SCRIPT_FROM_SOURCE to a
non-empty string.  In that case, 'xjh' in PATH has to be a symlink to
the script in the sources.  An initial installation is necessary, for
the JHBuild and Git-Python parts.
To ensure the callback executes the same script, there is now a check in
the main script that tries to find 'xjh' in $PATH and generates an error
if it cannot be found or is not the same real file. 

Module Selection
----------------
Instead of implementing a module selection interface in every plug-in,
the standard input stream (by means of 'modinfo', 'get-cwd-module',
'echo', etc) can be used to select modules.  Note, that this does
currently not apply to JHBuild subcommands. 
'modinfo' does include dependencies by default, now, as I thought that
that would be the common use case on the command line.  In scripts the
new, more intuitive option '--one' can be used to get attributes of a
single module.

Module Location Changes
-----------------------
The module pthread-stubs has been renamed to libpthread-stubs to comply
with the policy of using package-names for module-ids.  To allow reverse
mapping of a repository to its module-id, the location changed from
xcb/pthread-stubs to xcb/libpthread-stubs.

Other Changes:
==============
- The plug-in 'init' can now download a project configuration file,
  which will then be installed as .xjh/xjhrc.  This way projects can
  provide default modulesets and locations to download them from.
- Subcommand aliases can now include main command options.  Together with
  the new configuration mechanism, it is now possible to create
  aliases that change with the repository group.
- 'modinfo --deps' does no longer automatically swallow modules
  configured to be skipped.  There are new command line options (-s|-S)
  to include/exclude such modules explicitly.
- Generate error messages if the Git >= 1.7 dependency is not met.  The
  prompt sign will display an error message instead of the module name,
  and the CWD-Module adaption will fail.
- The implicit subcommand has been abolished in favor of explicitness.
- The plug-in 'edit-config' is no more.  I recommend writing a small
  plug-in that starts your favorite editor with all active configuration
  files -- all info for that is available from the environment.  See
  'show-env' or commit 184ace5.
- The plug-in 'show-config' is no more.  The 'config' plug-in combined
  with command-aliases is much more flexible.
- Plug-in names may not contain a dot because that is the section
  separator for configuration sections.  There is the policy, that all
  configuration sections in a plug-in should be assembled below the
  top-level section that has the same name as the plug-in.  This is
  necessary to let 'config' find that section.

Internal changes and additions:
===============================
- A current version of Git-Python is made available to all plug-ins.
  There is the class GitRepoInfoAccess in the library module gitaccess,
  which focuses on obtaining information about a repository.  Together
  with the comments accompanying that class it might be possible to
  accomplish small tasks without the need to delve into Git-Python
  completely.
  In addition it is possible to execute every Git command with Python
  syntax, by means of the git.cmd.Git class.
- The ModulesAccess interface has been revised.  Basically, all that
  applies to the selection of modules, that can be done with modinfo,
  has been assembled into one method of ModulesAccess, and is thus
  available to other plug-ins.
- Plug-in authors can easily use the new configuration mechanism.  The
  only task is to instantiate a section object that describes all
  options and their default values.  After that all active settings can
  be retrieved through one call.  The commit e1c6809 adds a
  configuration section to the status plug-in and is very instructive in
  this regard.
- Importing anything from xjhbuild will install an excepthook that will
  display only the message and not the traceback of exceptions with a name
  that ends in the term CommonError.  Setting the environment variable
  XJHBUILD_SHOW_TRACEBACK will show the traceback for these exceptions,
  too.

Let me drag you into this:
==========================
There are several areas which could need improvement and/or extension.
If you like this project and want to contribute, or if you already have
some useful modifications and/or plug-ins, just contact me directly or
through the trackers on sf.net.
Here are just a few ideas about what could be useful.  Of course, other
contribution are welcome, too.
* An installation procedure.
* A plug-in or library module that can start repo-browsers: A simple GUI
  window that lists modules (e.g. results from modinfo or status) and
  that can open a repository browser (gitk, qgit, giggle, etc) for any
  of those modules.
* A patch management plug-in: It could drive git-format-patch
  and change the subject to include the module-name and could save patch
  series in a sub-folder of a configurable location.  It could take a
  subject on the command line and use that for the sub-folder and put it
  in the cover letter automatically.  I guess this could benefit from
  further discussion what is needed.  For example there could be support
  for tags like v2, v7, revoked, etc to facilitate automatic Patchwork
  management.
* The plug-in 'spool' could be extended to save/restore repository
  states into/from an XML file.
* A plug-in that fetches upstream independent on the current state:
  'update' does only fetch changes when on a remote tracking branch and
  does that by means of a 'pull --rebase'.

Dirk Wallenstein (153):
      config: Update the documentation in the config template
      modaccess: Rename method to get_state_preselection
      modaccess: Move the dependency sorting function into modaccess
      modaccess: Comprehensive module selection with get_git_modules
      modinfo: Formulate the sort request variable positively
      modaccess: Relocate get_modules_by_name in the same scope
      modaccess: Always filter all non-git modules
      modaccess: Introduce a select-flags constant to omit filtering
      modules: remove the obsolete modules file my.modules
      doc: get_git_modules can take any empty sequence of modules
      modinfo: Make constants camel-case
      modinfo: Limit selection flags alias to scope of use
      Write the expand alias message to stderr
      modaccess: Improve selection flag comments
      modaccess: Add a 'skipped' select flag
      modinfo: Fix silent swallowing of skipped modules
      Use parentheses to 'from-import' multiple definitions
      depcomp: Style changes without behavioral impact
      Use the conventional 'Error' suffix for exceptions
      Compare the singleton None with 'is' or 'is not'
      utils: Introduce the NamedFlags class
      utils: Introduce a function to read from a stdin pipe
      utils: Introduce the FormattedTerminalOutput class
      Add a Git-Python submodule and install it
      lib: Introduce the gitaccess library module
      plugins: Introduce the 'status' plug-in
      Remove the period from every short help
      Add short help for the status plug-in
      Explicitly derive from 'object'
      Remove unused imports
      Prompt: Handle Git version without --show-toplevel option
      CWD-Module: Handle Git version without --show-toplevel option
      Improve the style of import statements
      Add a 'Requirements' section to the README file
      Add a JHBuild submodule and install it
      Add 'uninstall' target to the Makefile
      Remove Gnome specific help from tinderbox log
      lib: Rename the library module config to conf
      Implement 'show-modules-to-skip' as a plug-in
      Move all configuration constants into conf.py
      Move moduleset section into msspec lib module
      Implement moduleset translation errors with exceptions
      lib: Introduce the jhbuild_cmdline library module
      Generate CWD-Module errors on the call site
      Don't validate checkoutroot in get_cwd_module()
      Move CWD-Getter into cwdmodule library module
      cwdmodule: Simplify adding CWD-Module to JHBuild command line
      Sync import statement syntax
      Move RGLC section into the library module 'repogroup'
      hooks: Use the install-location for hooks directly
      hooks: Define the hooks subdir-name in the configuration
      hooks: Move Hooks class into hooks library module
      Move the PlugIns class into the new library module 'plugins'.
      Implement 'get-cwd-module' as a plug-in
      Implement 'set-cwd-module' as a plug-in
      Prevent whitespace in all parts of the callback
      Add envvar XJHBUILD_WANT_CWD_MODULE
      Implement 'expand-modules' as a plug-in
      Extract repo-group initialization into the lib module 'init'
      Rename the concept 'Internal Commands' to 'Built-ins'
      Clean up the main script
      Simplify command line parsing
      Add a patch to change JHBuild configuration defaults
      Move JHBuild config parse into jhbuild_config.py
      Move FormattedTerminalOutput into lib module 'termout'
      Define absolute paths to user config locations
      Let repogroup provide paths to locations within the xjh-dir
      doc: Add status argument in the tinderwrap help display
      Add executable bit to modinfo
      modinfo: remove version option
      plugins: Remove sort-u
      Upgrade subcommand aliases to full command aliases
      Install excepthook that omits a traceback for *CommonError
      repogroup: Throw *CommonError exception instead of failing
      Switch to a new configuration mechanism
      status: Split color-config at any non-word char-sequence
      status: Add a configuration section
      plugins: Ensure config compatible plug-in names
      Introduce the config plug-in
      init: Allow to create a new repogroup with path spec only
      Use *CommonError exceptions instead of error_out()
      config: Display string repr of options not python repr
      Add reload_config_files() method to XjhConfigParser
      Generalize location prefix names
      config: Generate an unknown sections warning
      repogroup: Add two helper functions
      Remove the implicit subcommand
      Rename option to want-cwd-module
      Add JHBuild buildroot layout patch
      Change boolean string representation to yes/no
      More reliable list item quotation in string representation
      Use the xjh config mechanism for build-configuration
      Relocate all main sections below 'core'
      Add a moduleset repository configuration section
      Add *CommonError base exception to the 'config' plug-in
      config: Add options to edit configuration files
      Overhaul xjhrc.example
      init: Make command line parsing consistent
      init: Add option to download project configuration
      Extract configsection library module from xjhconfig
      configsection: Remove xjhbuild.utils dependency
      status: Decode author name into unicode
      config: Allow to print only non-default values
      Remove the plug-in 'show-config'
      Install distributed hooks, plug-ins, modulesets into library
      Allow to run xjhbuild from the source directory
      config: Allow to suppress empty sections
      Add a --verbose option to the 'config' plug-in
      Overhaul the help output of the 'config' plug-in
      Remove checkoutroot arg from get_moduleset_uri()
      configsection: Add abstract method get_all_options()
      Make configuration available to non-Python plugins
      configsection: Rename get_option() to get_value()
      Make envvars available as XjhEnvironment properties
      Include dependencies by default for modinfo and status
      xjh.modules: Make use of a default repository
      Generate a CommonError for invalid moduleset specs
      Use a singleton meta-class
      modules: Add xkbcommon moduleset
      xjhconfig: Prevent accessing config from a different repo-group
      gitaccess: Add get_commit() to GitRepoInfoAccess
      Claim standard input for module selection
      status: Use only standard input for module selection
      env: Add properties for command line option envvars
      Implement callback with main command line option
      Deprecate the environment-switch envvar
      Make X-JHBuild envvars available in envvar config section
      gitaccess: Allow to directly query upstream relation
      status: Formulate color toggle config-option positively
      msspec: Add a function to determine the moduleset name/id
      jhbuild_call: Add lib module to execute JHBuild commands
      CommonError: Print only the basename of the script
      xjh.modules: Rename pthread-stubs to libpthread-stubs
      Persistently switch to a config-token and/or moduleset
      Introduce the plug-in 'diff-ms' to compare modulesets
      Add three small JHBuild patches
      gitaccess: Try to describe detached HEAD with tag
      init: Call expanduser() on the destination path argument
      Optionally hide version selections from JHBuild modules
      Restore configuration reload opportunity at JHBuild prompt
      Move Xorg configuration defaults into xjhrc
      repogroup: Less errors for the xjh directory structure
      init: Add the --restore option
      spool: Suppress SHA display by default
      spool: Config/Cmdline options for module switch requests
      msspec: Automatically clone the moduleset-repository
      init: Trigger moduleset-repo clone if necessary
      Use the new moduleset repository for Xorg
      Remove distro-modulesets
      README: Update for the next release
      config: Remove edit-configuration command line options
      Fix online-access error generation
      x-jhbuild-0.2

-- 
Greetings,
Dirk