Xvnc OpenGL troubles with -depth 16

Peter Åstrand astrand at cendio.se
Tue Jun 21 03:53:41 PDT 2005


I've tried building Xvnc with OpenGL/GLX support. I'm using RealVNC 4.1.1, 
Xorg 6.8.2 and the Fedora Core 4 RPM sources. I'm using the "fb" 
framebuffer; not "cfb". By setting "BuildGlxExt YES" in vnc.def, Xvnc is 
built with GLX support.

With "-depth 24", everything is working. With "-depth 16", however, I have 
some problems, which I believe might be due to bugs in xf86glx.c. It's 
"almost" working:

glxgears: works
chromium: Fails with Couldn't set GL mode: Couldnt find matching GLX visual
glean -t +blendFunc: FAILs
femlabviewer: works

Some other strangeness:

* Upon start, Xvnc says:

No matching visual for __GLcontextMode with visual class = 4 (32770), 
nplanes = 16

* glxinfo says:

    visual  x  bf lv rg d st colorbuffer ax dp st accumbuffer  ms  cav
  id dep cl sp sz l  ci b ro  r  g  b  a bf th cl  r  g  b  a ns b eat
----------------------------------------------------------------------
0x23 16 tc  1  0  0 c  .  .  0  0  0  0  0  0  0  0  0  0  0  0 0 None
0x24 16 tc  0 16  0 r  y  .  6  5  5  0  0 16  0  0  0  0  0  0 0 None
0x25 16 tc  0 16  0 r  y  .  6  5  5  0  0 16  8 16 16 16  0  0 0 None
0x26 16 tc  0 24  0 r  y  .  6  5  5  8  0 16  8 16 16 16 16  0 0 None
0x27 16 dc  0 16  0 r  y  .  6  5  5  0  0 16  0  0  0  0  0  0 0 None
0x28 16 dc  0 16  0 r  y  .  6  5  5  0  0 16  8 16 16 16  0  0 0 None
0x29 16 dc  0 24  0 r  y  .  6  5  5  8  0 16  8 16 16 16 16  0 0 None
0x2a 16 dc  0 24  0 r  .  .  6  5  5  8  0 16  8 16 16 16 16  0 0 None

The first visual, 0x23 (which is also default) is bogus, I think. There 
is no predefined config with transparency, and Xvnc only creates one 
TrueColor and one DirectColor visual: There's no room for a COLOR INDEX 
mode.

The same type of problem has been reported for Xprt: 
https://bugs.freedesktop.org/show_bug.cgi?id=2166. The bug comments 
suggests that the driver should call GlxSetVisualConfigs, but should this 
really be necessary? My conclusion is that xf86glx.c is wrong. The 
fixup_visuals function is called just before init_screen_visuals, but it's 
not "as smart" as init_screen_visuals.

Here's what I think is going on:

1. Xvnc defines two visuals:

0x21  TrueColor  red, green, blue masks:    0x3f, 0x7c0, 0xf800
0x22  DirectColor  red, green, blue masks:    0x3f, 0x7c0, 0xf800

2. xf86glx.c:init_visuals is called. It has 4 predefined GLX RGB 
configurations. It creates 4 new TrueColor and 4 new DirectColor visuals, 
and also 8 matching modes. Both the visuals and the modes have masks 0x3f, 
0x7c0, 0xf800.

3. xvnc.cc:vfbScreenInit changes the masks and offsets for the root 
visual. The masks are now 0xf800, 0x7e0, 0x1f for the default, first 
visual (0x23), but 0x3f, 0x7c0, 0xf800 for all the other 7.

4. xf86glx.c:fixup_visuals is executed. (Wouldn't fixup_modes be a better 
name?). But rather than doing an intelligent one-to-one modes<->visual 
assignement, this is a rather crude function: It sets the masks of all 
modes to the *last* encountered visual with the matching class. The 
result? All modes will have masks 0x3f, 0x7c0, 0xf800.

5. xf86glx.c:init_screen_visuals is executed. It joins the modes and 
visuals together, but only if the class and the masks match. Since 
fixup_visuals works the way it does, no mode will match the default, first 
visual, since it has the "special" masks 0xf800, 0x7e0, 0x1f. This leads 
to the error "No matching visual for __GLcontextMode...", and that the 
0x23 GLX visual is not fully initialized.


One solution seems to be patching xvnc.cc. Instead of only changing the 
masks for the first, default visual, I've tried changing *all* visuals. 
See my patch below. This makes xf86glx.c less confused.

I must admit, though, that I don't fully understand the code in xvnc.cc:

   if (pvfb->pixelFormatDefined) {
     VisualPtr vis;
     for (vis = pScreen->visuals; vis->vid != pScreen->rootVisual; vis++)
       ;

     <then change the masks>

Why only change the rootVisual?


In any case, it seems like xf86glx.c should be able to handle the 
situation better. I'm a GLX newbie, but xf86glx.c looks overly 
complicated. Why initialize some parts for the GLX visual in init_visuals,
and some parts in init_screen_visuals? Would it be possible to go with a 
simpler approach? What about something like my prototype patch below?


Patch for xvnc.cc:

--- ./programs/Xserver/vnc/Xvnc/xvnc.cc.org	2005-06-21 11:12:25.000000000 +0200
+++ ./programs/Xserver/vnc/Xvnc/xvnc.cc	2005-06-21 11:29:36.000000000 +0200
@@ -876,9 +876,9 @@

    if (pvfb->pixelFormatDefined) {
      VisualPtr vis;
-    for (vis = pScreen->visuals; vis->vid != pScreen->rootVisual; vis++)
-      ;
-
+    int i;
+    vis = pScreen->visuals;
+    for (i = 0; i < pScreen->numVisuals; i++) {
      if (pvfb->rgbNotBgr) {
        vis->offsetBlue = 0;
        vis->blueMask = (1 << pvfb->blueBits) - 1;
@@ -894,6 +894,8 @@
        vis->offsetBlue = vis->offsetGreen + pvfb->greenBits;
        vis->blueMask = ((1 << pvfb->blueBits) - 1) << vis->offsetBlue;
      }
+      vis++;
+    }
    }

  #ifdef VNC_USE_FB


Patch for xf86glx.c:

--- ./programs/Xserver/GL/mesa/X/xf86glx.c.org	2005-06-20 16:36:50.000000000 +0200
+++ ./programs/Xserver/GL/mesa/X/xf86glx.c	2005-06-21 11:08:56.000000000 +0200
@@ -399,28 +399,7 @@
  	    /* Initialize the glXVisual */
  	    _gl_copy_visual_to_context_mode( modes, & pNewVisualConfigs[k] );
  	    modes->visualID = pVisualNew[j].vid;
-
-	    /*
-	     * If the class is -1, then assume the X visual information
-	     * is identical to what GLX needs, and take them from the X
-	     * visual.  NOTE: if class != -1, then all other fields MUST
-	     * be initialized.
-	     */
-	    if (modes->visualType == GLX_NONE) {
-		modes->visualType = _gl_convert_from_x_visual_type( pVisual[i].class );
-		modes->redBits    = count_bits(pVisual[i].redMask);
-		modes->greenBits  = count_bits(pVisual[i].greenMask);
-		modes->blueBits   = count_bits(pVisual[i].blueMask);
-		modes->alphaBits  = modes->alphaBits;
-		modes->redMask    = pVisual[i].redMask;
-		modes->greenMask  = pVisual[i].greenMask;
-		modes->blueMask   = pVisual[i].blueMask;
-		modes->alphaMask  = modes->alphaMask;
-		modes->rgbBits = (is_rgb)
-		    ? (modes->redBits + modes->greenBits +
-		       modes->blueBits + modes->alphaBits)
-		    : rootDepth;
-	    }
+	    modes->rgbBits = rootDepth;

  	    /* Save the device-dependent private for this visual */
  	    glXVisualPriv[j] = pNewVisualPriv[k];
@@ -503,43 +482,11 @@
  			*ndepthp, *depthp, *rootDepthp);
  }

-static void fixup_visuals(int screen)
-{
-    ScreenPtr pScreen = screenInfo.screens[screen];
-    __MESA_screen *pMScr = &MESAScreens[screen];
-    int j;
-    __GLcontextModes *modes;
-
-    for ( modes = pMScr->modes ; modes != NULL ; modes = modes->next ) {
-	const int vis_class = _gl_convert_to_x_visual_type( modes->visualType );
-	const int nplanes = (modes->rgbBits - modes->alphaBits);
-	const VisualPtr pVis = pScreen->visuals;
-
-	/* Find a visual that matches the GLX visual's class and size */
-	for (j = 0; j < pScreen->numVisuals; j++) {
-	    if (pVis[j].class == vis_class &&
-		pVis[j].nplanes == nplanes) {
-
-		/* Fixup the masks */
-		modes->redMask   = pVis[j].redMask;
-		modes->greenMask = pVis[j].greenMask;
-		modes->blueMask  = pVis[j].blueMask;
-
-		/* Recalc the sizes */
-		modes->redBits   = count_bits(modes->redMask);
-		modes->greenBits = count_bits(modes->greenMask);
-		modes->blueBits  = count_bits(modes->blueMask);
-	    }
-	}
-    }
-}
-
  static void init_screen_visuals(int screen)
  {
      ScreenPtr pScreen = screenInfo.screens[screen];
      __GLcontextModes *modes;
      XMesaVisual *pXMesaVisual;
-    int *used;
      int i, j;

      /* Alloc space for the list of XMesa visuals */
@@ -548,30 +495,36 @@
      __glXMemset(pXMesaVisual, 0,
  		MESAScreens[screen].num_vis * sizeof(XMesaVisual));

-    /* FIXME: Change 'used' to be a array of bits (rather than of ints),
-     * FIXME: create a stack array of 8 or 16 bytes.  If 'numVisuals' is less
-     * FIXME: than 64 or 128 the stack array can be used instead of calling
-     * FIXME: __glXMalloc / __glXFree.  If nothing else, convert 'used' to
-     * FIXME: array of bytes instead of ints!
-     */
-    used = (int *)__glXMalloc(pScreen->numVisuals * sizeof(int));
-    __glXMemset(used, 0, pScreen->numVisuals * sizeof(int));
-
      i = 0;
      for ( modes = MESAScreens[screen].modes
  	  ; modes != NULL
  	  ; modes = modes->next ) {
-	const int vis_class = _gl_convert_to_x_visual_type( modes->visualType );
-	const int nplanes = (modes->rgbBits - modes->alphaBits);
  	const VisualPtr pVis = pScreen->visuals;
+        const int is_rgb = (pVis[i].class == TrueColor ||
+			    pVis[i].class == DirectColor);

  	for (j = 0; j < pScreen->numVisuals; j++) {
-	    if (pVis[j].class     == vis_class &&
-		pVis[j].nplanes   == nplanes &&
-		pVis[j].redMask   == modes->redMask &&
-		pVis[j].greenMask == modes->greenMask &&
-		pVis[j].blueMask  == modes->blueMask &&
-		!used[j]) {
+	    if (pVis[j].vid == modes->visualID) {
+
+		/*
+		 * If the class is -1, then assume the X visual information
+		 * is identical to what GLX needs, and take them from the X
+		 * visual.  NOTE: if class != -1, then all other fields MUST
+		 * be initialized.
+		 */
+		if (modes->visualType == GLX_NONE) {
+		    modes->visualType = _gl_convert_from_x_visual_type( pVis[j].class );
+		    modes->redBits    = count_bits(pVis[j].redMask);
+		    modes->greenBits  = count_bits(pVis[j].greenMask);
+		    modes->blueBits   = count_bits(pVis[j].blueMask);
+		    modes->alphaBits  = modes->alphaBits;
+		    modes->redMask    = pVis[j].redMask;
+		    modes->greenMask  = pVis[j].greenMask;
+		    modes->blueMask   = pVis[j].blueMask;
+		    modes->alphaMask  = modes->alphaMask;
+		    if (is_rgb)
+			modes->rgbBits = modes->redBits + modes->greenBits + modes->blueBits + modes->alphaBits;
+		}

  		/* Create the XMesa visual */
  		pXMesaVisual[i] =
@@ -591,31 +544,11 @@
  				      modes->samples,
  				      modes->level,
  				      modes->visualRating);
-		/* Set the VisualID */
-		modes->visualID = pVis[j].vid;
-
-		/* Mark this visual used */
-		used[j] = 1;
-		break;
-	    }
  	}
-
-	if ( j == pScreen->numVisuals ) {
-	    ErrorF("No matching visual for __GLcontextMode with "
-		   "visual class = %d (%d), nplanes = %u\n",
-		   vis_class, 
-		   modes->visualType,
-		   (modes->rgbBits - modes->alphaBits) );
  	}
-	else if ( modes->visualID == -1 ) {
-	    FatalError( "Matching visual found, but visualID still -1!\n" );
-	}
-
  	i++;
      }

-    __glXFree(used);
-
      MESAScreens[screen].xm_vis = pXMesaVisual;
  }

@@ -636,12 +569,6 @@
      __glDDXScreenInfo.createContext = __MESA_createContext;

      /*
-     * The ordering of the rgb compenents might have been changed by the
-     * driver after mi initialized them.
-     */
-    fixup_visuals(screen);
-
-    /*
       * Find the GLX visuals that are supported by this screen and create
       * XMesa's visuals.
       */

Thanks,
-- 
Peter Åstrand		Chief Developer
Cendio			www.thinlinc.com
Teknikringen 3		www.cendio.se
583 30 Linköping        Phone: +46-13-21 46 00


More information about the xorg mailing list