Antialiasing

[Carsten Haitzler (The Rasterman)]

On Sun, 29 Jan 2017 23:34:08 -0700 SRC SRC <src_s at rocketmail.com> said:

> Hi,
> 
> I was able to successfully create a translucent window using xlib and then
> set the transparency using the XRender extension, and make the window have
> rounded corners with the Xshape extension and the following code:
> 
> void rectAttributes(XRectangle *rect ,int width, int height, int x, int y) {
>   rect->height = height;
>   rect->width = width;
>   rect->x=x;
>   rect->y=y;
> }
> 
> void createRoundedEdges(int radius, Display *display, Window window) {
>   XWindowAttributes attr;
>   XGetWindowAttributes(display,window, &attr);
>   int windowHeight = attr.height,
>       windowWidth = attr.width,
>       cornerCoordinates[radius],
>       x_bar = windowWidth - radius;
>   XRectangle rect[(radius*4)]; //*4 because of 4 corners
>   /* Populate the firstCorner array */
>   for (int i=radius; i!=0; i--) {
>     int y = i,
>         x = sqrt((radius*radius)-(y*y));
>     cornerCoordinates[radius-i] = x;
>   }
>   for (int i=0,j=radius,k=2*j,l=3*j;i<radius;i++,j++,k++,l++) {
>     rectAttributes(&rect[i],windowWidth,1,x_bar+cornerCoordinates
> [i],i); //top-right rectAttributes(&rect[j],windowWidth,1,x_bar
> +cornerCoordinates[(2*radius)-1-j] //bottom-right ,windowHeight-(2*radius)+j);
>     rectAttributes(&rect[k],windowWidth,1,radius-windowWidth-cornerCoordinates
> [i],i); //top-left rectAttributes(&rect[l],windowWidth,1,rect
> [k].x,windowHeight-1-i); //bottom-left }
>   int sizeOfArray = sizeof(rect)/sizeof(rect[0]);
>   XShapeCombineRectangles
> (display,window,0,0,0,rect,sizeOfArray,ShapeSubtract,0); }
> 
> This is what the above code generates:
> 
> If you look closely at the rounded corners, they seem to be not smooth but
> rather jagged, and to make those corners smooth I’d have to do ant-aliasing
> on the window. I could do that in two ways, one is by writing the
> anti-aliasing algorithm and placing pixels with differing alpha values along
> the edges using Xrender, or by finding an extension that does the
> anti-aliasing on the window. So, I was wondering if you could point me to an
> anti-aliasing extension that I can use to make the corners smooth mostly
> because I really don’t want to spend the time writing an antialiasing
> algorithm from scratch if an extension already exists that does just that.

well first... the shape extension is not going to help you at all. it's
boolean. server-side it boils down to a list of rectangles your window occupies
(visibly and event-wise). this is by nature either "inside or outside" your
window on a pixel basis and thus you'll get jaggies.

that's why you have an argb window... you can draw rounded corners with alpha
channels. but you still need a compositor to composite them together... and you
say you cannot have one by definition. the shape extension works without a
wm/compositor as its the server literally clipping your window to this
rectangle set. 

your choices are:

1. hand write pixels yourself one at a time with xrender.
2. save time and just use cairo to draw a rect with rounded corners 
3. pre-draw the rounded corners as images and have a library render them for
you after turning them into a pixmap (xrender, cairo, imlib2 - based on your
previous mail can do that too).

you really want to just start using other libraries to do this work for you and
get yourself a compositor/wm. unless you  have studied/learned enough about x11
to do it all yourself of course... in which case you wouldn't be asking
here. :)

that *IS* what libraries are for. to encapsulate such knowledge and save you
the pain. of course if you wish to learn... you are likely wagging the dog by
it's tail with where you are starting from...  (based on your previous
stackoverflow post) and doing things by pretty much poking in the dark. you are
better off going FROM a higher level working environment TO a simpler one and
learning pieces of the puzzle along the way rather than building up from nil.

-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster at rasterman.com