Every newcomers to X think that servers and clients are reversed because they are

[Paul Dufresne]

I'll try to give a more precise description of  how I imagine things
works right now, and how I imagine they should work.

Currently, I believe things goes about like this:
When you start an X application (that we call the X client), it read
the $DISPLAY environment string to know the host where the X server it
should call is.
It then goes on to connect to this host, asking to speak on the X
server on this host.
The X server open a new window that talk to the X application, through
a new port.

What I would do:
-create a new program that I would call the X server. This program
would have a configuration that would looks like:
User_Group       Authorized X application
----------------      --------------------------------
secretary        LibreOffice
programmer  xterm
owner            xterm
owner            LibreOffice
-I would move the code that sits and wait for new clients to connect,
from what we was calling the X server, to this new X server
-I would move and modify a bit the code to authenticate, to this new X server
-I would rename the program we used to call the X server, to be called
the X Client, and make it be a program similar to xdm

So after my changes:
When you would start the X client, you would be asked to enter your
username, password, and select the X server you want to connect to.
After connecting and authenticating to the X server with your
username, the server would send you the list of authorized X
application you are allowed to run, depending on the user group the
username is in. Then, for each application you choose to open, the X
client would open a new window to speak with this X application on a
new port. It would also send a request to the X server, to request it
to launch the X application on the host where the X application is.

Not a very big change... but I guess it does change quite a bit how
you use the computer.