permissions issue with multiple desktops?

[mdew .]

under root, I'll "startx -- :0"

x11vnc -localhost -rfbport 5900 -forever -allow 192.168.1. -shared -display :0 &
fluxbox -display :0 &
/usr/local/bin/mythfrontend -display :0

and under a different user, using "startx -- :1"

x11vnc -display :1  -shared -rfbport 5902 -localhost -forever -allow
192.168.1. &
fluxbox -display :1

Though, as soon as launch the 2nd X session, it will lag out the first
xsession. Both sessions launching separately work fine (no issues with
vnc).

Anything I need to check for?