GLX

Also known as "indirect rendering". When accelerated 3D was first added to X they it worked by sending OpenGL? from the app-server to the x-server, and the x-server would do the 3D rendering with its own 3D hardware. This matched the common hardware arrangements, where the x-servers have the fancy video cards, and the app-servers are headless. It also scales well, since each x-server adds it's own rendering hardware. The problem is this requires the x-server to understand and know it can use the extra GLX extensions, and the app-server side to support using them. Not all 3D hardware is equal, so the app-server and x-server have to negotiate what extensions exist and try to work with what's available. Sometimes they fail to negotiate enough common functionality to give working 3D at all. Also, some extensions are nearly impossible to make work when the app-server and x-server are not on the same machine.

Virtual GL

This is a new thing that does the hardware rendering on the app-server side. It works by intercepting the GLX on the app-server, rendering it (maybe on yet another 3D-render-server?), and then sending the rendered results to the x-server. Sending the rendered results is more efficient than sending the OpenGL? (note: this was probably not true in the past when 3D scenes were simpler), and doing the rendering on the app-server allows fancy stuff that can't be done remotely. In theory this means the x-server doesn't need to understand 3D at all, but they apparently also use special x-server side stuff to more efficiently receive and display the rendered data. It allows you to efficiently share centralized rendering hardware on the app-server between multiple x-servers, but it doesn't scale as well without x-servers adding their own rendering hardware. I also bet it's a bitch to make the "share rendering hardware between multiple x-servers" bit work correctly, and I bet many video hardware/drivers assume they talk to a real screen. AFAIK there is no Debian packages of Virtual GL.

Old School: Software rendering

Before GLX, 3D had to be done using software rendering (swrast) on the app-server and sending the rendered results to the x-server. Note this is kind of what Virtual GL does, it just does it in a convoluted way via intercepting GLX to use hardware rendering. This means the x-server doesn't need to understand GLX extensions, but the app-server side needs to be able to do software rendering. Software rendering avoids any issues with trying to share 3D hardware and/or trick it into rendering to a virtual screen. It's old, but tried and tested.

Adding VNC into the mix

VNC is a really simple low level screen/keyboard over the network protocol. It was designed around the idea of screen-scraping and exporting an existing 2D display, and that's how it works on things like windows. In Debian there is x11vnc that does this and exports an existing x-server screen. This avoids all the rendering and protocol complications and leaves them up to whatever normally renders the screen. This is one way to do hardware rendering on the app-server for a remote display; run the app-server, x-server, and x11vnc on the same machine with 3D hardware. There might be some minor issues with VNC sending 3D updates, since the hardware rendering might bypass some of the hooks VNC uses to detect screen changes, but VNC polling options can usually work around this.

Using VNC to export an existing x-server doesn't nicely support multiple user logins, since each vnc-client connection shares the same x-server screen. For this reason most people use a vnc-server that is also its own x-server. These are also known as an x-proxy, since they translates X from the app-server into VNC for the vnc-client. These are configured to spawn a new x-server and session for each user that connects with a vnc-client. On Debian there is tightvncserver and vnc4server. In theory these could do harware 3D rendering using GLX inside their x-server implementation. I've seen ambiguous reports that vnc4server is capable of this, but its dependencies suggest otherwise, and tightvncsever definitely can't. However, even if your vnc-server can, it's normally running on the app-server, which is unlikely to have the fancy 3D hardware you have in the machine the screen is attached to.

There is also TurboVNC? which is an x-proxy vnc-server designed to work with Virtual GL. This is mostly just an x-proxy vnc-server with GLX support for Virtual GL, but also includes some optimizations to VNC for sending rendered 3D efficiently. There are no Debian packages of TurboVNC?.

Making it work

So to make 3D work, you need your app-server and your x-server to either both support GLX well enough, or do the rendering on the app-server side, either in software or using Virtual GL. If you are using VNC, then you either need to use a screen-scraping vnc-server (x11vnc) to scrape an x-server with working GLX, or an x-proxy vnc-server with working GLX (TurboVNC?), or do the rendering in the app-server (software rendering).

On Debian, the package libgl1-mesa-glx on the app-server will make all apps using 3D try to talk GLX to their x-server. This is the package that is generally preferred by anything that depends on OpenGL? support, since it gives you hardware acceleration.

Unfortunately it seems that the cygwin x-server doesn't quite support GLX well enough for things like qtcreator or evolution to run with libgl1-mesa-glx. It almost supports it enough for glxgears to run, and it renders the first frame, but then fails to update it. Note that cygwins x-server does support GLX well enough to run accelerated glxgears when run on the same cygwin machine. It also seems that libgl1-mesa-glx doesn't support falling back to software rendering if GLX negotiation with the x-server fails, hence libGL error: failed to load driver: swrast. Setting LIBGL_ALWAYS_INDIRECT=1 tries to force using GLX, which stops error about no swrast, but it still doesn't work.

Note Debians Xorg x-servers do seem to support GLX well enough to run qtcreator, so switching from cygwin to a proper linux desktop will fix this. If you want to stick with cygwin's x-server, you will need to switch your app-server to software rendering.

If you want to use a vnc-client, you either need to run x11vnc (and not have per-user sessions), or use tightvnc or vnc4server and switch your app-server to software rendering.

For software rendering on Debian, you need libgl1-mesa-swx11 installed on your app-server, and this conflicts with libgl1-mesa-glx, or any of the other libgl1-*-glx supporting libraries. This means you app-server can support either swrast software rendering or GLX, but not both.

Note this is a bit crap. If you have a mix of x-servers, some of which support GLX well and others that don't, then ideally the app-server should be able to fallback to software rendering only when required.

There is also a glx-diversions package which supports "plugging in accelerated implementations from GPU vendors via alternatives", but this appears to be more the x-server side than the app-server side.

I suspect some Debian bugs are in order here, but in the tangled web of different packages involved I'm not sure exactly what bugs to file against what.

Ugh! XKEYBOARD extension problems

Using software rendering fixed the GLX problem, but I still can't run qtcreator because it needs the XKEYBOARD extension to work correctly. Without it the keyboard mapping is all screwed. With tightvncserver it's so screwed the keyboard is not even "qwerty" any more. With vnc4server it's mostly querty, but things like the arrow keys do unexpected things. There may be a way to fix this by messing with x keyboard mappings but I feel like this is pushing shit uphill.

The sad reality I'm seeing searching online for solutions to this stuff is modern desktops (gnome3, unity) and libraries (Qt5, Compiz) absolutely depend on modern X11 extensions (XComposite?, GLX, XKEYBOARD) that are not included in tightvncserver or vnc4server. More and more applications are relying on these and will just not run on xservers that don't have them. Since those two vncservers, and even cygwin's xserver, are lagging, they are becoming obsolete and unusable. Right now x11vnc seems to be the most actively developed, and has the benefit of leaving all the xserver stuff to a real modern xserver. That means x11vnc is probably the future of vnc, and the others, including valiant efforts like TurboVNC? are inevitably going to fall behind xserver developments. Even x11vnc will have some issues as xservers evolve, so how can efforts to duplicate xservers hope to keep up? Right now xllvnc suffers because the XComposite extension breaks the XDamage? extension

I'm thinking of switching from vnc4server to a combo of x11vnc+xvfb or x11vnc+xserver-xorg-video-dummy behind xrdp, but that's going to take a bit of fiddling to make work.

What about NX or X2Go?

These use a totally different approach to accelerating remote X, effectively putting an xserver on both ends and having them keep in sync using a different protocol that requires less handshaking. This way most of the X "chatter" happens locally at both ends and only summary "digests" are sent over the network. This is definitely the most bandwidth/latency effective solution for a remote X display. Right now X2Go looks like the most recent pretty awesome open source project leading the efforts in this area (working sound!). Unfortunately, these projects are again going to be playing catchup with xserver development, and already X2Go cannot be used with the latest versions of Unity, Gnome3, or Cinnamon because of lack of support for XComposite? and too-old GLX. Personally, I think this is a strong argument for not using Unity, Gnome3, or Cinnamon, and embracing LXDE.

I think that qtcreator will probably work fine with X2Go, and I'll be definitely trying that option too. The main disadvantage right now is it requires a special client and thus doesn't work with the default rdp clients.

In the longer term I expect that only x11vnc will be able to keep up with xserver development (by not duplicating it), so it's always going to be the best solution for leading edge 3D gaming/desktops etc.