How to Use VNC to ‘Backdoor’ Access VMware Servers

In our Remotely Manage VMware Servers article, we talked about the different ways to remotely manage your VMware Server. A very cool (I believe) “backdoor” management method for VMware is VNC. And even better, its built-in to VMware – you don’t have to install it. All you have to do is turn it on. Let’s find out how…

What can the built-in VMware Server VNC Management tool do for you?

VNC is pretty well known but when it comes to understanding how it is going to manage your VMware virtual servers, it required some clarification. Here is how this works:

• VNC Server functionality can easily be enabled with a single command for each Virtual Machine (VM)
• You must enable VNC for each VM that will use it
• You would connect to the IP address or DNS name of the VMware Host – NOT of your VM guests
• The VM must be powered on to be able to connect to it
• The only function you can perform with the VNC tool is remote control of the console. You cannot perform all the other VM management and configuration tasks like you can with the VMware Server Console or the VMware MUI web interface.
• Each VM guest you connect to must have a different port number, starting at 5900
• When you connect to the VM you can abbreviate your port number. For example, to connect to server1, port 5900, you can just type server1:0
• If you connect a second VNC client to the same port number, you get two VNC clients controlling the same VM

Common VNC clients are:

• RealVNC
• TightVNC
• UltraVNC

All of these offer a free version.So how do you configure it? Let’s find out…

How does Remote Access work?

You can configure remote control for your VMware Server guests with as little as a single command. However, you probably want to enable a few of the optional features.To enable VNC for a certain VM, you need to edit its VMX file. For example, here is a VM of mine and I have circled the VMX file:

This is what the VMX file looks like:

As you can see, at the bottom, I have already inserted some VNC-related configurations. However, the one single line required to enable VNC to view your VMware Guests over the network is:

RemoteDisplay.vnc.enabled = “TRUE”

If you will insert that line at the bottom of your VMX file and start the VM, you will be able to connect with VNC. You should ensure that there is not a firewall in the way. By enabling that single option, you would connect on port 5900 (port 0 actually, like “servername:0”) and there would be no password. I would highly recommend configuring a password so that people cannot just connect to the console of your servers anytime they want to. To configure a password, add this command:

RemoteDisplay.vnc.password = “vncpassword”

If you will enable this on more than one VM, you will need to set a different port number inside the VMX of each VM, like this:

RemoteDisplay.vnc.port = “5900”

The first VM would be 5900, the second, 5901, and so on. To connect to them, they would be “0”, “1”, and so on. Next, save your VMX file, start your VM, and download your VNC client. I like the VNC viewer where you don’t have to do any install and you can just run an EXE to connect. Here is what my VNC viewer looked like before I connected:

Once connected, I could see the console of the remote virtual guest OS.

Many times, even better than seeing the console of the OS is being able to see the BIOS and boot up phases before the OS is functioning. This is very useful when your VM is having trouble booting the OS. Here is an example of one of my VM guests while using VNC to configure the BIOS:

If you like this feature, you will like the new VNC feature in VMware Workstation 6 where you can configure the same VNC functionality using a the VMware GUI interface. For more information on how VNC and VMware work, see this VMware Technology Network article: Using a VNC Client to Connect to Virtual Machines.

Summary

The ability to manage VMware virtual machines using the built-in VMware VNC server is a little known feature that could be helpful to many of you out there. This feature is easy to turn on and can even provide access to the VM without the OS booted. Disadvantages include that the VM must be powered on, basic security, and the inconvenience of having to use a different port for each VM. Overall, this is a very cool feature that all VMware Server users should know about. Even if you choose not to enable this feature today, keep it in your toolbox because you never know when you will need it.

Got a question? Post it on our VMware Forums!