LTSP - Linux Terminal Server Project

Jim McQuillan (jam@ltsp.org)

v1.11 18 September 1999


Linux makes a great platform for deploying diskless workstations that boot from a network server. We have started an open source project to create the administration tools that will make setting up a diskless workstatione easier. This document describes how to obtain the ltsp distribution and how to install it.

1. Introduction

This project was born out of the need to solve a problem for a customer who needed a terminal that could communicate with both an IBM AS/400 and a Unix application server. It needed to run TCP/IP, it needed to be inexpensive, and it needed to be easy to maintain. As a plus, it should allow the user to browse the web, and allow them to read and send email.

We could have used PC's running windows, the software is certainly available, but the cost would have been high. Both in terms of the initial investment, and the cost of supporting and maintaining the PC's over time.

We decided that a diskless workstation running the Linux kernel and X-Windows would fit nicely as a solution to the customer problem.

We didn't really invent anything new here, we searched the web and found the etherboot and netboot packages. Then we found an inexpensive ne2000 network interface card that had an eprom socket. We searched and found an affordable eprom burner, figured out what kind of eprom chips to buy, learned about bootp, xdm, nfs-root and all kinds of other things. Then, we put it all together. Initially we tried it on an old 486 PC we had lying around and it worked pretty well.

So, we installed a server and 11 workstations. The customer loved it. They ordered an additional 22 workstations, and they will probably add many more in the future.

After several months, and virtually no support problems with the workstations, we have decided to share our solution with the rest of the world.

The package we developed works very well for our uses. Hopefully, it will solve problems for others as well.

If you have any questions or comments about the package, please feel free to email me at jam@ltsp.org. I look forward to hearing how it is working for you.

2. Description of the diskless workstation

When the workstation is booted, it does the following:

  1. It gets it's ip address from a bootp server
  2. It downloads the kernel from a tftp server
  3. It mounts it's root filesystem from an nfs server
  4. It loads the X-Server software into memory and begins executing it
  5. It contacts an XDM server and allows the user to log into it

In most cases, the bootp server, tftp server, nfs server and xdm server will all be the same machine, we will refer to that simply as the 'Server', and we will refer to the diskless workstation as the 'Workstation'.

Once the workstation is booted, and the user logs in, any application programs that they invoke will be running on the server, while the output is being displayed on the workstation. This is a fundamental feature of X-Windows. The workstation is only running the Linux kernel, XFree86, Init and possibly an lpd daemon for printing to the local printer.

Because there is very little running on the workstation, you can get away with a fairly inexpensive, low-power machine. We found in our initial testing that an i486 with 16mb of ram was actually pretty good at being an X terminal.

We used the Etherboot package available from: The Etherboot Home Page. That takes care of the image that gets burned into the eprom, and it helps us prepare the kernel for downloading to a workstation, but it stops there.

There are several HOWTO's that explain how to setup a server for diskless booting of a single workstation, but they don't discuss the problems involved in serving many workstations from a single server. The problem is that when the workstation is running, it needs to write to some files on the server, so each workstation needs to mount it's own unique root filesystem. If you had 50 workstations, you would need 50 directory trees exported. This can be a real pain to try to manage.

We have developed a method of setting up the root filesystem hierarchy that can be shared among all of the workstations. The kernel mounts the filesystem in readonly mode, then mounts a 4mb ramdisk as it's /tmp filesystem. While the kernel and XFree86 are running, they like to update a few files, so we have placed those files on the ramdisk, and created symbolic links to them in the proper place within the hierarchy.

Additionally, we have created a configuration file and a program that runs as part of the workstation bootup sequence. Each workstation can include different hardware. Things such as Network board, Video Card and the type of mouse can be configured either as a default, or individually for each workstation.

Because there aren't any application programs running on the workstation, we don't need a swap device, although it is possible to configure the kernel to swap to an NFS filesystem.

This method of booting a workstation is being used very successfully on network with 36 workstations all running from a single server running RedHat 6.0 on a 400mhz Pentium-II. Each workstation is a 166mhz Pentium machine with 32mb of ram. We could have used a smaller processor, but these days it is getting pretty tough to find the low-end cpus.

In the future, we would like to offer the ability to run more programs on the workstation. Many people have suggested that they would like the window manager and a web browser such as Netscape running locally on the workstation.

3. Theory of operation

Booting a diskless workstation involves several steps. Understanding what is happening along the way will make it much easier to solve problems, should they arise.

This description of the events is assuming that there is a server configured to handle the booting of a workstation.

  1. When you turn on the workstation, it will go through it's power on self test (POST) and the bootcode in the eprom on the network card will begin executing.
  2. The bootcode will attempt to detect a network card. Once it detects the card, it will initialize it.
  3. The bootcode will then broadcast a bootp request to the local network. The bootp request will include the MAC address of the network card.
  4. The inetd process on the server will see the broadcast and invoke the bootpd daemon to respond to the request.
  5. The bootpd process will read it's configuration file, /etc/bootptab, and locate the entry that matches the MAC address that was sent in the request. Once the entry is found, it will be put into a reply packet and sent back to the workstation that requested the information. Several pieces of information will be passed back in the reply, the items that are important at this point are:
    1. IP address assigned to the workstation ('ip=')
    2. NETMASK setting for the local network ('sm=')
    3. Bootfile home directory ('hd=')
    4. The name of the kernel to download ('bf=')
  6. The bootcode will receive the reply from bootp and it will configure the TCP/IP interface in the network card with the parameters that were supplied.
  7. The bootcode will then send a TFTP request to the server to begin downloading the kernel from the server.
  8. Once the kernel has been completely downloaded to the workstation, the bootcode will do a jump to the starting code in the kernel.
  9. The kernel will then start executing, initializing the entire system and all of the peripherals.
  10. The kernel will issue another bootp broadcast to the network, looking for all of it's networking parameters. Note, the bootcode does NOT pass the information to the kernel, the kernel MUST request the information for itself.
  11. The server will respond with another reply packet, containing the information that the kernel needs to continue. The relevent items in the reply this time are:
    1. IP address assigned to the workstation ('ip=')
    2. NETMASK setting for the local network ('sm=')
    3. The root directory to be mounted via NFS ('rp=')
    4. The gateway ('gw=')
    5. The DNS server ('ds=')
    6. The hostname of the workstation (The value of the first field in the bootptab entry)
    Once all of the above parameters have been passed back to the workstation, the network interface will be configured and brought up.
  12. The root filesystem will be mounted via NFS. This filesystem will be mounted read-only. We do this because we may have many workstations mounting the same filesystem, and we don't want any of the workstations to modify the contents of the root filesystem.
  13. At this point, control will be passed from the kernel to the 'init' process.
  14. init will read the /etc/inittab file and begin setting up the environment.
  15. One of the first items in the inittab file is the rc.local command that will be run while the workstation is in the 'sysinit' state.
  16. The rc.local script will create a 4mb ramdisk to contain all of the things that need to be written to or modified in any way.
  17. This ramdisk will be mounted as the /tmp directory. Any files that need to be written will actually exist in the /tmp directory, and there are symbolic links pointing to these files. For example, when the workstation is running, it will try to modify the permissions on the /dev/tty0 device node. If the device node actually existed in the /dev directory, the permissions would not be modifiable, because the root filesystem is read-only. So, we have created symbolic links for all of the files and created the actual files/nodes in the /tmp directory (which is writeable).
  18. The /proc filesystem will be mounted.
  19. The loopback network interface will be configured.
  20. Several directories will be created under the /tmp filesystem for holding some of the transient files that are needed while the system is running. Directories such as:
    1. /tmp/compiled
    2. /tmp/var
    3. /tmp/var/run
    4. /tmp/var/log
    5. /tmp/var/lock
    6. /tmp/var/lock/subsys
    will all be created.
  21. The /etc/XF86Config file will be generated based on entries in the /tftpboot/lts/ltsroot/etc/lts.conf configuration file. This is where information about the type of mouse, and other X parameters are combined to create the config file for X.
  22. The /tmp/startx script will be created. This script will determine which X server to run, and the IP address of the server running xdm. This is Based on information found in the /tftpboot/lts/ltsroot/etc/lts.conf file.
  23. The /tmp/syslog.conf file will be created. This file will contain information telling the syslogd daemon which host on the network to send the logging information to. The syslog host is specified in the lts.conf file. There is a symbolic link called /etc/syslog.conf that points to the /tmp/syslog.conf file.
  24. The syslogd daemon is started, using the config file that was just created.
  25. Control is passed back to init. Init will then look at the initdefault entry to determine which runlevel to enter.
  26. If runlevel 3 is specified, then a shell will be started on the console. This is good for doing trouble shooting.
  27. If runlevel 5 is specified, then the /tmp/startx script will be invoked, which will bringup X.
  28. When X is started, it will send and XDMCP query to the server, which will cause the login dialog box to be displayed.
  29. Once the user logs in, they will be running an X session from the server. That is, if they bring up an xterm session, it will be running on the server, and it will be displaying it's output on the workstation.
Basically, we now have an X station.

4. Setting up the server

We have created RPM's containing all of the pieces you will need to setup a RedHat 6.0 system as the server.

You can download all of the software from the LTSP download page or you can ftp the files from the LTSP ftp server.

4.1 Packages available for download

XFree86-xfs-3.3.3.1-52.i386.rpm

Upgrade of the X Font server.

bootp-2.4.3-7.i386.rpm

Bootp daemon.

lts_core-1.0.rpm

Core LTS package, contains the root filesystem, including the configuration utilities and documentation for the workstations. The documentation will be installed in the /usr/doc/lts-1.0 directory. There are versions of this document in HTML, SGML, Postscript and Text.

lts_kernel_ne2000-1.0.rpm

Pre-compiled kernel for diskless booting with a NE2000 compatible network adapter. The .config file for this kernel is installed in /tftpboot/lts/kernel_configs/config.ne2000.

lts_kernel_rtl8139-1.0.rpm

Pre-compiled kernel for diskless booting with a network adapter containing a RealTek-8139 chipset. An example is an SMC-1211TX 10/100 adapter. The .config file for this kernel is installed in /tftpboot/lts/kernel_configs/config.rtl8139.

lts_xsvga-1.0.rpm

This package contains the Xserver for an SVGA video card. This was compiled from XFree86-3.3.5. This newer Xserver is needed to support the SiS-620 video chipset found in the current Jammin-50 workstations.

lts_xmach64-1.0.rpm

This package contains the Xserver for an Mach64 video card. This was obtained from the RedHat 6.0 distribution.

All of these packages are available at LTSP ftp server.

4.2 Planning the IP-Address scheme

Each machine in your network needs a unique IP address. We have chosen one of the reserved Class-C networks, 192.168.0.0. Of coarse, you are free to choose any addresses you want.

For the server, we chose 192.168.0.254, and for the workstations, we started with 192.168.0.1 and climb up from there, this gives us the possibility of having 253 workstations associated with a single server. If you need more workstations than that, you can either have multiple servers, each with a different Class-C, or you can go to a Class-B address, and have 65533 workstations all running from a single server. (Sounds pretty cool eh?)

We kept the names of the workstations simple, starting with 'ws001' and going up.

4.3 Upgrade the xfs package

The version of xfs that ships with Redhat-6.0 doesn't handle remote workstations. There is now an updated xfs that you can download from Redhat or from ftp.ltsp.org. You will need XFree86-xfs-3.3.3.1-52.i386.rpm or later.

The command you need to use to install the upgrade is:

rpm -U XFree86-xfs-3.3.3.1-52.i386.rpm

Once you install the upgrade, you will need to modify the startup script so that it will serve fonts to remote workstations. The file to edit is: /etc/rc.d/init.d/xfs. There are two lines that need to be modified. Look for the lines that start with daemon --check xfs There are two of them, one is on or around line 22 and the other is on or around line 41.

You need to modify the port from -1 to 7100, so the resulting line should read:

daemon --check xfs su xfs -c \"xfs -port 7100\" -s /bin/sh
This change will allow the X font server daemon to serve fonts to remote workstations.

A change MUST now be made to the /etc/X11/XF86Config file.

Look for the line that says:

FontPath   "unix/:-1"
You will need to change it to:
FontPath   "tcp/localhost:7100"

4.4 Install bootpd

Redhat doesn't include bootpd on the 6.0 distribution CD, so you will need to get it from an earlier version, download it from Redhat, or you can download it from ftp.ltsp.org.

The command to install the bootp package is:

rpm -i bootp-2.4.3-7.i386.rpm

This will install the bootp daemon and a sample configuration file.

4.5 Install the lts_core package

Download the lts_core-1.0.rpm file from ftp.ltsp.org and install it using the following command:

rpm -i lts_core-1.0.rpm
This sets up the /tftpboot/lts directory, which contains the basic root hierarchy that will be mounted as the root filesystem of the workstation. It will also add entries to the /etc/bootptab and the /etc/exports files.

4.6 Choose the appropriate kernel

When the workstation boots, it pulls a kernel from the server and loads it into memory. The kernel that it loads needs to be configured specifically for network booting, and it needs to have the proper network card driver included. It cannot load modules at this point. We have prepared a few kernels that are ready to go, you will just need to pick the correct one for your network card, download it and install it.

We have rpms available on our ftp site. Files such as: lts_kernel_ne2000-1.0.rpm and lts_kernel_ne2000-1.0.rpm can be downloaded and installed. For example, to install the Linux kernel for a NE2000 network card, do the following:

rpm -i lts_kernel_ne2000-1.0.rpm 
This will put the vmlinuz.ne2000 kernel in the /tftpboot/lts directory. You can install other workstation kernels the same way.

4.7 Choose the X server

You will need to pick the correct X server that is compatible with your video card. The XF86_SVGA server works with most video chipsets, but you may get better performance by using the server made specifically for your chipset.

You can use the Xservers that ship with Redhat, or you can download one of these RPM's from ftp.ltsp.org that will automatically install themselves in the proper location.

If you are using a SiS-620 card, you will need to get the 3.3.5 version of XF86_SVGA, earlier versions of the server don't properly support the SiS chipset.

We have rpms available on our ftp site. Files such as: lts_xmach64-1.0.rpm and lts_xsvga-1.0.rpm can be downloaded and installed.

For example, to install the X server for SVGA do the following:

rpm -i lts_xsvga-1.0.rpm 
This will put the XF86_SVGA server in the /tftpboot/lts/ltsroot/usr/X11R6/bin directory.

4.8 Edit the configuration files

The installation of the lts_core package will add entries to several configuration files, but these entries may need to be modified for your specific needs.

/etc/inetd.conf

By default, the lines in the inetd.conf file for the bootp and tftp servers are commented out. Search for the following lines in /etc/inetd.conf and remove the leading '#' characters.

#
#tftp   dgram   udp     wait    root    /usr/sbin/tcpd  in.tftpd
#bootps dgram   udp     wait    root    /usr/sbin/tcpd  bootpd
# 

/etc/X11/xdm/Xservers

You need to decide if you want X to run on the server automatically. If you want the graphical login screen on the console when you boot the system, then you don't have to worry about modifying the Xservers file. If you don't want X to startup automatically on the console, then you will need to comment out the line in the Xservers file that controls that.

The file to edit is /etc/X11/xdm/Xservers. Look for the line that shows:

:0 local /usr/X11R6/bin/X 
If you comment the line out by inserting a '#' character at the beginning of the line, that will prevent xdm from bringing up X on the console.

If you want to run X after the server has booted, you can run startx at the comment line.

/etc/inittab

The server needs to have xdm running. This is usually started from the /etc/inittab file. There are multiple xdm type servers available on a Redhat 6.0 system. (xdm, gdm and kdm)

We are using good old xdm.

In /etc/inittab, you will find a line that reads:

x:5:respawn:/etc/X11/prefdm -nodaemon 
By default, prefdm is a symbolic link to /usr/bin/gdm. You can change this by deleting the link, and creating a new one that points to xdm.
rm /etc/X11/prefdm
ln -s /usr/bin/X11/xdm /etc/X11/prefdm 
Normally, xdm runs when the system is in runlevel 5. Your system may be configured to run in runlevel 3. You can tell by looking at the line in /etc/inittab that contains initdefault. If it shows:
id:3:initdefault:

then you should change it to show:

id:5:initdefault: 

/etc/bootptab

When you install the lts_core package, it adds the following entries to the /etc/bootptab file:

## LTS-begin ####################################################
.ltsp:\
  :ht=ethernet:\
  :ds=192.168.0.254:\
  :gw=192.168.0.254:\
  :lg=192.168.0.254:\
  :sm=255.255.255.0:\
  :hn:\
  :hd=/tftpboot/lts:\
  :rp=/tftpboot/lts/ltsroot:
#
# The following is an example of a line needed for a workstation
#
# ws001:tc=.ltsp:ha=AABBCCDDEEFF:bf=vmlinuz.ne2000:ip=192.168.0.1:
## LTS-end ######################################################
You will need to create a line for each workstation. You can use the example line (commented out) as a template for the lines you add.

You will need to fill in the ha= entry with the ethernet address from the network card, you will need to fill in the bf= entry with the name of the kernel that you want the workstation to load, and you will need to fill in the ip= entry with the IP address that you want to assign to the workstation.

Also, if you are using a class-c other than the default of 192.168.0.0, then you will need to change the ds, gw and lg entries above.

/etc/hosts

Once you have added the workstation to the bootptab file, make sure you add the IP-Address and name of the workstation to either the /etc/hosts file, or set it up in your DNS tables either on the server, or somewhere on your network. The NFS server MUST be able to resolve the IP address to a name.

/etc/exports

The default entry that is put into this file should be Ok. Unless you are using a Class-C other than 192.168.0.0. If you are using a different class-c, then you need to modify this file to show it. The default entry looks like:

## LTS-begin ## 
/tftpboot/lts/ltsroot   192.168.0.0/255.255.255.0(ro,no_root_squash)
## LTS-end ##

/etc/rc.d/init.d/syslog

This is the script that starts syslogd when the system boots. You need to change it to allow other devices on the network to send log messages to the server.

Go to line 22 of the /etc/rc.d/init.d/syslog file, which reads:

daemon syslogd -m 0 
and change it to:
daemon syslogd -m 0 -r 

/tftpboot/lts/ltsroot/etc/lts.conf

This is the config file for the workstations. Most of the configurable parameters for the workstations can be specified here.

The config file is comprised of sections, each section represents a workstation, and there is a Default section.

Section headers contain the name of the workstation or the word 'Default', surrounded by square brackets ('[' and ']'). If all of the workstations are identical, then you could get away with just having a Default section.

Example of a /tftpboot/lts/ltsroot/etc/lts.conf file:

[Default]
        XSERVER            = XF86_SVGA
        SERVER             = 192.168.0.254
        X_MOUSE_PROTOCOL   = "PS/2"
        X_MOUSE_DEVICE     = "/dev/psaux"
        X_MOUSE_RESOLUTION = 400
        X_MOUSE_BUTTONS    = 3

[ws001]
        XSERVER            = XF86_SVGA
        X_MOUSE_PROTOCOL   = "Microsoft"
        X_MOUSE_DEVICE     = "/dev/ttyS1"
        X_MOUSE_RESOLUTION = 50
        X_MOUSE_BUTTONS    = 3
        X_MOUSE_BAUD       = 1200

[ws002]
        XSERVER            = XF86_Mach64

[ws003]
        XSERVER            = XF86_SVGA
        X_COLOR_DEPTH      = 24
        

Available parameters are:

SERVER

This is the server that is used for both the XDM_SERVER and SYSLOG_HOST, if one or both of those are not specified explicitly. If you have one machine that is acting as the server for everything, (XDM,SYSLOG), then you can just specify the address here and ommit the other server parameters.

XDM_SERVER

If you want to point XDM to a machine other than the default server, then you can specify the server here. If this parameter is NOT specified, then it will use the 'SERVER' parameter described above.

SYSLOG_HOST

If you want to send logging messages to a machine other than the default server, then you can specify the machine here. If this parameter is NOT specified, then it will use the 'SERVER' parameter described above.

XSERVER

This defines which X server the workstation will run. Possible values include: XF86_SVGA and XF86_Mach64. Any other XFree86 X server should work, as long as it has been installed in the /tftpboot/lts/ltsroot/usr/X11R6/bin directory. The default value for this is XF86_SVGA.

X_MOUSE_PROTOCOL

Any value that will work for the XFree86 Pointer Protocol keyword can be put here. Typical values include "Microsoft" and "PS/2". The default value for this is "PS/2".

X_MOUSE_DEVICE

This is the device node that the mouse is connected to. If it is a serial mouse, this would be a serial port, such as /dev/ttyS0 or /dev/ttyS1. If it is a PS/2 keyboard mouse, this value would be /dev/psaux. The default value for this is /dev/psaux.

X_MOUSE_RESOLUTION

This is the 'Resolution' value in the XF86Config file. A typical value for a serial mouse is 50 and a typical value for a PS/2 mouse would be 400. The default value for this is 400

X_BUTTONS

This tells the system how many buttons the mouse has. Usually set to 2 or 3. The default value for this is 3

X_MOUSE_BAUD

For serial mice, this defines the baud rate. The default value for this is 1200.

X_COLOR_DEPTH

This is the number of bits to use for the color depth. Possible values are 8, 15, 16, 24 and 32. 8 bits will give 256 colors, 16 will give 65536 colors, 24 will give 16 million colors and 32 bits will give 4.2 billion colors! Not all X servers support all of these values. The default value for this is 16.

Comments

Comments start with the hash '#' sign and continue through the end of the line.

4.9 Restarting the daemons

After creating and/or modifying the configuration files, several daemons need to be restarted, to take advantage of the changes. You can restart each of them individually, or you can simply reboot the system.

xfs

Stop the daemon by running:

/etc/rc.d/init.d/xfs  stop
Then, start the daemon by running:
/etc/rc.d/init.d/xfs  start
For some reason, 'xfs restart' doesn't work correctly. That is why we have to stop it first, then start it up.

NFS

exportfs -ra

bootpd

killall -q -HUP bootpd

syslogd

/etc/rc.d/init.d/syslog restart

inetd

killall -HUP inetd

5. Testing

Reboot the server after making all of the above changes. Then, just turn on the workstation and you should see it query the network for it's IP information, then you should see a tftp transfer of the kernel and the kernel will begin booting. You should then see X-Windows come up, and finally, a login window should appear.

If it fails somewhere along the way go back through the instructions above and double check the setup. If you are still having trouble, email me at jam@ltsp.org and I will try to help you out.

6. Troubleshooting

  1. The workstation displays:
       NE2000 base 0x0300, addr XX:XX:XX:XX:XX:XX
       Searching for server (BOOTP)...
       <sleep>
       <sleep> 
    
    This indicates that the workstation is unable to find a bootp server. Check the following:
    1. Cabling - See if you have a link light on the card and/or hub
    2. Bootpd daemon - Make sure the bootpd process is configured in the /etc/inetd.conf file on the server.
    3. /etc/bootptab file - Make sure the address shown on the screen exists in the /etc/bootptab file on the server.

7. Additional references

  1. Linux Terminal Server Project (LTSP) home page
  2. DisklessWorkstations.com home page
  3. Diskless-Nodes HOW-TO document for Linux
  4. Etherboot Home Page
  5. XFree86-Video-Timings-HOWTO