Optimizing the Linux Kernel to Work with the Processor of Your Choice
Novell Cool Solutions: Feature
By Kevin Burnett
Digg This -
Posted: 18 May 2005
- Enter Linux
- Linux Performance Optimization
- Speed Optimization
- Enter Linux
- Compiling the Kernel
- Recompile Critical Applications and Utilities
- Recompile Everything!
- Add RAM and more CPUs
- Disk Access
- Optimizing Memory
- Compile a Monolithic Kernel
- Loose X Windows
- Remove Virtual Consoles
- Remove Daemons
I would think that many of you have been in the situation where you have been waiting for your computer to finish something. I have a computer running a very common operating system that just stops processing for no reason at all, and stays frozen for several minutes before allowing me to resume using it. And the funny things is, I am using current hardware and software.
I would suspect that it would be an obvious question as to why you would want a fast and well tuned computer system. Granted, it may not be easy to diagnose performance problems, but having an understanding of things you can do to resolve these issues, and better yet, prevent these issues from ever occurring would be a good thing. Fortunately, most common operating systems provide some sort of tools to optimize the operating system to the hardware. This is where Linux in particular shines. By using a few basic and sometimes advanced tools, you can determine what the problems are with your system. In my case, hopefully figuring out a way to prevent the system stopping functioning incidents.
Granted, a slow-performing system can cause grief to common users, but will definitely be an unacceptable commodity to a software developer. If you are a system administrator, you have a greater burden to provide your users with well running computers. Typically excepting the engineering experts, IS people are tasked with keeping the computers of the lay employees in top shape. Slow or hesitant computers can not be tolerated. If the systems are not performing acceptably, you will soon know, as your phone will be ringing non-stop. Knowing how to effectively enhance performance will be a plus for you and can mean the difference between spending days and spending hours on performance problems.
Since I am a Novell employee, I realize that I should be pro-Linux, and I am, but less than a year ago I was a consultant traveling all over the country solving companies networking problems. Granted, some of these companies have made the move to Linux, and they love it. Linux does give you a lot of advantages over a commercial operating system such as Windows. Quickly put, Linux gives you access to the kernel source code, and most of the applications and kernel extensions are available with source. Another advantage is that you can get most of Novell's flagship products on Linux, with several of the products being open-sourced. Another neat extra with Linux is all of the Internet sites that specialize in help forums, documentation, and design comments.
Linux was so designed and is an excellent environment to troubleshoot problems. Once you get the hang of how the kernel works – processes information, and master a few debugging tools, it will be a piece of cake to find and fix performance problems. Contrast this to what you would have to do for a proprietary operating system. Can you say, "talk to an engineer?"
The remainder of this article will look at ways to improve the performance of a Linux system, both server and workstation. It will also look at optimization with a variety of different processors/hardware platforms.
If you have been in the software industry for a decade or more, you probably remember the early years of computing when software programs had to be written in assembly to run quickly and efficiently. Do you remember when 64K was a LOT of memory? 4.77mhz was an adequate computer? I remember when WordPerfect was the word processor to have. Most of that application was originally written in assembly. It wasn't a nicety; it had to be done for performance. Also, several versions of NetWare had the kernel written in assembly. Why, because of the need for fast performance. Boy do times change. Currently versions of Windows operating system are extremely fat. Fat, meaning needing significant amounts of RAM, disc space and other hardware resources. Add to that a sloppily written C++ application that requires enormous resources and where are you? Thank heavens hardware has advanced to support these new and improved programming practices!
With Linux, performance optimizations don't always mean significantly increasing the speed of the computer; sometimes it is about tuning the system to fit into a small memory footprint.
It is important to keep in mind that the purpose of a computer is to save you time. There is really not any point in spending hours tweaking a system, unless you actually enjoy doing that. So basically what we want to do is get the system running well, well enough, that is, to get things done efficiently.
When I think of speed, I am reminded of the classic Roadrunner, Wiley Coyote cartoons. Try as Mr. Coyote does, he never seems to catch his nemesis the Roadrunner.
The same is true with a computer system. I have some good friends that I would refer to as Linux-heads. They will try to squeeze every last ounce of performance from a Linux installation. However, this is not practical for a typical IS/IT staff to accomplish. What we need to look at is things that we can do to Linux to make it run well, run efficiently and get the job done.
I once remember reading a statement in one of those industry journals where they have quotable quotes. The quote went something like this, "It really doesn't matter whether an application is fast, as long as it looks fast." As I thought about this, it is so true. If the user interaction with an application: input and output, is fast, then the application code itself can suffer a bit and nobody will notice.
When you look at the typical Linux distribution, it is amazing how many CPU cycles are spent with screen manipulation. Putting those little pixels on your screen can really eat up the cycles! Granted, if you get a faster system, the Linux OS will run faster, but what is really happening is that the screen I/O will actually use less CPU cycles freeing up more cycles for the kernel threads and applications threads underneath.
Take for example a slower system, say a Pentium 200MHz box with 512Mb RAM. With a stock Linux desktop installation, you are going to probably see a somewhat sluggish performance when you use a GUI desktop. You may find out that switching GUI desktops from KDE to GNOME may make a slight difference. But the big key with this type of system is to get familiar with your hardware. In my case, I discovered that I had an Intel motherboard, which featured a built-in 82845G Extreme Graphics chip. One thing you are always told to do, but not very many people do, is keep you system drivers up to date. Since I was seeing poor performance from the GUI interface I was using. And since my Linux distribution was up to date, I went to Intel's site on the Internet, checking for video drivers. I located the video driver for the 8245G video chip and downloaded the latest version. Man! What a difference a little driver can make. My video increased in speed in an order that was very noticeable. My pokey hardware all of a sudden became acceptable hardware for my lab.
So, really, the first step in getting your system tweaked performance wise is to locate the latest versions of the device drivers required by your system. Might I suggest two Internet sites to help you: http://www.google.com/linux and http://groups.google.com.
NOTE: It is a good thing to kind of get a feel for how your system is running. Get a feeling for possible bottlenecks. Also get familiar with utilities such as KDE's ktop program, which is very similar to Window's Task Manager.
The first time you install the Linux desktop, if you install in verbose mode, you probably noticed the tons and tons of packages that are installed. But what you probably didn't notice is that there are specific compilations of the Linux kernel optimized for the processor of your choice. If you think about this, what if a Linux distribution vendor compiled all of their packages for the Pentium 4 processor? They would probably get a lot of comments from the Linux community from people who wanted to run his distribution on other processors such as Pentium II, III or even older chips like the 486. On the converse side of things, what if you have a state-of-the-art computer with a huge RAM configuration, XEON processor, etc.? You would be out of luck too! Luckily most Linux distributors offer Kernel compilations optimized for a variety of processors. Looking at the table, below, we see examples of kernel compilations optimized for different processors offered with the Red Hat Linux distribution.
|Kernel-2.4.18-14.athlon.rpm||Optimized for AMD Athlon processor|
|Kernel-2.4.18-14.i586.rpm||Optimized for Intel Pentium processor|
|Kernel-2.4.18-14.i686.rmp||Optimized for Intel XEON processor|
|Kernel-bigmem-2.4.18-14.i686.rpm||Optimized for Pentium machines with 4GB or more of RAM|
|Kernel-smp-2.4.18-14.athlon.rpm||Optimized for computers with two or more AMD Athlon processors|
|Kernel-smp-2.4.18-14.i686.rpm||Optimized for computers with two or more Pentium processors|
|Kernel-uml.2.4.18-14.i686.rpm||Linux kernel compiled as a conventional program (user-mode Linux)|
Be very careful when selecting the kernel compilation that you want to run. Selecting the wrong one can give you a serious performance hit. You may want to get adventurous and compile your own kernel. This has some very desired possibilities. One is that you can specify you're the processor precisely. For example, Pentium, Pentium-MMX, Pentium III, Pentium 4, Ahtlon, Duron, etc. Also, you can select processor specific options that will further refine the efficiency of the kernel's performance.
Recompiling applications and utilities can play a very important role in performance if you are a power user such as an accountant. If you spend the better part of your day in one application, such as a Word Processor or Spreadsheet, then most of your CPU cycles will be there too! With Linux, there is a very good chance that you use Open Office. And if you do, the best thing you can do in these type of circumstances would be to download the Open Office source code and recompile the application for your specific system. Granted, this would be the IS department's responsibility, but it can make a big difference to the users involved.
I know that there are those Engineers and IS people who want the very best for their organization. They are extremely dedicated and typically live at their companies. For these individuals, you may want to investigate a Linux distribution that has the options of recompiling everything, in the distribution, from scratch, for your processor. Talk about conforming to your particular setup! Most Linux distributions will allow you to do this.
OK, so I am going against what I said earlier about trying to get Linux to run well on older hardware, but when all else fails, it really doesn't hurt to add memory. You really can't have too much memory. I used to work for a small consulting firm. They provided me with a laptop with 2GB of RAM. When I left that company, I purchased a similar laptop with only 1GB of RAM (I couldn't afford the extra GB!) The difference in application performance was amazing! I am doing well with my 1GB, but I do miss the 2GB.
Also, in a lot of cases, if you have one of something, isn't it better to have 2 or more? Consider getting a system that supports two or more processors. As stated above, the Linux kernel supports multiple processors. Wouldn't it be nice to run the kernel threads on one processor and application threads on another? Sounds like Heaven to me!
I have noticed that you can buy rack-mounted computers from the likes of HP and Dell for prices that you would have paid for nice home computers a few years ago. This temps me to pick one up, add a few more processors, and set it up in the basement for my server.
Disk access can play a big part in system performance. When you think about it, disk access can be over 100,000 or more time slower than memory access. This is why your Windows system can crawl to a stop when accessing virtual memory. Simply put, using virtual memory is a bad idea. If you can avoid it, do. But we can't do without a hard drive, so let's investigate a few things that can be done to improve disk performance.
There is a Linux hard disk command called hdparm, which sets various flags and modes on the IDE disk driver subsystem. A couple of options that are worth investigating are the –c option, which can set 32-bit I/O support, and –d which can enable or disable the using_dma flag. Granted, most major Linux distributions will set the using_dma flag to 1, but if yours doesn't, you will be suffering a major performance hit. Try changing this flag by putting the following at the end of your /etc/rc.d/rc.local file:
hdparm –d 1 /dev/had
Also, most Linux systems are set up for 16-bit I/O. Switching this to 32-bit might speed up the disk subsystem a bit. Do this in the same file with this command:
hdparm –c 1 /dev/had
There are a couple of ways to gauge hard disk performance. You can use the following command:
to test read performance from the drives you make changes to. However, make sure you do this on either a test machine or one that is lightly loaded. Run this command several times and average the results from these runs.
A second option would be to look into a utility written by Russell Coker called bonnie++. This can be found at: http://www.coker.com.au/bonnie++/
NOTE: Be cautious about creating multiple file systems on a single physical drive. A terrible situation would be to have /usr at the outside of the drive and /home on the inside. Your poor hard drive would be continually seeking the whole space of the hard drive surface! Multiple drive arrays can be more efficient and minimize problems like this. See the next section.
RAID stands for Redundant Array of Inexpensive Drives. They are becoming much more affordable now and offer redundancy and reliability. Typically RAID 1 mirrors data on two or more hard drives. Reads are very quick since the data can be read from multiple drives simultaneously, but writes can be slower since the data must be written across all the drives in the array. RAID 5, which stripes data and parity across multiple drives, also offers great performance. But it is important to keep in mind that the software implemented RAID 1 support of the Linux kernel is nowhere near as effective as even a low-cost RAID hardware controller.
RAID 5 Illustrated
Optimizing memory can be good for fast machines, but can also keep your older boxes useful. I have a 90's 486/33 which I upgraded to a 486/66. With this machine, the problem is small memory and a slow processor. The following are things that can be done to optimize memory.
Typically the kernels supplied with current Linux distributions are quite modular. They can configure themselves at boot time or even at run time. This is done by loading the required drivers from among the hundreds that are shipped with the distribution. You can lighten your kernel immensely by building the required device driver code into the kernel at compile time. This avoids the overhead of run-time linking modules and the space occupied by stub code and the kernel module loader. The negative side of this option is that it can be difficult to implement. For example, change a network card and you will need to rebuild the kernel.
The X Window System uses a large amount of memory. Couple with that the desktop environment of either KDE or GNOME and you use a bunch more memory. Typically, uses for older machines include DNS or DHCP servers, they do not need a graphical interface. So get rid of it by editing /etc/inittab and setting the ID value to 3, instead of 5.
If you do need a GUI, then you have the option of getting a lightweight one. Most alternative ones are smaller than KDE or GNOME. Check out http://www.plig.org/xwinman/ for some options.
The typical Linux desktop will all you to use Alt+F1 through Alt+F6 to select six different virtual consoles. Alt+F7 will switch you to the graphical desktop. So, if you choose to get rid of X Windows (see above) you have the option to remove one or more of these virtual consoles, as they also consume a lot of memory. Do this as follows:
Edit /etc/inittab and scroll down to the end of the file. You will see several virtual consoles. Comment out four or more. Samples will be mingetty or mgetty.
There are several daemons that you probably don't need in order to run a DNS ot DHCP server. For example, you will not need sendmail running. Use ntsysv, ksysv, or chkconfig commands to disable as many daemons that you don't need. Good possibilities include apmd, autofs, gpm, identd, ip6tables, iptables, isdn, kudzu, lpd, nfs, portmao and talkd.
The best advice I can give you is probably the one item that most of you are not in a position to do; that is, take extreme care when installing your Linux distribution. You might as well be warned that letting a Linux distribution install by default probably will end up not performing the way you would like it to. Select individual packages during the install. This can take some time, but most Linux distributions have script-driven install utilities that will allow you to repeat the install with less effort, thereby fine-tuning the packages without too much trouble.
Also, one other thing to note is that using multiple file systems, while attractive for various reasons, will tend to waste a lot of disk space. You may want to stick with one favorite file system to maintain a better use of memory.
Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com