Two Paths to Server Performance
I/O scheduler and file system selection can boost SUSE Linux Enterprise server performance
Written by Matthias G. Eckermann and Bill Tobey
Making Per-Device I/O Scheduler Assignments
If you have multiple applications running on a server, using different storage environments, it’s possible to make separate I/O scheduler assignments to optimize the performance of each application-storage pair. These assignments can even be changed in production. You can check the I/O scheduler setting for individual storage devices with the following shell command: /sys/block/*DEV*/queue/iosched.
If desired, you can then re-set the I/O scheduler assignment for each device using this command: echo SCHEDNAME > /sys/block/*DEV*/queue/scheduler.
Integrity, Performance and the Barrier In Between
Barriers are a feature the kernel’s block I/O subsystem makes available to journaling file systems to protect data integrity. A barrier request temporarily locks the I/O scheduler’s execution queue, ensuring that a sequence of journal write requests are securely committed to physical media before any subsequent requests are served. Barriers protect metadata and ensure file system integrity in the event of a system crash, but they do so at the expense of a noticeable performance penalty. Novell assumes a higher value for data integrity than performance in enterprise computing environments, so barrier support is switched on by default in SUSE Linux Enterprise Server. It can be turned off to improve performance, but only by a knowledgeable administrator prepared to assume the risk.
- With reiserFS you can enable / disable barriers using the mount options: barrier=flush or barrier=none.
- With ext3 you can enable / disable barriers using the mount options: barriers=1 or barriers=0.
- With XFS you can enable / disable barriers using the mount options: barrier or nobarrier.
File System Selection for Server Performance
Another set-up decision that can significantly affect server performance is the choice of file systems. As is the case with I/O schedulers, SUSE Linux Enterprise Server ships with a number of file system alternatives, allowing administrators to match file systems and application workloads for optimum performance. Here are a few guidelines for making the right performance pick.
- Choose ReiserFS for small files – ReiserFS is best suited for applications that generate lots of small files, including mail, NFS and database servers, and for applications that use synchronous I/O.
- Choose ext3 for small file systems – Ext3, the default file system in SUSE Linux Enterprise Server 11, is best suited for small file systems of 100 gigabytes or less. If you’re planning to use ext3 with large numbers of files in a single directory, you should consider enabling btree support. This can be accomplished with the shell command: # mkfs.ext3 -O dir_index. Note that btree support is enabled by default in version 11 SP1.
- Choose XFS for large files and streaming media – XFS is an excellent choice for large files and medium to very large file systems (up to 16 terabytes on 32-bit systems, or a theoretical 8 exabytes on 64-bit systems). Its low latency transfer characteristics also make it an ideal selection for streaming media applications. SUSE Linux Enterprise has supported XFS since version 8, and Novell is working closely with SGI to optimize its performance with future releases of SUSE Linux Enterprise Server. It merits consideration for any file system likely to exceed 100 gigabytes, unless other factors (e.g. many small files) dictate another choice.
XFS also offers a number of special features that can be particularly useful, including dump / restore for backup and recovery, online file system check, and online defragmentation.
- Choose OCFS2 fo cluster performance or high availability – Oracle Cluster File System 2 (OCFS2) is a POSIX-compliant shared-disk cluster file system for Linux that is developed by the community under GPL. Because it provides local file system semantics, OCFS2 can be used with any application. Cluster-aware applications can leverage its parallel I/O support for higher performance, other applications can leverage its multi-node support to achieve higher availability through automated failover.
- Consider btrfs for the future – Btrfs is a new, copy on write file system for Linux aimed at bringing additional enterprise class file system features to the Linux kernel. Initially developed by Oracle, btrfs is licensed under the GPL and has quickly been adopted by the community. Long-awaited features include integrated volume management, copy on write, writable snapshots (and snapshots of snapshots), extents, dynamic inode allocation, checksums on data and metadata, online file system check and defragmentation, and integrated multiple device support.
Btrfs is still under intense development, but is included as a technology preview in SUSE Linux Enterprise Server 11 SP1.
Measuring I/O Scheduler and File System Performance
Once you’ve made your I/O scheduler and file system selections, there are many tools available to measure your configured system’s performance. A few favorites include:
- Bonnie is a simple but useful tool that provides a variety of benchmarks on the speed of your file system, OS caching, the underlying device and your libc. It is supported in the SUSE Linux Enterprise Server distribution.
- fio is an I/O tool meant to be used both for benchmark and stress/hardware verification. It has support for 13 different types of I/O engines. It is available at: http://freshmeat.net/projects/fio/. Packages for SUSE Linux Enterprise Server 11 are available at: http://download.opensuse.org/repositories/benchmark/SLE_11/.
- IOzone is a file system benchmark tool that generates and measures a variety of file operations. Iozone has been ported to many machines and runs under many operating systems. It is available at: http://www.iozone.org/ and http://download.opensuse.org/repositories/home:/mge1512:/benchmarking/SLE_11/.
Experiment to Find Your Optimum Configuration
I/O scheduler and file system selection can have major effects on the performance of SUSE Linux Enterprise Servers. We strongly recommend that you experiment with different configurations to gain first hand experience. And watch for our upcoming article on cgroups for kernel resource management.