Filesystem goes read-only without the underlying storage reporting errors

  • 3735930
  • 20-Sep-2007
  • 27-Apr-2012

Environment

Novell SUSE Linux Enterprise Server 10
Novell SUSE Linux Enterprise Desktop 10
Novell SUSE Linux Enterprise Server 9
Novell Open Enterprise Server 1(Linux based)
Novell Linux Desktop 9

ext3, reiserfs or udf filesystem


Situation

Filesystem (ext3, reiserfs or udf) is remounted read-only by the kernel.
No errors are reported from the storage layer underlying the affected filesystem.

Resolution

For SLE 10 (Novell SUSE Linux Enterprise 10), this issue has been resolved for the ext3 filesystem as of kernel update 2.6.16.27-0.9, released Feb 23, 2007. For the reiserfs and udf filesystems, this issue is fixed as of the release of Service Pack 1 (kernel 2.6.16.46-0.12).

For SLE 9 (Novell SUSE Linux Enterprise 9) and OES 1, this issue is resolved for ext3, reiserfs and udf as of kernel 2.6.5-7.286.

Additional Information

When the filesystem code detects the occurrence of an error from which recovery without an fsck is not possible, it changes state to read-only to prevent further corruption.

Normally, this behavior occurs when the underlying storage layer (like SCSI) reported an error when the filesystem code was trying to write out vital data. Errors or warnings reported by the underlying storage layer will precede the messages from the filesystem layer and the issue can only be addressed by fixing the hardware and/or configuration problem with the storage layer.

With a system under heavy load, a bug in the read-ahead optimization part of the ext3, reiserfs and udf code could be triggered in which it mistook a temporary error condition from the storage layer ("please retry again later") for an unrecoverable error.In this case, only the filesystem layer reports errors; these errors are not preceded by errors or warnings reported by the underlying storage layer.


In the case of ext3, messages similar to the following are
logged:
ERROR: EXT3-fs error (device sda1): ext3_readdir: directory #133120 containsa hole at offset 4096
ERROR: Aborting journal on device sda1.
ERROR: ext3_abort called.
ERROR: EXT3-fs abort (device sda1):
ERROR: ext3_journal_start: Detected aborted journal
ERROR: Remounting filesystem read-only