Deadlocks occur when one thread holds a resource lock (A) that it needs to update and is waiting on another resource (lock B), which a different thread has already locked and cannot abandon because it cannot finish its work. If the work of the second thread includes gaining access to lock A, which is held by the first thread, the deadlock is a classic ABBA. The first thread has A and waits on B, while the second thread has B and is waiting on A.
You can avoid deadlocks in an NLM™ in one of two ways:
(Preferred) Design a hierarchy of lock acquisition into the implementation that restricts access to locks to a sequence or policy that is identical for all threads.
NOTE:In the future, LibC should help to prototype and prove such policies by means of a debugging mechanism in its mutex and reader-writer lock interfaces. These will be compile-time debugging features to help find policy infractions, but they will not prevent them from happening.
Use try lock functions to test lock availability.
However, this approach is costly and potentially sloppy. In the case of ABBA, if we acquire A and check for B, we must release A if B cannot be acquired in order to avoid the deadlock. Consequently, an access loop must be set up.
Nevertheless, situations exist in which a perfect hierarchical lock acquisition policy cannot be set up or must be violated. Try locking is useful in these situations.
Because they are so difficult to debug, the most serious deadlocks are those of inter-NLM blocking and cancellation. If a thread winds its way through unknown NLM code, acquires a lock, and is then cancelled by unloading or suspending, the NLM that owns the locking code could be damaged in its ability to access an important resource. As NLM shutdown code is executed by the system console thread, in the case of an unload request from the command line, the console can become useless if it cannot suspend or kill the NLM because one of its threads is hanging on a mutex. This scenario happens if the thread that acquired the mutex was killed while still holding it.
The cardinal rule is that no thread can exit while holding any lock.