Bug 1341621 - Avoid deadlocks in DMD when forking. r=njn

In order to avoid the possibility of a deadlock if the DMD state lock is
currently acquired when forking a |pthread_atfork| hook is added to wait for
and acquire the lock prior to forking, then release it after forking.

memory/replace/dmd/DMD.cpp

@@ -1546,6 +1546,22 @@ NopStackWalkCallback(uint32_t aFrameNumber, void* aPc, void* aSp,
 }
 #endif
 
+static void
+prefork()
+{
+  if (gStateLock) {
+    gStateLock->Lock();
+  }
+}
+
+static void
+postfork()
+{
+  if (gStateLock) {
+    gStateLock->Unlock();
+  }
+}
+
 // WARNING: this function runs *very* early -- before all static initializers
 // have run.  For this reason, non-scalar globals such as gStateLock and
 // gStackTraceTable are allocated dynamically (so we can guarantee their
@@ -1556,6 +1572,16 @@ Init(const malloc_table_t* aMallocTable)
   gMallocTable = aMallocTable;
   gDMDBridge = InfallibleAllocPolicy::new_<DMDBridge>();
 
+#ifndef XP_WIN
+  // Avoid deadlocks when forking by acquiring our state lock prior to forking
+  // and releasing it after forking. See |LogAlloc|'s |replace_init| for
+  // in-depth details.
+  //
+  // Note: This must run after attempting an allocation so as to give the
+  // system malloc a chance to insert its own atfork handler.
+  pthread_atfork(prefork, postfork, postfork);
+#endif
+
   // DMD is controlled by the |DMD| environment variable.
   const char* e = getenv("DMD");