Bug 1300974 - Work around race condition leading to deadlock on fork when enabling LogAlloc. r=njn
authorMike Hommey <mh+mozilla@glandium.org>
Wed, 07 Sep 2016 13:51:34 +0900
changeset 313059 f3aec6a563866b1ea2e8621b3db91513ed889e60
parent 313058 4c87875a0befd207544a65313cf3b1a603c11f4f
child 313060 54ba4ea8bdc1a427cc7f085e8e3183171c8d6104
push id81538
push usermh@glandium.org
push dateWed, 07 Sep 2016 22:56:41 +0000
treeherdermozilla-inbound@f3aec6a56386 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersnjn
bugs1300974
milestone51.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1300974 - Work around race condition leading to deadlock on fork when enabling LogAlloc. r=njn
memory/replace/logalloc/LogAlloc.cpp
--- a/memory/replace/logalloc/LogAlloc.cpp
+++ b/memory/replace/logalloc/LogAlloc.cpp
@@ -81,17 +81,38 @@ replace_init(const malloc_table_t* aTabl
 #ifndef _WIN32
   /* When another thread has acquired a lock before forking, the child
    * process will inherit the lock state but the thread, being nonexistent
    * in the child process, will never release it, leading to a dead-lock
    * whenever the child process gets the lock. We thus need to ensure no
    * other thread is holding the lock before forking, by acquiring it
    * ourselves, and releasing it after forking, both in the parent and child
    * processes.
-   * Windows doesn't have this problem since there is no fork(). */
+   * Windows doesn't have this problem since there is no fork().
+   * The real allocator, however, might be doing the same thing (jemalloc
+   * does). But pthread_atfork `prepare` handlers (first argument) are
+   * processed in reverse order they were established. But replace_init
+   * runs before the real allocator has had any chance to initialize and
+   * call pthread_atfork itself. This leads to its prefork running before
+   * ours. This leads to a race condition that can lead to a deadlock like
+   * the following:
+   *   - thread A forks.
+   *   - libc calls real allocator's prefork, so thread A holds the real
+   *     allocator lock.
+   *   - thread B calls malloc, which calls our replace_malloc.
+   *   - consequently, thread B holds our lock.
+   *   - thread B then proceeds to call the real allocator's malloc, and
+   *     waits for the real allocator's lock, which thread A holds.
+   *   - libc calls our prefork, so thread A waits for our lock, which
+   *     thread B holds.
+   * To avoid this race condition, the real allocator's prefork must be
+   * called after ours, which means it needs to be registered before ours.
+   * So trick the real allocator into initializing itself without more side
+   * effects by calling malloc with a size it can't possibly allocate. */
+  sFuncs->malloc(-1);
   pthread_atfork(prefork, postfork, postfork);
 #endif
 
   /* Initialize output file descriptor from the MALLOC_LOG environment
    * variable. Numbers up to 9999 are considered as a preopened file
    * descriptor number. Other values are considered as a file name. */
   char* log = getenv("MALLOC_LOG");
   if (log && *log) {