Bug 1272782 - Don't wait forever for child process to exit; r=ahal

I believe this is the source of hangs/timeouts in automation.
join() waits forever. We add code to wait at most N seconds before
force terminating the process. The timeout is a bit high. But it is
better than infinite.

MozReview-Commit-ID: KwyO4RZ9OqL

--HG--
extra : rebase_source : 767d8ff5b48d7e75ab8fe72b18145446a38d439a

testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py

@@ -312,8 +312,17 @@ class SystemResourceMonitor(object):
             self.measurements.append(SystemResourceUsage(start_time, end_time,
                 cpu_times, cpu_percent, io, virt, swap))
 
-        self._process.join()
-        assert done
+        # We establish a timeout so we don't hang forever if the child
+        # process has crashed.
+        self._process.join(10)
+        if self._process.is_alive():
+            self._process.terminate()
+            self._process.join(10)
+        else:
+            # We should have received a "done" message from the
+            # child indicating it shut down properly. This only
+            # happens if the child shuts down cleanly.
+            assert done
 
         if len(self.measurements):
             self.start_time = self.measurements[0].start