Bug 1766684 - Use compositor device for hardware video decoding only with hardware WebRender r=jrmuizel,gfx-reviewers

Do not use compositor device for decoder device when SW-WR + D3D11 is used.

When SW-WR + D3D11 is used, there is a case that RenderDXGITextureHost::MapPlane() is called. It readbacks video frame data from ID3D11Texture2D. Before doing the readback, copying video data/decoding video data needs to be completed. But we could not use keyed mutex for the ID3D11Texture2D.

To prevent the problem, SyncObjectD3D11Client::Synchronize() is called in D3D11DXVA2Manager::CopyToImage(). But the Synchronize() is called only when decoder device is not compositor device. Then when compositor device is not used for decoder device, the readback problem does not happen.

Differential Revision: https://phabricator.services.mozilla.com/D147049

dom/media/platforms/wmf/DXVA2Manager.cpp

@@ -912,7 +912,10 @@ D3D11DXVA2Manager::InitInternal(layers::KnowsCompositor* aKnowsCompositor,
   mDevice = aDevice;
 
   if (!mDevice) {
-    mDevice = gfx::DeviceManagerDx::Get()->CreateDecoderDevice();
+    bool useHardwareWebRender =
+        aKnowsCompositor && aKnowsCompositor->UsingHardwareWebRender();
+    mDevice =
+        gfx::DeviceManagerDx::Get()->CreateDecoderDevice(useHardwareWebRender);
     if (!mDevice) {
       aFailureReason.AssignLiteral("Failed to create D3D11 device for decoder");
       return E_FAIL;

gfx/thebes/DeviceManagerDx.cpp

@@ -862,7 +862,8 @@ FeatureStatus DeviceManagerDx::CreateContentDevice() {
   return FeatureStatus::Available;
 }
 
-RefPtr<ID3D11Device> DeviceManagerDx::CreateDecoderDevice() {
+RefPtr<ID3D11Device> DeviceManagerDx::CreateDecoderDevice(
+    bool aHardwareWebRender) {
   bool isAMD = false;
   {
     MutexAutoLock lock(mDeviceLock);
@@ -884,8 +885,9 @@ RefPtr<ID3D11Device> DeviceManagerDx::CreateDecoderDevice() {
   }
 
   if (reuseDevice) {
-    if (mCompositorDevice && mCompositorDeviceSupportsVideo &&
-        !mDecoderDevice) {
+    // Use mCompositorDevice for decoder device only for hardware WebRender.
+    if (aHardwareWebRender && mCompositorDevice &&
+        mCompositorDeviceSupportsVideo && !mDecoderDevice) {
       mDecoderDevice = mCompositorDevice;
 
       RefPtr<ID3D10Multithread> multi;

gfx/thebes/DeviceManagerDx.h

@@ -61,7 +61,7 @@ class DeviceManagerDx final {
   RefPtr<ID3D11Device> GetImageDevice();
   RefPtr<IDCompositionDevice2> GetDirectCompositionDevice();
   RefPtr<ID3D11Device> GetVRDevice();
-  RefPtr<ID3D11Device> CreateDecoderDevice();
+  RefPtr<ID3D11Device> CreateDecoderDevice(bool aHardwareWebRender);
   IDirectDraw7* GetDirectDraw();
 
   unsigned GetCompositorFeatureLevel() const;