ASoC: SOF: simplify snd_sof_device_remove()

The commit "ASoC: SOF: core: Unregister machine driver before IPC and
debugfs" moved the call to unregister the machine driver to be done
before freeing the IPC. With this change, we can safely move the call to
snd_sof_remove() inside the same if() block just above.

Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
Reviewed-by: Bard Liao <yung-chuan.liao@linux.intel.com>
Reviewed-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Link: https://lore.kernel.org/r/20220307181111.49392-3-ranjani.sridharan@linux.intel.com
Signed-off-by: Mark Brown <broonie@kernel.org>

sound/soc/sof/core.c

@@ -442,16 +442,8 @@ int snd_sof_device_remove(struct device *dev)
 
 		snd_sof_ipc_free(sdev);
 		snd_sof_free_debug(sdev);
-	}
-
-	/*
-	 * Unregistering the machine driver results in unloading the topology.
-	 * Some widgets, ex: scheduler, attempt to power down the core they are
-	 * scheduled on, when they are unloaded. Therefore, the DSP must be
-	 * removed only after the topology has been unloaded.
-	 */
-	if (sdev->fw_state > SOF_FW_BOOT_NOT_STARTED)
 		snd_sof_remove(sdev);
+	}
 
 	/* release firmware */
 	snd_sof_fw_unload(sdev);