You can now connect to WSL distros when you're connected to a remote Windows machine by using Remote - SSH. The usual entrypoints, including the Remote Explorer and the WSL: Reopen in WSL command, work as expected.