As the Wine PE conversion process marches toward the finish line, a question of mine has stuck around for a long time:
Why is the *.sys modules, which emulate the Windows kernel drivers, being converted to PE?
As lots of efforts have been put into the process so far, I asume there is a valid reason for it; however, I'm yet to find any explanations on the list. Here are the best explanations I found so far:
- "PE Conversion", Giovanni Mascellani, https://www.winehq.org/pipermail/wine-devel/2021-August/193127.html - "wine modules", Eric Pouech, https://www.winehq.org/pipermail/wine-devel/2022-April/213677.html
To begin with, the rationale of the conversion process *in general* is twofold:
1. Accurate emulation of Windows userspace. This includes the system call interface. Such effort is necessary to appease some DRM and anti-cheat technologies.
2. Wow64 support. Some platforms (e.g. macOS) are either planning to drop, or have already dropped support for 32-bit binaries. To work around this, Wine adopts a "32-on-64" model where 32-bit code is run on 64-bit processes thanks to CPU and operating system support.
Now, here's the reasons I think that both of the above do not necessarily apply to drivers:
1. Accurate emulation of Windows userspace. As kernel drivers run in kernel mode, a syscall interface does not make sense at all.
2. Wow64 support. Note that 64-bit Windows does not support running 32-bit drivers. I don't necessary see a significant value in supporting 32-bit binaries on a Wow64 wineprefix, but others may have different opinion.
Also, drivers tend to interface with the host platform *a lot*, so it's much more difficult than, say, purely native modules whose PE conversion is only a matter of getting rid of "-mcygwin".
Looking forward for your input. Thanks a lot!
Hi,
I am not directly involved in any of those, but here are my best guesses:
Am 03.07.2022 um 21:33 schrieb Jinoh Kang jinoh.kang.kr@gmail.com:
- Accurate emulation of Windows userspace. As kernel drivers run in kernel
mode, a syscall interface does not make sense at all.
Not all of them. See e.g. winspool.drv, which is something applications can link against. Might have inherited the "wrong" file ending somewhere, but it is mostly a regular DLL otherwise.
- Wow64 support. Note that 64-bit Windows does not support running 32-bit
drivers. I don't necessary see a significant value in supporting 32-bit binaries on a Wow64 wineprefix, but others may have different opinion.
I think there are two things affecting this:
*) Application-provided drivers (DRM etc) still need a Windows-ish environment. They run in the same space our .drv's run. Afaics those drivers can also link against our .sys modules, e.g. the dinput tests contain a driver that links against hidclass.sys.
*) aarch64 and x86_64 (on mac) still need a non-trivial syscall thunk for TLS registers. I am not sure if there is any kind of TLS support in a .drv in the Windows kernel though.
*) We might want to run 32 bit DRM drivers on 64 bit even if Windows doesn't support that. We can currently load old 32 bit safedisk 2 drivers and want to retain that. Windows solved this problem by providing a rewrite of the safedisk drivers. I don't know how they handle kernel DRM modules on aarch64.
I think it is the combination of those points why we want to have a windows-y kernel environment and not take the host environment 1:1.
On 7/4/22 05:19, Stefan Dösinger wrote:
Hi,
I am not directly involved in any of those, but here are my best guesses:
Am 03.07.2022 um 21:33 schrieb Jinoh Kang jinoh.kang.kr@gmail.com:
- Accurate emulation of Windows userspace. As kernel drivers run in kernel
mode, a syscall interface does not make sense at all.
Not all of them. See e.g. winspool.drv, which is something applications can link against. Might have inherited the "wrong" file ending somewhere, but it is mostly a regular DLL otherwise.
I'm aware that a few drivers (e.g. some *.drv files) are actually userland modules mostly as a relic from Windows 9x days. Right now, my main focus are on _kernel_ drivers, i.e. the ones directly loaded alongside NTOSKRNL (or, on winedevice.exe in case of Wine). Apologies if it wasn't clear enough.
- Wow64 support. Note that 64-bit Windows does not support running 32-bit
drivers. I don't necessary see a significant value in supporting 32-bit binaries on a Wow64 wineprefix, but others may have different opinion.
I think there are two things affecting this:
*) Application-provided drivers (DRM etc) still need a Windows-ish environment. They run in the same space our .drv's run. Afaics those drivers can also link against our .sys modules, e.g. the dinput tests contain a driver that links against hidclass.sys.
The thing is, the environment of NT Kernel/Executive differs greatly from that of the Win32 userspace, so we already kind of fail when we don't have proper IRQL / POPF / privileged instruction semantics.
That said, IMHO fakedll may not matter much unless the driver is doing something really weird (beyond hotpatching) with those modules or reading from that PE file directly. I can see how one could want to do that with NTOSKRNL.EXE or HAL.DLL, but other PnP drivers / filters less so (modulo hotpatching, of course).
*) aarch64 and x86_64 (on mac) still need a non-trivial syscall thunk for TLS registers. I am not sure if there is any kind of TLS support in a .drv in the Windows kernel though.
I see how thunk for transition of thread register would be necessary for anything that interact with custom drivers, although it doesn't have to be in a form of a syscall (if a custom thunk would be simpler to implement than a full unixlib split). Also, the "TLS" in NT Kernel is actually per-cpu rather than per-thread.
*) We might want to run 32 bit DRM drivers on 64 bit even if Windows doesn't support that. We can currently load old 32 bit safedisk 2 drivers and want to retain that. Windows solved this problem by providing a rewrite of the safedisk drivers. I don't know how they handle kernel DRM modules on aarch64.
I think more work would be involved in translating 64-bit calls to 32-bit I/O requests (FastIO or IRP), but a basic support would be an intereseting idea.
Ideally the device driver shall be integrated with host, with Wine merely acting as a translation layer. Not sure if it could happen, though...
I think it is the combination of those points why we want to have a windows-y kernel environment and not take the host environment 1:1.
Thank you very much for your thoughtful response!
Am 04.07.2022 um 00:17 schrieb Jinoh Kang jinoh.kang.kr@gmail.com:
Ideally the device driver shall be integrated with host, with Wine merely acting as a translation layer. Not sure if it could happen, though...
Yeah we don't even do that for the limited amount of driver support we have. Transforming Wine into a super-ndiswrapper that can integrate all kinds of device drivers with Linux is a huge undertaking and I am not sure I would be a fan of it.
If necessary it could be done on a case by case basis. FUSE for file systems, iSCSI / vhba for block devices, tun/tap for network adapters, etc. I think integrating Windows printer drivers with CUPS might be an achievable and useful goal. Wine once upon a time could use Win16 printer drivers.
I think it is the combination of those points why we want to have a windows-y kernel environment and not take the host environment 1:1.
Thank you very much for your thoughtful response!
You seem to have better understanding of the NT kernel world than I do; Maybe someone else can contribute more than my guesswork.
On 7/5/22 18:44, Stefan Dösinger wrote:
Am 04.07.2022 um 00:17 schrieb Jinoh Kang jinoh.kang.kr@gmail.com:
Ideally the device driver shall be integrated with host, with Wine merely acting as a translation layer. Not sure if it could happen, though...
Yeah we don't even do that for the limited amount of driver support we have. Transforming Wine into a super-ndiswrapper that can integrate all kinds of device drivers with Linux is a huge undertaking and I am not sure I would be a fan of it.
If necessary it could be done on a case by case basis. FUSE for file systems, iSCSI / vhba for block devices, tun/tap for network adapters, etc. I think integrating Windows printer drivers with CUPS might be an achievable and useful goal. Wine once upon a time could use Win16 printer drivers.
I didn't mean NDISwrapper. I simply meant that the host operating system should add proper device support, barring any licensing restrictions. Perhaps the "translation layer" part was misleading.
A more interesting endeavor would be continuing the "wineserver-inside-kernel" work.
I think it is the combination of those points why we want to have a windows-y kernel environment and not take the host environment 1:1.
Thank you very much for your thoughtful response!
You seem to have better understanding of the NT kernel world than I do; Maybe someone else can contribute more than my guesswork.
-
Jinoh Kang -
Stefan Dösinger