Hi,
I have an interesting bug (maybe?) for you. I don't know what's the best way to explain so I will just retrace how I found out what's happening.
I was investigating Bug 58216 (completely unrelated, but this is how I encountered this problem). When I started the game, it just hangs with a black screen. It didn't crash but it didn't start either. Nothing stood out in the logs, so I attached winedbg. It showed a thread stuck in mmdevapi.dll::DriverProc, handling a DRV_FREE. The notify_thread wasn't stopping, clearly the midi_release call isn't causing the midi_notify_wait to return.
So I added debug logging to alsamidi.c, and they weren't being hit. Turning on +mmdevapi debug channel shows winealsa.drv was being loaded as the MIDI driver. What is going on? I was perplexed. Single stepping into the unix call to midi_release showed it somehow called into pulse_not_implemented. Turning on +module and I found this:
0024:trace:module:load_native_dll found L"C:\windows\system32\winepulse.drv" for L"\??\C:\windows\system32\winealsa.drv"
This is bizarre. Reading the code I saw a few server calls for memory mapping stuff, so I turned on +server. And I saw a create_mapping call made with a file handle for winealsa.drv, but a later get_mapping_info call on that mapping returned winepulse.drv. Okay, I was reading wineserver code now. In the create_mapping function we have:
mapping->fd = get_fd_object_for_mapping( ... )
And get_fd_object_for_mapping can replace the given fd with another fd of the same underlying inode#. Ah ha! I look at the inode of winealsa.drv and winepulse.drv, and yeah they are identical.
* * *
But how? I run NixOS. On NixOS all installed software packages come out of the "nix store", and nix store has an "optimise" feature which replaces identical files in the nix store with hardlinks. winealsa.drv and winepulse.drv are empty dlls only there for loading their corresponding .so unixlibs, that's why they are identical, and were replaced by hardlinks.
Question: what's the right thing to do here? Considering hardlinked files indistinguishable for mapping purposes seems reasonable, and I won't be surprised if there are testcases testing specifically this. But adding unixlibs to the mix and things change. Now the path from which a module is accessed becomes significant too, because it will be used to find the .so. In this case the result was the wrong unixlib being used.
Help? Thanks.
On Saturday, 20 September 2025 01:03:22 CDT Yuxuan Shui wrote:
Question: what's the right thing to do here? Considering hardlinked files indistinguishable for mapping purposes seems reasonable, and I won't be surprised if there are testcases testing specifically this. But adding unixlibs to the mix and things change. Now the path from which a module is accessed becomes significant too, because it will be used to find the .so. In this case the result was the wrong unixlib being used.
I think NixOS is doing the wrong thing here personally. I don't know if I can point to spec language, but if I create two different files with the same content, they are still different files and should by definition have different serial numbers. I don't think it's wrong that Wine depends on this.
On the other hand, I am reminded of an unrelated bug, involving a program (I don't have the name unfortunately) which copies ws2_32.dll to a temp file with a randomly generated name and attempts to load the copy. This fails on Wine because ws2_32 has a unixlib, and we derive the name of the unixlib from the path of the DLL. In order to fix this bug we would probably need to embed the name of the unixlib in the DLL itself rather than relying on its path. If we did that the DLLs for winepulse and winealsa would no longer be identical.
On 9/22/25 12:14, Elizabeth Figura wrote:
On the other hand, I am reminded of an unrelated bug, involving a program (I don't have the name unfortunately) which copies ws2_32.dll to a temp file with a randomly generated name and attempts to load the copy. This fails on Wine because ws2_32 has a unixlib, and we derive the name of the unixlib from the path of the DLL. In order to fix this bug we would probably need to embed the name of the unixlib in the DLL itself rather than relying on its path. If we did that the DLLs for winepulse and winealsa would no longer be identical.
That doesn't look quite related to the discussed issue? This part is about how we find Unix library. I didn't check exactly this case lately but doesn't Wine search for Unix libraries in the configured or determined on start Wine paths now? I'd guess it might just work?
For the original issue, having the same inodes for different files looks clearly like a bug in NixOS to me. First, yes, if there are two differently named files they are different files even if the contents happens to be the same. For one, the contents is identical right now but may change, and if those are differently named files the contents is not synchronous between those. Then, inode is supposed to be unique within the same file system: https://man7.org/linux/man-pages/man7/inode.7.html, "Inode number": "Each file in a filesystem has a unique inode number.".
On 22. Sep 2025, at 20:31, Paul Gofman pgofman@codeweavers.com wrote:
On 9/22/25 12:14, Elizabeth Figura wrote:
On the other hand, I am reminded of an unrelated bug, involving a program (I don't have the name unfortunately) which copies ws2_32.dll to a temp file with a randomly generated name and attempts to load the copy. This fails on Wine because ws2_32 has a unixlib, and we derive the name of the unixlib from the path of the DLL. In order to fix this bug we would probably need to embed the name of the unixlib in the DLL itself rather than relying on its path. If we did that the DLLs for winepulse and winealsa would no longer be identical.
That doesn't look quite related to the discussed issue? This part is about how we find Unix library. I didn't check exactly this case lately but doesn't Wine search for Unix libraries in the configured or determined on start Wine paths now? I'd guess it might just work?
For the original issue, having the same inodes for different files looks clearly like a bug in NixOS to me. First, yes, if there are two differently named files they are different files even if the contents happens to be the same. For one, the contents is identical right now but may change, and if those are differently named files the contents is not synchronous between those. Then, inode is supposed to be unique within the same file system: https://man7.org/linux/man-pages/man7/inode.7.html, "Inode number": "Each file in a filesystem has a unique inode number.".
I think they can mostly get away with this because the nix store is by design read-only (and there are in practice many duplicates there). Though I agree that such a form of deduplication should happen on the filesystem layer ideally.
On 9/22/25 12:37, Marc-Aurel Zent wrote:
On 22. Sep 2025, at 20:31, Paul Gofman pgofman@codeweavers.com wrote:
On 9/22/25 12:14, Elizabeth Figura wrote:
On the other hand, I am reminded of an unrelated bug, involving a program (I don't have the name unfortunately) which copies ws2_32.dll to a temp file with a randomly generated name and attempts to load the copy. This fails on Wine because ws2_32 has a unixlib, and we derive the name of the unixlib from the path of the DLL. In order to fix this bug we would probably need to embed the name of the unixlib in the DLL itself rather than relying on its path. If we did that the DLLs for winepulse and winealsa would no longer be identical.
That doesn't look quite related to the discussed issue? This part is about how we find Unix library. I didn't check exactly this case lately but doesn't Wine search for Unix libraries in the configured or determined on start Wine paths now? I'd guess it might just work?
For the original issue, having the same inodes for different files looks clearly like a bug in NixOS to me. First, yes, if there are two differently named files they are different files even if the contents happens to be the same. For one, the contents is identical right now but may change, and if those are differently named files the contents is not synchronous between those. Then, inode is supposed to be unique within the same file system: https://man7.org/linux/man-pages/man7/inode.7.html, "Inode number": "Each file in a filesystem has a unique inode number.".
I think they can mostly get away with this because the nix store is by design read-only (and there are in practice many duplicates there). Though I agree that such a form of deduplication should happen on the filesystem layer ideally.
But also I don't quite understand from the description why that happens to winepulse.drv and winealsa.drv, these files are not indentical and have different contents??
On 9/22/25 12:39, Paul Gofman wrote:
On 9/22/25 12:37, Marc-Aurel Zent wrote:
On 22. Sep 2025, at 20:31, Paul Gofman pgofman@codeweavers.com wrote:
On 9/22/25 12:14, Elizabeth Figura wrote:
On the other hand, I am reminded of an unrelated bug, involving a program (I don't have the name unfortunately) which copies ws2_32.dll to a temp file with a randomly generated name and attempts to load the copy. This fails on Wine because ws2_32 has a unixlib, and we derive the name of the unixlib from the path of the DLL. In order to fix this bug we would probably need to embed the name of the unixlib in the DLL itself rather than relying on its path. If we did that the DLLs for winepulse and winealsa would no longer be identical.
That doesn't look quite related to the discussed issue? This part is about how we find Unix library. I didn't check exactly this case lately but doesn't Wine search for Unix libraries in the configured or determined on start Wine paths now? I'd guess it might just work?
For the original issue, having the same inodes for different files looks clearly like a bug in NixOS to me. First, yes, if there are two differently named files they are different files even if the contents happens to be the same. For one, the contents is identical right now but may change, and if those are differently named files the contents is not synchronous between those. Then, inode is supposed to be unique within the same file system: https://man7.org/linux/man-pages/man7/inode.7.html, "Inode number": "Each file in a filesystem has a unique inode number.".
I think they can mostly get away with this because the nix store is by design read-only (and there are in practice many duplicates there). Though I agree that such a form of deduplication should happen on the filesystem layer ideally.
But also I don't quite understand from the description why that happens to winepulse.drv and winealsa.drv, these files are not indentical and have different contents??
In any case, different files can't have same inode number in the same FS outside of a bug. The same file may have different names hardlinked to the same file. Inode is exactly what identifies the file per se on Unix filesystems.
On 22. Sep 2025, at 20:39, Paul Gofman pgofman@codeweavers.com wrote:
On 9/22/25 12:37, Marc-Aurel Zent wrote:
On 22. Sep 2025, at 20:31, Paul Gofman pgofman@codeweavers.com wrote:
On 9/22/25 12:14, Elizabeth Figura wrote:
On the other hand, I am reminded of an unrelated bug, involving a program (I don't have the name unfortunately) which copies ws2_32.dll to a temp file with a randomly generated name and attempts to load the copy. This fails on Wine because ws2_32 has a unixlib, and we derive the name of the unixlib from the path of the DLL. In order to fix this bug we would probably need to embed the name of the unixlib in the DLL itself rather than relying on its path. If we did that the DLLs for winepulse and winealsa would no longer be identical.
That doesn't look quite related to the discussed issue? This part is about how we find Unix library. I didn't check exactly this case lately but doesn't Wine search for Unix libraries in the configured or determined on start Wine paths now? I'd guess it might just work?
For the original issue, having the same inodes for different files looks clearly like a bug in NixOS to me. First, yes, if there are two differently named files they are different files even if the contents happens to be the same. For one, the contents is identical right now but may change, and if those are differently named files the contents is not synchronous between those. Then, inode is supposed to be unique within the same file system: https://man7.org/linux/man-pages/man7/inode.7.html, "Inode number": "Each file in a filesystem has a unique inode number.".
I think they can mostly get away with this because the nix store is by design read-only (and there are in practice many duplicates there). Though I agree that such a form of deduplication should happen on the filesystem layer ideally.
But also I don't quite understand from the description why that happens to winepulse.drv and winealsa.drv, these files are not indentical and have different contents??
For me at least winepulse.drv and winealsa.drv are in content exactly identical. If nix were to deduplicate only cross-derivation this would not happen I think.
For me at least winepulse.drv and winealsa.drv are in content exactly identical.
Maybe this could be a side-effect of reproducible builds? Here the hashes aren't identical for those two libraries
For me at least winepulse. drv and winealsa. drv are in content exactly identical. Maybe this could be a side-effect of reproducible builds? Here the hashes aren't identical for those two libraries
and winealsa.drv are in content exactly identical.
Maybe this could be a side-effect of reproducible builds? Here the hashes aren't identical for those two libraries
Probably, yes - in PE builds, the binaries would not be exactly identical, because https://gitlab.winehq.org/wine/wine/-/commit/b0908666639d5dfe825b5d5f3b9893b... was only a partial fix. Winebuild writes the export table with a reproducible hash (of the filename) to produce the export table timetime, but mingw-ld from binutils still introduces a duplicate export table (named "dll stuff" if you add -Wl,-Map=) that gets populated at https://github.com/bminor/binutils-gdb/blob/f86c456bec7969779dd044c313e419f8...
This is just junk data, but it ends up in the file following the one generated by winebuild that readers actually use. Now that winegcc uses Wl,--exclude-all-symbols this extra table now has zero symbols in it, but it still makes a duplicate export directory table, but that header still has a timestamp in it https://github.com/bminor/binutils-gdb/blob/f86c456bec7969779dd044c313e419f8...
However, NixOS is all about reproducible builds, so I'd expect their binutils to use SOURCE_DATE_EPOCH. --no-insert-timestamp, or something, like that to remove the BFD timestamps in general, and then one might get identical binaries.
I dug into this at one point curious about the large/non-reproducible .edata section, found !3173, but decided the few remaining bytes of "dll stuff" junk data were basically harmless, so I never bothered to report it... but it seems topical to this thread. Certainly I think we'd rather not have the duplicate export directory table, if there was some way to get binutils to skip producing it. But there doesn't seem to be anything obvious...
Public
Hi,
On Mon, Sep 22, 2025 at 7:49 PM Marc-Aurel Zent mzent@codeweavers.com wrote:
On 22. Sep 2025, at 20:39, Paul Gofman pgofman@codeweavers.com wrote:
On 9/22/25 12:37, Marc-Aurel Zent wrote:
On 22. Sep 2025, at 20:31, Paul Gofman pgofman@codeweavers.com wrote:
On 9/22/25 12:14, Elizabeth Figura wrote:
On the other hand, I am reminded of an unrelated bug, involving a program (I don't have the name unfortunately) which copies ws2_32.dll to a temp file with a randomly generated name and attempts to load the copy. This fails on Wine because ws2_32 has a unixlib, and we derive the name of the unixlib from the path of the DLL. In order to fix this bug we would probably need to embed the name of the unixlib in the DLL itself rather than relying on its path. If we did that the DLLs for winepulse and winealsa would no longer be identical.
That doesn't look quite related to the discussed issue? This part is about how we find Unix library. I didn't check exactly this case lately but doesn't Wine search for Unix libraries in the configured or determined on start Wine paths now? I'd guess it might just work?
For the original issue, having the same inodes for different files looks clearly like a bug in NixOS to me. First, yes, if there are two differently named files they are different files even if the contents happens to be the same. For one, the contents is identical right now but may change, and if those are differently named files the contents is not synchronous between those. Then, inode is supposed to be unique within the same file system: https://man7.org/linux/man-pages/man7/inode.7.html, "Inode number": "Each file in a filesystem has a unique inode number.".
I think they can mostly get away with this because the nix store is by design read-only (and there are in practice many duplicates there). Though I agree that such a form of deduplication should happen on the filesystem layer ideally.
But also I don't quite understand from the description why that happens to winepulse.drv and winealsa.drv, these files are not indentical and have different contents??
For me at least winepulse.drv and winealsa.drv are in content exactly identical. If nix were to deduplicate only cross-derivation this would not happen I think.
Yes, this will be a solution indeed. I think I saw people mentioning this on the nix issue tracker.
BTW the nixpkgs issue discussion is here: https://github.com/NixOS/nixpkgs/issues/444543
On 9/22/25 19:20, Yuxuan Shui wrote:
Hi,
On Mon, Sep 22, 2025 at 7:49 PM Marc-Aurel Zentmzent@codeweavers.com wrote:
On 22. Sep 2025, at 20:39, Paul Gofmanpgofman@codeweavers.com wrote:
On 9/22/25 12:37, Marc-Aurel Zent wrote:
On 22. Sep 2025, at 20:31, Paul Gofmanpgofman@codeweavers.com wrote:
On 9/22/25 12:14, Elizabeth Figura wrote:
On the other hand, I am reminded of an unrelated bug, involving a program (I don't have the name unfortunately) which copies ws2_32.dll to a temp file with a randomly generated name and attempts to load the copy. This fails on Wine because ws2_32 has a unixlib, and we derive the name of the unixlib from the path of the DLL. In order to fix this bug we would probably need to embed the name of the unixlib in the DLL itself rather than relying on its path. If we did that the DLLs for winepulse and winealsa would no longer be identical.
That doesn't look quite related to the discussed issue? This part is about how we find Unix library. I didn't check exactly this case lately but doesn't Wine search for Unix libraries in the configured or determined on start Wine paths now? I'd guess it might just work?
For the original issue, having the same inodes for different files looks clearly like a bug in NixOS to me. First, yes, if there are two differently named files they are different files even if the contents happens to be the same. For one, the contents is identical right now but may change, and if those are differently named files the contents is not synchronous between those. Then, inode is supposed to be unique within the same file system:https://man7.org/linux/man-pages/man7/inode.7.html, "Inode number": "Each file in a filesystem has a unique inode number.".
I think they can mostly get away with this because the nix store is by design read-only (and there are in practice many duplicates there). Though I agree that such a form of deduplication should happen on the filesystem layer ideally.
But also I don't quite understand from the description why that happens to winepulse.drv and winealsa.drv, these files are not indentical and have different contents??
For me at least winepulse.drv and winealsa.drv are in content exactly identical. If nix were to deduplicate only cross-derivation this would not happen I think.
Yes, this will be a solution indeed. I think I saw people mentioning this on the nix issue tracker.
This is not a solution, this is a workaround for a particular case of the problem. The similar issue may happen with app's dll loaded from different directories, or other (more special) cases not related to loading dlls. FWIW Wine behaviour checking underlying file identity is not random. For instance, LdrGetModuleHandle called with full DLL path will distinguish dll which was renamed (while loaded) and replaced by different file, we have a test for that (and that's one example where correct file identification is needed).
The problem is not Wine specific, identifying files by inode within FS is what other Unix apps can rightfully do, making deduplication look like a hard link (made on behalf of the app) which then gets "unlinked" when either file is modified is severe change of Unix inode semantics and IMO should be fixed.
On Monday, 22 September 2025 13:31:49 CDT Paul Gofman wrote:
On 9/22/25 12:14, Elizabeth Figura wrote:
On the other hand, I am reminded of an unrelated bug, involving a program (I don't have the name unfortunately) which copies ws2_32.dll to a temp file with a randomly generated name and attempts to load the copy. This fails on Wine because ws2_32 has a unixlib, and we derive the name of the unixlib from the path of the DLL. In order to fix this bug we would probably need to embed the name of the unixlib in the DLL itself rather than relying on its path. If we did that the DLLs for winepulse and winealsa would no longer be identical.
That doesn't look quite related to the discussed issue? This part is about how we find Unix library.
The point is that if we were to do this, winepulse and winealsa would no longer have the exact same contents, which would sidestep this problem.
I didn't check exactly this case lately but doesn't Wine search for Unix libraries in the configured or determined on start Wine paths now? I'd guess it might just work?
The problem isn't the path, it's the base name. ws2_32.dll gets renamed to "tmp1234.dll" or something, and we look for (and fail to find) "tmp1234.so".
On 9/22/25 12:43, Elizabeth Figura wrote:
On Monday, 22 September 2025 13:31:49 CDT Paul Gofman wrote:
On 9/22/25 12:14, Elizabeth Figura wrote:
On the other hand, I am reminded of an unrelated bug, involving a program (I don't have the name unfortunately) which copies ws2_32.dll to a temp file with a randomly generated name and attempts to load the copy. This fails on Wine because ws2_32 has a unixlib, and we derive the name of the unixlib from the path of the DLL. In order to fix this bug we would probably need to embed the name of the unixlib in the DLL itself rather than relying on its path. If we did that the DLLs for winepulse and winealsa would no longer be identical.
That doesn't look quite related to the discussed issue? This part is about how we find Unix library.
The point is that if we were to do this, winepulse and winealsa would no longer have the exact same contents, which would sidestep this problem.
I see now why are those are the same, the PE part may be identical indeed. But it is only a particular case, the bug will stay in general, there will be other cases when things are broken due to that (while probably not reproducible across majority of apps under Wine).
I didn't check exactly this case lately but doesn't Wine search for Unix libraries in the configured or determined on start Wine paths now? I'd guess it might just work?
The problem isn't the path, it's the base name. ws2_32.dll gets renamed to "tmp1234.dll" or something, and we look for (and fail to find) "tmp1234.so".
Ah, it is renamed...
Elizabeth Figura zfigura@codeweavers.com writes:
On Monday, 22 September 2025 13:31:49 CDT Paul Gofman wrote:
On 9/22/25 12:14, Elizabeth Figura wrote:
On the other hand, I am reminded of an unrelated bug, involving a program (I don't have the name unfortunately) which copies ws2_32.dll to a temp file with a randomly generated name and attempts to load the copy. This fails on Wine because ws2_32 has a unixlib, and we derive the name of the unixlib from the path of the DLL. In order to fix this bug we would probably need to embed the name of the unixlib in the DLL itself rather than relying on its path. If we did that the DLLs for winepulse and winealsa would no longer be identical.
That doesn't look quite related to the discussed issue? This part is about how we find Unix library.
The point is that if we were to do this, winepulse and winealsa would no longer have the exact same contents, which would sidestep this problem.
For this specific case, the plan is to get rid of these empty PE files and make it possible for mmdevapi to load the appropriate Unix backend directly. Presumably the same mechanism could then be used to handle the ws2_32 case.
Of course that doesn't change the fact that hardlinking files behind our back is not a good idea on NixOS's part.
-
Alexandre Julliard -
DodoGTA GT -
Elizabeth Figura -
Kevin Puetz -
Marc-Aurel Zent -
Paul Gofman -
Yuxuan Shui