wine-devel

wine-devel@winehq.org

1 participants
84478 discussions

[PATCH v6 00/28] NT synchronization primitive driver
by Elizabeth Figura 09 Dec '24

09 Dec '24

This patch series implements a new char misc driver, /dev/ntsync, which is used to implement Windows NT synchronization primitives. NT synchronization primitives are unique in that the wait functions both are vectored, operate on multiple types of object with different behaviour (mutex, semaphore, event), and affect the state of the objects they wait on. This model is not compatible with existing kernel synchronization objects or interfaces, and therefore the ntsync driver implements its own wait queues and locking. This patch series is rebased against the "char-misc-next" branch of gregkh/char-misc.git. == Background == The Wine project emulates the Windows API in user space. One particular part of that API, namely the NT synchronization primitives, have historically been implemented via RPC to a dedicated "kernel" process. However, more recent applications use these APIs more strenuously, and the overhead of RPC has become a bottleneck. The NT synchronization APIs are too complex to implement on top of existing primitives without sacrificing correctness. Certain operations, such as NtPulseEvent() or the "wait-for-all" mode of NtWaitForMultipleObjects(), require direct control over the underlying wait queue, and implementing a wait queue sufficiently robust for Wine in user space is not possible. This proposed driver, therefore, implements the problematic interfaces directly in the Linux kernel. This driver was presented at Linux Plumbers Conference 2023. For those further interested in the history of synchronization in Wine and past attempts to solve this problem in user space, a recording of the presentation can be viewed here: https://www.youtube.com/watch?v=NjU4nyWyhU8 == Performance == The performance measurements described below are copied from earlier versions of the patch set. While some of the code has changed, I do not currently anticipate that it has changed drastically enough to affect those measurements. The gain in performance varies wildly depending on the application in question and the user's hardware. For some games NT synchronization is not a bottleneck and no change can be observed, but for others frame rate improvements of 50 to 150 percent are not atypical. The following table lists frame rate measurements from a variety of games on a variety of hardware, taken by users Dmitry Skvortsov, FuzzyQuils, OnMars, and myself: Game Upstream ntsync improvement =========================================================================== Anger Foot 69 99 43% Call of Juarez 99.8 224.1 125% Dirt 3 110.6 860.7 678% Forza Horizon 5 108 160 48% Lara Croft: Temple of Osiris 141 326 131% Metro 2033 164.4 199.2 21% Resident Evil 2 26 77 196% The Crew 26 51 96% Tiny Tina's Wonderlands 130 360 177% Total War Saga: Troy 109 146 34% =========================================================================== == Patches == The intended semantics of the patches are broadly intended to match those of the corresponding Windows functions. For those not already familiar with the Windows functions (or their undocumented behaviour), patch 27/28 provides a detailed specification, and individual patches also include a brief description of the API they are implementing. The patches making use of this driver in Wine can be retrieved or browsed here: https://repo.or.cz/wine/zf.git/shortlog/refs/heads/ntsync5 == Previous versions == No changes were made from v5 other than rebasing on top of the 6.13-rc1 char-misc-next tree. I would like to repeat a question from the last round of review, though. Two changes were suggested related to API design, which I did not make because the APIs in question were already released in upstream Linux. However, the driver is also completely nonfunctional and hidden behind BROKEN, so would this be acceptable anyway? The changes in question are: * rename NTSYNC_IOC_SEM_POST to NTSYNC_IOC_SEM_RELEASE (matching the NT terminology instead of POSIX), * change object creation ioctls to return the fds directly in the return value instead of through the args struct. I would also still appreciate a clarification on the advice in [1], which is why I didn't do this in the first place. [1] https://docs.kernel.org/driver-api/ioctl.html#return-code * Link to v5: https://lore.kernel.org/lkml/20240519202454.1192826-1-zfigura@codeweavers.c… * Link to v4: https://lore.kernel.org/lkml/20240416010837.333694-1-zfigura@codeweavers.co… * Link to v3: https://lore.kernel.org/lkml/20240329000621.148791-1-zfigura@codeweavers.co… * Link to v2: https://lore.kernel.org/lkml/20240219223833.95710-1-zfigura@codeweavers.com/ * Link to v1: https://lore.kernel.org/lkml/20240214233645.9273-1-zfigura@codeweavers.com/ * Link to RFC v2: https://lore.kernel.org/lkml/20240131021356.10322-1-zfigura@codeweavers.com/ * Link to RFC v1: https://lore.kernel.org/lkml/20240124004028.16826-1-zfigura@codeweavers.com/ Elizabeth Figura (28): ntsync: Introduce NTSYNC_IOC_WAIT_ANY. ntsync: Introduce NTSYNC_IOC_WAIT_ALL. ntsync: Introduce NTSYNC_IOC_CREATE_MUTEX. ntsync: Introduce NTSYNC_IOC_MUTEX_UNLOCK. ntsync: Introduce NTSYNC_IOC_MUTEX_KILL. ntsync: Introduce NTSYNC_IOC_CREATE_EVENT. ntsync: Introduce NTSYNC_IOC_EVENT_SET. ntsync: Introduce NTSYNC_IOC_EVENT_RESET. ntsync: Introduce NTSYNC_IOC_EVENT_PULSE. ntsync: Introduce NTSYNC_IOC_SEM_READ. ntsync: Introduce NTSYNC_IOC_MUTEX_READ. ntsync: Introduce NTSYNC_IOC_EVENT_READ. ntsync: Introduce alertable waits. selftests: ntsync: Add some tests for semaphore state. selftests: ntsync: Add some tests for mutex state. selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ANY. selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ALL. selftests: ntsync: Add some tests for wakeup signaling with WINESYNC_IOC_WAIT_ANY. selftests: ntsync: Add some tests for wakeup signaling with WINESYNC_IOC_WAIT_ALL. selftests: ntsync: Add some tests for manual-reset event state. selftests: ntsync: Add some tests for auto-reset event state. selftests: ntsync: Add some tests for wakeup signaling with events. selftests: ntsync: Add tests for alertable waits. selftests: ntsync: Add some tests for wakeup signaling via alerts. selftests: ntsync: Add a stress test for contended waits. maintainers: Add an entry for ntsync. docs: ntsync: Add documentation for the ntsync uAPI. ntsync: No longer depend on BROKEN. Documentation/userspace-api/index.rst | 1 + Documentation/userspace-api/ntsync.rst | 398 +++++ MAINTAINERS | 9 + drivers/misc/Kconfig | 1 - drivers/misc/ntsync.c | 989 +++++++++++- include/uapi/linux/ntsync.h | 39 + tools/testing/selftests/Makefile | 1 + .../selftests/drivers/ntsync/.gitignore | 1 + .../testing/selftests/drivers/ntsync/Makefile | 7 + tools/testing/selftests/drivers/ntsync/config | 1 + .../testing/selftests/drivers/ntsync/ntsync.c | 1407 +++++++++++++++++ 11 files changed, 2850 insertions(+), 4 deletions(-) create mode 100644 Documentation/userspace-api/ntsync.rst create mode 100644 tools/testing/selftests/drivers/ntsync/.gitignore create mode 100644 tools/testing/selftests/drivers/ntsync/Makefile create mode 100644 tools/testing/selftests/drivers/ntsync/config create mode 100644 tools/testing/selftests/drivers/ntsync/ntsync.c base-commit: cdd30ebb1b9f36159d66f088b61aee264e649d7a -- 2.45.2

1 28

Wine staging 10.0-rc1 release
by Alistair Leslie-Hughes 06 Dec '24

06 Dec '24

Binary packages for various distributions will be available from: https://www.winehq.org/download Summary since last release * Rebased to current wine 10.0-rc1 (350 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * windows.networking.connectivity: Add stub dll. * indows.networking.connectivity: Implement IActivationFactory stubs. * windows.networking.connectivity: Implement INetworkInformationStatics stubs. * windows.networking.connectivity: Implement INetworkInformationStatics GetInternetConnectionProfile. * windows.networking.connectivity: IConnectionProfile GetNetworkConnectivityLevel always return internet access. Removed (No longer required). * None Added: * None Updated: * vkd3d-latest Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

1 0

Unable to completely replace vkd3d with vkd3d-proton?
by zhengxianwei 28 Nov '24

28 Nov '24

I tried replacing Wine’s built-in DLLs with the d3d12.dll compiled from vkd3d-proton and the dxgi.dll compiled from dxvk. However, I noticed that Wine’s built-in D3DCompile2 still calls vkd3d_shader_compiler( https://gitlab.winehq.org/wine/wine/-/blob/master/dlls/d3dcompiler_43/compi…), which belongs to vkd3d, not vkd3d-proton. Does this mean that even in Proton, vkd3d and vkd3d-proton coexist, and it’s not possible to completely replace vkd3d? So if there’s a bug in it, I would still need to fix the issue in vkd3d rather than being able to replace it with functions from vkd3d-proton. Is that correct?

3 8

How could I use my own build vkd3d in wine ?
by zhengxianwei 26 Nov '24

26 Nov '24

I want to try modifying and building vkd3d myself, but I’m not sure how to make Wine use my custom-built vkd3d. My development environment is Ubuntu 24.04 + x64.

1 1

Wine staging 9.22 release
by Alistair Leslie-Hughes 23 Nov '24

23 Nov '24

Binary packages for various distributions will be available from: https://www.winehq.org/download Summary since last release * Rebased to current wine 9.22 (350 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * ntdll: Implement HashLinks field in LDR module data. * ntdll: Use HashLinks when searching for a dll using the basename. Removed (No longer required). * None Added: * None Updated: * vkd3d-latest Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

1 0

How to use DLL files from vkd3d-proton in Wine?
by zhengxianwei 22 Nov '24

22 Nov '24

I am a beginner with Wine and currently using it to run Photoshop 23. I noticed that vkd3d seems to lack support for Tiled Resource, which causes Photoshop’s compatibility check to fail in displaying the correct GPU information. I found that vkd3d-proton appears to implement this feature, so I compiled vkd3d-proton and copied the resulting d3d12.dll and d3d12core.dll directly into Wine’s directory. I also ensured that the required d3d12.dll was loaded correctly. However, when launching Photoshop, it crashes when using certain swapchain features. Setting aside the swapchain crash for now, is my approach of directly using the d3d12.dll generated by vkd3d-proton correct?

2 3

Pending Reviews page
by Alexandre Julliard 19 Nov '24

19 Nov '24

Folks, In our grand tradition of shaming people into doing work, after the Regressions page I'm proud to present the Pending Reviews page: https://source.winehq.org/reviews As with the regressions page, the goal should be to *not* be number one on the list. But feel free to congratulate Huw anyway ;-) Note that the list is a little overestimated, commenting on the diff counts as a review but simply adding a note doesn't, so you may see some MRs in your list that you have actually commented on already. If people find this thing useful we can try to make it smarter. Feedback is welcome! -- Alexandre Julliard julliard(a)winehq.org

2 3

Wine staging 9.21 release
by Alistair Leslie-Hughes 08 Nov '24

08 Nov '24

Binary packages for various distributions will be available from: https://www.winehq.org/download Summary since last release * Rebased to current wine 9.21 (358 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * d3dx9_36: Add semi-stub for D3DXOptimizeVertices. Removed (No longer required). * None Added: * [56699] dmime: Semi-Implement IDirectMusicSegment8::GetAudioPathConfig * [55982] dmscript: IDirectMusicScript::EnumRoutine return S_FALSE by default Updated: * vkd3d-latest * win32u-NtGdiExtTextOutW-rotation Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

1 0

Release plans
by Alexandre Julliard 31 Oct '24

31 Oct '24

Folks, Since there has been little activity this week, and I'm travelling on Friday, I'm going to delay Wine 9.21 until next Friday, November 8. There will then be a 9.22 release two weeks later, and two weeks after that, on December 6, the 10.0-rc1 release will mark the beginning of our annual code freeze period. As usual, the freeze will last until the final 10.0 is released sometime around mid-January. -- Alexandre Julliard julliard(a)winehq.org

1 0

Re: Helping Wine use 64 bit Mesa OGL drivers for 32-bit Windows applications
by Derek Lesho 31 Oct '24

31 Oct '24

Hi Christian, Thanks for exploring this option, it's great to have a better understanding of the capabilities/limits of mremap. First, a few practical questions: - How would Wine detect the updated mremap, I suppose we would want to create a test persistent mapping on init and see if mremap works on that page? - In my experience Mesa will sometimes return malloc'd pointers from glMapBuffer when using the transfer helper, which I assume has to do with buffer textures. If wine blindly uses remap for all glMapBuffer calls, we'll then end up hitting the private anonymous path, where from what I understand page faults are use to preserve the old page, potentially slowing things down, thoughts? Second, in regards to going forward: I don't have the final call on what path we end up taking 😅. I think it would be good if others from wine-devel pitched into what they think, but here's my opinion: I think the mremap approach should definitely be pursued since it looks like such a simple Kernel patch, but it may also be good to pursue a simple Mesa-Integrated path as well, maybe with Mesa calling into a Wine library to allocate 32-bit pages, as we support non-linux OS's as well. Thanks, Derek Am 10/30/24 um 14:03 schrieb Christian König: > Hi guys, > > so I looked a bit deeper into the problem of duplicating graphics > driver mappings with mremap(). > > This use case of duplicating a mapping into a fixed address is already > supported quite well using mremap(). This is used by a couple of > different emulators to re-create the address space like you would find > it in the specific environment. > > The only problem is that this only works for files and shared memory > at the moment. Graphic driver mappings on the other hand have the > VM_DONTEXPAND and VM_PFNMAP flag set because their mappings shouldn't > grow and can also include VRAM. > > The attached patch changes this restriction for the mremap() function > and so also allows duplicating the VMAs of graphics drivers into the > lower 32bit address space managed by Wine. > > I've tested this with some of AMD's GPU unit tests and it actually > seems to work quite fine. > > Derek please let me know if that solution works for you and if you're > interested in using it. If yes I would go ahead and send the patch to > the Linux memory management folks for discussion. > > Regards, > Christian. > > Am 24.10.24 um 17:06 schrieb Christian König: >> Darek we are unfortunately both partially right. >> >> Linux supports cloning VMAs using mremap() from userspace by using a >> zero old size, but unfortunately only for SHM areas. >> >> See the code in mm/mremap.c: >> /* >> * We allow a zero old-len as a special case >> * for DOS-emu "duplicate shm area" thing. But >> * a zero new-len is nonsensical. >> */ >> if (!new_len) >> return ret; >> >> Going to take a closer look to figure out what would be necessary to >> solve that for GPU drivers as well. >> >> Regards, >> Christian. >> >> Am 24.10.24 um 14:56 schrieb Christian König: >>> I haven't tested it but as far as I know that isn't correct. >>> >>> As far as I know you can map the same VMA at a different location >>> even without MREMAP_DONTUNMAP. And yes MREMAP_DONTUNMAP only work >>> with private mappings, but that isn't needed here. >>> >>> Give me a moment to test this. >>> >>> Regards, >>> Christian. >>> >>> Am 24.10.24 um 10:03 schrieb Derek Lesho: >>>> In my last mail I responded to this approach all the way at the >>>> bottom, so it probably got lost: mremap on Linux as it exists now >>>> won't work as it only supports private anonymous mappings (in >>>> conjunction with MREMAP_DONTUNMAP), which GPU mappings are not. >>>> >>>> Am 10/24/24 um 01:06 schrieb James Jones: >>>>> That makes sense. Reading the man page myself, it does seem like: >>>>> >>>>> -If the drivers can guarantee they set MAP_SHARED when creating >>>>> their initial mapping. >>>>> >>>>> -If WINE is fine rounding down to page boundaries to deal with >>>>> mappings of suballocations and either using some lookup structure >>>>> to avoid duplicate remappings (probably needed to handle unmap >>>>> anyway per below) or just living with the perf cost and address >>>>> space overconsumption for duplicate remappings. >>>>> >>>>> -If mremap() preserves the cache attributes of the original mapping. >>>>> >>>>> Then no GL API change would be needed. WINE would just have to do >>>>> an if (addrAbove4G) { mremapStuff() } on map and presumably add >>>>> some tracking to perform an equivalent munmap() when unmapping. I >>>>> assume WINE already has a bunch of vaddr tracking logic in use to >>>>> manage the <4G address space as described elsewhere in the thread. >>>>> That would be pretty ideal from a driver vendor perspective. >>>>> >>>>> Does that work? >>>>> >>>>> Thanks, >>>>> -James >>>>> >>>>> On 10/23/24 06:12, Christian König wrote: >>>>>> I haven't read through the whole mail thread, but if you manage >>>>>> the address space using mmap() then you always run into this issue. >>>>>> >>>>>> If you manage the whole 4GiB address space by Wine then you never >>>>>> run into this issue. You would just allocate some address range >>>>>> internally and mremap() into that. >>>>>> >>>>>> Regards, >>>>>> Christian. >>>>>> >>>>>> Am 22.10.24 um 19:32 schrieb James Jones: >>>>>>> This sounds interesting, but does it come with the same "Only >>>>>>> gets 2GB VA" downside Derek pointed out in the thread fork where >>>>>>> he was responding to Michel? >>>>>>> >>>>>>> Thanks, >>>>>>> -James >>>>>>> >>>>>>> On 10/22/24 07:14, Christian König wrote: >>>>>>>> Hi guys, >>>>>>>> >>>>>>>> one theoretical alternative not mentioned in this thread is the >>>>>>>> use of mremap(). >>>>>>>> >>>>>>>> In other words you reserve some address space below 2G by using >>>>>>>> mmap(NULL, length, PROT_NONE, MAP_32BIT | MAP_ANONYMOUS, 0, 0) >>>>>>>> and then use mremap(addr64bit, 0, length, MREMAP_FIXED, >>>>>>>> reserved_addr). >>>>>>>> >>>>>>>> I haven't tested this but at least in theory it should give you >>>>>>>> a duplicate of the 64bit mapping in the lower 2G of the address >>>>>>>> space. >>>>>>>> >>>>>>>> Important is that you give 0 as oldsize to mremap() so that the >>>>>>>> old mapping isn't unmapped but rather just a new mapping of the >>>>>>>> existing VMA created. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Christian. >>>>>>>> >>>>>>>> >>>>>>>> Am 18.10.24 um 23:55 schrieb Derek Lesho: >>>>>>>>> Hey everyone 👋, >>>>>>>>> >>>>>>>>> I'm Derek from the Wine project, and wanted to start a >>>>>>>>> discussion with y'all about potentially extending the Mesa OGL >>>>>>>>> drivers to help us with a functionality gap we're facing. >>>>>>>>> >>>>>>>>> Problem Space: >>>>>>>>> >>>>>>>>> In the last few years Wine's support for running 32-bit >>>>>>>>> windows apps in a 64-bit host environment (wow64) has almost >>>>>>>>> reached feature completion, but there remains a pain point >>>>>>>>> with OpenGL applications: Namely that Wine can't return a >>>>>>>>> 64-bit GL implementation's buffer mappings to a 32 bit >>>>>>>>> application when the address is outside of the 32-bit range. >>>>>>>>> >>>>>>>>> Currently, we have a workaround that will copy any changes to >>>>>>>>> the mapping back to the host upon glBufferUnmap, but this of >>>>>>>>> course is slow when the implementation directly returns mapped >>>>>>>>> memory, and doesn't work for GL_PERSISTENT_BIT, where directly >>>>>>>>> mapped memory is required. >>>>>>>>> >>>>>>>>> A few years ago we also faced this problem with Vulkan's, >>>>>>>>> which was solved through the VK_EXT_map_memory_placed >>>>>>>>> extension Faith drafted, allowing us to use our Wine-internal >>>>>>>>> allocator to provide the pages the driver maps to. I'm now >>>>>>>>> wondering if an GL equivalent would also be seen as feasible >>>>>>>>> amongst the devs here. >>>>>>>>> >>>>>>>>> Proposed solution: >>>>>>>>> >>>>>>>>> As the GL backend handles host mapping in its own code, only >>>>>>>>> giving suballocations from its mappings back to the App, the >>>>>>>>> problem is a little bit less straight forward in comparison to >>>>>>>>> our Vulkan solution: If we just allowed the application to set >>>>>>>>> its own placed mapping when calling glMapBuffer, the driver >>>>>>>>> might then have to handle moving buffers out of already mapped >>>>>>>>> ranges, and would lose control over its own memory management >>>>>>>>> schemes. >>>>>>>>> >>>>>>>>> Therefore, I propose a GL extension that allows the GL client >>>>>>>>> to provide a mapping and unmapping callback to the >>>>>>>>> implementation, to be used whenever the driver needs to >>>>>>>>> perform such operations. This way the driver remains in full >>>>>>>>> control of its memory management affairs, and the amount of >>>>>>>>> work for an implementation as well as potential for bugs is >>>>>>>>> kept minimal. I've written a draft implementation in Zink >>>>>>>>> using map_memory_placed [1] and a corresponding Wine MR >>>>>>>>> utilizing it [2], and would be curious to hear your thoughts. >>>>>>>>> I don't have experience in the Mesa codebase, so I apologize >>>>>>>>> if the branch is a tad messy. >>>>>>>>> >>>>>>>>> In theory, the only requirement from drivers from the >>>>>>>>> extension would be that glMapBuffer always return a pointer >>>>>>>>> from within a page allocated through the provided callbacks, >>>>>>>>> so that it can be guaranteed to be positioned within the >>>>>>>>> required address space. Wine would then use it's existing >>>>>>>>> workaround for other types of buffers, but as Mesa seems to >>>>>>>>> often return directly mapped buffers in other cases as well, >>>>>>>>> Wine could also avoid the slowdown that comes with copying in >>>>>>>>> these cases as well. >>>>>>>>> >>>>>>>>> Why not use Zink?: >>>>>>>>> >>>>>>>>> There's also a proposal to use a 32-bit PE build of Zink in >>>>>>>>> Wine bypassing the need for an extension; I brought this to >>>>>>>>> discussion in this Wine-Devel thread last week [3], which has >>>>>>>>> some arguments against this approach. >>>>>>>>> >>>>>>>>> >>>>>>>>> If any of you have thoughts, concerns, or questions about this >>>>>>>>> potential approach, please let me know, thanks! >>>>>>>>> >>>>>>>>> 1: >>>>>>>>> https://gitlab.freedesktop.org/Guy1524/mesa/-/commits/placed_allocation >>>>>>>>> >>>>>>>>> >>>>>>>>> 2: https://gitlab.winehq.org/wine/wine/-/merge_requests/6663 >>>>>>>>> >>>>>>>>> 3: https://marc.info/?t=172883260300002&r=1&w=2 >>>>>>>>> >>>>>>>> >>>>>> >>> >>

1 0

← Newer
1
...
7
8
9
10
11
12
13
...
8448
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

wine-devel