[Bug 19561] New: Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

Summary: Very large memory leak when doing overlapped reads
           Product: Wine
           Version: 1.1.26
          Platform: PC
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: major
          Priority: P2
         Component: wineserver
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]

Created an attachment (id=22796)
 --> (http://bugs.winehq.org/attachment.cgi?id=22796)
Program compiled and ready to run using mingw

WINE is buffering serial ports which is in conflict with using overlapped
reads. Additionally, when using overlapped reads on a serial port, there is a
large memory leak. Every time we receive a block of data on the port, (say 800
bytes in our case,) wineserver allocates many pages on the heap and does not
free them afterwards. We are seeing as many as 21 pages lost to the heap every
time we read a small block of data.

My company is creating a Monitor & Control product that works through serial
ports, and we want it to be able to run on Linux netbooks (using USB to serial
port adapters.) This memory leak is the only thing holding us up. The memory
leak is too large for us to recommend to customers that our application can be
used on Linux platforms using WINE.

We are including an attachment for a small program that captures the leak. The
problem is a result of our call to readfile() in the do-while loop. It is using
the overlappedRead feature. Every time this Win32 function call is made, 84K
bytes disappears forever.

-Ron F.

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

--- Comment #1 from Ron Fischler   2009-08-03 19:15:59 ---
Created an attachment (id=22797)
 --> (http://bugs.winehq.org/attachment.cgi?id=22797)
Source code that captures the problem

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

Ron Fischler  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #22796|Program compiled and ready  |Program compiled (using
        description|to run using mingw          |mingw)

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |download, source

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

Vitaliy Margolen  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|download, source            |testcase
              Alias|RonF.                       |

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

--- Comment #2 from Dmitry Timoshkov   2009-08-03 22:43:56 ---
How have you figured out that it's wineserver who leaks the memory?
Is there any particular place in wineserver you think is to blame?

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

--- Comment #3 from Ron Fischler   2009-08-03 23:39:10 ---
Dear Dmitry,

I don't know for sure if it is in wineserver, and even if it is I certainly do
not know where - I have never looked through WINE code before, and I would not
know where to look. However, if we use readfile() without the overlappedRead
option the program works fine, and there is no growth in wineserver's heap.
When overlappedRead is used with readfile(), winserver's heap grows at a
fantastic rate (about 84K bytes with each read, even if only a few bytes are
being read each time.)

Additionally, the program source provided in the attachment has been set up
with no buffering, and yet readfile() is still returning data in blocks of
approximately 400 bytes, so there appears to be a buffering layer of some kind
down below that I suspect could be related to the memory leak.

Best Regards,
Ron

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

Dmitry Timoshkov  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|wineserver                  |-unknown

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

Dmitry Timoshkov  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|major                       |minor

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

--- Comment #4 from Ron Fischler   2009-08-04 19:01:28 ---
Note: A popular open-source terminal program, TeraTerm Pro, (see
http://www.tucows.com/preview/195282) exhibits exactly the same explosive heap
growth in wineserver. Looking at it's source, it is using readfile() with
overlapped reads.

-Ron

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

--- Comment #5 from Ron Fischler   2009-08-04 19:10:40 ---
A correction to my last note: the best place to get Tera Term Pro:
    http://en.sourceforge.jp/projects/ttssh2/releases/

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |download

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

--- Comment #6 from [e-mail] 2013-05-25 09:32:51 CDT ---
No update for over 900 days. Is this still and issue in 1.5.31 or higher or is
this  abandoned ?

[Bug 19561] Very large memory leak when doing overlapped reads

http://bugs.winehq.org/show_bug.cgi?id=19561

Helder 'Lthere' Magalhães  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 19561] Very large memory leak when doing overlapped reads

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=19561

joaopa  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #7 from joaopa  ---
Does the bug still occur with wine-5.0-rc2?

[Bug 19561] Very large memory leak when doing overlapped reads

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=19561

--- Comment #8 from joaopa  ---
Created attachment 66599
  --> https://bugs.winehq.org/attachment.cgi?id=66599
compiled test

Compiled test. But I do not know how to use it.

[Bug 19561] Very large memory leak when doing overlapped reads

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=19561

--- Comment #9 from joaopa  ---
Does the bug still occur with wine-7.0-rc3?

[Bug 19561] Very large memory leak when doing overlapped reads

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=19561

--- Comment #10 from joaopa  ---
Anyone want to test? I do not know what program does (or should do)

[Bug 19561] Very large memory leak when doing overlapped reads

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=19561

Janne  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #11 from Janne  ---
Hello,

i tested the original program with USB-serial port.
Program seems to print what comes to serial port.
In my test string was written to serial port with 1 second interval.
String format: hh:mm:ss ABCDEFG \r \n

Tested with wine version 8.5
Linux is running on virtual box.
I tested this also on windows and output was similar.

terminal log:
jehu@jehu-VirtualBox:~/.wine/drive_c/temp$ wine sertest.exe -c2 -b19200
SerTest V1.01, using Overlapped Reads
using comm port [\.\COM2] at 19200 baud
Press [Control-C] to close this program
00:00:01 ABCDEFG
00:00:02 ABCDEFG
00:00:03 ABCDEFG
00:00:04 ABCDEFG
00:00:05 ABCDEFG
^C0130:fixme:console:default_ctrl_handler Terminating process 11c on event 0
wine client error:120: write: Bad file descriptor.
>

I have no idea how to monitor heap size of wineserver.

[Bug 19561] Very large memory leak when doing overlapped reads

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=19561

Bernhard Übelacker  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
      Fixed by SHA1|                            |577cb166d48e5c6f270237179cd
                   |                            |beb82a68b1ac0
                 CC|                            |[e-mail]
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #12 from Bernhard Übelacker  ---
Because my ancient Debian-8-Jessie VM was already up, I tried
to attach two virtual USB adapters, connected via "null-modem" cable.
Running there wine-1.1.26 with the test application shows a small
increase of RSS in the output of ps in processes sertest.exe and wineserver
over a few minutes.

After following commit I could this increase no longer see.
  577cb166d48e5c6f270237179cdbeb82a68b1ac0 is the first not-leaking commit
  commit 577cb166d48e5c6f270237179cdbeb82a68b1ac0
  Author: Alexandre Julliard 
  Date:   Mon Mar 2 17:53:40 2015 +0900
    ntdll: Don't queue a user APC when there is no callback function.

This commit is included since wine-1.7.38.

With the same setup with wine-10.0 I could also not see such an increase.

[Bug 19561] Very large memory leak when doing overlapped reads

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=19561

--- Comment #13 from Bernhard Übelacker  ---
Created attachment 78869
  --> http://bugs.winehq.org/attachment.cgi?id=78869
Some notes on the git bisect.