http://bugs.winehq.org/show_bug.cgi?id=13606
folks, hi,
am running a configure script under wine, and it's _literally_ one to two seconds per sh.exe instance. i started running ./configure over two hours ago and there are about 100 lines of output so far, with a further 200 to go i am better off leaving this running overnight. as this is development work i am doing, not "building something that's already known to work" i have to call a halt to the development, because it is crazy to expect to have to wait four to six hours if i make a single change to configure.in or Makefile.pre.in - god help me if i have to run autoconf (fortunately i can do that from linux).
... so i did some investigation:
strace -o log.txt -ff wine cmd
followed by running c:\msys\msys.bat
ignoring the fact that sh.exe crashed on me (which it doesn't do when you don't use strace) i was able to obtain some system call usage when firing up sh.exe, and it's _horrendous_.
X11 keyboard files, X11 locale files, iconv, windows fonts, unix fonts - all to fire up c:/msys/bin/sh.exe ?
further up the strace files, i'm looking at the biggest offender and it looks like it's reading files one byte at a time. reconstructing some of the sentences i can make out "remember that we get here even under command.com\r\n". ohhh i remember where i've seen that - that's in msys.bat. .... why is it being read one byte at a time?? there are _thousands_ of lines like this:
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigprocmask(SIG_BLOCK, [HUP INT USR1 USR2 ALRM CHLD IO], [], 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 read(13, "t", 1) = 1 rt_sigprocmask(SIG_BLOCK, [HUP INT USR1 USR2 ALRM CHLD IO], [], 8) = 0 write(3, "7\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 64) = 64 read(6, "\r\0\0\300\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 64) = 64 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigprocmask(SIG_BLOCK, [HUP INT USR1 USR2 ALRM CHLD IO], [], 8) = 0 write(3, "4\0\0\0\0\0\0\0\0\0\0\0?\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 64) = 64 read(6, "$\0\0\300\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 64) = 64 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigprocmask(SIG_BLOCK, [HUP INT USR1 USR2 ALRM CHLD IO], [], 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
checking earlier in the same strace log file (the one that loaded msys.bat one byte at a time) _again_ i see font-loading (every single one on my system).
there are _forty_ instances of libraries being loaded - per process!
open("/usr/lib/locale/locale-archive", O_RDONLY) = 3 open("/usr/lib/gconv/gconv-modules.cache", O_RDONLY) = 3 open("/usr/lib/libwine.so.1", O_RDONLY) = 3 open("/usr/lib/wine/wine-pthread", O_RDONLY) = 3 open("/usr/lib/libwine.so.1", O_RDONLY) = 3 open("/usr/lib/wine/ntdll.dll.so", O_RDONLY) = 3 open("/usr/lib/wine/kernel32.dll.so", O_RDONLY) = 7 open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 7 open("/usr/lib/wine/cmd.exe.so", O_RDONLY) = 10 open("/usr/lib/wine/shell32.dll.so", O_RDONLY) = 10 open("/usr/lib/wine/shlwapi.dll.so", O_RDONLY) = 10 open("/usr/lib/wine/user32.dll.so", O_RDONLY) = 10 open("/usr/lib/wine/gdi32.dll.so", O_RDONLY) = 10 open("/usr/lib/wine/advapi32.dll.so", O_RDONLY) = 10 open("/usr/lib/wine/comctl32.dll.so", O_RDONLY) = 10 open("/usr/lib/wine/krnl386.exe16", O_RDONLY|O_LARGEFILE) = 10 open("/usr/lib/wine/system.drv16", O_RDONLY|O_LARGEFILE) = 10 open("/usr/lib/wine/gdi.exe16", O_RDONLY|O_LARGEFILE) = 10 open("/usr/lib32/libfreetype.so.6", O_RDONLY) = 10 open("/usr/lib32/libz.so.1", O_RDONLY) = 10 open("/usr/lib32/libexpat.so.1", O_RDONLY) = 10 open("/usr/lib/wine/user.exe16", O_RDONLY|O_LARGEFILE) = 10 open("/usr/lib/wine/winex11.drv.so", O_RDONLY) = 10 open("/usr/lib32/libSM.so.6", O_RDONLY) = 10 open("/usr/lib32/libICE.so.6", O_RDONLY) = 10 open("/usr/lib32/libXxf86vm.so.1", O_RDONLY) = 10 open("/usr/lib32/libXext.so.6", O_RDONLY) = 10 open("/usr/lib32/libX11.so.6", O_RDONLY) = 10 open("/usr/lib32/libXau.so.6", O_RDONLY) = 10 open("/usr/lib32/libxcb-xlib.so.0", O_RDONLY) = 10 open("/usr/lib32/libxcb.so.1", O_RDONLY) = 10 open("/usr/lib32/libXdmcp.so.6", O_RDONLY) = 10 open("/usr/lib/wine/imm32.dll.so", O_RDONLY) = 10 open("/usr/lib32/libXinerama.so.1", O_RDONLY) = 11 open("/usr/lib32/libXrender.so.1", O_RDONLY) = 11 open("/usr/lib32/libXrandr.so.2", O_RDONLY) = 11 open("/usr/lib32/libXcomposite.so.1", O_RDONLY) = 11 open("/usr/lib32/libXfixes.so.3", O_RDONLY) = 11 open("/usr/lib32/libXcursor.so.1", O_RDONLY) = 11 open("/usr/lib/wine/uxtheme.dll.so", O_RDONLY) = 11 open("/usr/lib/wine/ole32.dll.so", O_RDONLY) = 11 open("/usr/lib/wine/rpcrt4.dll.so", O_RDONLY) = 11 open("/usr/lib/wine/iphlpapi.dll.so", O_RDONLY) = 11 open("/usr/lib/wine/start.exe.so", O_RDONLY|O_LARGEFILE) = 7
surely, on a console-based command shell, all of the fonts and the X11 libraries can be skipped?
l.
On Fri, Jan 09, 2009 at 06:39:27PM +0000, Luke Kenneth Casson Leighton wrote:
http://bugs.winehq.org/show_bug.cgi?id=13606
folks, hi,
am running a configure script under wine, and it's _literally_ one to two seconds per sh.exe instance. i started running ./configure over two hours ago and there are about 100 lines of output so far, with a further 200 to go i am better off leaving this running overnight. as this is development work i am doing, not "building something that's already known to work" i have to call a halt to the development, because it is crazy to expect to have to wait four to six hours if i make a single change to configure.in or Makefile.pre.in - god help me if i have to run autoconf (fortunately i can do that from linux).
... so i did some investigation:
strace -o log.txt -ff wine cmd
followed by running c:\msys\msys.bat
ignoring the fact that sh.exe crashed on me (which it doesn't do when you don't use strace) i was able to obtain some system call usage when firing up sh.exe, and it's _horrendous_.
X11 keyboard files, X11 locale files, iconv, windows fonts, unix fonts
- all to fire up c:/msys/bin/sh.exe ?
further up the strace files, i'm looking at the biggest offender and it looks like it's reading files one byte at a time. reconstructing some of the sentences i can make out "remember that we get here even under command.com\r\n". ohhh i remember where i've seen that - that's in msys.bat. .... why is it being read one byte at a time?? there are _thousands_ of lines like this:
If it starts up the wine environment every time this might take some time.
Try to keep a "notepad" open in the background so the wine environment stays running.
Ciao, Marcus
If it starts up the wine environment every time this might take some time.
Try to keep a "notepad" open in the background so the wine environment stays running.
hiya marcus, good advice - sounds reasonable. tried it: wineserver is the top-running process, stays with the same process id. so i don't believe it makes a difference: we're still looking at 20-30 seconds per line from configure.
what i did was utterly utterly nastily cheat: i #ifdef'd most of the lines from python's configure script, for a win32 build! this would normally be an absolute recipe for a horror-story disaster, but, given that there's a preconfigured hard-coded pcbuild.h for visual studio to use, to replace what configure normally does _anyway_, it's a one-off acceptable thing.
this cuts the ./configure time from 2 hours to 10 minutes... :)
l.
On Fri, Jan 09, 2009 at 06:39:27PM +0000, Luke Kenneth Casson Leighton wrote:
further up the strace files, i'm looking at the biggest offender and it looks like it's reading files one byte at a time.
Certainly a normal unix shell has to read input (from stdin) byte by byte since it never knows when it might have to fork a process passing stdin through - and the child needs the correct file offset. (And the unix kernel doesn't have a 'read until newline' for files.)
If stdin is seekable or mapable there are other options, but that is difficult to detect and code for.
David
On Saturday 10 January 2009 01:05:07 David Laight wrote:
On Fri, Jan 09, 2009 at 06:39:27PM +0000, Luke Kenneth Casson Leighton
wrote:
further up the strace files, i'm looking at the biggest offender and it looks like it's reading files one byte at a time.
Certainly a normal unix shell has to read input (from stdin) byte by byte since it never knows when it might have to fork a process passing stdin through - and the child needs the correct file offset. (And the unix kernel doesn't have a 'read until newline' for files.)
If stdin is seekable or mapable there are other options, but that is difficult to detect and code for.
David
Hi, I don't think this is true. When the shell forks to create a child, it doesn't have to give the child the same standard file descriptors - rather it creates pipes, and it passes stdin to the child by reading its own stdin than writing the data to the child's stdin pipe. This way it can filter the input (like for control sequences). So, if it has any excess input data when it creates a child, it simply writes that data to the stdin pipe first. That is how I think things are and should be done.
Ambroz
On Sat, Jan 10, 2009 at 10:23:49AM +0100, Ambroz Bizjak wrote:
I don't think this is true. When the shell forks to create a child, it doesn't have to give the child the same standard file descriptors - rather it creates pipes, and it passes stdin to the child by reading its own stdin than writing the data to the child's stdin pipe. This way it can filter the input (like for control sequences). So, if it has any excess input data when it creates a child, it simply writes that data to the stdin pipe first. That is how I think things are and should be done.
Not in any shells I've fixed. There is no need for the shell itself to pre-process process input/output. There are no control sequences for the shell. (^C etc are detected by the kernel line-discipline code and sent as signals to the session leader.)
David
On Sat, Jan 10, 2009 at 12:05 AM, David Laight david@l8s.co.uk wrote:
On Fri, Jan 09, 2009 at 06:39:27PM +0000, Luke Kenneth Casson Leighton wrote:
further up the strace files, i'm looking at the biggest offender and it looks like it's reading files one byte at a time.
Certainly a normal unix shell has to read input (from stdin) byte by byte since it never knows when it might have to fork a process passing stdin through - and the child needs the correct file offset.
ohhh, ok :) it just looked... mad!
-
Ambroz Bizjak -
David Laight -
Luke Kenneth Casson Leighton -
Marcus Meissner