Issue 7310094: code review 7310094: runtime: Do not reserve huge amount of swap on 32 bit a... (Closed)
|
Created:
13 years, 5 months ago by snabb Modified:
13 years, 5 months ago Reviewers:
CC:
golang-dev, minux1, iant, dave_cheney.net, gofrontend-dev_googlegroups.com Visibility:
Public. |
Descriptionruntime: Do not reserve huge amount of swap on 32 bit architectures.
The mmap() call which reserves the arena should have MAP_NORESERVE
flag as in typical cases this memory will never be (fully) needed.
This matters in environments which do not do Linux style memory
overcommit, such as OpenIndiana/OpenSolaris/Solaris.
The MAP_NORESERVE flag does not exist on all operating systems
(for example FreeBSD). Therefore we define it to zero value in
case it does not exist.
Fixes issue 21.
Patch Set 1 #Patch Set 2 : diff -r e1872b6db171 https://code.google.com/p/gofrontend/ #Patch Set 3 : diff -r e1872b6db171 https://code.google.com/p/gofrontend/ #MessagesTotal messages: 9
Hello golang-dev@googlegroups.com (cc: golang-dev@googlegroups.com), I'd like you to review this change to https://code.google.com/p/gofrontend/
Sign in to reply to this message.
please send gccgo CLs to the gofrontend-dev ML.
i think we'd better patch this to master Go runtime as it's not specific to
gccgo runtime.
add the macro definition to pkg/runtime/defs_linux_{386,amd64,arm}.h and
pkg/runtime/defs_linux.go
and then use it in pkg/runtime/mem_linux.c.
PS: if you're sure the value of the macro is uniform across all
architectures supported
by Linux, you could add the definition to os_linux.h instead of defs_linux*.
Sign in to reply to this message.
On 2013-02-14 18:52, minux wrote: > please send gccgo CLs to the gofrontend-dev ML. Oops, sorry. I did not know about this list. It is not mentioned at the relevant page: http://golang.org/doc/gccgo_contribute.html > i think we'd better patch this to master Go runtime as it's not specific > to gccgo runtime. > > add the macro definition to pkg/runtime/defs_linux_{386,amd64,arm}.h and > pkg/runtime/defs_linux.go > and then use it in pkg/runtime/mem_linux.c. > > PS: if you're sure the value of the macro is uniform across all > architectures supported > by Linux, you could add the definition to os_linux.h instead of defs_linux*. I tried this out on 32 bit Linux and it did not solve any problem there even if running with "vm.overcommit_memory=2" (which should disable memory overcommit). The mmap() without MAP_NORESERVE does not seem to reserve the swap space on Linux (which seems to contradict with the mmap(2) manual page). The flag makes no difference there. I am able to start a big amount of Go program processes even when the overcommit is disabled. The virtual memory consumption number in ps output (VSZ) and the "pmap PID" output remains the same no matter if this flag is specified or not. Therefore I thought I should not suggest patching mem_linux.c. -- Janne Snabb / EPIPE Communications snabb@epipe.com - http://epipe.com/
Sign in to reply to this message.
On 2013/02/14 17:16:08, snabb wrote: > > I tried this out on 32 bit Linux and it did not solve any problem there > even if running with "vm.overcommit_memory=2" (which should disable > memory overcommit). > > The mmap() without MAP_NORESERVE does not seem to reserve the swap space > on Linux (which seems to contradict with the mmap(2) manual page). The > flag makes no difference there. I am able to start a big amount of Go > program processes even when the overcommit is disabled. > > The virtual memory consumption number in ps output (VSZ) and the "pmap > PID" output remains the same no matter if this flag is specified or not. Given that we don't need MAP_NORESERVE on GNU/Linux, I'm not sure we need to change the master mem_linux.c. So I think it is OK to only patch the gofrontend file. As far as I can see, adding MAP_NORESERVE can't hurt, but I'm not an expert; does anybody think it can? Ian
Sign in to reply to this message.
On 2013-02-14 19:35, iant@golang.org wrote: > Given that we don't need MAP_NORESERVE on GNU/Linux, I'm not sure we > need to change the master mem_linux.c. If the file was named mem_generic.c or something like that instead of mem_linux.c I would have probably suggested patching it :). Because it is mem_linux.c I am thinking "If it ain't broke, don't fix it". I have some additional observations about Linux: I read the documentation about "vm.overcommit_memory" in Documentation/vm/overcommit-accounting in Linux source tree. It states the following: "In mode 2 the MAP_NORESERVE flag is ignored." Therefore my previous testing on Linux was invalid (as I was using mode 2). If the MAP_NORESERVE flag is put in pkg/runtime/mem_linux.c then the flag may not be defined in pkg/runtime/defs_linux.go but must be in the architecture specified files in pkg/runtime/defs_linux_{386,amd64,arm}.h. This is because the value of the flag varies in the Linux uapi header files for different architectures. It is 0x4000 in the "generic" version (which is apparently used for 386, amd64 and arm) but in the alpha uapi it is 0x10000, powerpc and sparc 0x40 and on mips, tile and extensa it is 0x400. I will do a bit more testing on 32 bit Linux and have a look at the Linux source code. I will get back if I find anything to add. -- Janne Snabb / EPIPE Communications snabb@epipe.com - http://epipe.com/
Sign in to reply to this message.
I think this CL has merit, the common problems with running Go or Java on VPSs like OpenVZ is the large reservation for the GC bitmap which will go unused. This is related to issue http://golang.org/issue/4447 On 15 Feb 2013 07:00, "Janne Snabb" <snabb@epipe.com> wrote: > On 2013-02-14 19:35, iant@golang.org wrote: > > Given that we don't need MAP_NORESERVE on GNU/Linux, I'm not sure we > > need to change the master mem_linux.c. > > If the file was named mem_generic.c or something like that instead of > mem_linux.c I would have probably suggested patching it :). > > Because it is mem_linux.c I am thinking "If it ain't broke, don't fix it". > > > I have some additional observations about Linux: > > I read the documentation about "vm.overcommit_memory" in > Documentation/vm/overcommit-accounting in Linux source tree. It states > the following: > > "In mode 2 the MAP_NORESERVE flag is ignored." > > Therefore my previous testing on Linux was invalid (as I was using mode 2). > > > If the MAP_NORESERVE flag is put in pkg/runtime/mem_linux.c then the > flag may not be defined in pkg/runtime/defs_linux.go but must be in the > architecture specified files in > pkg/runtime/defs_linux_{386,amd64,arm}.h. This is because the value of > the flag varies in the Linux uapi header files for different > architectures. It is 0x4000 in the "generic" version (which is > apparently used for 386, amd64 and arm) but in the alpha uapi it is > 0x10000, powerpc and sparc 0x40 and on mips, tile and extensa it is 0x400. > > > I will do a bit more testing on 32 bit Linux and have a look at the > Linux source code. I will get back if I find anything to add. > > -- > Janne Snabb / EPIPE Communications > snabb@epipe.com - http://epipe.com/ > > -- > > --- > You received this message because you are subscribed to the Google Groups > "golang-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > >
Sign in to reply to this message.
On 2013-02-14 22:05, Dave Cheney wrote: > I think this CL has merit, the common problems with running Go or Java > on VPSs like OpenVZ is the large reservation for the GC bitmap which > will go unused. This is related to issue http://golang.org/issue/4447 I tried to run OpenVZ (current version) on 32 and 64 bit configurations and created a VM with 128 MB of mem ("vzctl set NNN --ram 128M --save") but I could not find any issues running simple Go programs with the "tip" Go. It would be useful to have the contents of "/proc/user_beancounters" and other relevant information to be able to reproduce the issues. I could not find this information in the thread about the OpenVZ issues. The most useful bit of related information that I encountered is the accountable_mapping() function in mm/mmap.c in Linux sources. It defines that the memory usage of mappings which are writable and not shared and not mapped with MAP_NORESERVE flag are accounted for (for whatever accounting purposes :). On 32 bit Linux with Go "tip" a typical memory map of a small Go program looks like this: 08048000 252K r-x-- /home/foobar/foo 08087000 12K rw--- /home/foobar/foo 0808a000 4156K rw--- [ anon ] 08500000 262016K ----- [ anon ] 184e0000 1152K rwx-- [ anon ] 18600000 523264K ----- [ anon ] b7621000 1608K rwx-- [ anon ] b77b3000 4K r-x-- [ anon ] bfe81000 84K rw--- [ stack ] total 792548K The big chunks of 262016K + 523264K should not be accounted for by the kernel because they are not writable. No problems there. On 64 bit Linux with Go "tip" a typical memory map looks like this: 0000000000400000 316K r-x-- /home/snabb/foo 000000000044f000 16K rw--- /home/snabb/foo 0000000000453000 262232K rw--- [ anon ] 000000c1ffff0000 1088K rwx-- [ anon ] 00007f01ef84b000 1612K rwx-- [ anon ] 00007fffe4c2c000 132K rw--- [ stack ] 00007fffe4dd5000 4K r-x-- [ anon ] ffffffffff600000 4K r-x-- [ anon ] total 265404K Here the big mapping of 262232K is writable which probably means that it is accounted for and could cause issues on environments which have some memory related limit set at a low value. This particular mapping is *not* done by calling mmap() in runtime/mem_linux.c but instead it is mapped by ".noptrbss" section of the ELF binary produced by Go toolchain ld. The space is already mapped by the kernel when the binary starts executing. Thus changing the current mmap() calls in mem_linux.c will not change it. It might be a good idea to map all big writable memory chunks which remain mostly unused with MAP_NORESERVE but in this case it is not possible. I suppose there is some reason for having this big section in the ELF binary instead of mmap()ing it at runtime? I think the Linux OpenVZ issues are a separate issue from my OpenIndiana issues, although related. I propose patching the gofrontend mem.c to fix the OpenIndiana/Solaris issues and leaving Go main repo mem_linux.c as it is until someone can give specifics for reproducing the OpenVZ problem (and handle it as a separate issue -- if it seems still relevant, looks like there are some recent changes in this regard in OpenVZ? I am not familiar with OpenVZ - this is the first time ever I touched it.). For reference, the interesting OpenVZ thread is here: https://groups.google.com/forum/?fromgroups=#!msg/golang-nuts/4Y-SO5QnGyY/BMe... Related issue: https://code.google.com/p/go/issues/detail?id=4447 -- Janne Snabb / EPIPE Communications snabb@epipe.com - http://epipe.com/
Sign in to reply to this message.
LGTM
Sign in to reply to this message.
*** Submitted as https://code.google.com/p/gofrontend/source/detail?r=29b742a2ed37 *** runtime: Do not reserve huge amount of swap on 32 bit architectures. The mmap() call which reserves the arena should have MAP_NORESERVE flag as in typical cases this memory will never be (fully) needed. This matters in environments which do not do Linux style memory overcommit, such as OpenIndiana/OpenSolaris/Solaris. The MAP_NORESERVE flag does not exist on all operating systems (for example FreeBSD). Therefore we define it to zero value in case it does not exist. Fixes issue 21. R=golang-dev, minux.ma, iant, dave CC=gofrontend-dev, golang-dev https://codereview.appspot.com/7310094 Committer: Ian Lance Taylor <iant@golang.org>
Sign in to reply to this message.
|
