Error cannot allocate memory errno 12

Another possibility (which I encountered just now) would be bad settings for «overcommit memory» on linux.

In my situation, /proc/sys/vm/overcommit_memory was set to «2» and /proc/sys/vm/overcommit_ratio to «50» , meaning «don’t ever overcommit and only allow allocation of 50% of the available RAM+Swap».

That’s a pretty deceptive problem, since there can be a lot of memory available, but allocations still fail for apparently no reason.

The settings can be changed to the default (overcommit in a sensible way) for now (until a restart):

echo 0 >/proc/sys/vm/overcommit_memory

… or permanently:

echo "vm.overcommit_memory=0 >> /etc/sysctl.conf
sysctl -p /etc/sysctl.conf # apply it immediately

Note: this can also partly be diagnosed by looking at the output of /proc/meminfo:

...
CommitLimit:    45329388 kB
Committed_AS:   44818080 kB 
...

In the example in the question, Committed_AS is much higher than CommitLimit, indicating (together with the fact that allocations fail) that overcommit is enabled, while here both values are close together, meaning that the limit is strictly enforced.

An excellent detailed explanation of these settings and their effect (as well as when it makes sense to modify them) can be found in this pivotal blog entry. (Tl;dr: messing with overcommit is useful if you don’t want critical processes to use swap)

We all know that while doing any operations on file CPU uses resources from our system. It may be memory or some I/O devices. Moreover, depending on the load of the operations, it sometimes shares heavy or graphical operations to GPUs. Now, it may be possible that the system can always fulfill the memory or I/O requirement every time. In that case, the CPU may stop its execution and wait for the fulfillment of the resources, or sometimes it may terminate the program. In this article, we will keep some light on one such issue named OsError: [errno 12] cannot allocate memory.

Oserror: [errno 12] cannot allocate memory is raised by the system when CPU won’t get enough memory resources to process pipelined operations and execute them. It may be possible that the system doesn’t have enough memory for storing intermediaries while the program is in the process. Or, it may run out of available memory due to its usage by other pipelined operations.

Oserror: [errno 12] cannot allocate memory is the memory unavailability error when we run our python program, and the system does not fulfill the memory requirement. This unavailability of resources leads to the termination of our python program with the raised error Oserror: [errno 12] cannot allocate memory.

Why do I get “Oserror: [errno 12] cannot allocate memory” Error?

There is no other reason than mentioned above for the error. Any operation we perform on our computer requires some memory resources and some input/output devices. These memory resources are used to store the intermediate files or variables created while executing our program. CPU fetches these files and variables from the main memory for further execution. Now, when the system cannot save those files in the main memory due to unavailable space, it raises the given error.

[Note: The availability of Secondary memory cannot fulfill the memory requirement by default as it is much slower than main memory. This increases the throughput of the program and can be durably expensive.]

Solution to Oserror: [errno 12] cannot allocate memory

The solution to the given error is that we need to increase the system’s memory space so that it can store the intermediate files there. Now, increasing RAM memory may not be practically possible for everyone. In that case, we need to check for some other alternative.

Now, it may not be necessary to store intermediate files. We necessarily need to extend our RAM as it is not a feasible option. What we can do we create a virtual memory in our system. This virtual memory is the part of hard disk memory allocated to work as the main memory( RAM). The use of this virtual memory is that the CPU can store temporary files between the operations. The virtual memory is also called swap memory. We need to assign 64 GB of swap memory to the system so that the CPU can use it whenever there is a requirement for that. It can be a feasible option and requires no extra cost for that. The only obstacle in this process is that it is slower whenever the CPU uses this swap memory because fetching data from the main memory is faster than a hard disk.

Python subprocess.Popen “OSError: [Errno 12] Cannot allocate memory”

However, you may also get the error while using subprocess.Popen. While using the subprocess.Popen() we somehow call the fork instance, which means that we are creating a child process or subprocess, and it is going to consume the same amount of memory that is already getting consumed by the python. Now, there can be hundreds of MB of memory space required for that. Subsequently, there may be a memory shortage, and we get the error. Now, to solve the issue, there may be two options: to increase the memory space, or the second option is to write efficient code controlling the script memory.

However, to do that, we have the alternative of using vfork and posix_spawn. But maybe we don’t want to rewrite the code for the subprocess.Popen in terms of posix_spawn/vfork. In that condition, one can use a subprocess.Popen but at the beginning of the script so that the memory that will be consumed is minimum due to fewer resources, and we got the shell script. Now, we can use the shell script for our parallel processing with the lighter tasks such as free/ps/sleep, whatever else is in a loop.

Recommended Reading | Python Memory Error | How to Solve Memory Error in Python

Conclusion

So, today in this article, we learned about Oserror: [errno 12] cannot allocate memory Error. We have seen what the possible reasons for the given error are. Then we have discussed the possible solutions for that. After following the above explanation, you will be able to solve the error.

I hope this article has helped you. Thank you.

Hello,
I am getting Cannot allocate memory error;I understand this is something related to my GPU. But it is quite surprising that I should get this error because, I am training this on 3 1080TI GPUs, with a batch size of 64.

Traceback (most recent call last):
  File "train.py", line 162, in <module>
    train()
  File "train.py", line 113, in train
    for batch_idx, (data, target) in enumerate(train_loader):
  File "/usr/local/torch3/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 310, in __iter__
    return DataLoaderIter(self)
  File "/usr/local/torch3/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 167, in __init__
    w.start()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.5/multiprocessing/context.py", line 212, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/lib/python3.5/multiprocessing/context.py", line 267, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.5/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.5/multiprocessing/popen_fork.py", line 67, in _launch
    self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory

CUDA_VISIBLE_DEVICES=0,1,2 python train.py ~/DATASETS/cifar.python cifar10 -s ./snapshots —log ./logs —ngpu 3 —learning_rate 0.05 -b 64

Please suggest what I could do to avoid this issue.
Thank You!

Bitbucket Support

Knowledge base

Products

Jira Software

Project and issue tracking

Jira Service Management

Service management and customer support

Jira Work Management

Manage any business project

Confluence

Bitbucket

Git code management

Resources

Documentation

Usage and admin help

Answers, support, and inspiration

Suggestions and bugs

Feature suggestions and bug reports

Marketplace

Billing and licensing

Frequently asked questions

Viewport

Confluence

Forking JVM: error=12, Cannot allocate memory or error=12, Not enough space

Related content

Still need help?

The Atlassian Community is here for you.

Symptoms

Cause

If you received error=12, Cannot allocate memory or error=12, Not enough space , this means your system ran out of memory or swap space when Java tried to fork a process.

The problem is inherent with the way Java allocates memory when executing processes. When Java executes a process, it must fork() then exec(). Forking creates a child process by duplicating the current process. By duplicating the current process, the new child process will request approximately the same amount of memory as its parent process, essentially doubling the memory required. However, this does not mean all the memory allocated will be used, as exec() is immediately called to execute the different code within the child process, freeing up this memory. As an example, when Bitbucket Server tries to locate git, the Bitbucket Server JVM process must be forked, approximately doubling the memory required by Bitbucket Server.

Note that this issue is made worse by hosting multiple webapps in the same Tomcat container as Bitbucket Server (which is not supported), because more memory is used by the Java process.

Please see Forking JVM for a more detailed explanation.

Resolution

If you are hosting multiple products in the same Tomcat container as Bitbucket Server, move Bitbucket Server to its own Tomcat container.

For Linux, this can be resolved by enabling over-committing memory.

For Solaris, we recommend increasing your swap space. How much swap space do you need? This will also depend on the other applications running on the same machine. We recommend allocating at least swap equal to 4 x JVM_MAXIMUM_MEMORY , or 2 x PHYSICAL RAM, whichever is larger. The JVM_MAXIMUM_MEMORY is set in your /bin/setenv.sh . You can increase your swap space if you are still running out of memory.

setenv and environment variable changes in Bitbucket Server 5.0+

Starting with Bitbucket Server 5.0, setenv.sh and setenv.bat have been removed. The options that were set in this file can now be set via environment variables. Where to set the environment variable depends on which Operating System you’re running on.

Linux

When using the atlbitbucket service on Linux, the environment variables are ignored. You must set the parameters in _start-webapp.sh (or start-bitbucket.sh ) . These values will be read when the service starts.

As an example, to set JVM_SUPPORT_RECOMMENDED_ARGS , you would add this line to the file:

Windows

Set the parameter as an environment variable for the user running Bitbucket Server. For example, if you want to set JVM_SUPPORT_RECOMMENDED_ARGS , create it as an environment variable and assign the appropriate value to it. When Bitbucket Server starts using the startup scripts or service, it will pick up and apply this value.

Источник

Java error=12, cannot allocate memory (also chmod) #1403

Please use the following bug reporting template to help produce actionable and reproducible issues. Please try to ensure that the reproduction is minimal so that the team can go through more bugs!

Java, chmod & other tools fail with «Cannot allocate memory» when trying to build an open source project (Apache Ranger). Same code builds fine on non-MS unix systems with less memory (ie 4GB — this system has 16)

Apache Ranger builds ok

• Actual results (with terminal output if applicable)

/src/ranger$ cat ../ranger100.log | grep allocate
java.io.IOException: Cannot run program «chmod»: error=12, Cannot allocate memory
Caused by: java.io.IOException: error=12, Cannot allocate memory
Caused by: java.io.IOException: Cannot run program «chmod»: error=12, Cannot allocate memory
Caused by: java.io.IOException: error=12, Cannot allocate memory
java.io.IOException: Cannot run program «chmod»: error=12, Cannot allocate memory
Caused by: java.io.IOException: error=12, Cannot allocate memory
Caused by: java.io.IOException: Cannot run program «chmod»: error=12, Cannot allocate memory
Caused by: java.io.IOException: error=12, Cannot allocate memory
Caused by: java.io.IOException: Cannot run program «chmod»: error=12, Cannot allocate memory
Caused by: java.io.IOException: error=12, Cannot allocate memory

(will attach full log)

Your Windows build number
14971

Steps / All commands required to reproduce the error from a brand new installation
Build apache ranger as per http://ranger.apache.org/quick_start_guide.html
(Note — open jdk 1.8 is installed, and as of 21/11/16 a 1.8 patch is needed as per https://reviews.apache.org/r/53924/ — business as usual. . )

Strace of the failing command
not obtained

Required packages and commands to install
See apache page above

The text was updated successfully, but these errors were encountered:

See http://pastebin.com/26fA3XHX for the ranger build logs. Search for «allocate»

/src$ free -m
total used free shared buffers cached
Mem: 15946 10281 5664 17 33 184
-/+ buffers/cache: 10064 5882
Swap: 6873 473 6399

And here’s the fork failure from the log:

Caused by: java.io.IOException: error=12, Cannot allocate memory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.(UNIXProcess.java:247)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
. 40 more

And some version info:

jonesn@DESKTOP-2V27KS0:/mnt/c/tmp$ dpkg -l | grep openjdk
ii openjdk-7-jdk:amd64 7u121-2.6.8-1ubuntu0.14.04.1 amd64 OpenJDK Development Kit (JDK)
ii openjdk-7-jre:amd64 7u121-2.6.8-1ubuntu0.14.04.1 amd64 OpenJDK Java runtime, using Hotspot JIT
ii openjdk-7-jre-headless:amd64 7u121-2.6.8-1ubuntu0.14.04.1 amd64 OpenJDK Java runtime, using Hotspot JIT (headless)
ii openjdk-8-jdk:amd64 8u111-b14-3 14.04.1 amd64 OpenJDK Development Kit (JDK)
ii openjdk-8-jdk-headless:amd64 8u111-b14-3 14.04.1 amd64 OpenJDK Development Kit (JDK) (headless)
ii openjdk-8-jre:amd64 8u111-b14-3 14.04.1 amd64 OpenJDK Java runtime, using Hotspot JIT
ii openjdk-8-jre-headless:amd64 8u111-b14-3 14.04.1 amd64 OpenJDK Java runtime, using Hotspot JIT (headless)
jonesn@DESKTOP-2V27KS0:/mnt/c/tmp$ java -version
openjdk version «1.8.0_111»
OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3

14.04.1-b14)
OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)

Hi @planetf1 — thanks for reporting this! I suspect that it might have the same underlying cause as #1019 ; specifically, some operations in WSL currently don’t work properly if your system’s page file is small as compared to your available RAM. Could you try the instructions in the last comment on that ticket to see if they help your situation?

Thanks. My virtual memory settings currently are
c: System Managed (11205 MB)
d: None

I have 16GB ram

Unfortunately I’m also rather constrainted on storage space — it’s a mere 120 GB SSD. I’ll however try setting it to a fixed size, say 16GB & report back

Setting to fixed 16GB allowed the compile to progress past this step. However it failed later on with a different error (one of the test cases seemed to exit prematurely with no obvious error). No obvious cause that I can precisely describe here for debugging the ms bash system in particular as the open source project is a new one for me that I’m getting up to speed with. I can only say the build is fine in regular ubuntu in hyper-v, or a remote system, or CentOS, so I’m going to have to revert to those environments for now. It would be good to understand more about the initial memory issue going on here though — I would have thought things should work just fine with a system managed page file? Thanks.

I assume (though I don’t know for sure) that the memory limitation is due to the difference in the behavior of the NT kernel vs the Linux kernel:

Windows assumes that, if an application maps («asks the kernel to let it use in the future») a block of virtual-memory pages, it does actually intend to use them eventually. It therefore requires that enough memory currently be available, either as real physical memory or as swap, to allow the entirety of any given mapping to be allocated and to have data assigned to it. If it gets some unmanageably-large allocations (I’m sure there are specific rules but I don’t know the rules offhand), it doesn’t necessarily grow the page file; it may instead choose to return an error code to indicate to the application that it probably doesn’t want to be allocating that much memory on this machine. This approach can be a little clunky/limiting, but it has some nice correctness guarantees and it tends to encourage good application behavior.

Linux pretty much always lets applications map whatever they want, whether or not it’s reasonable on the current machine. Therefore, for simplicity, some applications’ runtimes (notably including Java; I’m not personally aware of any other major applications that do this to such a large extent) tend to map giant chunks of memory up front, presumably so that they don’t have to think about mapping more memory later. They then allocate and use only as much memory as they actually use. If they allocate more memory than is available on the system (and the swap file runs out of space, etc), which they can do because the kernel has already promised the mapping, at that point it’s too late for the kernel to provide an error code; the only correct thing it can do is to force-kill the application.

Supporting the Linux behavior on Windows would therefore require either allowing the Windows pagefile to grow much larger than the amount of memory that Java is likely to actually use, just in case it does use it (which would be problematic on machines like yours with limited disk space), or implementing the Linux out-of-memory killer in Windows, which I would expect to require a change to the core NT kernel (not WSL-specific code).

(Disclaimer: I’m not actually a WSL dev, just a Windows and Linux user; this is just my best guess at the cause based on the behavior of the two kernels’ public APIs, it’s certainly possible that there’s a different limitation going on under the hood.)

Источник

Forking JVM: error=12, Cannot allocate memory or error=12, Not enough space

Troubleshooting Git

On this page

Related content

Still need help?

The Atlassian Community is here for you.

Symptoms

Cause

If you received error=12, Cannot allocate memory or error=12, Not enough space , this means your system ran out of memory or swap space when Java tried to fork a process.

The problem is inherent with the way Java allocates memory when executing processes. When Java executes a process, it must fork() then exec(). Forking creates a child process by duplicating the current process. By duplicating the current process, the new child process will request approximately the same amount of memory as its parent process, essentially doubling the memory required. However, this does not mean all the memory allocated will be used, as exec() is immediately called to execute the different code within the child process, freeing up this memory. As an example, when Stash tries to locate git, the Stash JVM process must be forked, approximately doubling the memory required by Stash.

Note that this issue is made worse by hosting multiple webapps in the same Tomcat container as Stash (which is not supported), because more memory is used by the Java process.

Please see Forking JVM for a more detailed explanation.

Resolution

If you are hosting multiple products in the same Tomcat container as Stash, move Stash to its own Tomcat container.

For Linux, this can be resolved by enabling over-committing memory (see option 3 of Forking JVM).

Источник

Suddenly error=’Cannot allocate memory’ (errno=12) [closed]

Questions should demonstrate reasonable business information technology management practices. Questions that relate to unsupported hardware or software platforms or unmaintained environments may not be suitable for Server Fault — see the help center.

Closed 7 years ago .

So this morning I noticed my website is down, went on my server and saw that the webserver has been stopped.

When I try to start it again I get this error

What is confusing me is that yesterday the webserver ran without problems, and suddenly this morning it is down? Debian 7.8 is runnin on the server.

cat /proc/meminfo shows

It seems that suddenly I don’t have enough memory right? I really don`t know what to do, to make it work again, hopefully someone can help me out.

Here the full error log file:

1 Answer 1

Your log sais that when the application is crashing, there is 224852k memory free:

Memory: 4k page, physical 1027040k(224852k free), swap 0k(0k free)

Also I see that you have no swap partition added. It’s obvious that your system starves of memory. So you have to increase physical memory or add swap space.

To add swap space while the system runs, create a partition, marked as Linux Swap. That the partition id 82. Then use that command to set up the partition as a swap area:

Where /dev/sda4 , is the device file.

To activate the partition use:

After that you should see SwapTotal and SwapFree in /proc/meminfo .

To make this permanent add a line like this in /etc/fstab :

Источник

Fisheye Support

Knowledge base

Products

Jira Software

Project and issue tracking

Jira Service Management

Service management and customer support

Jira Work Management

Manage any business project

Confluence

Bitbucket

Git code management

Resources

Documentation

Usage and admin help

Answers, support, and inspiration

Suggestions and bugs

Feature suggestions and bug reports

Marketplace

Billing and licensing

Frequently asked questions

Viewport

Confluence

Fix Out of Memory Errors

Related content

Still need help?

The Atlassian Community is here for you.

On this page:

Updating JVM options when running Fisheye / Crucible on Windows

The Apache procrun monitor application is used for editing the service and it is installed automatically. You can usually open it by going to Windows Start Menu > All Programs > Fisheye > Configure Fisheye .
If this path cannot be found in the Windows Start Menu:
Open the Windows services list and open the Properties panel for the Fisheye service in order to copy the Service Name to clipboard (it’s usually named Atlassian Fisheye).

In a Command Prompt window, navigate to /bin and run a command like this:

Where SERVICE_NAME needs to be replaced by what has been copied to clipboard, for example:

If it’s a Fisheye environment variable (such as FISHEYE_INST , FISHEYE_OPTS , etc), add -D at the beginning of the line and type the option name in lowercase. The option value can be camel cased. Example:

If it’s a regular JVM option there is no need to add -D at the beginning. Example:

1. Modify JVM options as needed in the FISHEYE_OPTS environment variable (see the page on Environment variables).
2. Fisheye’s own variables such as FISHEYE_ARGS are declared separately as environment variable.

To check if the environment variable is set and what its value currently is, use the echo command. For example:

Updating JVM options when running Fisheye / Crucible on Linux or MacOS

At Installing Fisheye on Linux and Mac we recommend setting environment variables in the /etc/environment file on Linux distributions and in the

/.profile file for the current user on MacOS.

Environment variables can be set in multiple different bash configuration files though. In most distributions, when you start a new session the environment variables are read from the following files:

/etc/environment : Use this file to set up system-wide environment variables. Variables in this file are set in the following format:

/etc/profile : Variables set in this file are loaded whenever a bash login shell is entered. When declaring environment variables in this file you need to use the export command:

/.profile: Per-user shell specific configuration files. When declaring environment variables in this file you need to use the export command:

To load the new environment variables into the current shell session use the source command. For example:

To check if the environment variable is set and what its value currently is, use the echo command. For example:

Troubleshooting

There are a number of different Out Of Memory Errors that the JVM can encounter. Solving these errors might require that you update the default memory settings by following the steps from the previous sections of this article, according to Fisheye’s host operating system.

The most common types of Out Of Memory Errors are listed below:

OutOfMemoryError: Java Heap Space

• Meaning: The JVM had to allocate more memory than it is currently allowed to.
• Default value: 1024 MB / 1 GB
• Known scenarios:
  • During the initial repository scan period
  • While syncing a large number of users from an external user directory
  • While a large backup is being performed
  • When working on large reviews
  • Installed add-ons
• How to fix it: This usually indicates you need to increase the amount of memory the JVM can allocate (also known as memory heap) to Fisheye, however doing so may mask and / or postpone the real issue. In addition to that, there are many things to consider when increasing the available memory to your application. For example, the amount of memory being allowed cannot be greater than the RAM configured on the server and also needs to take into consideration the memory required by all the other applications installed on the system and the amount of memory required by the operating system itself. It may be needed to increase the RAM on the server before actually increasing the amount of memory the JVM used by Fisheye can allocate.
  When needed, though, to increase the amount of memory the JVM can allocate you need to:
  • If Fisheye runs on Windows and the Windows service for Fisheye exists, set / increase the Maximum memory pool field in the Apache procrun monitor application (see instructions in the section above).
  • If Fisheye runs on Windows and the Windows service for Fisheye does not exist, set / increase the -Xmx option in the FISHEYE_OPTS environment variable (see instructions in the section above).
  • If Fisheye runs on Linux or Mac, set / increase the -Xmx option in the FISHEYE_OPTS environment variable (see instructions in the section above).
  • Restart Fisheye afterwards.

OutOfMemoryError: unable to create new native thread

• Meaning: This error occurs when the operating system is unable to create new threads.
• Default value: not set by default.
• Known scenarios:
  • The JVM heap may be taking up the available RAM on the system. Big heaps take away from the space that can be allocated for the stack of a new thread.
  • The OS limits for the maximum number of threads are too low
• How to fix it:
  • Option 1: Reduce the heap:

    The size of the stack per thread ( -Xss ) can also contribute to this problem. The stack size can reduce the number of threads that can be created.
    The default value of -Xss depends on the platform:
    — On Windows the default value depends on virtual memory.

    — On Linux / MacOS the default value is 1024 KB.

    To fix this problem, you should reduce the amount of memory the JVM can allocate ( -Xmx ) and also reduce the size of the JVM stack per thread ( -Xss ).

    • If Fisheye runs on Windows and the Windows service for Fisheye exists, open the Apache procrun monitor application and:
      • Set / decrease the Maximum memory pool field
      • Set the -Xss option in the Java Options list to, for example, -Xss512k .
      • See instructions in the section above.
    • If Fisheye runs on Windows and the Windows service for Fisheye does not exist, or if Fisheye runs on Linux or MacOS:
      • Set / decrease the -Xmx option in the FISHEYE_OPTS environment variable
      • Set the -Xss option to -Xss512k in the FISHEYE_OPTS environment variable
      • See instructions in the section above for each operating system.
    • Restart Fisheye afterwards.
• • Option 2: Increase the number of allowed threads:
    In Linux / MacOS systems the maximum number of allowed threads can be checked by executing ulimit -u as the user running Fisheye/Crucible.
    This limit can be increased by executing ulimit -u 2048 for example, which will set the maximum number of threads to 2048.
    Restart Fisheye afterwards.

OutOfMemoryError: GC overhead limit exceeded

• Meaning: This error indicates the JVM took too long to free up memory during its garbage collection process.
• Default value: not set by default.
• Known scenarios: The Oracle Java documentation states that this error is reported «if more than 98% of the total time is spent in garbage collection and less than 2% of the heap is recovered», and that «this feature is designed to prevent applications from running for an extended period of time while making little or no progress because the heap is too small». This may happen if the JVM starts to use the swapped memory for its heap. This will cause the JVM to take longer than normal to perform normal GC operations which will eventually cause a timeout to occur.

How to fix it: Even though this feature can be disabled by adding the -XX:-UseGCOverheadLimit JVM option to stop this kind of error to be written in the Fisheye log files, it is not a good idea to do so. To overcome this issue, you actually need to make sure that all processes can’t allocate more memory than the RAM available on the system. In practice this is impossible to calculate that for all processes so at a minimum you should make sure that the JVM does not have a total maximum memory allocation than your normally available system memory.
To analyze the garbage collection activity and to explore the contents the application’s memory, the JVM options below should be used. Please check in the sections above how to add these options according to your use case:

Restart your Fisheye instance after adding these options above and enable debug logging via Fisheye web interface by going to Administration > Global Settings > Server > Debug Logging and clicking the Turn Debugging ON button.

The atlassian-fecru-gc- .log file will be created automatically on restart, however the heap dump file named like java_pid

.hprof (e.g. java_pid12345.hprof ) that is the size of the heap will only be created on the next OutOfMemoryError .

If you open a support request with the following items the Atlassian Support team will be able to investigate this problem further:
— The atlassian-fecru-gc- .log
— The java_pid

.hprof
— The support zip with debug logging enabled before the error was reported.

The garbage collection settings (excluding the -Xloggc one) can be enabled at runtime as explained on the How to change JVM arguments at runtime to avoid application restart guide.

java.lang.OutOfMemoryError: requested 32756 bytes for ChunkPool::allocate. Out of swap space?

• Meaning: The native objects do not have enough memory to use.
• Known scenarios: This usually happens when you have allocated too much memory to your heap, reducing the amount available for native objects. See this article.
• How to fix it: Reduce the amount of heap memory you have allocated. For example, if you currently allow the JVM to allocate up to 4096m of memory, you should consider reducing this to 3072m or even 2048m.

Additional reading

The following documentation provides additional clarifications on adjusting limits and performance settings in Fisheye:

Источник

So this morning I noticed my website is down, went on my server and saw that the webserver has been stopped.

When I try to start it again I get this error

 start -Dhttp.port=80 -Dapplication.secret=**********
[info] Wrote /home/Jakob/Apps/Botlanegg/target/scala-2.11/botlanegg_2.11-1.0.pom

(Starting server. Type Ctrl+D to exit logs, the server will remain in background)

Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000d5550000, 715849728, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 715849728 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /home/Jakob/Apps/Botlanegg/hs_err_pid3892.log

What is confusing me is that yesterday the webserver ran without problems, and suddenly this morning it is down? Debian 7.8 is runnin on the server.

cat /proc/meminfo shows

 MemTotal:        1027040 kB
MemFree:          626340 kB
Buffers:           16304 kB
Cached:           233932 kB
SwapCached:            0 kB
Active:           249580 kB
Inactive:         121680 kB
Active(anon):     121076 kB
Inactive(anon):      164 kB
Active(file):     128504 kB
Inactive(file):   121516 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                48 kB
Writeback:             0 kB
AnonPages:        121056 kB
Mapped:            21108 kB
Shmem:               216 kB
Slab:              17412 kB
SReclaimable:      10892 kB
SUnreclaim:         6520 kB
KernelStack:        1168 kB
PageTables:         2704 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      513520 kB
Committed_AS:     438980 kB
VmallocTotal:   34359738367 kB
VmallocUsed:        4972 kB
VmallocChunk:   34359733071 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       30708 kB
DirectMap2M:     1017856 kB

It seems that suddenly I don’t have enough memory right? I really don`t know what to do, to make it work again, hopefully someone can help me out.

Here the full error log file:

# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 715849728 bytes for committing reserved memory.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (os_linux.cpp:2671), pid=3139, tid=140373617166080
#
# JRE version:  (8.0_31-b13) (build )
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#

---------------  T H R E A D  ---------------

Current thread (0x0000000001cfa000):  JavaThread "Unknown thread" [_thread_in_vm, id=3194, stack(0x00007fab47864000,0x00007fab47965000)]

Stack: [0x00007fab47864000,0x00007fab47965000],  sp=0x00007fab47963390,  free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xa79e0a]  VMError::report_and_die()+0x2ca
V  [libjvm.so+0x4e57bb]  report_vm_out_of_memory(char const*, int, unsigned long, VMErrorType, char const*)+0x8b
V  [libjvm.so+0x8e5a23]  os::Linux::commit_memory_impl(char*, unsigned long, bool)+0x103
V  [libjvm.so+0x8e5f79]  os::pd_commit_memory(char*, unsigned long, unsigned long, bool)+0x29
V  [libjvm.so+0x8dfc4a]  os::commit_memory(char*, unsigned long, unsigned long, bool)+0x2a
V  [libjvm.so+0xa75d39]  VirtualSpace::expand_by(unsigned long, bool)+0x1c9
V  [libjvm.so+0xa768ce]  VirtualSpace::initialize(ReservedSpace, unsigned long)+0xee
V  [libjvm.so+0x5d2e21]  CardGeneration::CardGeneration(ReservedSpace, unsigned long, int, GenRemSet*)+0xf1
V  [libjvm.so+0xa209bc]  TenuredGeneration::TenuredGeneration(ReservedSpace, unsigned long, int, GenRemSet*)+0x5c
V  [libjvm.so+0x5d409b]  GenerationSpec::init(ReservedSpace, int, GenRemSet*)+0x33b
V  [libjvm.so+0x5c2bee]  GenCollectedHeap::initialize()+0x1ee
V  [libjvm.so+0xa45d4d]  Universe::initialize_heap()+0xfd
V  [libjvm.so+0xa4616f]  universe_init()+0x3f
V  [libjvm.so+0x6128d5]  init_globals()+0x65
V  [libjvm.so+0xa2bdbd]  Threads::create_vm(JavaVMInitArgs*, bool*)+0x24d
V  [libjvm.so+0x6a9b34]  JNI_CreateJavaVM+0x74
C  [libjli.so+0x736e]  JavaMain+0x9e
C  [libpthread.so.0+0x6b50]  start_thread+0xd0


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )

Other Threads:

=>0x0000000001cfa000 (exited) JavaThread "Unknown thread" [_thread_in_vm, id=3194, stack(0x00007fab47864000,0x00007fab47965000)]

VM state:not at safepoint (not fully initialized)

VM Mutex/Monitor currently owned by a thread: None

GC Heap History (0 events):
No events

Deoptimization events (0 events):
No events

Internal exceptions (0 events):
No events

Events (0 events):
No events


Dynamic libraries:
00400000-00401000 r-xp 00000000 fe:02 884862                             /usr/lib/jvm/jdk1.8.0_31/bin/java
00600000-00601000 rw-p 00000000 fe:02 884862                             /usr/lib/jvm/jdk1.8.0_31/bin/java
01cea000-01d2c000 rw-p 00000000 00:00 0                                  [heap]
c0000000-d5550000 rw-p 00000000 00:00 0 
7fab3c47e000-7fab3c79f000 rw-p 00000000 00:00 0 
7fab3c79f000-7fab3c8f4000 ---p 00000000 00:00 0 
7fab3c8f4000-7fab3c8ff000 rw-p 00000000 00:00 0 
7fab3c8ff000-7fab3caf5000 ---p 00000000 00:00 0 
7fab3caf5000-7fab3cd65000 rwxp 00000000 00:00 0 
7fab3cd65000-7fab44af5000 ---p 00000000 00:00 0 
7fab44af5000-7fab44b0f000 r-xp 00000000 fe:02 901189                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/libzip.so
7fab44b0f000-7fab44d0f000 ---p 0001a000 fe:02 901189                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/libzip.so
7fab44d0f000-7fab44d10000 rw-p 0001a000 fe:02 901189                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/libzip.so
7fab44d10000-7fab44d1b000 r-xp 00000000 fe:02 510300                     /lib/x86_64-linux-gnu/libnss_files-2.13.so
7fab44d1b000-7fab44f1a000 ---p 0000b000 fe:02 510300                     /lib/x86_64-linux-gnu/libnss_files-2.13.so
7fab44f1a000-7fab44f1b000 r--p 0000a000 fe:02 510300                     /lib/x86_64-linux-gnu/libnss_files-2.13.so
7fab44f1b000-7fab44f1c000 rw-p 0000b000 fe:02 510300                     /lib/x86_64-linux-gnu/libnss_files-2.13.so
7fab44f1c000-7fab44f26000 r-xp 00000000 fe:02 510330                     /lib/x86_64-linux-gnu/libnss_nis-2.13.so
7fab44f26000-7fab45125000 ---p 0000a000 fe:02 510330                     /lib/x86_64-linux-gnu/libnss_nis-2.13.so
7fab45125000-7fab45126000 r--p 00009000 fe:02 510330                     /lib/x86_64-linux-gnu/libnss_nis-2.13.so
7fab45126000-7fab45127000 rw-p 0000a000 fe:02 510330                     /lib/x86_64-linux-gnu/libnss_nis-2.13.so
7fab45127000-7fab4513c000 r-xp 00000000 fe:02 510320                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fab4513c000-7fab4533b000 ---p 00015000 fe:02 510320                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fab4533b000-7fab4533c000 r--p 00014000 fe:02 510320                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fab4533c000-7fab4533d000 rw-p 00015000 fe:02 510320                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fab4533d000-7fab4533f000 rw-p 00000000 00:00 0 
7fab4533f000-7fab45346000 r-xp 00000000 fe:02 510361                     /lib/x86_64-linux-gnu/libnss_compat-2.13.so
7fab45346000-7fab45545000 ---p 00007000 fe:02 510361                     /lib/x86_64-linux-gnu/libnss_compat-2.13.so
7fab45545000-7fab45546000 r--p 00006000 fe:02 510361                     /lib/x86_64-linux-gnu/libnss_compat-2.13.so
7fab45546000-7fab45547000 rw-p 00007000 fe:02 510361                     /lib/x86_64-linux-gnu/libnss_compat-2.13.so
7fab45547000-7fab45571000 r-xp 00000000 fe:02 901184                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/libjava.so
7fab45571000-7fab45771000 ---p 0002a000 fe:02 901184                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/libjava.so
7fab45771000-7fab45773000 rw-p 0002a000 fe:02 901184                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/libjava.so
7fab45773000-7fab45780000 r-xp 00000000 fe:02 901188                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/libverify.so
7fab45780000-7fab45980000 ---p 0000d000 fe:02 901188                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/libverify.so
7fab45980000-7fab45982000 rw-p 0000d000 fe:02 901188                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/libverify.so
7fab45982000-7fab45989000 r-xp 00000000 fe:02 510380                     /lib/x86_64-linux-gnu/librt-2.13.so
7fab45989000-7fab45b88000 ---p 00007000 fe:02 510380                     /lib/x86_64-linux-gnu/librt-2.13.so
7fab45b88000-7fab45b89000 r--p 00006000 fe:02 510380                     /lib/x86_64-linux-gnu/librt-2.13.so
7fab45b89000-7fab45b8a000 rw-p 00007000 fe:02 510380                     /lib/x86_64-linux-gnu/librt-2.13.so
7fab45b8a000-7fab45c0b000 r-xp 00000000 fe:02 510365                     /lib/x86_64-linux-gnu/libm-2.13.so
7fab45c0b000-7fab45e0a000 ---p 00081000 fe:02 510365                     /lib/x86_64-linux-gnu/libm-2.13.so
7fab45e0a000-7fab45e0b000 r--p 00080000 fe:02 510365                     /lib/x86_64-linux-gnu/libm-2.13.so
7fab45e0b000-7fab45e0c000 rw-p 00081000 fe:02 510365                     /lib/x86_64-linux-gnu/libm-2.13.so
7fab45e0c000-7fab46a7a000 r-xp 00000000 fe:02 901174                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/server/libjvm.so
7fab46a7a000-7fab46c79000 ---p 00c6e000 fe:02 901174                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/server/libjvm.so
7fab46c79000-7fab46d4f000 rw-p 00c6d000 fe:02 901174                     /usr/lib/jvm/jdk1.8.0_31/jre/lib/amd64/server/libjvm.so
7fab46d4f000-7fab46d92000 rw-p 00000000 00:00 0 
7fab46d92000-7fab46f14000 r-xp 00000000 fe:02 510355                     /lib/x86_64-linux-gnu/libc-2.13.so
7fab46f14000-7fab47114000 ---p 00182000 fe:02 510355                     /lib/x86_64-linux-gnu/libc-2.13.so
7fab47114000-7fab47118000 r--p 00182000 fe:02 510355                     /lib/x86_64-linux-gnu/libc-2.13.so
7fab47118000-7fab47119000 rw-p 00186000 fe:02 510355                     /lib/x86_64-linux-gnu/libc-2.13.so
7fab47119000-7fab4711e000 rw-p 00000000 00:00 0 
7fab4711e000-7fab47120000 r-xp 00000000 fe:02 510423                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fab47120000-7fab47320000 ---p 00002000 fe:02 510423                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fab47320000-7fab47321000 r--p 00002000 fe:02 510423                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fab47321000-7fab47322000 rw-p 00003000 fe:02 510423                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fab47322000-7fab47337000 r-xp 00000000 fe:02 892931                     /usr/lib/jvm/jdk1.8.0_31/lib/amd64/jli/libjli.so
7fab47337000-7fab47536000 ---p 00015000 fe:02 892931                     /usr/lib/jvm/jdk1.8.0_31/lib/amd64/jli/libjli.so
7fab47536000-7fab47537000 rw-p 00014000 fe:02 892931                     /usr/lib/jvm/jdk1.8.0_31/lib/amd64/jli/libjli.so
7fab47537000-7fab4754e000 r-xp 00000000 fe:02 510416                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fab4754e000-7fab4774d000 ---p 00017000 fe:02 510416                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fab4774d000-7fab4774e000 r--p 00016000 fe:02 510416                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fab4774e000-7fab4774f000 rw-p 00017000 fe:02 510416                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fab4774f000-7fab47753000 rw-p 00000000 00:00 0 
7fab47753000-7fab47773000 r-xp 00000000 fe:02 510324                     /lib/x86_64-linux-gnu/ld-2.13.so
7fab4785c000-7fab47864000 rw-s 00000000 fe:02 655376                     /tmp/hsperfdata_root/3139
7fab47864000-7fab47867000 ---p 00000000 00:00 0 
7fab47867000-7fab47969000 rw-p 00000000 00:00 0 
7fab4796b000-7fab4796f000 rw-p 00000000 00:00 0 
7fab4796f000-7fab47970000 r--p 00000000 00:00 0 
7fab47970000-7fab47972000 rw-p 00000000 00:00 0 
7fab47972000-7fab47973000 r--p 0001f000 fe:02 510324                     /lib/x86_64-linux-gnu/ld-2.13.so
7fab47973000-7fab47974000 rw-p 00020000 fe:02 510324                     /lib/x86_64-linux-gnu/ld-2.13.so
7fab47974000-7fab47975000 rw-p 00000000 00:00 0 
7fff37ee7000-7fff37f0a000 rw-p 00000000 00:00 0                          [stack]
7fff37fff000-7fff38000000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

VM Arguments:
jvm_args: -Xms1024m -Xmx1024m -XX:ReservedCodeCacheSize=128m -Duser.dir=/home/Jakob/Apps/Botlanegg/target/universal/stage -Dhttp.port=80 -Dapplication.secret=8a7s9df7as9d -Dhttp.port=80 
java_command: play.core.server.NettyServer
java_class_path (initial): /home/Jakob/Apps/Botlanegg/target/universal/stage/lib/botlanegg.botlanegg-1.0.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/org.scala-lang.scala-library-2.11.1.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.typesafe.play.twirl-api_2.11-1.0.2.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/org.apache.commons.commons-lang3-3.1.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/org.scala-lang.modules.scala-xml_2.11-1.0.1.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.typesafe.play.play_2.11-2.3.7.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.typesafe.play.build-link-2.3.7.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.typesafe.play.play-exceptions-2.3.7.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/org.javassist.javassist-3.18.2-GA.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.typesafe.play.play-iteratees_2.11-2.3.7.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/org.scala-stm.scala-stm_2.11-0.7.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.typesafe.config-1.2.1.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.typesafe.play.play-json_2.11-2.3.7.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.typesafe.play.play-functional_2.11-2.3.7.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.typesafe.play.play-datacommons_2.11-2.3.7.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/joda-time.joda-time-2.3.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/org.joda.joda-convert-1.6.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.fasterxml.jackson.core.jackson-annotations-2.3.2.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.fasterxml.jackson.core.jackson-core-2.3.2.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/com.fasterxml.jackson.core.jackson-databind-2.3.2.jar:/home/Jakob/Apps/Botlanegg/target/universal/stage/lib/or
Launcher Type: SUN_STANDARD

Environment Variables:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SHELL=/bin/bash

Signal Handlers:
SIGSEGV: [libjvm.so+0xa7a650], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGBUS: [libjvm.so+0xa7a650], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGFPE: [libjvm.so+0x8e28f0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGPIPE: [libjvm.so+0x8e28f0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGXFSZ: [libjvm.so+0x8e28f0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGILL: [libjvm.so+0x8e28f0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGUSR1: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGUSR2: [libjvm.so+0x8e4120], sa_mask[0]=00100000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
SIGHUP: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGINT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGTERM: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGQUIT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none


---------------  S Y S T E M  ---------------

OS:PRETTY_NAME="Debian GNU/Linux 7 (wheezy)"
NAME="Debian GNU/Linux"
VERSION_ID="7"
VERSION="7 (wheezy)"
ID=debian
ANSI_COLOR="1;31"
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support/"
BUG_REPORT_URL="http://bugs.debian.org/"

uname:Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.65-1+deb7u1 x86_64
libc:glibc 2.13 NPTL 2.13 
rlimit: STACK 8192k, CORE 0k, NPROC 7919, NOFILE 4096, AS infinity
load average:0.32 0.08 0.03

/proc/meminfo:
MemTotal:        1027040 kB
MemFree:          224852 kB
Buffers:           13844 kB
Cached:           211228 kB
SwapCached:            0 kB
Active:           615288 kB
Inactive:         154744 kB
Active(anon):     545032 kB
Inactive(anon):      244 kB
Active(file):      70256 kB
Inactive(file):   154500 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:               836 kB
Writeback:             0 kB
AnonPages:        545016 kB
Mapped:            46884 kB
Shmem:               292 kB
Slab:              17012 kB
SReclaimable:       9844 kB
SUnreclaim:         7168 kB
KernelStack:        1464 kB
PageTables:         5752 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      513520 kB
Committed_AS:    2056244 kB
VmallocTotal:   34359738367 kB
VmallocUsed:        4972 kB
VmallocChunk:   34359733071 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       30708 kB
DirectMap2M:     1017856 kB


CPU:total 1 (1 cores per cpu, 1 threads per core) family 6 model 15 stepping 11, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, aes, erms, tsc

/proc/cpuinfo:
processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 15
model name  : Intel(R) Core(TM)2 Duo CPU     T7700  @ 2.40GHz
stepping    : 11
microcode   : 0x1
cpu MHz     : 3400.022
cache size  : 4096 KB
fpu     : yes
fpu_exception   : yes
cpuid level : 10
wp      : yes
flags       : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx lm constant_tsc up rep_good nopl pni vmx ssse3 cx16 sse4_1 sse4_2 popcnt aes hypervisor lahf_lm fsgsbase smep erms
bogomips    : 6800.04
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:



Memory: 4k page, physical 1027040k(224852k free), swap 0k(0k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (25.31-b07) for linux-amd64 JRE (1.8.0_31-b13), built on Dec 17 2014 20:40:15 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)

time: Fri Feb 27 13:17:50 2015
elapsed time: 0 seconds (0d 0h 0m 0s)

Symptoms

error=12, Cannot allocate memory (Linux)

com.atlassian.bitbucket.exception.ServerException: An error occurred while executing an external process: java.io.IOException: error=12, Cannot allocate memory

error=12, Not enough space (Solaris)

com.atlassian.bitbucket.exception.ServerException: An error occurred while executing an external process: error=12, Not enough space

Cause

If you received error=12, Cannot allocate memory or error=12, Not enough space, this means your system ran out of memory or swap space when Java tried to fork a process. 

The problem is inherent with the way Java allocates memory when executing processes. When Java executes a process, it must fork() then exec(). Forking creates a child process by duplicating the current process. By duplicating the current process, the new child process will request approximately the same amount of memory as its parent process, essentially doubling the memory required. However, this does not mean all the memory allocated will be used, as exec() is immediately called to execute the different code within the child process, freeing up this memory. As an example, when Bitbucket Server tries to locate git, the Bitbucket Server JVM process must be forked, approximately doubling the memory required by Bitbucket Server.

Note that this issue is made worse by hosting multiple webapps in the same Tomcat container as Bitbucket Server (which is not supported), because more memory is used by the Java process. 

Please see Forking JVM for a more detailed explanation.

Resolution

If you are hosting multiple products in the same Tomcat container as Bitbucket Server, move Bitbucket Server to its own Tomcat container.

For Linux, this can be resolved by enabling over-committing memory.

For Solaris, we recommend increasing your swap space. How much swap space do you need? This will also depend on the other applications running on the same machine. We recommend allocating at least swap equal to 4 x JVM_MAXIMUM_MEMORY, or 2 x PHYSICAL RAM, whichever is larger. The JVM_MAXIMUM_MEMORY is set in your  <Bitbucket Server installation directory>/bin/setenv.sh.  You can increase your swap space if you are still running out of memory.

JVM_SUPPORT_RECOMMENDED_ARGS=-XX:+HeapDumpOnOutOfMemoryError

I am not competent on server issues, any help is much appreciated. When try to start a python/django shell on a linux box, I am getting OSError: [Errno 12] Cannot allocate memory.

free -m seems to confirm I am out of memory:

             total       used       free     shared    buffers     cached
Mem:           590        560         29          0          3         37
-/+ buffers/cache:        518         71
Swap:            0          0          0

But I cannot see what is eating up the memory with top or ps aux:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
    1 root      20   0 24336  908    0 S  0.0  0.2   0:00.68 init               
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd           
    3 root      20   0     0    0    0 S  0.0  0.0   0:04.85 ksoftirqd/0        

How can I identify the leak? Thanks.

BTW, I am not sure if it is relevant, but the machine I am talking about is an AWS EC2 instance with Ubuntu 12 running.