Heap space OutOfMemoryError: available vs. requestedHow to use java.net.URLConnection to fire and handle HTTP requestsNegative Desired survivor size in GC LogsJava memory/gc issues (insufficient JRE memory, heap space, and full gc)Java big object garbage collectionDirect allocation to old generationFull GC is occurring frequentlyGarbage collection doesnt seem to work in javaGC pause duration from GC logsFrequent FULL GC when reduceByKeyGC Issue when running tomcat with TLSv1.2
Can a German sentence have two subjects?
Why CLRS example on residual networks does not follows its formula?
How to make payment on the internet without leaving a money trail?
Why are 150k or 200k jobs considered good when there are 300k+ births a month?
How to add power-LED to my small amplifier?
Example of a relative pronoun
Why can't I see bouncing of a switch on an oscilloscope?
Are tax years 2016 & 2017 back taxes deductible for tax year 2018?
Motorized valve interfering with button?
What would happen to a modern skyscraper if it rains micro blackholes?
Is the month field really deprecated?
Why is the design of haulage companies so “special”?
Is it possible to make sharp wind that can cut stuff from afar?
How can bays and straits be determined in a procedurally generated map?
Question about Goedel's incompleteness Proof
How to get the available space of $HOME as a variable in shell scripting?
Why is an old chain unsafe?
Book about a traveler who helps planets in need
How to type dʒ symbol (IPA) on Mac?
Email Account under attack (really) - anything I can do?
Compute hash value according to multiplication method
Can I make popcorn with any corn?
Accidentally leaked the solution to an assignment, what to do now? (I'm the prof)
Is it possible to do 50 km distance without any previous training?
Heap space OutOfMemoryError: available vs. requested
How to use java.net.URLConnection to fire and handle HTTP requestsNegative Desired survivor size in GC LogsJava memory/gc issues (insufficient JRE memory, heap space, and full gc)Java big object garbage collectionDirect allocation to old generationFull GC is occurring frequentlyGarbage collection doesnt seem to work in javaGC pause duration from GC logsFrequent FULL GC when reduceByKeyGC Issue when running tomcat with TLSv1.2
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
I have a spark application which ran into OutOfMemory Error.The GC log is as follows. Is there a way I can tell how much memory was requested and how much memory was available at the time of request? Thank you!
3429.458: [Full GC (Allocation Failure) [PSYoungGen: 3495918K->3495908K(6990848K)] [ParOldGen: 2096
9872K->20969870K(20971520K)] 24465790K->24465778K(27962368K), [Metaspace: 56727K->56723K(1101824K)]
, 0.4597426 secs] [Times: user=0.90 sys=0.01, real=0.46 secs]
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill %p"
# Executing /bin/sh -c "kill 16095"...
os::fork_and_exec failed: Cannot allocate memory (12)
3429.990: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495906K(6990848K)] [ParOldGen: 20969937K->20969933K(20971520K)] 24465873K->24465840K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 5.4942216 secs] [Times: user=34.79 sys=0.58, real=5.50 secs]
3435.485: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495907K(6990848K)] [ParOldGen: 20969933K->20969933K(20971520K)] 24465869K->24465840K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 2.3213689 secs] [Times: user=14.01 sys=0.22, real=2.32 secs]
3437.823: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495908K(6990848K)] [ParOldGen: 20970002K->20969998K(20971520K)] 24465938K->24465907K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 0.4090566 secs] [Times: user=0.83 sys=0.01, real=0.40 secs]
java apache-spark pyspark
asked Mar 9 at 3:37
sgusgu
497317
add a comment |
I have a spark application which ran into OutOfMemory Error.The GC log is as follows. Is there a way I can tell how much memory was requested and how much memory was available at the time of request? Thank you!
3429.458: [Full GC (Allocation Failure) [PSYoungGen: 3495918K->3495908K(6990848K)] [ParOldGen: 2096
9872K->20969870K(20971520K)] 24465790K->24465778K(27962368K), [Metaspace: 56727K->56723K(1101824K)]
, 0.4597426 secs] [Times: user=0.90 sys=0.01, real=0.46 secs]
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill %p"
# Executing /bin/sh -c "kill 16095"...
os::fork_and_exec failed: Cannot allocate memory (12)
3429.990: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495906K(6990848K)] [ParOldGen: 20969937K->20969933K(20971520K)] 24465873K->24465840K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 5.4942216 secs] [Times: user=34.79 sys=0.58, real=5.50 secs]
3435.485: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495907K(6990848K)] [ParOldGen: 20969933K->20969933K(20971520K)] 24465869K->24465840K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 2.3213689 secs] [Times: user=14.01 sys=0.22, real=2.32 secs]
3437.823: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495908K(6990848K)] [ParOldGen: 20970002K->20969998K(20971520K)] 24465938K->24465907K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 0.4090566 secs] [Times: user=0.83 sys=0.01, real=0.40 secs]
java apache-spark pyspark
asked Mar 9 at 3:37
sgusgu
497317
add a comment |
I have a spark application which ran into OutOfMemory Error.The GC log is as follows. Is there a way I can tell how much memory was requested and how much memory was available at the time of request? Thank you!
3429.458: [Full GC (Allocation Failure) [PSYoungGen: 3495918K->3495908K(6990848K)] [ParOldGen: 2096
9872K->20969870K(20971520K)] 24465790K->24465778K(27962368K), [Metaspace: 56727K->56723K(1101824K)]
, 0.4597426 secs] [Times: user=0.90 sys=0.01, real=0.46 secs]
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill %p"
# Executing /bin/sh -c "kill 16095"...
os::fork_and_exec failed: Cannot allocate memory (12)
3429.990: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495906K(6990848K)] [ParOldGen: 20969937K->20969933K(20971520K)] 24465873K->24465840K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 5.4942216 secs] [Times: user=34.79 sys=0.58, real=5.50 secs]
3435.485: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495907K(6990848K)] [ParOldGen: 20969933K->20969933K(20971520K)] 24465869K->24465840K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 2.3213689 secs] [Times: user=14.01 sys=0.22, real=2.32 secs]
3437.823: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495908K(6990848K)] [ParOldGen: 20970002K->20969998K(20971520K)] 24465938K->24465907K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 0.4090566 secs] [Times: user=0.83 sys=0.01, real=0.40 secs]
java apache-spark pyspark
asked Mar 9 at 3:37
sgusgu
497317
I have a spark application which ran into OutOfMemory Error.The GC log is as follows. Is there a way I can tell how much memory was requested and how much memory was available at the time of request? Thank you!
3429.458: [Full GC (Allocation Failure) [PSYoungGen: 3495918K->3495908K(6990848K)] [ParOldGen: 2096
9872K->20969870K(20971520K)] 24465790K->24465778K(27962368K), [Metaspace: 56727K->56723K(1101824K)]
, 0.4597426 secs] [Times: user=0.90 sys=0.01, real=0.46 secs]
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill %p"
# Executing /bin/sh -c "kill 16095"...
os::fork_and_exec failed: Cannot allocate memory (12)
3429.990: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495906K(6990848K)] [ParOldGen: 20969937K->20969933K(20971520K)] 24465873K->24465840K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 5.4942216 secs] [Times: user=34.79 sys=0.58, real=5.50 secs]
3435.485: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495907K(6990848K)] [ParOldGen: 20969933K->20969933K(20971520K)] 24465869K->24465840K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 2.3213689 secs] [Times: user=14.01 sys=0.22, real=2.32 secs]
3437.823: [Full GC (Ergonomics) [PSYoungGen: 3495936K->3495908K(6990848K)] [ParOldGen: 20970002K->20969998K(20971520K)] 24465938K->24465907K(27962368K), [Metaspace: 56723K->56723K(1101824K)], 0.4090566 secs] [Times: user=0.83 sys=0.01, real=0.40 secs]
java apache-spark pyspark
java apache-spark pyspark
asked Mar 9 at 3:37
sgusgu
497317
asked Mar 9 at 3:37
sgusgu
497317
asked Mar 9 at 3:37
sgusgu
497317
asked Mar 9 at 3:37
sgusgu
497317
asked Mar 9 at 3:37
sgusgu
497317
497317
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Is there a way I can tell how much memory was requested
Not from the information provided. Possibly not at all.
and how much memory was available at the time of request?
According to the GC log, this is when you ran out of memory:
3429.458: [Full GC (Allocation Failure)
[PSYoungGen: 3495918K->3495908K(6990848K)]
[ParOldGen: 20969872K->20969870K(20971520K)]
24465790K->24465778K(27962368K),
[Metaspace: 56727K->56723K(1101824K)], 0.4597426 secs]
[Times: user=0.90 sys=0.01, real=0.46 secs]
This 24465790K->24465778K(27962368K) line says that:
- space used before GC 24465790K
- space used after GC 24465778K
- heap space after GC 27962368K
So it looks like 3496590K is free out of 27962368K.
(The other triples can be read the same way ...)
It looks like the old generation is full. I suspect the GC failure is due to the GC being unable to tenure objects from eden space because of that. Also note that neither the young or old generation collectors managed to free much memory, and neither did the Full GC.
So the overall diagnosis is that your heap is full. For some reason the allocator could not put the new object into eden space. It might have been larger than the available free space in eden, or it might have been bigger than the large object threshold.
answered Mar 9 at 4:09
Stephen CStephen C
526k72586944
I am just curious. Why jvm cannot show the amount of request memory when throwing OOM exception? This is a very useful information.
– sgu
Mar 11 at 18:10
1) Check the OpenJDK source code to satisfy your curiosity. 2) If after your investigation you think it should be different, submit a feature request ... or a patch.
– Stephen C
Mar 11 at 22:32
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55073749%2fheap-space-outofmemoryerror-available-vs-requested%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Is there a way I can tell how much memory was requested
Not from the information provided. Possibly not at all.
and how much memory was available at the time of request?
According to the GC log, this is when you ran out of memory:
3429.458: [Full GC (Allocation Failure)
[PSYoungGen: 3495918K->3495908K(6990848K)]
[ParOldGen: 20969872K->20969870K(20971520K)]
24465790K->24465778K(27962368K),
[Metaspace: 56727K->56723K(1101824K)], 0.4597426 secs]
[Times: user=0.90 sys=0.01, real=0.46 secs]
This 24465790K->24465778K(27962368K) line says that:
- space used before GC 24465790K
- space used after GC 24465778K
- heap space after GC 27962368K
So it looks like 3496590K is free out of 27962368K.
(The other triples can be read the same way ...)
It looks like the old generation is full. I suspect the GC failure is due to the GC being unable to tenure objects from eden space because of that. Also note that neither the young or old generation collectors managed to free much memory, and neither did the Full GC.
So the overall diagnosis is that your heap is full. For some reason the allocator could not put the new object into eden space. It might have been larger than the available free space in eden, or it might have been bigger than the large object threshold.
answered Mar 9 at 4:09
Stephen CStephen C
526k72586944
I am just curious. Why jvm cannot show the amount of request memory when throwing OOM exception? This is a very useful information.
– sgu
Mar 11 at 18:10
1) Check the OpenJDK source code to satisfy your curiosity. 2) If after your investigation you think it should be different, submit a feature request ... or a patch.
– Stephen C
Mar 11 at 22:32
add a comment |
Is there a way I can tell how much memory was requested
Not from the information provided. Possibly not at all.
and how much memory was available at the time of request?
According to the GC log, this is when you ran out of memory:
3429.458: [Full GC (Allocation Failure)
[PSYoungGen: 3495918K->3495908K(6990848K)]
[ParOldGen: 20969872K->20969870K(20971520K)]
24465790K->24465778K(27962368K),
[Metaspace: 56727K->56723K(1101824K)], 0.4597426 secs]
[Times: user=0.90 sys=0.01, real=0.46 secs]
This 24465790K->24465778K(27962368K) line says that:
- space used before GC 24465790K
- space used after GC 24465778K
- heap space after GC 27962368K
So it looks like 3496590K is free out of 27962368K.
(The other triples can be read the same way ...)
It looks like the old generation is full. I suspect the GC failure is due to the GC being unable to tenure objects from eden space because of that. Also note that neither the young or old generation collectors managed to free much memory, and neither did the Full GC.
So the overall diagnosis is that your heap is full. For some reason the allocator could not put the new object into eden space. It might have been larger than the available free space in eden, or it might have been bigger than the large object threshold.
answered Mar 9 at 4:09
Stephen CStephen C
526k72586944
I am just curious. Why jvm cannot show the amount of request memory when throwing OOM exception? This is a very useful information.
– sgu
Mar 11 at 18:10
1) Check the OpenJDK source code to satisfy your curiosity. 2) If after your investigation you think it should be different, submit a feature request ... or a patch.
– Stephen C
Mar 11 at 22:32
add a comment |
Is there a way I can tell how much memory was requested
Not from the information provided. Possibly not at all.
and how much memory was available at the time of request?
According to the GC log, this is when you ran out of memory:
3429.458: [Full GC (Allocation Failure)
[PSYoungGen: 3495918K->3495908K(6990848K)]
[ParOldGen: 20969872K->20969870K(20971520K)]
24465790K->24465778K(27962368K),
[Metaspace: 56727K->56723K(1101824K)], 0.4597426 secs]
[Times: user=0.90 sys=0.01, real=0.46 secs]
This 24465790K->24465778K(27962368K) line says that:
- space used before GC 24465790K
- space used after GC 24465778K
- heap space after GC 27962368K
So it looks like 3496590K is free out of 27962368K.
(The other triples can be read the same way ...)
It looks like the old generation is full. I suspect the GC failure is due to the GC being unable to tenure objects from eden space because of that. Also note that neither the young or old generation collectors managed to free much memory, and neither did the Full GC.
So the overall diagnosis is that your heap is full. For some reason the allocator could not put the new object into eden space. It might have been larger than the available free space in eden, or it might have been bigger than the large object threshold.
answered Mar 9 at 4:09
Stephen CStephen C
526k72586944
Is there a way I can tell how much memory was requested
Not from the information provided. Possibly not at all.
and how much memory was available at the time of request?
According to the GC log, this is when you ran out of memory:
3429.458: [Full GC (Allocation Failure)
[PSYoungGen: 3495918K->3495908K(6990848K)]
[ParOldGen: 20969872K->20969870K(20971520K)]
24465790K->24465778K(27962368K),
[Metaspace: 56727K->56723K(1101824K)], 0.4597426 secs]
[Times: user=0.90 sys=0.01, real=0.46 secs]
This 24465790K->24465778K(27962368K) line says that:
- space used before GC 24465790K
- space used after GC 24465778K
- heap space after GC 27962368K
So it looks like 3496590K is free out of 27962368K.
(The other triples can be read the same way ...)
It looks like the old generation is full. I suspect the GC failure is due to the GC being unable to tenure objects from eden space because of that. Also note that neither the young or old generation collectors managed to free much memory, and neither did the Full GC.
So the overall diagnosis is that your heap is full. For some reason the allocator could not put the new object into eden space. It might have been larger than the available free space in eden, or it might have been bigger than the large object threshold.
answered Mar 9 at 4:09
Stephen CStephen C
526k72586944
answered Mar 9 at 4:09
Stephen CStephen C
526k72586944
answered Mar 9 at 4:09
Stephen CStephen C
526k72586944
answered Mar 9 at 4:09
Stephen CStephen C
526k72586944
526k72586944
I am just curious. Why jvm cannot show the amount of request memory when throwing OOM exception? This is a very useful information.
– sgu
Mar 11 at 18:10
1) Check the OpenJDK source code to satisfy your curiosity. 2) If after your investigation you think it should be different, submit a feature request ... or a patch.
– Stephen C
Mar 11 at 22:32
add a comment |
I am just curious. Why jvm cannot show the amount of request memory when throwing OOM exception? This is a very useful information.
– sgu
Mar 11 at 18:10
1) Check the OpenJDK source code to satisfy your curiosity. 2) If after your investigation you think it should be different, submit a feature request ... or a patch.
– Stephen C
Mar 11 at 22:32
I am just curious. Why jvm cannot show the amount of request memory when throwing OOM exception? This is a very useful information.
– sgu
Mar 11 at 18:10
I am just curious. Why jvm cannot show the amount of request memory when throwing OOM exception? This is a very useful information.
– sgu
Mar 11 at 18:10
1) Check the OpenJDK source code to satisfy your curiosity. 2) If after your investigation you think it should be different, submit a feature request ... or a patch.
– Stephen C
Mar 11 at 22:32
1) Check the OpenJDK source code to satisfy your curiosity. 2) If after your investigation you think it should be different, submit a feature request ... or a patch.
– Stephen C
Mar 11 at 22:32
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55073749%2fheap-space-outofmemoryerror-available-vs-requested%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
