File Output: Virtual Machine Crashing for files larger than ~5 MBstd::fstream buffering vs manual buffering (why 10x gain with manual buffering)?How to redirect output to a file and stdoutAddressing localhost from a VirtualBox virtual machineC++ program memory leak on SIGINTValgrind and Memory Leaksgstreamer memory leakuse signal handler to release resources?Boost memory leaks in threadsHow is Docker different from a virtual machine?NCurses memory allocation valgrind messageValgrind does not detect dangerous freeing memory
Mortal danger in mid-grade literature
Checking @@ROWCOUNT failing
Asserting that Atheism and Theism are both faith based positions
A seasonal riddle
Put the phone down / Put down the phone
"Oh no!" in Latin
Did I make a mistake by ccing email to boss to others?
1 John in Luther’s Bibel
Has the laser at Magurele, Romania reached a tenth of the Sun's power?
Turning a hard to access nut?
Does capillary rise violate hydrostatic paradox?
Why would five hundred and five same as one?
Trouble reading roman numeral notation with flats
If the Dominion rule using their Jem'Hadar troops, why is their life expectancy so low?
Why is participating in the European Parliamentary elections used as a threat?
Why can't I get pgrep output right to variable on bash script?
Should I warn a new PhD Student?
Why does the frost depth increase when the surface temperature warms up?
How would a solely written language work mechanically
Rendered textures different to 3D View
Calculate Pi using Monte Carlo
What can I do if I am asked to learn different programming languages very frequently?
Why does a 97 / 92 key piano exist by Bosendorfer?
"Marked down as someone wanting to sell shares." What does that mean?
File Output: Virtual Machine Crashing for files larger than ~5 MB
std::fstream buffering vs manual buffering (why 10x gain with manual buffering)?How to redirect output to a file and stdoutAddressing localhost from a VirtualBox virtual machineC++ program memory leak on SIGINTValgrind and Memory Leaksgstreamer memory leakuse signal handler to release resources?Boost memory leaks in threadsHow is Docker different from a virtual machine?NCurses memory allocation valgrind messageValgrind does not detect dangerous freeing memory
Who knew writing a text file would be so difficult?
I've attempted to write a text file on a Fedora 29 Virtual Machine using a number of different approaches, however when the file size grows large enough (~5 MB), the VM crashes, and on restart Fedora gives the following error message:
GNOME Shell
Application Crash
Sampling of crash details:
Likely crash reason: Jump to an invalid address
Exploitable rating (0-9 scale): 6
reason gnome-shell killed by SIGSEGV
…
General details:
-Compiling using gcc
-Using Oracle VirtualBox
-Host Machine is Windows 10
-On crash, the Virtual Box immediately restarts with the abrt-applet process running
-The host machine has no error messages
Here's my most recent approach, inspired by this great question:
#include <fstream>
#include <string>
int main()
std::string data(5 * 1024 * 1024, 'b'); // The huge-ish data string
std::ofstream file; // Manually change the size of the internal buffer
const unsigned int bufferSize = 4096;
// I have played around with various powers of 2, 2024
// has been the most stable, but not stable enough to
// write past 10 MB
char buffer[bufferSize];
file.rdbuf()->pubsetbuf(buffer, bufferSize);
file.open("datafile.txt", std::ios::trunc
I've also attempted flushing the internal buffer line by line, once every 10, 50, 100 lines etc. My first and most naïve attempt was throwing the whole string into the ofstream at once with the insertion operator and hoping the ofstream's internal buffering would chunk up the string appropriately.
The text files that are written prior to the VM crash are typically mostly complete, with some portions of the screen turning black while browsing. However the underlying text can be seen by highlighting, oddly enough...
(FWIW I have used the insertion operator, which from what I understand performs some formatting prior to the write. I probably don't need to use write(…) since I'm not writing binary data.)
Edit: Valgrind output:
I managed to run the program, which happens to be a qt program, under valgrind and pipe the valgrind output to a log file. It is of course wildly verbose, but it might be helpful:
> ==14585== HEAP SUMMARY:
> ==14585== in use at exit: 40,627,110 bytes in 445,534 blocks
> ==14585== total heap usage: 1,411,986 allocs, 966,452 frees, 422,404,746 bytes allocated
> ==14585==
> ==14585== LEAK SUMMARY:
> ==14585== definitely lost: 26,288 bytes in 381 blocks
> ==14585== indirectly lost: 11,674 bytes in 126 blocks
> ==14585== possibly lost: 224,995 bytes in 1,918 blocks
> ==14585== still reachable: 40,293,921 bytes in 442,533 blocks
> ==14585== of which reachable via heuristic:
> ==14585== length64 : 230,064 bytes in 238 blocks
> ==14585== newarray : 479,176 bytes in 736 blocks
> ==14585== multipleinheritance: 697,832 bytes in 1,161 blocks
> ==14585== suppressed: 0 bytes in 0 blocks
> ==14585== Rerun with --leak-check=full to see details of leaked memory
> ==14585==
> ==14585== For counts of detected and suppressed errors, rerun with: -v
> ==14585== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
> ==14590==
> ==14590== HEAP SUMMARY:
> ==14590== in use at exit: 40,633,631 bytes in 445,628 blocks
> ==14590== total heap usage: 1,412,264 allocs, 966,636 frees, 422,424,764 bytes allocated
> ==14590==
> ==14590== LEAK SUMMARY:
> ==14590== definitely lost: 26,288 bytes in 381 blocks
> ==14590== indirectly lost: 11,675 bytes in 126 blocks
> ==14590== possibly lost: 224,995 bytes in 1,918 blocks
> ==14590== still reachable: 40,300,441 bytes in 442,627 blocks
> ==14590== of which reachable via heuristic:
> ==14590== length64 : 230,064 bytes in 238 blocks
> ==14590== newarray : 479,176 bytes in 736 blocks
> ==14590== multipleinheritance: 697,832 bytes in 1,161 blocks
> ==14590== suppressed: 0 bytes in 0 blocks
> ==14590== Rerun with --leak-check=full to see details of leaked memory
> ==14590==
> ==14590== For counts of detected and suppressed errors, rerun with: -v
> ==14590== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
> ==14563== Conditional jump or move depends on uninitialised value(s)
> ==14563== at 0x2F8BCBA3: ???
> ==14563== by 0x2D488327: ???
(Lots more of these last two lines but at different memory locations)
> ==14563== More than 1000 different errors detected. I'm not reporting any more.
> ==14563== Final error counts will be inaccurate. Go fix your program!
> ==14563== Rerun with --error-limit=no to disable this cutoff. Note
> ==14563== that errors may occur in your program without prior warning from
> ==14563== Valgrind, because errors are no longer being displayed.
> ==14563==
> ==14563==
> ==14563== Process terminating with default action of signal 1 (SIGHUP)
> ==14563== at 0x61C2421: poll (in /usr/lib64/libc-2.28.so)
> ==14563== by 0x8ADB3A5: ??? (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x8ADB4CF: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x5AA3592: QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
> (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5A51E0A: QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (in
> /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x58B9E85: QThread::exec() (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x6558F58: ??? (in /usr/lib64/libQt5Qml.so.5.11.3)
> ==14563== by 0x58C32FA: ??? (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5D7D58D: start_thread (in /usr/lib64/libpthread-2.28.so)
> ==14563== by 0x61CD6A2: clone (in /usr/lib64/libc-2.28.so)
> ==14563==
> ==14563== Process terminating with default action of signal 1 (SIGHUP)
> ==14563== at 0x61C2421: poll (in /usr/lib64/libc-2.28.so)
> ==14563== by 0x8ADB3A5: ??? (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x8ADB4CF: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x5AA3592: QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
> (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5A51E0A: QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (in
> /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x58B9E85: QThread::exec() (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x6558F58: ??? (in /usr/lib64/libQt5Qml.so.5.11.3)
> ==14563== by 0x58C32FA: ??? (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5D7D58D: start_thread (in /usr/lib64/libpthread-2.28.so)
==14563== by 0x61CD6A2: clone (in /usr/lib64/libc-2.28.so)
c++ file-io virtual-machine fedora-29
asked Mar 7 at 20:25
Andres SalasAndres Salas
927
|
show 8 more comments
Who knew writing a text file would be so difficult?
I've attempted to write a text file on a Fedora 29 Virtual Machine using a number of different approaches, however when the file size grows large enough (~5 MB), the VM crashes, and on restart Fedora gives the following error message:
GNOME Shell
Application Crash
Sampling of crash details:
Likely crash reason: Jump to an invalid address
Exploitable rating (0-9 scale): 6
reason gnome-shell killed by SIGSEGV
…
General details:
-Compiling using gcc
-Using Oracle VirtualBox
-Host Machine is Windows 10
-On crash, the Virtual Box immediately restarts with the abrt-applet process running
-The host machine has no error messages
Here's my most recent approach, inspired by this great question:
#include <fstream>
#include <string>
int main()
std::string data(5 * 1024 * 1024, 'b'); // The huge-ish data string
std::ofstream file; // Manually change the size of the internal buffer
const unsigned int bufferSize = 4096;
// I have played around with various powers of 2, 2024
// has been the most stable, but not stable enough to
// write past 10 MB
char buffer[bufferSize];
file.rdbuf()->pubsetbuf(buffer, bufferSize);
file.open("datafile.txt", std::ios::trunc
I've also attempted flushing the internal buffer line by line, once every 10, 50, 100 lines etc. My first and most naïve attempt was throwing the whole string into the ofstream at once with the insertion operator and hoping the ofstream's internal buffering would chunk up the string appropriately.
The text files that are written prior to the VM crash are typically mostly complete, with some portions of the screen turning black while browsing. However the underlying text can be seen by highlighting, oddly enough...
(FWIW I have used the insertion operator, which from what I understand performs some formatting prior to the write. I probably don't need to use write(…) since I'm not writing binary data.)
Edit: Valgrind output:
I managed to run the program, which happens to be a qt program, under valgrind and pipe the valgrind output to a log file. It is of course wildly verbose, but it might be helpful:
> ==14585== HEAP SUMMARY:
> ==14585== in use at exit: 40,627,110 bytes in 445,534 blocks
> ==14585== total heap usage: 1,411,986 allocs, 966,452 frees, 422,404,746 bytes allocated
> ==14585==
> ==14585== LEAK SUMMARY:
> ==14585== definitely lost: 26,288 bytes in 381 blocks
> ==14585== indirectly lost: 11,674 bytes in 126 blocks
> ==14585== possibly lost: 224,995 bytes in 1,918 blocks
> ==14585== still reachable: 40,293,921 bytes in 442,533 blocks
> ==14585== of which reachable via heuristic:
> ==14585== length64 : 230,064 bytes in 238 blocks
> ==14585== newarray : 479,176 bytes in 736 blocks
> ==14585== multipleinheritance: 697,832 bytes in 1,161 blocks
> ==14585== suppressed: 0 bytes in 0 blocks
> ==14585== Rerun with --leak-check=full to see details of leaked memory
> ==14585==
> ==14585== For counts of detected and suppressed errors, rerun with: -v
> ==14585== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
> ==14590==
> ==14590== HEAP SUMMARY:
> ==14590== in use at exit: 40,633,631 bytes in 445,628 blocks
> ==14590== total heap usage: 1,412,264 allocs, 966,636 frees, 422,424,764 bytes allocated
> ==14590==
> ==14590== LEAK SUMMARY:
> ==14590== definitely lost: 26,288 bytes in 381 blocks
> ==14590== indirectly lost: 11,675 bytes in 126 blocks
> ==14590== possibly lost: 224,995 bytes in 1,918 blocks
> ==14590== still reachable: 40,300,441 bytes in 442,627 blocks
> ==14590== of which reachable via heuristic:
> ==14590== length64 : 230,064 bytes in 238 blocks
> ==14590== newarray : 479,176 bytes in 736 blocks
> ==14590== multipleinheritance: 697,832 bytes in 1,161 blocks
> ==14590== suppressed: 0 bytes in 0 blocks
> ==14590== Rerun with --leak-check=full to see details of leaked memory
> ==14590==
> ==14590== For counts of detected and suppressed errors, rerun with: -v
> ==14590== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
> ==14563== Conditional jump or move depends on uninitialised value(s)
> ==14563== at 0x2F8BCBA3: ???
> ==14563== by 0x2D488327: ???
(Lots more of these last two lines but at different memory locations)
> ==14563== More than 1000 different errors detected. I'm not reporting any more.
> ==14563== Final error counts will be inaccurate. Go fix your program!
> ==14563== Rerun with --error-limit=no to disable this cutoff. Note
> ==14563== that errors may occur in your program without prior warning from
> ==14563== Valgrind, because errors are no longer being displayed.
> ==14563==
> ==14563==
> ==14563== Process terminating with default action of signal 1 (SIGHUP)
> ==14563== at 0x61C2421: poll (in /usr/lib64/libc-2.28.so)
> ==14563== by 0x8ADB3A5: ??? (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x8ADB4CF: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x5AA3592: QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
> (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5A51E0A: QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (in
> /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x58B9E85: QThread::exec() (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x6558F58: ??? (in /usr/lib64/libQt5Qml.so.5.11.3)
> ==14563== by 0x58C32FA: ??? (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5D7D58D: start_thread (in /usr/lib64/libpthread-2.28.so)
> ==14563== by 0x61CD6A2: clone (in /usr/lib64/libc-2.28.so)
> ==14563==
> ==14563== Process terminating with default action of signal 1 (SIGHUP)
> ==14563== at 0x61C2421: poll (in /usr/lib64/libc-2.28.so)
> ==14563== by 0x8ADB3A5: ??? (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x8ADB4CF: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x5AA3592: QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
> (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5A51E0A: QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (in
> /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x58B9E85: QThread::exec() (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x6558F58: ??? (in /usr/lib64/libQt5Qml.so.5.11.3)
> ==14563== by 0x58C32FA: ??? (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5D7D58D: start_thread (in /usr/lib64/libpthread-2.28.so)
==14563== by 0x61CD6A2: clone (in /usr/lib64/libc-2.28.so)
c++ file-io virtual-machine fedora-29
asked Mar 7 at 20:25
Andres SalasAndres Salas
927
I'd love to, however I'm not sure what parts of the question don't satisfy which of those three criteria. Do you mean that I ought to write code that loads ~5MB of characters into the data string? Or perhaps that I should make the loop that explicitly chunks the buffer with characters from the data stream? Or would that hurt the minimal part...FWIW the linked question similarly sidestepped these more wildly explicit steps @NathanOliver
– Andres Salas
Mar 7 at 20:34
2
Valgrind should at least let you know what's wrong with your program up to the write operation, which might (and probably does) play a role here. For future reference : a MCV would look something like this, and it does produce a file of exactly 5MiB in size on my machine.
– Daniel Kamil Kozar
Mar 7 at 20:52
2
does your VM and your host machine have enough disk space?
– Alan Birtles
Mar 7 at 21:00
3
Q: What kind of "VM" is this? VMWare? VirtualBox? Other? Q: The entire VM "crashes", correct? Is there any kind of error message? On the host OS? In the Fedora VM's syslog, after you bring it back up? The guest OS is Fedora 29? What's the host OS/ What's the C++ compiler?
– paulsm4
Mar 7 at 21:02
1
Throw it all away and use a proper copy loop that actually copies, and handles the last partial buffer correctly. What you've written here makes no sense at all.
– user207421
Mar 8 at 2:36
|
show 8 more comments
Who knew writing a text file would be so difficult?
I've attempted to write a text file on a Fedora 29 Virtual Machine using a number of different approaches, however when the file size grows large enough (~5 MB), the VM crashes, and on restart Fedora gives the following error message:
GNOME Shell
Application Crash
Sampling of crash details:
Likely crash reason: Jump to an invalid address
Exploitable rating (0-9 scale): 6
reason gnome-shell killed by SIGSEGV
…
General details:
-Compiling using gcc
-Using Oracle VirtualBox
-Host Machine is Windows 10
-On crash, the Virtual Box immediately restarts with the abrt-applet process running
-The host machine has no error messages
Here's my most recent approach, inspired by this great question:
#include <fstream>
#include <string>
int main()
std::string data(5 * 1024 * 1024, 'b'); // The huge-ish data string
std::ofstream file; // Manually change the size of the internal buffer
const unsigned int bufferSize = 4096;
// I have played around with various powers of 2, 2024
// has been the most stable, but not stable enough to
// write past 10 MB
char buffer[bufferSize];
file.rdbuf()->pubsetbuf(buffer, bufferSize);
file.open("datafile.txt", std::ios::trunc
I've also attempted flushing the internal buffer line by line, once every 10, 50, 100 lines etc. My first and most naïve attempt was throwing the whole string into the ofstream at once with the insertion operator and hoping the ofstream's internal buffering would chunk up the string appropriately.
The text files that are written prior to the VM crash are typically mostly complete, with some portions of the screen turning black while browsing. However the underlying text can be seen by highlighting, oddly enough...
(FWIW I have used the insertion operator, which from what I understand performs some formatting prior to the write. I probably don't need to use write(…) since I'm not writing binary data.)
Edit: Valgrind output:
I managed to run the program, which happens to be a qt program, under valgrind and pipe the valgrind output to a log file. It is of course wildly verbose, but it might be helpful:
> ==14585== HEAP SUMMARY:
> ==14585== in use at exit: 40,627,110 bytes in 445,534 blocks
> ==14585== total heap usage: 1,411,986 allocs, 966,452 frees, 422,404,746 bytes allocated
> ==14585==
> ==14585== LEAK SUMMARY:
> ==14585== definitely lost: 26,288 bytes in 381 blocks
> ==14585== indirectly lost: 11,674 bytes in 126 blocks
> ==14585== possibly lost: 224,995 bytes in 1,918 blocks
> ==14585== still reachable: 40,293,921 bytes in 442,533 blocks
> ==14585== of which reachable via heuristic:
> ==14585== length64 : 230,064 bytes in 238 blocks
> ==14585== newarray : 479,176 bytes in 736 blocks
> ==14585== multipleinheritance: 697,832 bytes in 1,161 blocks
> ==14585== suppressed: 0 bytes in 0 blocks
> ==14585== Rerun with --leak-check=full to see details of leaked memory
> ==14585==
> ==14585== For counts of detected and suppressed errors, rerun with: -v
> ==14585== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
> ==14590==
> ==14590== HEAP SUMMARY:
> ==14590== in use at exit: 40,633,631 bytes in 445,628 blocks
> ==14590== total heap usage: 1,412,264 allocs, 966,636 frees, 422,424,764 bytes allocated
> ==14590==
> ==14590== LEAK SUMMARY:
> ==14590== definitely lost: 26,288 bytes in 381 blocks
> ==14590== indirectly lost: 11,675 bytes in 126 blocks
> ==14590== possibly lost: 224,995 bytes in 1,918 blocks
> ==14590== still reachable: 40,300,441 bytes in 442,627 blocks
> ==14590== of which reachable via heuristic:
> ==14590== length64 : 230,064 bytes in 238 blocks
> ==14590== newarray : 479,176 bytes in 736 blocks
> ==14590== multipleinheritance: 697,832 bytes in 1,161 blocks
> ==14590== suppressed: 0 bytes in 0 blocks
> ==14590== Rerun with --leak-check=full to see details of leaked memory
> ==14590==
> ==14590== For counts of detected and suppressed errors, rerun with: -v
> ==14590== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
> ==14563== Conditional jump or move depends on uninitialised value(s)
> ==14563== at 0x2F8BCBA3: ???
> ==14563== by 0x2D488327: ???
(Lots more of these last two lines but at different memory locations)
> ==14563== More than 1000 different errors detected. I'm not reporting any more.
> ==14563== Final error counts will be inaccurate. Go fix your program!
> ==14563== Rerun with --error-limit=no to disable this cutoff. Note
> ==14563== that errors may occur in your program without prior warning from
> ==14563== Valgrind, because errors are no longer being displayed.
> ==14563==
> ==14563==
> ==14563== Process terminating with default action of signal 1 (SIGHUP)
> ==14563== at 0x61C2421: poll (in /usr/lib64/libc-2.28.so)
> ==14563== by 0x8ADB3A5: ??? (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x8ADB4CF: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x5AA3592: QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
> (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5A51E0A: QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (in
> /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x58B9E85: QThread::exec() (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x6558F58: ??? (in /usr/lib64/libQt5Qml.so.5.11.3)
> ==14563== by 0x58C32FA: ??? (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5D7D58D: start_thread (in /usr/lib64/libpthread-2.28.so)
> ==14563== by 0x61CD6A2: clone (in /usr/lib64/libc-2.28.so)
> ==14563==
> ==14563== Process terminating with default action of signal 1 (SIGHUP)
> ==14563== at 0x61C2421: poll (in /usr/lib64/libc-2.28.so)
> ==14563== by 0x8ADB3A5: ??? (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x8ADB4CF: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x5AA3592: QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
> (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5A51E0A: QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (in
> /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x58B9E85: QThread::exec() (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x6558F58: ??? (in /usr/lib64/libQt5Qml.so.5.11.3)
> ==14563== by 0x58C32FA: ??? (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5D7D58D: start_thread (in /usr/lib64/libpthread-2.28.so)
==14563== by 0x61CD6A2: clone (in /usr/lib64/libc-2.28.so)
c++ file-io virtual-machine fedora-29
asked Mar 7 at 20:25
Andres SalasAndres Salas
927
Who knew writing a text file would be so difficult?
I've attempted to write a text file on a Fedora 29 Virtual Machine using a number of different approaches, however when the file size grows large enough (~5 MB), the VM crashes, and on restart Fedora gives the following error message:
GNOME Shell
Application Crash
Sampling of crash details:
Likely crash reason: Jump to an invalid address
Exploitable rating (0-9 scale): 6
reason gnome-shell killed by SIGSEGV
…
General details:
-Compiling using gcc
-Using Oracle VirtualBox
-Host Machine is Windows 10
-On crash, the Virtual Box immediately restarts with the abrt-applet process running
-The host machine has no error messages
Here's my most recent approach, inspired by this great question:
#include <fstream>
#include <string>
int main()
std::string data(5 * 1024 * 1024, 'b'); // The huge-ish data string
std::ofstream file; // Manually change the size of the internal buffer
const unsigned int bufferSize = 4096;
// I have played around with various powers of 2, 2024
// has been the most stable, but not stable enough to
// write past 10 MB
char buffer[bufferSize];
file.rdbuf()->pubsetbuf(buffer, bufferSize);
file.open("datafile.txt", std::ios::trunc
I've also attempted flushing the internal buffer line by line, once every 10, 50, 100 lines etc. My first and most naïve attempt was throwing the whole string into the ofstream at once with the insertion operator and hoping the ofstream's internal buffering would chunk up the string appropriately.
The text files that are written prior to the VM crash are typically mostly complete, with some portions of the screen turning black while browsing. However the underlying text can be seen by highlighting, oddly enough...
(FWIW I have used the insertion operator, which from what I understand performs some formatting prior to the write. I probably don't need to use write(…) since I'm not writing binary data.)
Edit: Valgrind output:
I managed to run the program, which happens to be a qt program, under valgrind and pipe the valgrind output to a log file. It is of course wildly verbose, but it might be helpful:
> ==14585== HEAP SUMMARY:
> ==14585== in use at exit: 40,627,110 bytes in 445,534 blocks
> ==14585== total heap usage: 1,411,986 allocs, 966,452 frees, 422,404,746 bytes allocated
> ==14585==
> ==14585== LEAK SUMMARY:
> ==14585== definitely lost: 26,288 bytes in 381 blocks
> ==14585== indirectly lost: 11,674 bytes in 126 blocks
> ==14585== possibly lost: 224,995 bytes in 1,918 blocks
> ==14585== still reachable: 40,293,921 bytes in 442,533 blocks
> ==14585== of which reachable via heuristic:
> ==14585== length64 : 230,064 bytes in 238 blocks
> ==14585== newarray : 479,176 bytes in 736 blocks
> ==14585== multipleinheritance: 697,832 bytes in 1,161 blocks
> ==14585== suppressed: 0 bytes in 0 blocks
> ==14585== Rerun with --leak-check=full to see details of leaked memory
> ==14585==
> ==14585== For counts of detected and suppressed errors, rerun with: -v
> ==14585== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
> ==14590==
> ==14590== HEAP SUMMARY:
> ==14590== in use at exit: 40,633,631 bytes in 445,628 blocks
> ==14590== total heap usage: 1,412,264 allocs, 966,636 frees, 422,424,764 bytes allocated
> ==14590==
> ==14590== LEAK SUMMARY:
> ==14590== definitely lost: 26,288 bytes in 381 blocks
> ==14590== indirectly lost: 11,675 bytes in 126 blocks
> ==14590== possibly lost: 224,995 bytes in 1,918 blocks
> ==14590== still reachable: 40,300,441 bytes in 442,627 blocks
> ==14590== of which reachable via heuristic:
> ==14590== length64 : 230,064 bytes in 238 blocks
> ==14590== newarray : 479,176 bytes in 736 blocks
> ==14590== multipleinheritance: 697,832 bytes in 1,161 blocks
> ==14590== suppressed: 0 bytes in 0 blocks
> ==14590== Rerun with --leak-check=full to see details of leaked memory
> ==14590==
> ==14590== For counts of detected and suppressed errors, rerun with: -v
> ==14590== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
> ==14563== Conditional jump or move depends on uninitialised value(s)
> ==14563== at 0x2F8BCBA3: ???
> ==14563== by 0x2D488327: ???
(Lots more of these last two lines but at different memory locations)
> ==14563== More than 1000 different errors detected. I'm not reporting any more.
> ==14563== Final error counts will be inaccurate. Go fix your program!
> ==14563== Rerun with --error-limit=no to disable this cutoff. Note
> ==14563== that errors may occur in your program without prior warning from
> ==14563== Valgrind, because errors are no longer being displayed.
> ==14563==
> ==14563==
> ==14563== Process terminating with default action of signal 1 (SIGHUP)
> ==14563== at 0x61C2421: poll (in /usr/lib64/libc-2.28.so)
> ==14563== by 0x8ADB3A5: ??? (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x8ADB4CF: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x5AA3592: QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
> (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5A51E0A: QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (in
> /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x58B9E85: QThread::exec() (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x6558F58: ??? (in /usr/lib64/libQt5Qml.so.5.11.3)
> ==14563== by 0x58C32FA: ??? (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5D7D58D: start_thread (in /usr/lib64/libpthread-2.28.so)
> ==14563== by 0x61CD6A2: clone (in /usr/lib64/libc-2.28.so)
> ==14563==
> ==14563== Process terminating with default action of signal 1 (SIGHUP)
> ==14563== at 0x61C2421: poll (in /usr/lib64/libc-2.28.so)
> ==14563== by 0x8ADB3A5: ??? (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x8ADB4CF: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.5800.3)
> ==14563== by 0x5AA3592: QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
> (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5A51E0A: QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (in
> /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x58B9E85: QThread::exec() (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x6558F58: ??? (in /usr/lib64/libQt5Qml.so.5.11.3)
> ==14563== by 0x58C32FA: ??? (in /usr/lib64/libQt5Core.so.5.11.3)
> ==14563== by 0x5D7D58D: start_thread (in /usr/lib64/libpthread-2.28.so)
==14563== by 0x61CD6A2: clone (in /usr/lib64/libc-2.28.so)
c++ file-io virtual-machine fedora-29
c++ file-io virtual-machine fedora-29
asked Mar 7 at 20:25
Andres SalasAndres Salas
927
asked Mar 7 at 20:25
Andres SalasAndres Salas
927
edited Mar 8 at 17:03
Andres Salas
asked Mar 7 at 20:25
Andres SalasAndres Salas
927
asked Mar 7 at 20:25
Andres SalasAndres Salas
927
asked Mar 7 at 20:25
Andres SalasAndres Salas
927
927
I'd love to, however I'm not sure what parts of the question don't satisfy which of those three criteria. Do you mean that I ought to write code that loads ~5MB of characters into the data string? Or perhaps that I should make the loop that explicitly chunks the buffer with characters from the data stream? Or would that hurt the minimal part...FWIW the linked question similarly sidestepped these more wildly explicit steps @NathanOliver
– Andres Salas
Mar 7 at 20:34
2
Valgrind should at least let you know what's wrong with your program up to the write operation, which might (and probably does) play a role here. For future reference : a MCV would look something like this, and it does produce a file of exactly 5MiB in size on my machine.
– Daniel Kamil Kozar
Mar 7 at 20:52
2
does your VM and your host machine have enough disk space?
– Alan Birtles
Mar 7 at 21:00
3
Q: What kind of "VM" is this? VMWare? VirtualBox? Other? Q: The entire VM "crashes", correct? Is there any kind of error message? On the host OS? In the Fedora VM's syslog, after you bring it back up? The guest OS is Fedora 29? What's the host OS/ What's the C++ compiler?
– paulsm4
Mar 7 at 21:02
1
Throw it all away and use a proper copy loop that actually copies, and handles the last partial buffer correctly. What you've written here makes no sense at all.
– user207421
Mar 8 at 2:36
|
show 8 more comments
I'd love to, however I'm not sure what parts of the question don't satisfy which of those three criteria. Do you mean that I ought to write code that loads ~5MB of characters into the data string? Or perhaps that I should make the loop that explicitly chunks the buffer with characters from the data stream? Or would that hurt the minimal part...FWIW the linked question similarly sidestepped these more wildly explicit steps @NathanOliver
– Andres Salas
Mar 7 at 20:34
2
Valgrind should at least let you know what's wrong with your program up to the write operation, which might (and probably does) play a role here. For future reference : a MCV would look something like this, and it does produce a file of exactly 5MiB in size on my machine.
– Daniel Kamil Kozar
Mar 7 at 20:52
2
does your VM and your host machine have enough disk space?
– Alan Birtles
Mar 7 at 21:00
3
Q: What kind of "VM" is this? VMWare? VirtualBox? Other? Q: The entire VM "crashes", correct? Is there any kind of error message? On the host OS? In the Fedora VM's syslog, after you bring it back up? The guest OS is Fedora 29? What's the host OS/ What's the C++ compiler?
– paulsm4
Mar 7 at 21:02
1
Throw it all away and use a proper copy loop that actually copies, and handles the last partial buffer correctly. What you've written here makes no sense at all.
– user207421
Mar 8 at 2:36
I'd love to, however I'm not sure what parts of the question don't satisfy which of those three criteria. Do you mean that I ought to write code that loads ~5MB of characters into the data string? Or perhaps that I should make the loop that explicitly chunks the buffer with characters from the data stream? Or would that hurt the minimal part...FWIW the linked question similarly sidestepped these more wildly explicit steps @NathanOliver
– Andres Salas
Mar 7 at 20:34
I'd love to, however I'm not sure what parts of the question don't satisfy which of those three criteria. Do you mean that I ought to write code that loads ~5MB of characters into the data string? Or perhaps that I should make the loop that explicitly chunks the buffer with characters from the data stream? Or would that hurt the minimal part...FWIW the linked question similarly sidestepped these more wildly explicit steps @NathanOliver
– Andres Salas
Mar 7 at 20:34
2
2
Valgrind should at least let you know what's wrong with your program up to the write operation, which might (and probably does) play a role here. For future reference : a MCV would look something like this, and it does produce a file of exactly 5MiB in size on my machine.
– Daniel Kamil Kozar
Mar 7 at 20:52
Valgrind should at least let you know what's wrong with your program up to the write operation, which might (and probably does) play a role here. For future reference : a MCV would look something like this, and it does produce a file of exactly 5MiB in size on my machine.
– Daniel Kamil Kozar
Mar 7 at 20:52
2
2
does your VM and your host machine have enough disk space?
– Alan Birtles
Mar 7 at 21:00
does your VM and your host machine have enough disk space?
– Alan Birtles
Mar 7 at 21:00
3
3
Q: What kind of "VM" is this? VMWare? VirtualBox? Other? Q: The entire VM "crashes", correct? Is there any kind of error message? On the host OS? In the Fedora VM's syslog, after you bring it back up? The guest OS is Fedora 29? What's the host OS/ What's the C++ compiler?
– paulsm4
Mar 7 at 21:02
Q: What kind of "VM" is this? VMWare? VirtualBox? Other? Q: The entire VM "crashes", correct? Is there any kind of error message? On the host OS? In the Fedora VM's syslog, after you bring it back up? The guest OS is Fedora 29? What's the host OS/ What's the C++ compiler?
– paulsm4
Mar 7 at 21:02
1
1
Throw it all away and use a proper copy loop that actually copies, and handles the last partial buffer correctly. What you've written here makes no sense at all.
– user207421
Mar 8 at 2:36
Throw it all away and use a proper copy loop that actually copies, and handles the last partial buffer correctly. What you've written here makes no sense at all.
– user207421
Mar 8 at 2:36
|
show 8 more comments
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55052253%2ffile-output-virtual-machine-crashing-for-files-larger-than-5-mb%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55052253%2ffile-output-virtual-machine-crashing-for-files-larger-than-5-mb%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I'd love to, however I'm not sure what parts of the question don't satisfy which of those three criteria. Do you mean that I ought to write code that loads ~5MB of characters into the data string? Or perhaps that I should make the loop that explicitly chunks the buffer with characters from the data stream? Or would that hurt the minimal part...FWIW the linked question similarly sidestepped these more wildly explicit steps @NathanOliver
– Andres Salas
Mar 7 at 20:34
2
Valgrind should at least let you know what's wrong with your program up to the write operation, which might (and probably does) play a role here. For future reference : a MCV would look something like this, and it does produce a file of exactly 5MiB in size on my machine.
– Daniel Kamil Kozar
Mar 7 at 20:52
2
does your VM and your host machine have enough disk space?
– Alan Birtles
Mar 7 at 21:00
3
Q: What kind of "VM" is this? VMWare? VirtualBox? Other? Q: The entire VM "crashes", correct? Is there any kind of error message? On the host OS? In the Fedora VM's syslog, after you bring it back up? The guest OS is Fedora 29? What's the host OS/ What's the C++ compiler?
– paulsm4
Mar 7 at 21:02
1
Throw it all away and use a proper copy loop that actually copies, and handles the last partial buffer correctly. What you've written here makes no sense at all.
– user207421
Mar 8 at 2:36