Ggtcf

Is this Paypal Github SDK reference really a dangerous site?

Why is a very small peak with larger m/z not considered to be the molecular ion?

When Schnorr signatures are part of Bitcoin will it be possible validate each block with only one signature validation?

I reported the illegal activity of my boss to his boss. My boss found out. Now I am being punished. What should I do?

QQ Plot and Shapiro Wilk Test Disagree

Why restrict private health insurance?

Dynamic Linkage of LocatorPane and InputField

Finitely many repeated replacements

Does an unused member variable take up memory?

ER diagram relationship node size adjustment

Plausibility of Mushroom Buildings

Expressing logarithmic equations without logs

Doubts in understanding some concepts of potential energy

Can the alpha, lambda values of a glmnet object output determine whether ridge or Lasso?

What do *foreign films* mean for an American?

Virginia employer terminated employee and wants signing bonus returned

How to resolve: Reviewer #1 says remove section X vs. Reviewer #2 says expand section X

What sort of fish is this

Why do phishing e-mails use faked e-mail addresses instead of the real one?

I can't die. Who am I?

For which categories of spectra is there an explicit description of the fibrant objects via lifting properties?

Power Strip for Europe

Called into a meeting and told we are being made redundant (laid off) and "not to share outside". Can I tell my partner?

Which classes are needed to have access to every spell in the PHB?

is 'sed' thread safe

What should someone know about using Python scripts in the shell?Nexenta bash script uses /usr/sun/bin/sed instead of /usr/bin/sedLeft and right square brackets treated differently by sed/bashsystemd daemon & python getting the wrong timesed multiple statements within a single command not workingHow to use sed to substitute strings which has “” in it?Python process can't create a file in a directory, keeps getting `permission denied` IOErrorIs a light weight process attached to a kernel thread in Linux?Extract every 2 lines from a 40 lines file and create a new fileRHEL upload / pull scripts from people without access to the server itseld

3

If I have a shell/python script that uses sed to modify a file in place based on user inputs, and then two users run the same script at the same time or approx. same time, is 'sed' thread safe ? Or perhaps it is not an issue because the file_descripor that was opened by the first thread will be used to lock the file anyway ? thx

linux sed python

share|improve this question

edited Mar 6 at 21:52

Jeff Schaller

43.2k1159138

asked Mar 6 at 21:34

terreysterreys

2113

• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  
  Slightly more info wanted. How do you use sed from Python (and why, can't Python do things like that fairly effortlessly?).
  
  – Kusalananda
  Mar 6 at 21:38
  

  
  
  
  

add a comment | 

3

If I have a shell/python script that uses sed to modify a file in place based on user inputs, and then two users run the same script at the same time or approx. same time, is 'sed' thread safe ? Or perhaps it is not an issue because the file_descripor that was opened by the first thread will be used to lock the file anyway ? thx

linux sed python

share|improve this question

edited Mar 6 at 21:52

Jeff Schaller

43.2k1159138

asked Mar 6 at 21:34

terreysterreys

2113

• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  
  Slightly more info wanted. How do you use sed from Python (and why, can't Python do things like that fairly effortlessly?).
  
  – Kusalananda
  Mar 6 at 21:38
  

  
  
  
  

add a comment | 

If I have a shell/python script that uses sed to modify a file in place based on user inputs, and then two users run the same script at the same time or approx. same time, is 'sed' thread safe ? Or perhaps it is not an issue because the file_descripor that was opened by the first thread will be used to lock the file anyway ? thx

linux sed python

share|improve this question

edited Mar 6 at 21:52

Jeff Schaller

43.2k1159138

asked Mar 6 at 21:34

terreysterreys

2113

share|improve this question

edited Mar 6 at 21:52

Jeff Schaller

43.2k1159138

asked Mar 6 at 21:34

terreysterreys

2113

share|improve this question

edited Mar 6 at 21:52

Jeff Schaller

43.2k1159138

asked Mar 6 at 21:34

terreysterreys

2113

edited Mar 6 at 21:52

Jeff Schaller

43.2k1159138

edited Mar 6 at 21:52

Jeff Schaller

43.2k1159138

asked Mar 6 at 21:34

terreysterreys

2113

asked Mar 6 at 21:34

terreysterreys

2113

1 Answer
1

active

oldest

votes

7

I'm not going to nitpick on the awful terminology, but yes, GNU sed with its -i ("in-place") flag could be safely used by more than one process at the same time without any extra locking, because sed is not actually modifying the file in-place, but it's redirecting the output to a temporary file, and if everything goes well, it will rename(2) (move) the temporary file to the original file, and the rename(2) is guaranteed to be atomic:

$ strace sed -i s/o/e/g foo.txt
open("foo.txt", O_RDONLY) = 3
...
open("./sedDe80VL", O_RDWR|O_CREAT|O_EXCL, 0600) = 4
...
read(3, "foon", 4096) = 4
...
write(4, "feen", 4) = 4
read(3, "", 4096) = 0
...
close(3) = 0
close(4) = 0
rename("./sedDe80VL", "foo.txt") = 0

At any point, foo.txt will refer either to the complete original file or to the complete processed file, never to something in between the two.

share|improve this answer

edited Mar 6 at 21:54

answered Mar 6 at 21:47

mosvymosvy

8,0421531

• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  
  But what can happen is that two copies of sed start reading the file, make different changes, storing them to their respective temporary files, which are then renamed into place one after other, without regard for the fact that there was another sed working on the file at the same time. The changes made by one of the sed processes would be lost.
  
  – ilkkachu
  Mar 6 at 22:05
  
  

  
  
  
  


• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  
  @ilkkachu that will still be completely consistent -- the file will be either modified by both processes in turn (in any order) or by just one of them. At no point will the file contain garbage resulting from both processes modifying it at the same time.
  
  – mosvy
  Mar 6 at 22:15
  

  
  
  
  


• 
  
  
  

  
  8
  

  
  
  
  
  
  

  
  
  it won't be total garbage, but having changes lost can still be an issue. The question refers to the file "being locked", and locking would usually refer to the second sed process waiting until the first completed. That might just be awful terminology on their part; it's hard to say what they really care about, but that particular issue is still possible.
  
  – ilkkachu
  Mar 6 at 22:25
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f504810%2fis-sed-thread-safe%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

7

I'm not going to nitpick on the awful terminology, but yes, GNU sed with its -i ("in-place") flag could be safely used by more than one process at the same time without any extra locking, because sed is not actually modifying the file in-place, but it's redirecting the output to a temporary file, and if everything goes well, it will rename(2) (move) the temporary file to the original file, and the rename(2) is guaranteed to be atomic:

$ strace sed -i s/o/e/g foo.txt
open("foo.txt", O_RDONLY) = 3
...
open("./sedDe80VL", O_RDWR|O_CREAT|O_EXCL, 0600) = 4
...
read(3, "foon", 4096) = 4
...
write(4, "feen", 4) = 4
read(3, "", 4096) = 0
...
close(3) = 0
close(4) = 0
rename("./sedDe80VL", "foo.txt") = 0

At any point, foo.txt will refer either to the complete original file or to the complete processed file, never to something in between the two.

share|improve this answer

edited Mar 6 at 21:54

answered Mar 6 at 21:47

mosvymosvy

8,0421531

• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  
  But what can happen is that two copies of sed start reading the file, make different changes, storing them to their respective temporary files, which are then renamed into place one after other, without regard for the fact that there was another sed working on the file at the same time. The changes made by one of the sed processes would be lost.
  
  – ilkkachu
  Mar 6 at 22:05
  
  

  
  
  
  


• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  
  @ilkkachu that will still be completely consistent -- the file will be either modified by both processes in turn (in any order) or by just one of them. At no point will the file contain garbage resulting from both processes modifying it at the same time.
  
  – mosvy
  Mar 6 at 22:15
  

  
  
  
  


• 
  
  
  

  
  8
  

  
  
  
  
  
  

  
  
  it won't be total garbage, but having changes lost can still be an issue. The question refers to the file "being locked", and locking would usually refer to the second sed process waiting until the first completed. That might just be awful terminology on their part; it's hard to say what they really care about, but that particular issue is still possible.
  
  – ilkkachu
  Mar 6 at 22:25
  

  
  
  
  

add a comment | 

7

I'm not going to nitpick on the awful terminology, but yes, GNU sed with its -i ("in-place") flag could be safely used by more than one process at the same time without any extra locking, because sed is not actually modifying the file in-place, but it's redirecting the output to a temporary file, and if everything goes well, it will rename(2) (move) the temporary file to the original file, and the rename(2) is guaranteed to be atomic:

$ strace sed -i s/o/e/g foo.txt
open("foo.txt", O_RDONLY) = 3
...
open("./sedDe80VL", O_RDWR|O_CREAT|O_EXCL, 0600) = 4
...
read(3, "foon", 4096) = 4
...
write(4, "feen", 4) = 4
read(3, "", 4096) = 0
...
close(3) = 0
close(4) = 0
rename("./sedDe80VL", "foo.txt") = 0

At any point, foo.txt will refer either to the complete original file or to the complete processed file, never to something in between the two.

share|improve this answer

edited Mar 6 at 21:54

answered Mar 6 at 21:47

mosvymosvy

8,0421531

• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  
  But what can happen is that two copies of sed start reading the file, make different changes, storing them to their respective temporary files, which are then renamed into place one after other, without regard for the fact that there was another sed working on the file at the same time. The changes made by one of the sed processes would be lost.
  
  – ilkkachu
  Mar 6 at 22:05
  
  

  
  
  
  


• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  
  @ilkkachu that will still be completely consistent -- the file will be either modified by both processes in turn (in any order) or by just one of them. At no point will the file contain garbage resulting from both processes modifying it at the same time.
  
  – mosvy
  Mar 6 at 22:15
  

  
  
  
  


• 
  
  
  

  
  8
  

  
  
  
  
  
  

  
  
  it won't be total garbage, but having changes lost can still be an issue. The question refers to the file "being locked", and locking would usually refer to the second sed process waiting until the first completed. That might just be awful terminology on their part; it's hard to say what they really care about, but that particular issue is still possible.
  
  – ilkkachu
  Mar 6 at 22:25
  

  
  
  
  

add a comment | 

I'm not going to nitpick on the awful terminology, but yes, GNU sed with its -i ("in-place") flag could be safely used by more than one process at the same time without any extra locking, because sed is not actually modifying the file in-place, but it's redirecting the output to a temporary file, and if everything goes well, it will rename(2) (move) the temporary file to the original file, and the rename(2) is guaranteed to be atomic:

$ strace sed -i s/o/e/g foo.txt
open("foo.txt", O_RDONLY) = 3
...
open("./sedDe80VL", O_RDWR|O_CREAT|O_EXCL, 0600) = 4
...
read(3, "foon", 4096) = 4
...
write(4, "feen", 4) = 4
read(3, "", 4096) = 0
...
close(3) = 0
close(4) = 0
rename("./sedDe80VL", "foo.txt") = 0

At any point, foo.txt will refer either to the complete original file or to the complete processed file, never to something in between the two.

share|improve this answer

edited Mar 6 at 21:54

answered Mar 6 at 21:47

mosvymosvy

8,0421531

$ strace sed -i s/o/e/g foo.txt
open("foo.txt", O_RDONLY) = 3
...
open("./sedDe80VL", O_RDWR|O_CREAT|O_EXCL, 0600) = 4
...
read(3, "foon", 4096) = 4
...
write(4, "feen", 4) = 4
read(3, "", 4096) = 0
...
close(3) = 0
close(4) = 0
rename("./sedDe80VL", "foo.txt") = 0

share|improve this answer

edited Mar 6 at 21:54

answered Mar 6 at 21:47

mosvymosvy

8,0421531

answered Mar 6 at 21:47

mosvymosvy

8,0421531

answered Mar 6 at 21:47

mosvymosvy

8,0421531