Unix command to replace first column of a .csv fileUnix shell script find out which directory the script file resides?sed command find and replace in file and overwrite file doesn't work, it empties the fileMerging very large csv files with common columnWant to sort large csv file having date in the format ddMONYYYY using unix sort commandConverting a file of dates into unix time with awkHow to remove line `v u` from a file when line `u v` already exists using unix commandAdd a date column in an existing CSV using awkReplace date in a txt file having xmlAdding a new column to a CSV fileUsing awk on a CSV to find date pattern in first column

Biological Blimps: Propulsion

I found an audio circuit and I built it just fine, but I find it a bit too quiet. How do I amplify the output so that it is a bit louder?

Which Article Helped Get Rid of Technobabble in RPGs?

Why is the "ls" command showing permissions of files in a FAT32 partition?

Do we have to expect a queue for the shuttle from Watford Junction to Harry Potter Studio?

Shouldn’t conservatives embrace universal basic income?

How do you make your own symbol when Detexify fails?

Why do ¬, ∀ and ∃ have the same precedence?

The IT department bottlenecks progress, how should I handle this?

Why is so much work done on numerical verification of the Riemann Hypothesis?

awk assign to multiple variables at once

What does "Scientists rise up against statistical significance" mean? (Comment in Nature)

How to draw a matrix with arrows in limited space

What to do when eye contact makes your coworker uncomfortable?

Why can't the Brexit deadlock in the UK parliament be solved with a plurality vote?

Stack Interview Code methods made from class Node and Smart Pointers

How to make money from a browser who sees 5 seconds into the future of any web page?

Permission on Database

Has the laser at Magurele, Romania reached a tenth of the Sun's power?

Are Captain Marvel's powers affected by Thanos breaking the Tesseract and claiming the stone?

How does electrical safety system work on ISS?

Delete multiple columns using awk or sed

Non-trope happy ending?

Can I turn my anal-retentiveness into a career?



Unix command to replace first column of a .csv file


Unix shell script find out which directory the script file resides?sed command find and replace in file and overwrite file doesn't work, it empties the fileMerging very large csv files with common columnWant to sort large csv file having date in the format ddMONYYYY using unix sort commandConverting a file of dates into unix time with awkHow to remove line `v u` from a file when line `u v` already exists using unix commandAdd a date column in an existing CSV using awkReplace date in a txt file having xmlAdding a new column to a CSV fileUsing awk on a CSV to find date pattern in first column













-1















I want a unix command (that I will call in a ControlM job) that changes the value of the first column of my .csv file (not the header line), with the date of the previous day (expected format : YYYY-MM-DD).



I tried many commands but none of them do want I want :



tmp=$(mktemp) && awk -F| -v val=`date -d yesterday +%F` 'NR>1 gsub($1,val)' file.csv > "$tmp" && mv "$tmp" file.csv


or :



awk -F| -v val=`date -d yesterday +%F` 'gsub($1, val)1' file.csv


even tried gensub but not working.



Example of what I want :
Input :



VALUE_DATE;TRADE_DATE;DESCR1;DESCR2
2019-03-05;2017-11-15;BRIDGE;HELLO
2019-03-05;2018-03-17;WORK;DATA


Output I want (as today is 2019-03-07):



VALUE_DATE;TRADE_DATE;DESCR1;DESCR2
2019-03-06;2017-11-15;BRIDGE;HELLO
2019-03-06;2018-03-17;WORK;DATA


Can you help please and give me examples of commands that should work, I'm not finding a solution.



Thanks a lot










share|improve this question
























  • Welcome to SO, it is recommended to wrap your samples/codes in CODE TAGS, you could do it by using button while editing your post.Thanks to jxc who has done it for this one but please make a note for future posts.

    – RavinderSingh13
    Mar 8 at 0:23















-1















I want a unix command (that I will call in a ControlM job) that changes the value of the first column of my .csv file (not the header line), with the date of the previous day (expected format : YYYY-MM-DD).



I tried many commands but none of them do want I want :



tmp=$(mktemp) && awk -F| -v val=`date -d yesterday +%F` 'NR>1 gsub($1,val)' file.csv > "$tmp" && mv "$tmp" file.csv


or :



awk -F| -v val=`date -d yesterday +%F` 'gsub($1, val)1' file.csv


even tried gensub but not working.



Example of what I want :
Input :



VALUE_DATE;TRADE_DATE;DESCR1;DESCR2
2019-03-05;2017-11-15;BRIDGE;HELLO
2019-03-05;2018-03-17;WORK;DATA


Output I want (as today is 2019-03-07):



VALUE_DATE;TRADE_DATE;DESCR1;DESCR2
2019-03-06;2017-11-15;BRIDGE;HELLO
2019-03-06;2018-03-17;WORK;DATA


Can you help please and give me examples of commands that should work, I'm not finding a solution.



Thanks a lot










share|improve this question
























  • Welcome to SO, it is recommended to wrap your samples/codes in CODE TAGS, you could do it by using button while editing your post.Thanks to jxc who has done it for this one but please make a note for future posts.

    – RavinderSingh13
    Mar 8 at 0:23













-1












-1








-1








I want a unix command (that I will call in a ControlM job) that changes the value of the first column of my .csv file (not the header line), with the date of the previous day (expected format : YYYY-MM-DD).



I tried many commands but none of them do want I want :



tmp=$(mktemp) && awk -F| -v val=`date -d yesterday +%F` 'NR>1 gsub($1,val)' file.csv > "$tmp" && mv "$tmp" file.csv


or :



awk -F| -v val=`date -d yesterday +%F` 'gsub($1, val)1' file.csv


even tried gensub but not working.



Example of what I want :
Input :



VALUE_DATE;TRADE_DATE;DESCR1;DESCR2
2019-03-05;2017-11-15;BRIDGE;HELLO
2019-03-05;2018-03-17;WORK;DATA


Output I want (as today is 2019-03-07):



VALUE_DATE;TRADE_DATE;DESCR1;DESCR2
2019-03-06;2017-11-15;BRIDGE;HELLO
2019-03-06;2018-03-17;WORK;DATA


Can you help please and give me examples of commands that should work, I'm not finding a solution.



Thanks a lot










share|improve this question
















I want a unix command (that I will call in a ControlM job) that changes the value of the first column of my .csv file (not the header line), with the date of the previous day (expected format : YYYY-MM-DD).



I tried many commands but none of them do want I want :



tmp=$(mktemp) && awk -F| -v val=`date -d yesterday +%F` 'NR>1 gsub($1,val)' file.csv > "$tmp" && mv "$tmp" file.csv


or :



awk -F| -v val=`date -d yesterday +%F` 'gsub($1, val)1' file.csv


even tried gensub but not working.



Example of what I want :
Input :



VALUE_DATE;TRADE_DATE;DESCR1;DESCR2
2019-03-05;2017-11-15;BRIDGE;HELLO
2019-03-05;2018-03-17;WORK;DATA


Output I want (as today is 2019-03-07):



VALUE_DATE;TRADE_DATE;DESCR1;DESCR2
2019-03-06;2017-11-15;BRIDGE;HELLO
2019-03-06;2018-03-17;WORK;DATA


Can you help please and give me examples of commands that should work, I'm not finding a solution.



Thanks a lot







unix awk gsub






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Mar 8 at 0:08









jxc

953139




953139










asked Mar 7 at 23:51









Guillaume CGuillaume C

31




31












  • Welcome to SO, it is recommended to wrap your samples/codes in CODE TAGS, you could do it by using button while editing your post.Thanks to jxc who has done it for this one but please make a note for future posts.

    – RavinderSingh13
    Mar 8 at 0:23

















  • Welcome to SO, it is recommended to wrap your samples/codes in CODE TAGS, you could do it by using button while editing your post.Thanks to jxc who has done it for this one but please make a note for future posts.

    – RavinderSingh13
    Mar 8 at 0:23
















Welcome to SO, it is recommended to wrap your samples/codes in CODE TAGS, you could do it by using button while editing your post.Thanks to jxc who has done it for this one but please make a note for future posts.

– RavinderSingh13
Mar 8 at 0:23





Welcome to SO, it is recommended to wrap your samples/codes in CODE TAGS, you could do it by using button while editing your post.Thanks to jxc who has done it for this one but please make a note for future posts.

– RavinderSingh13
Mar 8 at 0:23












1 Answer
1






active

oldest

votes


















0














Could you please try following first?(not saving output into file.csv itself it will print output on terminal once happy then you could use answer
provided at last of this post)



awk -v val=$(date -d yesterday +%F) 'BEGINFS=OFS=";"FNR>1$1=val 1' file.csv


Problems identified in OP's code(and fixed in my suggestion):



1- Use of backtick is depreciated now to save shell variable's values, so instead use val=$(date....) for declaring awk's variable named val.



2- Use of -F, you have set your field separator as | which is pipe but when we see your provided sample Input_file carefully it is delimited with ;(semi colon) NOT | so that is also one of the reason why it is not reflecting in output.



3- Since use of gsub($1,val), replaces whole line to only with value of variable val
because
syntax of gsub is something like: gsub(your_regex/value_needs_to_be_replaced,"new_value"/variable_which_should_be_there_after_replacement,current_line/variable). Since you have defined wrong field separator so whole line being treated as $1 and thus when you print it by doing awk -F| -v val=$(date -d yesterday +%F) 'NR>1 gsub($1,val) 1' file.csv it will only print previous dates.



4- 4th and main issue is you have NOT printed anything, so even you did mistakes you will NOT see any output either on terminal or in output file.





If happy then you could run your own command to make changes into Input_file itself.(I am assuming that you are having propervaluein your tmp variable here)



tmp=$(mktemp) && awk -v val=$(date -d yesterday +%F) 'BEGINFS=OFS=";"FNR>1$1=val 1' file.csv > "$tmp" && mv "$tmp" file.csv





share|improve this answer




















  • 1





    This is brilliant. Thanks a lot @RavinderSingh13

    – Guillaume C
    Mar 8 at 20:37










Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55054660%2funix-command-to-replace-first-column-of-a-csv-file%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









0














Could you please try following first?(not saving output into file.csv itself it will print output on terminal once happy then you could use answer
provided at last of this post)



awk -v val=$(date -d yesterday +%F) 'BEGINFS=OFS=";"FNR>1$1=val 1' file.csv


Problems identified in OP's code(and fixed in my suggestion):



1- Use of backtick is depreciated now to save shell variable's values, so instead use val=$(date....) for declaring awk's variable named val.



2- Use of -F, you have set your field separator as | which is pipe but when we see your provided sample Input_file carefully it is delimited with ;(semi colon) NOT | so that is also one of the reason why it is not reflecting in output.



3- Since use of gsub($1,val), replaces whole line to only with value of variable val
because
syntax of gsub is something like: gsub(your_regex/value_needs_to_be_replaced,"new_value"/variable_which_should_be_there_after_replacement,current_line/variable). Since you have defined wrong field separator so whole line being treated as $1 and thus when you print it by doing awk -F| -v val=$(date -d yesterday +%F) 'NR>1 gsub($1,val) 1' file.csv it will only print previous dates.



4- 4th and main issue is you have NOT printed anything, so even you did mistakes you will NOT see any output either on terminal or in output file.





If happy then you could run your own command to make changes into Input_file itself.(I am assuming that you are having propervaluein your tmp variable here)



tmp=$(mktemp) && awk -v val=$(date -d yesterday +%F) 'BEGINFS=OFS=";"FNR>1$1=val 1' file.csv > "$tmp" && mv "$tmp" file.csv





share|improve this answer




















  • 1





    This is brilliant. Thanks a lot @RavinderSingh13

    – Guillaume C
    Mar 8 at 20:37















0














Could you please try following first?(not saving output into file.csv itself it will print output on terminal once happy then you could use answer
provided at last of this post)



awk -v val=$(date -d yesterday +%F) 'BEGINFS=OFS=";"FNR>1$1=val 1' file.csv


Problems identified in OP's code(and fixed in my suggestion):



1- Use of backtick is depreciated now to save shell variable's values, so instead use val=$(date....) for declaring awk's variable named val.



2- Use of -F, you have set your field separator as | which is pipe but when we see your provided sample Input_file carefully it is delimited with ;(semi colon) NOT | so that is also one of the reason why it is not reflecting in output.



3- Since use of gsub($1,val), replaces whole line to only with value of variable val
because
syntax of gsub is something like: gsub(your_regex/value_needs_to_be_replaced,"new_value"/variable_which_should_be_there_after_replacement,current_line/variable). Since you have defined wrong field separator so whole line being treated as $1 and thus when you print it by doing awk -F| -v val=$(date -d yesterday +%F) 'NR>1 gsub($1,val) 1' file.csv it will only print previous dates.



4- 4th and main issue is you have NOT printed anything, so even you did mistakes you will NOT see any output either on terminal or in output file.





If happy then you could run your own command to make changes into Input_file itself.(I am assuming that you are having propervaluein your tmp variable here)



tmp=$(mktemp) && awk -v val=$(date -d yesterday +%F) 'BEGINFS=OFS=";"FNR>1$1=val 1' file.csv > "$tmp" && mv "$tmp" file.csv





share|improve this answer




















  • 1





    This is brilliant. Thanks a lot @RavinderSingh13

    – Guillaume C
    Mar 8 at 20:37













0












0








0







Could you please try following first?(not saving output into file.csv itself it will print output on terminal once happy then you could use answer
provided at last of this post)



awk -v val=$(date -d yesterday +%F) 'BEGINFS=OFS=";"FNR>1$1=val 1' file.csv


Problems identified in OP's code(and fixed in my suggestion):



1- Use of backtick is depreciated now to save shell variable's values, so instead use val=$(date....) for declaring awk's variable named val.



2- Use of -F, you have set your field separator as | which is pipe but when we see your provided sample Input_file carefully it is delimited with ;(semi colon) NOT | so that is also one of the reason why it is not reflecting in output.



3- Since use of gsub($1,val), replaces whole line to only with value of variable val
because
syntax of gsub is something like: gsub(your_regex/value_needs_to_be_replaced,"new_value"/variable_which_should_be_there_after_replacement,current_line/variable). Since you have defined wrong field separator so whole line being treated as $1 and thus when you print it by doing awk -F| -v val=$(date -d yesterday +%F) 'NR>1 gsub($1,val) 1' file.csv it will only print previous dates.



4- 4th and main issue is you have NOT printed anything, so even you did mistakes you will NOT see any output either on terminal or in output file.





If happy then you could run your own command to make changes into Input_file itself.(I am assuming that you are having propervaluein your tmp variable here)



tmp=$(mktemp) && awk -v val=$(date -d yesterday +%F) 'BEGINFS=OFS=";"FNR>1$1=val 1' file.csv > "$tmp" && mv "$tmp" file.csv





share|improve this answer















Could you please try following first?(not saving output into file.csv itself it will print output on terminal once happy then you could use answer
provided at last of this post)



awk -v val=$(date -d yesterday +%F) 'BEGINFS=OFS=";"FNR>1$1=val 1' file.csv


Problems identified in OP's code(and fixed in my suggestion):



1- Use of backtick is depreciated now to save shell variable's values, so instead use val=$(date....) for declaring awk's variable named val.



2- Use of -F, you have set your field separator as | which is pipe but when we see your provided sample Input_file carefully it is delimited with ;(semi colon) NOT | so that is also one of the reason why it is not reflecting in output.



3- Since use of gsub($1,val), replaces whole line to only with value of variable val
because
syntax of gsub is something like: gsub(your_regex/value_needs_to_be_replaced,"new_value"/variable_which_should_be_there_after_replacement,current_line/variable). Since you have defined wrong field separator so whole line being treated as $1 and thus when you print it by doing awk -F| -v val=$(date -d yesterday +%F) 'NR>1 gsub($1,val) 1' file.csv it will only print previous dates.



4- 4th and main issue is you have NOT printed anything, so even you did mistakes you will NOT see any output either on terminal or in output file.





If happy then you could run your own command to make changes into Input_file itself.(I am assuming that you are having propervaluein your tmp variable here)



tmp=$(mktemp) && awk -v val=$(date -d yesterday +%F) 'BEGINFS=OFS=";"FNR>1$1=val 1' file.csv > "$tmp" && mv "$tmp" file.csv






share|improve this answer














share|improve this answer



share|improve this answer








edited Mar 8 at 0:37

























answered Mar 8 at 0:10









RavinderSingh13RavinderSingh13

30.2k41639




30.2k41639







  • 1





    This is brilliant. Thanks a lot @RavinderSingh13

    – Guillaume C
    Mar 8 at 20:37












  • 1





    This is brilliant. Thanks a lot @RavinderSingh13

    – Guillaume C
    Mar 8 at 20:37







1




1





This is brilliant. Thanks a lot @RavinderSingh13

– Guillaume C
Mar 8 at 20:37





This is brilliant. Thanks a lot @RavinderSingh13

– Guillaume C
Mar 8 at 20:37



















draft saved

draft discarded
















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55054660%2funix-command-to-replace-first-column-of-a-csv-file%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Thal And Out Agency railway station See also References External links Navigation menuOfficial Web Site of Pakistan RailwaysArchivedOfficial Web Site of Pakistan Railwayseeexpanding ite

Understanding generators in Python2019 Community Moderator ElectionGenerator function not working pythonFor loop not executing two timesGenerators - Printing generated valuesWhy can a python generator only be used once?What exactly do generators do?What does the “yield” keyword do?What does “list comprehension” mean? How does it work and how can I use it?sklearn Kfold acces single fold instead of for loopIs a generator the callable? Which is the generator?Apply Border To Range Of Cells Using OpenpyxlCalling an external command in PythonWhat are metaclasses in Python?What is the difference between @staticmethod and @classmethod?Finding the index of an item given a list containing it in PythonDifference between append vs. extend list methods in PythonHow can I safely create a nested directory in Python?Does Python have a ternary conditional operator?Understanding slice notationUnderstanding Python super() with __init__() methodsDoes Python have a string 'contains' substring method?

How can I change the color of pagination dots of UIPageControl?How to change UIPageControl dotsIs there a way to change page indicator dots colorCustomize dot with image of UIPageControl at index 0 of UIPageControlNo visible @interface for 'NSObject<PageControlDelegate>' declares the selector 'pageControlPageDidChange:'How to change the color of pagination dots in UIPageControl with a different color per pagepagecontrol indicator custom image instead of DefaultChanging the colour of UIPageControl dots in MonoTouchpagecontrol selectable page visibility color?Alternative way to load ViewControllers on a UIPageControlHow to set only layer.border-color for UIpage control dots in swiftHow can I develop for iPhone using a Windows development machine?How to change the name of an iOS app?UITableView - change section header coloruipagecontrol indicator(dot)issueCustom UIPageControl dots color not changingchange the interspace between UIPageControl dotsHow to change Status Bar text color in iOSHow can I change image tintColor in iOS and WatchKitUIPageControl dots with larger space in between each dotsHow to change the color of pagination dots in UIPageControl with a different color per page