How to iterate over N files concurrently to count occurrence of unique word The Next CEO of Stack OverflowHow to read/write from/to file using Go?Iterating over all the keys of a mapHow to move a file in PythonHow to check if a file exists in Go?Is there a way to iterate over a slice in reverse in Go?Perl script to split a file and process then concatenate based on size and a stringHow to get the directory of the currently running file?Go String after variable declarationIs there a way to iterate over a range of integers in Golang?Break out of input.Scan()
Is it okay to majorly distort historical facts while writing a fiction story?
Easy to read palindrome checker
Which one is the true statement?
A Man With a Stainless Steel Endoskeleton (like The Terminator) Fighting Cloaked Aliens Only He Can See
How to delete every two lines after 3rd lines in a file contains very large number of lines?
How to check if all elements of 1 list are in the *same quantity* and in any order, in the list2?
Chain wire methods together in Lightning Web Components
What did we know about the Kessel run before the prequels?
How to install OpenCV on Raspbian Stretch?
Why does standard notation not preserve intervals (visually)
Find non-case sensitive string in a mixed list of elements?
How to invert MapIndexed on a ragged structure? How to construct a tree from rules?
Why is information "lost" when it got into a black hole?
Legal workarounds for testamentary trust perceived as unfair
Does soap repel water?
Bartok - Syncopation (1): Meaning of notes in between Grand Staff
Yu-Gi-Oh cards in Python 3
What was the first Unix version to run on a microcomputer?
Is there a way to save my career from absolute disaster?
Axiom Schema vs Axiom
How to edit “Name” property in GCI output?
Where do students learn to solve polynomial equations these days?
How to get from Geneva Airport to Metabief, Doubs, France by public transport?
Is it ever safe to open a suspicious HTML file (e.g. email attachment)?
How to iterate over N files concurrently to count occurrence of unique word
The Next CEO of Stack OverflowHow to read/write from/to file using Go?Iterating over all the keys of a mapHow to move a file in PythonHow to check if a file exists in Go?Is there a way to iterate over a slice in reverse in Go?Perl script to split a file and process then concatenate based on size and a stringHow to get the directory of the currently running file?Go String after variable declarationIs there a way to iterate over a range of integers in Golang?Break out of input.Scan()
This is my code to count the occurrence of all the unique word in a file:
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main()
file, err := os.Open("file1.txt")
if err != nil
log.Fatal(err)
words := make(map[string]int)
/*asking scanner to split into words*/
scanner := bufio.NewScanner(file)
scanner.Split(bufio.ScanWords)
count := 0
//scan the inpurt
for scanner.Scan()
//get input token - in our case a word and update it's frequence
words[scanner.Text()]++
count++
if err := scanner.Err(); err != nil
fmt.Fprintln(os.Stderr, "reading input:", err)
for k, v := range words
fmt.Printf("%s:%dn", k, v)
I have to iterate this map over N files concurrently in order to calculate the occurrence of all the unique words.
go file-handling
edited Mar 8 at 16:32
Flimzy
40.2k1367100
asked Mar 8 at 15:33
SachinSachin
13
add a comment |
This is my code to count the occurrence of all the unique word in a file:
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main()
file, err := os.Open("file1.txt")
if err != nil
log.Fatal(err)
words := make(map[string]int)
/*asking scanner to split into words*/
scanner := bufio.NewScanner(file)
scanner.Split(bufio.ScanWords)
count := 0
//scan the inpurt
for scanner.Scan()
//get input token - in our case a word and update it's frequence
words[scanner.Text()]++
count++
if err := scanner.Err(); err != nil
fmt.Fprintln(os.Stderr, "reading input:", err)
for k, v := range words
fmt.Printf("%s:%dn", k, v)
I have to iterate this map over N files concurrently in order to calculate the occurrence of all the unique words.
go file-handling
edited Mar 8 at 16:32
Flimzy
40.2k1367100
asked Mar 8 at 15:33
SachinSachin
13
4
Then the Tour of Go is a great start to learn and practice.
– Volker
Mar 8 at 16:03
add a comment |
This is my code to count the occurrence of all the unique word in a file:
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main()
file, err := os.Open("file1.txt")
if err != nil
log.Fatal(err)
words := make(map[string]int)
/*asking scanner to split into words*/
scanner := bufio.NewScanner(file)
scanner.Split(bufio.ScanWords)
count := 0
//scan the inpurt
for scanner.Scan()
//get input token - in our case a word and update it's frequence
words[scanner.Text()]++
count++
if err := scanner.Err(); err != nil
fmt.Fprintln(os.Stderr, "reading input:", err)
for k, v := range words
fmt.Printf("%s:%dn", k, v)
I have to iterate this map over N files concurrently in order to calculate the occurrence of all the unique words.
go file-handling
edited Mar 8 at 16:32
Flimzy
40.2k1367100
asked Mar 8 at 15:33
SachinSachin
13
This is my code to count the occurrence of all the unique word in a file:
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main()
file, err := os.Open("file1.txt")
if err != nil
log.Fatal(err)
words := make(map[string]int)
/*asking scanner to split into words*/
scanner := bufio.NewScanner(file)
scanner.Split(bufio.ScanWords)
count := 0
//scan the inpurt
for scanner.Scan()
//get input token - in our case a word and update it's frequence
words[scanner.Text()]++
count++
if err := scanner.Err(); err != nil
fmt.Fprintln(os.Stderr, "reading input:", err)
for k, v := range words
fmt.Printf("%s:%dn", k, v)
I have to iterate this map over N files concurrently in order to calculate the occurrence of all the unique words.
go file-handling
go file-handling
edited Mar 8 at 16:32
Flimzy
40.2k1367100
asked Mar 8 at 15:33
SachinSachin
13
edited Mar 8 at 16:32
Flimzy
40.2k1367100
asked Mar 8 at 15:33
SachinSachin
13
edited Mar 8 at 16:32
Flimzy
40.2k1367100
edited Mar 8 at 16:32
Flimzy
40.2k1367100
edited Mar 8 at 16:32
Flimzy
40.2k1367100
40.2k1367100
asked Mar 8 at 15:33
SachinSachin
13
asked Mar 8 at 15:33
SachinSachin
13
asked Mar 8 at 15:33
SachinSachin
13
13
4
Then the Tour of Go is a great start to learn and practice.
– Volker
Mar 8 at 16:03
add a comment |
4
Then the Tour of Go is a great start to learn and practice.
– Volker
Mar 8 at 16:03
4
4
Then the Tour of Go is a great start to learn and practice.
– Volker
Mar 8 at 16:03
Then the Tour of Go is a great start to learn and practice.
– Volker
Mar 8 at 16:03
add a comment |
1 Answer
1
active
oldest
votes
You could use an errgroup.Group to make it concurrent.
Keep in mind to handle concurrent writes to the map properly if you make it accessible for all goroutines.
answered Mar 8 at 19:37
morimori
996
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55066386%2fhow-to-iterate-over-n-files-concurrently-to-count-occurrence-of-unique-word%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You could use an errgroup.Group to make it concurrent.
Keep in mind to handle concurrent writes to the map properly if you make it accessible for all goroutines.
answered Mar 8 at 19:37
morimori
996
add a comment |
You could use an errgroup.Group to make it concurrent.
Keep in mind to handle concurrent writes to the map properly if you make it accessible for all goroutines.
answered Mar 8 at 19:37
morimori
996
add a comment |
You could use an errgroup.Group to make it concurrent.
Keep in mind to handle concurrent writes to the map properly if you make it accessible for all goroutines.
answered Mar 8 at 19:37
morimori
996
You could use an errgroup.Group to make it concurrent.
Keep in mind to handle concurrent writes to the map properly if you make it accessible for all goroutines.
answered Mar 8 at 19:37
morimori
996
answered Mar 8 at 19:37
morimori
996
answered Mar 8 at 19:37
morimori
996
answered Mar 8 at 19:37
morimori
996
996
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55066386%2fhow-to-iterate-over-n-files-concurrently-to-count-occurrence-of-unique-word%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
4
Then the Tour of Go is a great start to learn and practice.
– Volker
Mar 8 at 16:03