summing positive and negative numbers in MapReduce Iterable IntWritable2019 Community Moderator ElectionHow do I efficiently iterate over each entry in a Java Map?Iterate through a HashMapUsing Hadoop for the First Time, MapReduce Job does not run Reduce PhaseExample for running mapreduce on hdfs files and storing reducer results in hbase tableHadoop: maintain memory cache between iterated map jobsHadoop mapreduce with input size ~ 2Mb slowDeciding key value pair for deduplication using hadoop mapreduceMapreduce Error: Outputting Map Data instead of Reducehadoop job with single mapper and two different reducersHow do you handle different value types in reducer
Animating wave motion in water
Why is there so much iron?
Can other pieces capture a threatening piece and prevent a checkmate?
Why does Surtur say that Thor is Asgard's doom?
Does convergence of polynomials imply that of its coefficients?
Does the Shadow Magic sorcerer's Eyes of the Dark feature work on all Darkness spells or just his/her own?
How to test the sharpness of a knife?
Why do I have a large white artefact on the rendered image?
How can an organ that provides biological immortality be unable to regenerate?
How old is Nick Fury?
The English Debate
What kind of footwear is suitable for walking in micro gravity environment?
Fair way to split coins
Help with identifying unique aircraft over NE Pennsylvania
How to read string as hex number in bash?
Justification failure in beamer enumerate list
Why is this tree refusing to shed its dead leaves?
Was World War I a war of liberals against authoritarians?
How can a new country break out from a developed country without war?
Interior of Set Notation
is this saw blade faulty?
Why is "la Gestapo" feminine?
Do people actually use the word "kaputt" in conversation?
Single word to change groups
summing positive and negative numbers in MapReduce Iterable IntWritable
2019 Community Moderator ElectionHow do I efficiently iterate over each entry in a Java Map?Iterate through a HashMapUsing Hadoop for the First Time, MapReduce Job does not run Reduce PhaseExample for running mapreduce on hdfs files and storing reducer results in hbase tableHadoop: maintain memory cache between iterated map jobsHadoop mapreduce with input size ~ 2Mb slowDeciding key value pair for deduplication using hadoop mapreduceMapreduce Error: Outputting Map Data instead of Reducehadoop job with single mapper and two different reducersHow do you handle different value types in reducer
I'm using MapReduce in Java Hadoop to calculate the difference between the outgoing edges and incoming edges.
For instance node 2 has 2 outgoing edges and 3 incoming edges, then the difference is -1.
src, tgt
1, 2
2, 1
2, 3
3, 2
4, 2
4, 3
So the output should be: node 2, difference: -1.
In the Mapper function I assigned the IntWritable +1 if 2 is the source and -1 if 2 is the target.
so the mapper is producing this following Iterable IntWritable and sending it to the Reducer.
(node 2, values( -1, +1, +1, -1, -1)) and the reducer is returning (0, -1) instead of just -1.
Output below:
1 0
2 0
2 -1
3 0
3 -1
4 2
I tried it several times with different scenarios and that's what I believe is happening. I don't how to get the reducer to simply add all numbers and return one value??
java hadoop mapreduce
asked Mar 7 at 18:54
rabi26rabi26
102
add a comment |
I'm using MapReduce in Java Hadoop to calculate the difference between the outgoing edges and incoming edges.
For instance node 2 has 2 outgoing edges and 3 incoming edges, then the difference is -1.
src, tgt
1, 2
2, 1
2, 3
3, 2
4, 2
4, 3
So the output should be: node 2, difference: -1.
In the Mapper function I assigned the IntWritable +1 if 2 is the source and -1 if 2 is the target.
so the mapper is producing this following Iterable IntWritable and sending it to the Reducer.
(node 2, values( -1, +1, +1, -1, -1)) and the reducer is returning (0, -1) instead of just -1.
Output below:
1 0
2 0
2 -1
3 0
3 -1
4 2
I tried it several times with different scenarios and that's what I believe is happening. I don't how to get the reducer to simply add all numbers and return one value??
java hadoop mapreduce
asked Mar 7 at 18:54
rabi26rabi26
102
can you post your code?
– vefthym
Mar 12 at 3:01
It was solved. the problem was unseen characters in the text document. So -1 in one row was not the same as -1 in the other row.
– rabi26
Mar 14 at 3:54
add a comment |
I'm using MapReduce in Java Hadoop to calculate the difference between the outgoing edges and incoming edges.
For instance node 2 has 2 outgoing edges and 3 incoming edges, then the difference is -1.
src, tgt
1, 2
2, 1
2, 3
3, 2
4, 2
4, 3
So the output should be: node 2, difference: -1.
In the Mapper function I assigned the IntWritable +1 if 2 is the source and -1 if 2 is the target.
so the mapper is producing this following Iterable IntWritable and sending it to the Reducer.
(node 2, values( -1, +1, +1, -1, -1)) and the reducer is returning (0, -1) instead of just -1.
Output below:
1 0
2 0
2 -1
3 0
3 -1
4 2
I tried it several times with different scenarios and that's what I believe is happening. I don't how to get the reducer to simply add all numbers and return one value??
java hadoop mapreduce
asked Mar 7 at 18:54
rabi26rabi26
102
I'm using MapReduce in Java Hadoop to calculate the difference between the outgoing edges and incoming edges.
For instance node 2 has 2 outgoing edges and 3 incoming edges, then the difference is -1.
src, tgt
1, 2
2, 1
2, 3
3, 2
4, 2
4, 3
So the output should be: node 2, difference: -1.
In the Mapper function I assigned the IntWritable +1 if 2 is the source and -1 if 2 is the target.
so the mapper is producing this following Iterable IntWritable and sending it to the Reducer.
(node 2, values( -1, +1, +1, -1, -1)) and the reducer is returning (0, -1) instead of just -1.
Output below:
1 0
2 0
2 -1
3 0
3 -1
4 2
I tried it several times with different scenarios and that's what I believe is happening. I don't how to get the reducer to simply add all numbers and return one value??
java hadoop mapreduce
java hadoop mapreduce
asked Mar 7 at 18:54
rabi26rabi26
102
asked Mar 7 at 18:54
rabi26rabi26
102
asked Mar 7 at 18:54
rabi26rabi26
102
asked Mar 7 at 18:54
rabi26rabi26
102
asked Mar 7 at 18:54
rabi26rabi26
102
102
can you post your code?
– vefthym
Mar 12 at 3:01
It was solved. the problem was unseen characters in the text document. So -1 in one row was not the same as -1 in the other row.
– rabi26
Mar 14 at 3:54
add a comment |
can you post your code?
– vefthym
Mar 12 at 3:01
It was solved. the problem was unseen characters in the text document. So -1 in one row was not the same as -1 in the other row.
– rabi26
Mar 14 at 3:54
can you post your code?
– vefthym
Mar 12 at 3:01
can you post your code?
– vefthym
Mar 12 at 3:01
It was solved. the problem was unseen characters in the text document. So -1 in one row was not the same as -1 in the other row.
– rabi26
Mar 14 at 3:54
It was solved. the problem was unseen characters in the text document. So -1 in one row was not the same as -1 in the other row.
– rabi26
Mar 14 at 3:54
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55050923%2fsumming-positive-and-negative-numbers-in-mapreduce-iterable-intwritable%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55050923%2fsumming-positive-and-negative-numbers-in-mapreduce-iterable-intwritable%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
can you post your code?
– vefthym
Mar 12 at 3:01
It was solved. the problem was unseen characters in the text document. So -1 in one row was not the same as -1 in the other row.
– rabi26
Mar 14 at 3:54