Which we use different Tokenizers for input and target data in seq2seq LSTM model?What does size of the GRU or LSTM cell in the TensorFlow seq2seq tutorial represent?Seq2seq multiple input features (Passing multiple word/word tokens as input)Implementing Luong and Manning's hybrid modelkeras - seq2seq model predicting same output for all test inputsSeq2seq lstm tensorflow implMultilayer Seq2Seq model with LSTM in KerasPreprocessing for seq2seq modelDimensionality of the `Y` input for LSTM in KerasTraining with tensorflow seq2seq modelAm I applying embedding layer in seq2seq correctly in inference model?
Does the fruit of Mantra Japa automatically go to Indra if Japa Samarpana Mantra is not chanted?
What do the dots in this tr command do: tr .............A-Z A-ZA-Z <<< "JVPQBOV" (with 13 dots)
Why does Kotter return in Welcome Back Kotter?
What is the difference between 元 (圓), 幣, and 金?
Creating a document with mixed languages
Why doesn't Newton's third law mean a person bounces back to where they started when they hit the ground?
Which models of the Boeing 737 are still in production?
Is this a crack on the carbon frame?
Can a German sentence have two subjects?
Roll the carpet
Why was the small council so happy for Tyrion to become the Master of Coin?
Mathematical cryptic clues
Do I have a twin with permutated remainders?
Service Entrance Breakers Rain Shield
is it possible to make sharp wind that can cut stuff from afar?
Have astronauts in space suits ever taken selfies? If so, how?
How old can references or sources in a thesis be?
In Japanese, what’s the difference between “Tonari ni” (となりに) and “Tsugi” (つぎ)? When would you use one over the other?
Arthur Somervell: 1000 Exercises - Meaning of this notation
How does strength of boric acid solution increase in presence of salicylic acid?
Is it possible to do 50 km distance without any previous training?
Shell script not opening as desktop application
How do I create uniquely male characters?
How is the claim "I am in New York only if I am in America" the same as "If I am in New York, then I am in America?
Which we use different Tokenizers for input and target data in seq2seq LSTM model?
What does size of the GRU or LSTM cell in the TensorFlow seq2seq tutorial represent?Seq2seq multiple input features (Passing multiple word/word tokens as input)Implementing Luong and Manning's hybrid modelkeras - seq2seq model predicting same output for all test inputsSeq2seq lstm tensorflow implMultilayer Seq2Seq model with LSTM in KerasPreprocessing for seq2seq modelDimensionality of the `Y` input for LSTM in KerasTraining with tensorflow seq2seq modelAm I applying embedding layer in seq2seq correctly in inference model?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
I am working on a project which generates a headline from a given news article. Currently, I am preparing two tokenizers for the news and headlines. Hence, two vocabularies are created. Both, the news and the headlines are in English.
Should I use a single tokenizer to create a single vocabulary of words and use it to tokenize the news and the headlines> Would that have any effect of the seq2seq model? Or the existing method of different tokenizers is okay?
tensorflow neural-network tokenize seq2seq
asked Mar 9 at 3:19
Shubham PanchalShubham Panchal
604212
add a comment |
I am working on a project which generates a headline from a given news article. Currently, I am preparing two tokenizers for the news and headlines. Hence, two vocabularies are created. Both, the news and the headlines are in English.
Should I use a single tokenizer to create a single vocabulary of words and use it to tokenize the news and the headlines> Would that have any effect of the seq2seq model? Or the existing method of different tokenizers is okay?
tensorflow neural-network tokenize seq2seq
asked Mar 9 at 3:19
Shubham PanchalShubham Panchal
604212
It's not entirely clear what you're asking. Do you intend to train a model using one vocabulary (mapping from words to IDs), and then use that model with data with an entirely different vocabulary? That won't work well.
– Peteris
Mar 9 at 3:47
That's my question whether to use one vocab or not.
– Shubham Panchal
Mar 9 at 4:57
add a comment |
I am working on a project which generates a headline from a given news article. Currently, I am preparing two tokenizers for the news and headlines. Hence, two vocabularies are created. Both, the news and the headlines are in English.
Should I use a single tokenizer to create a single vocabulary of words and use it to tokenize the news and the headlines> Would that have any effect of the seq2seq model? Or the existing method of different tokenizers is okay?
tensorflow neural-network tokenize seq2seq
asked Mar 9 at 3:19
Shubham PanchalShubham Panchal
604212
I am working on a project which generates a headline from a given news article. Currently, I am preparing two tokenizers for the news and headlines. Hence, two vocabularies are created. Both, the news and the headlines are in English.
Should I use a single tokenizer to create a single vocabulary of words and use it to tokenize the news and the headlines> Would that have any effect of the seq2seq model? Or the existing method of different tokenizers is okay?
tensorflow neural-network tokenize seq2seq
tensorflow neural-network tokenize seq2seq
asked Mar 9 at 3:19
Shubham PanchalShubham Panchal
604212
asked Mar 9 at 3:19
Shubham PanchalShubham Panchal
604212
asked Mar 9 at 3:19
Shubham PanchalShubham Panchal
604212
asked Mar 9 at 3:19
Shubham PanchalShubham Panchal
604212
asked Mar 9 at 3:19
Shubham PanchalShubham Panchal
604212
604212
It's not entirely clear what you're asking. Do you intend to train a model using one vocabulary (mapping from words to IDs), and then use that model with data with an entirely different vocabulary? That won't work well.
– Peteris
Mar 9 at 3:47
That's my question whether to use one vocab or not.
– Shubham Panchal
Mar 9 at 4:57
add a comment |
It's not entirely clear what you're asking. Do you intend to train a model using one vocabulary (mapping from words to IDs), and then use that model with data with an entirely different vocabulary? That won't work well.
– Peteris
Mar 9 at 3:47
That's my question whether to use one vocab or not.
– Shubham Panchal
Mar 9 at 4:57
It's not entirely clear what you're asking. Do you intend to train a model using one vocabulary (mapping from words to IDs), and then use that model with data with an entirely different vocabulary? That won't work well.
– Peteris
Mar 9 at 3:47
It's not entirely clear what you're asking. Do you intend to train a model using one vocabulary (mapping from words to IDs), and then use that model with data with an entirely different vocabulary? That won't work well.
– Peteris
Mar 9 at 3:47
That's my question whether to use one vocab or not.
– Shubham Panchal
Mar 9 at 4:57
That's my question whether to use one vocab or not.
– Shubham Panchal
Mar 9 at 4:57
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55073668%2fwhich-we-use-different-tokenizers-for-input-and-target-data-in-seq2seq-lstm-mode%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55073668%2fwhich-we-use-different-tokenizers-for-input-and-target-data-in-seq2seq-lstm-mode%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It's not entirely clear what you're asking. Do you intend to train a model using one vocabulary (mapping from words to IDs), and then use that model with data with an entirely different vocabulary? That won't work well.
– Peteris
Mar 9 at 3:47
That's my question whether to use one vocab or not.
– Shubham Panchal
Mar 9 at 4:57