How to load an aligned word2vec model in Gensim?
1
I have a time series dataset. therefore, for each time period I trained a word2vec model and realigned the models.
However, when I try to load the aligned word2vec models as follows I get the below mentioned error.
#Load model
model = word2vec.Word2Vec.load('model_1970')
Error:
train_words_pow += wv.vocab[wv.index2word[word_index]].count**power
KeyError: 'ctrx'
Is there a way to resolve this error? :)
I have attached a sample trained word2vec model that gives error for testing purposes
Link: https://drive.google.com/file/d/1IBbUgeAubr2xzNYLKZgPt34xOEsW92bO/view?usp=sharing
EDIT:
Mention below is the log of my program.
2018-11-30 14:23:43,897 : INFO : loading Word2Vec object from model_1970
2018-11-30 14:23:43,961 : INFO : loading wv recursively from model_1970.wv.* with mmap=None
2018-11-30 14:23:43,965 : INFO : loading vectors from model_1970.wv.vectors.npy with mmap=None
2018-11-30 14:23:44,005 : INFO : setting ignored attribute vectors_norm to None
2018-11-30 14:23:44,009 : INFO : loading vocabulary recursively from model_1970.vocabulary.* with mmap=None
2018-11-30 14:23:44,009 : INFO : loading trainables recursively from model_1970.trainables.* with mmap=None
2018-11-30 14:23:44,009 : INFO : loading syn1neg from model_1970.trainables.syn1neg.npy with mmap=None
2018-11-30 14:23:44,053 : INFO : setting ignored attribute cum_table to None
2018-11-30 14:23:44,053 : INFO : loaded model_1970
Reloaded modules: __mp_main__
Traceback (most recent call last):
File "<ipython-input-3-3b9230dacba9>", line 1, in <module>
runfile('C:/Users/Emi/Desktop/code/word2vec_distance.py', wdir='C:/Users/Emi/Desktop/code')
File "C:UsersEmiAnaconda3libsite-packagesspyder_kernelscustomizespydercustomize.py", line 668, in runfile
execfile(filename, namespace)
File "C:UsersEmiAnaconda3libsite-packagesspyder_kernelscustomizespydercustomize.py", line 108, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Emi/Desktop/code/word2vec_distance.py", line 26, in <module>
model_1 = word2vec.Word2Vec.load(word2vec_model_name_1)
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsword2vec.py", line 975, in load
return super(Word2Vec, cls).load(*args, **kwargs)
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsbase_any2vec.py", line 631, in load
model.vocabulary.make_cum_table(model.wv) # rebuild cum_table from vocabulary
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsword2vec.py", line 1383, in make_cum_table
train_words_pow += wv.vocab[wv.index2word[word_index]].count**power
KeyError: 'cmnx'
python nlp gensim word2vec
asked Nov 26 '18 at 1:58
EmiEmi
332215
|
show 6 more comments
1
I have a time series dataset. therefore, for each time period I trained a word2vec model and realigned the models.
However, when I try to load the aligned word2vec models as follows I get the below mentioned error.
#Load model
model = word2vec.Word2Vec.load('model_1970')
Error:
train_words_pow += wv.vocab[wv.index2word[word_index]].count**power
KeyError: 'ctrx'
Is there a way to resolve this error? :)
I have attached a sample trained word2vec model that gives error for testing purposes
Link: https://drive.google.com/file/d/1IBbUgeAubr2xzNYLKZgPt34xOEsW92bO/view?usp=sharing
EDIT:
Mention below is the log of my program.
2018-11-30 14:23:43,897 : INFO : loading Word2Vec object from model_1970
2018-11-30 14:23:43,961 : INFO : loading wv recursively from model_1970.wv.* with mmap=None
2018-11-30 14:23:43,965 : INFO : loading vectors from model_1970.wv.vectors.npy with mmap=None
2018-11-30 14:23:44,005 : INFO : setting ignored attribute vectors_norm to None
2018-11-30 14:23:44,009 : INFO : loading vocabulary recursively from model_1970.vocabulary.* with mmap=None
2018-11-30 14:23:44,009 : INFO : loading trainables recursively from model_1970.trainables.* with mmap=None
2018-11-30 14:23:44,009 : INFO : loading syn1neg from model_1970.trainables.syn1neg.npy with mmap=None
2018-11-30 14:23:44,053 : INFO : setting ignored attribute cum_table to None
2018-11-30 14:23:44,053 : INFO : loaded model_1970
Reloaded modules: __mp_main__
Traceback (most recent call last):
File "<ipython-input-3-3b9230dacba9>", line 1, in <module>
runfile('C:/Users/Emi/Desktop/code/word2vec_distance.py', wdir='C:/Users/Emi/Desktop/code')
File "C:UsersEmiAnaconda3libsite-packagesspyder_kernelscustomizespydercustomize.py", line 668, in runfile
execfile(filename, namespace)
File "C:UsersEmiAnaconda3libsite-packagesspyder_kernelscustomizespydercustomize.py", line 108, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Emi/Desktop/code/word2vec_distance.py", line 26, in <module>
model_1 = word2vec.Word2Vec.load(word2vec_model_name_1)
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsword2vec.py", line 975, in load
return super(Word2Vec, cls).load(*args, **kwargs)
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsbase_any2vec.py", line 631, in load
model.vocabulary.make_cum_table(model.wv) # rebuild cum_table from vocabulary
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsword2vec.py", line 1383, in make_cum_table
train_words_pow += wv.vocab[wv.index2word[word_index]].count**power
KeyError: 'cmnx'
python nlp gensim word2vec
asked Nov 26 '18 at 1:58
EmiEmi
332215
Would it be possible to make one of the models available to test?
– Pedro Torres
Nov 26 '18 at 2:05
@PedroTorres Thanks a lot for the comment. I am using the following code to align my word2vec models gist.github.com/quadrismegistus/… My real models are very big. I will do a test model and attach it soon :)
– Emi
Nov 26 '18 at 2:14
@PedroTorres updated the question :)
– Emi
Nov 26 '18 at 2:40
Tried it here. It does show the error. It seems like the problem is with the vocabulary dictionary given by wv.vocab. For some reason it does not have all the words
– Pedro Torres
Nov 26 '18 at 2:49
@PedroTorres I did a modification to the model's vocabulary (i.e aligning the model) using the gensim code in gist.github.com/quadrismegistus/… Any idea how to resolve this issue? :)
– Emi
Nov 26 '18 at 5:23
|
show 6 more comments
1
1
1
I have a time series dataset. therefore, for each time period I trained a word2vec model and realigned the models.
However, when I try to load the aligned word2vec models as follows I get the below mentioned error.
#Load model
model = word2vec.Word2Vec.load('model_1970')
Error:
train_words_pow += wv.vocab[wv.index2word[word_index]].count**power
KeyError: 'ctrx'
Is there a way to resolve this error? :)
I have attached a sample trained word2vec model that gives error for testing purposes
Link: https://drive.google.com/file/d/1IBbUgeAubr2xzNYLKZgPt34xOEsW92bO/view?usp=sharing
EDIT:
Mention below is the log of my program.
2018-11-30 14:23:43,897 : INFO : loading Word2Vec object from model_1970
2018-11-30 14:23:43,961 : INFO : loading wv recursively from model_1970.wv.* with mmap=None
2018-11-30 14:23:43,965 : INFO : loading vectors from model_1970.wv.vectors.npy with mmap=None
2018-11-30 14:23:44,005 : INFO : setting ignored attribute vectors_norm to None
2018-11-30 14:23:44,009 : INFO : loading vocabulary recursively from model_1970.vocabulary.* with mmap=None
2018-11-30 14:23:44,009 : INFO : loading trainables recursively from model_1970.trainables.* with mmap=None
2018-11-30 14:23:44,009 : INFO : loading syn1neg from model_1970.trainables.syn1neg.npy with mmap=None
2018-11-30 14:23:44,053 : INFO : setting ignored attribute cum_table to None
2018-11-30 14:23:44,053 : INFO : loaded model_1970
Reloaded modules: __mp_main__
Traceback (most recent call last):
File "<ipython-input-3-3b9230dacba9>", line 1, in <module>
runfile('C:/Users/Emi/Desktop/code/word2vec_distance.py', wdir='C:/Users/Emi/Desktop/code')
File "C:UsersEmiAnaconda3libsite-packagesspyder_kernelscustomizespydercustomize.py", line 668, in runfile
execfile(filename, namespace)
File "C:UsersEmiAnaconda3libsite-packagesspyder_kernelscustomizespydercustomize.py", line 108, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Emi/Desktop/code/word2vec_distance.py", line 26, in <module>
model_1 = word2vec.Word2Vec.load(word2vec_model_name_1)
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsword2vec.py", line 975, in load
return super(Word2Vec, cls).load(*args, **kwargs)
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsbase_any2vec.py", line 631, in load
model.vocabulary.make_cum_table(model.wv) # rebuild cum_table from vocabulary
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsword2vec.py", line 1383, in make_cum_table
train_words_pow += wv.vocab[wv.index2word[word_index]].count**power
KeyError: 'cmnx'
python nlp gensim word2vec
asked Nov 26 '18 at 1:58
EmiEmi
332215
I have a time series dataset. therefore, for each time period I trained a word2vec model and realigned the models.
However, when I try to load the aligned word2vec models as follows I get the below mentioned error.
#Load model
model = word2vec.Word2Vec.load('model_1970')
Error:
train_words_pow += wv.vocab[wv.index2word[word_index]].count**power
KeyError: 'ctrx'
Is there a way to resolve this error? :)
I have attached a sample trained word2vec model that gives error for testing purposes
Link: https://drive.google.com/file/d/1IBbUgeAubr2xzNYLKZgPt34xOEsW92bO/view?usp=sharing
EDIT:
Mention below is the log of my program.
2018-11-30 14:23:43,897 : INFO : loading Word2Vec object from model_1970
2018-11-30 14:23:43,961 : INFO : loading wv recursively from model_1970.wv.* with mmap=None
2018-11-30 14:23:43,965 : INFO : loading vectors from model_1970.wv.vectors.npy with mmap=None
2018-11-30 14:23:44,005 : INFO : setting ignored attribute vectors_norm to None
2018-11-30 14:23:44,009 : INFO : loading vocabulary recursively from model_1970.vocabulary.* with mmap=None
2018-11-30 14:23:44,009 : INFO : loading trainables recursively from model_1970.trainables.* with mmap=None
2018-11-30 14:23:44,009 : INFO : loading syn1neg from model_1970.trainables.syn1neg.npy with mmap=None
2018-11-30 14:23:44,053 : INFO : setting ignored attribute cum_table to None
2018-11-30 14:23:44,053 : INFO : loaded model_1970
Reloaded modules: __mp_main__
Traceback (most recent call last):
File "<ipython-input-3-3b9230dacba9>", line 1, in <module>
runfile('C:/Users/Emi/Desktop/code/word2vec_distance.py', wdir='C:/Users/Emi/Desktop/code')
File "C:UsersEmiAnaconda3libsite-packagesspyder_kernelscustomizespydercustomize.py", line 668, in runfile
execfile(filename, namespace)
File "C:UsersEmiAnaconda3libsite-packagesspyder_kernelscustomizespydercustomize.py", line 108, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Emi/Desktop/code/word2vec_distance.py", line 26, in <module>
model_1 = word2vec.Word2Vec.load(word2vec_model_name_1)
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsword2vec.py", line 975, in load
return super(Word2Vec, cls).load(*args, **kwargs)
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsbase_any2vec.py", line 631, in load
model.vocabulary.make_cum_table(model.wv) # rebuild cum_table from vocabulary
File "C:UsersEmiAnaconda3libsite-packagesgensimmodelsword2vec.py", line 1383, in make_cum_table
train_words_pow += wv.vocab[wv.index2word[word_index]].count**power
KeyError: 'cmnx'
python nlp gensim word2vec
python nlp gensim word2vec
asked Nov 26 '18 at 1:58
EmiEmi
332215
asked Nov 26 '18 at 1:58
EmiEmi
332215
edited Nov 30 '18 at 4:00
Emi
asked Nov 26 '18 at 1:58
EmiEmi
332215
asked Nov 26 '18 at 1:58
EmiEmi
332215
asked Nov 26 '18 at 1:58
EmiEmi
332215
332215
Would it be possible to make one of the models available to test?
– Pedro Torres
Nov 26 '18 at 2:05
@PedroTorres Thanks a lot for the comment. I am using the following code to align my word2vec models gist.github.com/quadrismegistus/… My real models are very big. I will do a test model and attach it soon :)
– Emi
Nov 26 '18 at 2:14
@PedroTorres updated the question :)
– Emi
Nov 26 '18 at 2:40
Tried it here. It does show the error. It seems like the problem is with the vocabulary dictionary given by wv.vocab. For some reason it does not have all the words
– Pedro Torres
Nov 26 '18 at 2:49
@PedroTorres I did a modification to the model's vocabulary (i.e aligning the model) using the gensim code in gist.github.com/quadrismegistus/… Any idea how to resolve this issue? :)
– Emi
Nov 26 '18 at 5:23
|
show 6 more comments
Would it be possible to make one of the models available to test?
– Pedro Torres
Nov 26 '18 at 2:05
@PedroTorres Thanks a lot for the comment. I am using the following code to align my word2vec models gist.github.com/quadrismegistus/… My real models are very big. I will do a test model and attach it soon :)
– Emi
Nov 26 '18 at 2:14
@PedroTorres updated the question :)
– Emi
Nov 26 '18 at 2:40
Tried it here. It does show the error. It seems like the problem is with the vocabulary dictionary given by wv.vocab. For some reason it does not have all the words
– Pedro Torres
Nov 26 '18 at 2:49
@PedroTorres I did a modification to the model's vocabulary (i.e aligning the model) using the gensim code in gist.github.com/quadrismegistus/… Any idea how to resolve this issue? :)
– Emi
Nov 26 '18 at 5:23
Would it be possible to make one of the models available to test?
– Pedro Torres
Nov 26 '18 at 2:05
Would it be possible to make one of the models available to test?
– Pedro Torres
Nov 26 '18 at 2:05
@PedroTorres Thanks a lot for the comment. I am using the following code to align my word2vec models gist.github.com/quadrismegistus/… My real models are very big. I will do a test model and attach it soon :)
– Emi
Nov 26 '18 at 2:14
@PedroTorres Thanks a lot for the comment. I am using the following code to align my word2vec models gist.github.com/quadrismegistus/… My real models are very big. I will do a test model and attach it soon :)
– Emi
Nov 26 '18 at 2:14
@PedroTorres updated the question :)
– Emi
Nov 26 '18 at 2:40
@PedroTorres updated the question :)
– Emi
Nov 26 '18 at 2:40
Tried it here. It does show the error. It seems like the problem is with the vocabulary dictionary given by wv.vocab. For some reason it does not have all the words
– Pedro Torres
Nov 26 '18 at 2:49
Tried it here. It does show the error. It seems like the problem is with the vocabulary dictionary given by wv.vocab. For some reason it does not have all the words
– Pedro Torres
Nov 26 '18 at 2:49
@PedroTorres I did a modification to the model's vocabulary (i.e aligning the model) using the gensim code in gist.github.com/quadrismegistus/… Any idea how to resolve this issue? :)
– Emi
Nov 26 '18 at 5:23
@PedroTorres I did a modification to the model's vocabulary (i.e aligning the model) using the gensim code in gist.github.com/quadrismegistus/… Any idea how to resolve this issue? :)
– Emi
Nov 26 '18 at 5:23
|
show 6 more comments
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53473868%2fhow-to-load-an-aligned-word2vec-model-in-gensim%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53473868%2fhow-to-load-an-aligned-word2vec-model-in-gensim%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Would it be possible to make one of the models available to test?
– Pedro Torres
Nov 26 '18 at 2:05
@PedroTorres Thanks a lot for the comment. I am using the following code to align my word2vec models gist.github.com/quadrismegistus/… My real models are very big. I will do a test model and attach it soon :)
– Emi
Nov 26 '18 at 2:14
@PedroTorres updated the question :)
– Emi
Nov 26 '18 at 2:40
Tried it here. It does show the error. It seems like the problem is with the vocabulary dictionary given by wv.vocab. For some reason it does not have all the words
– Pedro Torres
Nov 26 '18 at 2:49
@PedroTorres I did a modification to the model's vocabulary (i.e aligning the model) using the gensim code in gist.github.com/quadrismegistus/… Any idea how to resolve this issue? :)
– Emi
Nov 26 '18 at 5:23