text segmentation by the number of words in python
0
can anyone tell me what is the problem with my own code?
I want to segment a big text into small texts by words. for example, each segment contains 60 words each.
file=r'C:UsersNujouDesktopMasterthesissteganalysisdataseteconomy2.txt'
openFile= open(file, 'r', encoding='utf-8-sig')
words= openFile.read().split()
#print (words)
i = 0
for idx, w in enumerate(words, start=0):
textNum = 1
while textNum <= 20:
wordAsText =
print("word list before:", wordAsText)
while i<idx+60:
wordAsText.append(words[i])
i+=1
print ("word list after:", wordAsText)
textSeg=' '.join(wordAsText)
print (textNum, textSeg)
files = open(r"C:UsersNujouDesktopMasterthesissteganalysisdatasetdatasetEcoEco" + str(textNum) + ".txt", "w", encoding='utf-8-sig')
files.write(textSeg)
files.close()
idx+=60
if textNum!=20:
continue
textNum+=1
my big file (economy2) contains more than 12K words.
EDIT:
thanks for all responses. I tried what I found here and it is achieved my require.
Edited Code:
file=r'C:UsersNujouDesktopMasterthesissteganalysisdataseteconomy2.txt'
openFile= open(file, 'r', encoding='utf-8-sig')
words= openFile.read().split()
#print (words)
n=60
segments=[' '.join(words[i:i+n]) for i in range(0,len(words),n)] #from link
i=1
for s in segments:
seg = open(r"C:UsersNujouDesktopMasterthesissteganalysisdatasetdatasetEcoEco" + str(i) + ".txt", "w", encoding='utf-8-sig')
seg.write(s)
seg.close()
i+=1
python-3.x
asked Nov 20 at 12:02
Nujud Ali
356
add a comment |
0
can anyone tell me what is the problem with my own code?
I want to segment a big text into small texts by words. for example, each segment contains 60 words each.
file=r'C:UsersNujouDesktopMasterthesissteganalysisdataseteconomy2.txt'
openFile= open(file, 'r', encoding='utf-8-sig')
words= openFile.read().split()
#print (words)
i = 0
for idx, w in enumerate(words, start=0):
textNum = 1
while textNum <= 20:
wordAsText =
print("word list before:", wordAsText)
while i<idx+60:
wordAsText.append(words[i])
i+=1
print ("word list after:", wordAsText)
textSeg=' '.join(wordAsText)
print (textNum, textSeg)
files = open(r"C:UsersNujouDesktopMasterthesissteganalysisdatasetdatasetEcoEco" + str(textNum) + ".txt", "w", encoding='utf-8-sig')
files.write(textSeg)
files.close()
idx+=60
if textNum!=20:
continue
textNum+=1
my big file (economy2) contains more than 12K words.
EDIT:
thanks for all responses. I tried what I found here and it is achieved my require.
Edited Code:
file=r'C:UsersNujouDesktopMasterthesissteganalysisdataseteconomy2.txt'
openFile= open(file, 'r', encoding='utf-8-sig')
words= openFile.read().split()
#print (words)
n=60
segments=[' '.join(words[i:i+n]) for i in range(0,len(words),n)] #from link
i=1
for s in segments:
seg = open(r"C:UsersNujouDesktopMasterthesissteganalysisdatasetdatasetEcoEco" + str(i) + ".txt", "w", encoding='utf-8-sig')
seg.write(s)
seg.close()
i+=1
python-3.x
asked Nov 20 at 12:02
Nujud Ali
356
3
What is the problem? How does this not work as expected?
– mkrieger1
Nov 20 at 12:12
it gives this error "IndexError: list index out of range", and did not write the segmented files. I want just 20 small files with 60 words each not whole the big file.
– Nujud Ali
Nov 20 at 12:33
1
Have you tried stepping through your code with your debugger? Find out where you're indexing a list incorrectly?
– TrebuchetMS
Nov 20 at 12:41
"IndexError: list index out of range" You never increase textNum before comparing it to 20, you might just have an endless loop in there
– Pinkie Pie
Nov 20 at 14:23
add a comment |
0
0
0
0
can anyone tell me what is the problem with my own code?
I want to segment a big text into small texts by words. for example, each segment contains 60 words each.
file=r'C:UsersNujouDesktopMasterthesissteganalysisdataseteconomy2.txt'
openFile= open(file, 'r', encoding='utf-8-sig')
words= openFile.read().split()
#print (words)
i = 0
for idx, w in enumerate(words, start=0):
textNum = 1
while textNum <= 20:
wordAsText =
print("word list before:", wordAsText)
while i<idx+60:
wordAsText.append(words[i])
i+=1
print ("word list after:", wordAsText)
textSeg=' '.join(wordAsText)
print (textNum, textSeg)
files = open(r"C:UsersNujouDesktopMasterthesissteganalysisdatasetdatasetEcoEco" + str(textNum) + ".txt", "w", encoding='utf-8-sig')
files.write(textSeg)
files.close()
idx+=60
if textNum!=20:
continue
textNum+=1
my big file (economy2) contains more than 12K words.
EDIT:
thanks for all responses. I tried what I found here and it is achieved my require.
Edited Code:
file=r'C:UsersNujouDesktopMasterthesissteganalysisdataseteconomy2.txt'
openFile= open(file, 'r', encoding='utf-8-sig')
words= openFile.read().split()
#print (words)
n=60
segments=[' '.join(words[i:i+n]) for i in range(0,len(words),n)] #from link
i=1
for s in segments:
seg = open(r"C:UsersNujouDesktopMasterthesissteganalysisdatasetdatasetEcoEco" + str(i) + ".txt", "w", encoding='utf-8-sig')
seg.write(s)
seg.close()
i+=1
python-3.x
asked Nov 20 at 12:02
Nujud Ali
356
can anyone tell me what is the problem with my own code?
I want to segment a big text into small texts by words. for example, each segment contains 60 words each.
file=r'C:UsersNujouDesktopMasterthesissteganalysisdataseteconomy2.txt'
openFile= open(file, 'r', encoding='utf-8-sig')
words= openFile.read().split()
#print (words)
i = 0
for idx, w in enumerate(words, start=0):
textNum = 1
while textNum <= 20:
wordAsText =
print("word list before:", wordAsText)
while i<idx+60:
wordAsText.append(words[i])
i+=1
print ("word list after:", wordAsText)
textSeg=' '.join(wordAsText)
print (textNum, textSeg)
files = open(r"C:UsersNujouDesktopMasterthesissteganalysisdatasetdatasetEcoEco" + str(textNum) + ".txt", "w", encoding='utf-8-sig')
files.write(textSeg)
files.close()
idx+=60
if textNum!=20:
continue
textNum+=1
my big file (economy2) contains more than 12K words.
EDIT:
thanks for all responses. I tried what I found here and it is achieved my require.
Edited Code:
file=r'C:UsersNujouDesktopMasterthesissteganalysisdataseteconomy2.txt'
openFile= open(file, 'r', encoding='utf-8-sig')
words= openFile.read().split()
#print (words)
n=60
segments=[' '.join(words[i:i+n]) for i in range(0,len(words),n)] #from link
i=1
for s in segments:
seg = open(r"C:UsersNujouDesktopMasterthesissteganalysisdatasetdatasetEcoEco" + str(i) + ".txt", "w", encoding='utf-8-sig')
seg.write(s)
seg.close()
i+=1
python-3.x
python-3.x
asked Nov 20 at 12:02
Nujud Ali
356
asked Nov 20 at 12:02
Nujud Ali
356
edited Nov 21 at 1:47
asked Nov 20 at 12:02
Nujud Ali
356
asked Nov 20 at 12:02
Nujud Ali
356
asked Nov 20 at 12:02
Nujud Ali
356
356
3
What is the problem? How does this not work as expected?
– mkrieger1
Nov 20 at 12:12
it gives this error "IndexError: list index out of range", and did not write the segmented files. I want just 20 small files with 60 words each not whole the big file.
– Nujud Ali
Nov 20 at 12:33
1
Have you tried stepping through your code with your debugger? Find out where you're indexing a list incorrectly?
– TrebuchetMS
Nov 20 at 12:41
"IndexError: list index out of range" You never increase textNum before comparing it to 20, you might just have an endless loop in there
– Pinkie Pie
Nov 20 at 14:23
add a comment |
3
What is the problem? How does this not work as expected?
– mkrieger1
Nov 20 at 12:12
it gives this error "IndexError: list index out of range", and did not write the segmented files. I want just 20 small files with 60 words each not whole the big file.
– Nujud Ali
Nov 20 at 12:33
1
Have you tried stepping through your code with your debugger? Find out where you're indexing a list incorrectly?
– TrebuchetMS
Nov 20 at 12:41
"IndexError: list index out of range" You never increase textNum before comparing it to 20, you might just have an endless loop in there
– Pinkie Pie
Nov 20 at 14:23
3
3
What is the problem? How does this not work as expected?
– mkrieger1
Nov 20 at 12:12
What is the problem? How does this not work as expected?
– mkrieger1
Nov 20 at 12:12
it gives this error "IndexError: list index out of range", and did not write the segmented files. I want just 20 small files with 60 words each not whole the big file.
– Nujud Ali
Nov 20 at 12:33
it gives this error "IndexError: list index out of range", and did not write the segmented files. I want just 20 small files with 60 words each not whole the big file.
– Nujud Ali
Nov 20 at 12:33
1
1
Have you tried stepping through your code with your debugger? Find out where you're indexing a list incorrectly?
– TrebuchetMS
Nov 20 at 12:41
Have you tried stepping through your code with your debugger? Find out where you're indexing a list incorrectly?
– TrebuchetMS
Nov 20 at 12:41
"IndexError: list index out of range" You never increase textNum before comparing it to 20, you might just have an endless loop in there
– Pinkie Pie
Nov 20 at 14:23
"IndexError: list index out of range" You never increase textNum before comparing it to 20, you might just have an endless loop in there
– Pinkie Pie
Nov 20 at 14:23
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53392589%2ftext-segmentation-by-the-number-of-words-in-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53392589%2ftext-segmentation-by-the-number-of-words-in-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
3
What is the problem? How does this not work as expected?
– mkrieger1
Nov 20 at 12:12
it gives this error "IndexError: list index out of range", and did not write the segmented files. I want just 20 small files with 60 words each not whole the big file.
– Nujud Ali
Nov 20 at 12:33
1
Have you tried stepping through your code with your debugger? Find out where you're indexing a list incorrectly?
– TrebuchetMS
Nov 20 at 12:41
"IndexError: list index out of range" You never increase textNum before comparing it to 20, you might just have an endless loop in there
– Pinkie Pie
Nov 20 at 14:23