StringBuilder sometimes results in mangled lines
up vote
0
down vote
favorite
We have a Java application running which processes large amount of data and then writes it to CSV files. We have noticed that around 2% of the lines written are mangled and we have no idea why.
We use the following function for writing the lines:
static void writeLineToCSV(List<String> fields, Writer csvWriter) throws IOException {
StringBuilder line = new StringBuilder();
for (int i = 0; i < fields.size() - 1; i++) line.append(fields.get(i)).append(",");
line.append(fields.get(fields.size() - 1)).append("n");
csvWriter.write(line.toString());
csvWriter.flush();
}
Which results in lines which are mangled as follows:
2018-09-21T23:00:11.555Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:11.917Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:14592322,0.57377,-100,0.57644,1178.59788311,0.57375,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
Notice how in the third line, for some reason, the first and second field are mangled together with the separating comma gone without a trace.
EDIT: for completion's sake, the schema of the CSV file is: Instant.toString(), timestamp (long), BigDecimal, ..., BigDecimal
java csv stringbuilder
asked Nov 19 at 10:32
user1870238
105315
|
show 4 more comments
up vote
0
down vote
favorite
We have a Java application running which processes large amount of data and then writes it to CSV files. We have noticed that around 2% of the lines written are mangled and we have no idea why.
We use the following function for writing the lines:
static void writeLineToCSV(List<String> fields, Writer csvWriter) throws IOException {
StringBuilder line = new StringBuilder();
for (int i = 0; i < fields.size() - 1; i++) line.append(fields.get(i)).append(",");
line.append(fields.get(fields.size() - 1)).append("n");
csvWriter.write(line.toString());
csvWriter.flush();
}
Which results in lines which are mangled as follows:
2018-09-21T23:00:11.555Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:11.917Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:14592322,0.57377,-100,0.57644,1178.59788311,0.57375,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
Notice how in the third line, for some reason, the first and second field are mangled together with the separating comma gone without a trace.
EDIT: for completion's sake, the schema of the CSV file is: Instant.toString(), timestamp (long), BigDecimal, ..., BigDecimal
java csv stringbuilder
asked Nov 19 at 10:32
user1870238
105315
1
Interesting, but how could we reproduce the issue? What does theList<String>contain? How do you open/print the resulting file?
– Konstantin Yovkov
Nov 19 at 10:35
1
YourList<String> fieldsis presumably wrong.
– Michael
Nov 19 at 10:36
2
It looks to me like multithreaded access gone wrong
– Milo Bem
Nov 19 at 10:40
1
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
Nov 19 at 10:41
2
Why do you think the problem is in thelistedcode rather than whereList<String> fieldsis created ???
– Bruce Martin
Nov 19 at 10:55
|
show 4 more comments
up vote
0
down vote
favorite
up vote
0
down vote
favorite
We have a Java application running which processes large amount of data and then writes it to CSV files. We have noticed that around 2% of the lines written are mangled and we have no idea why.
We use the following function for writing the lines:
static void writeLineToCSV(List<String> fields, Writer csvWriter) throws IOException {
StringBuilder line = new StringBuilder();
for (int i = 0; i < fields.size() - 1; i++) line.append(fields.get(i)).append(",");
line.append(fields.get(fields.size() - 1)).append("n");
csvWriter.write(line.toString());
csvWriter.flush();
}
Which results in lines which are mangled as follows:
2018-09-21T23:00:11.555Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:11.917Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:14592322,0.57377,-100,0.57644,1178.59788311,0.57375,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
Notice how in the third line, for some reason, the first and second field are mangled together with the separating comma gone without a trace.
EDIT: for completion's sake, the schema of the CSV file is: Instant.toString(), timestamp (long), BigDecimal, ..., BigDecimal
java csv stringbuilder
asked Nov 19 at 10:32
user1870238
105315
We have a Java application running which processes large amount of data and then writes it to CSV files. We have noticed that around 2% of the lines written are mangled and we have no idea why.
We use the following function for writing the lines:
static void writeLineToCSV(List<String> fields, Writer csvWriter) throws IOException {
StringBuilder line = new StringBuilder();
for (int i = 0; i < fields.size() - 1; i++) line.append(fields.get(i)).append(",");
line.append(fields.get(fields.size() - 1)).append("n");
csvWriter.write(line.toString());
csvWriter.flush();
}
Which results in lines which are mangled as follows:
2018-09-21T23:00:11.555Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:11.917Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:14592322,0.57377,-100,0.57644,1178.59788311,0.57375,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
Notice how in the third line, for some reason, the first and second field are mangled together with the separating comma gone without a trace.
EDIT: for completion's sake, the schema of the CSV file is: Instant.toString(), timestamp (long), BigDecimal, ..., BigDecimal
java csv stringbuilder
java csv stringbuilder
asked Nov 19 at 10:32
user1870238
105315
asked Nov 19 at 10:32
user1870238
105315
edited Nov 19 at 10:36
asked Nov 19 at 10:32
user1870238
105315
asked Nov 19 at 10:32
user1870238
105315
asked Nov 19 at 10:32
user1870238
105315
105315
1
Interesting, but how could we reproduce the issue? What does theList<String>contain? How do you open/print the resulting file?
– Konstantin Yovkov
Nov 19 at 10:35
1
YourList<String> fieldsis presumably wrong.
– Michael
Nov 19 at 10:36
2
It looks to me like multithreaded access gone wrong
– Milo Bem
Nov 19 at 10:40
1
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
Nov 19 at 10:41
2
Why do you think the problem is in thelistedcode rather than whereList<String> fieldsis created ???
– Bruce Martin
Nov 19 at 10:55
|
show 4 more comments
1
Interesting, but how could we reproduce the issue? What does theList<String>contain? How do you open/print the resulting file?
– Konstantin Yovkov
Nov 19 at 10:35
1
YourList<String> fieldsis presumably wrong.
– Michael
Nov 19 at 10:36
2
It looks to me like multithreaded access gone wrong
– Milo Bem
Nov 19 at 10:40
1
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
Nov 19 at 10:41
2
Why do you think the problem is in thelistedcode rather than whereList<String> fieldsis created ???
– Bruce Martin
Nov 19 at 10:55
1
1
Interesting, but how could we reproduce the issue? What does the
List<String> contain? How do you open/print the resulting file?– Konstantin Yovkov
Nov 19 at 10:35
Interesting, but how could we reproduce the issue? What does the
List<String> contain? How do you open/print the resulting file?– Konstantin Yovkov
Nov 19 at 10:35
1
1
Your
List<String> fields is presumably wrong.– Michael
Nov 19 at 10:36
Your
List<String> fields is presumably wrong.– Michael
Nov 19 at 10:36
2
2
It looks to me like multithreaded access gone wrong
– Milo Bem
Nov 19 at 10:40
It looks to me like multithreaded access gone wrong
– Milo Bem
Nov 19 at 10:40
1
1
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
Nov 19 at 10:41
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
Nov 19 at 10:41
2
2
Why do you think the problem is in the
listed code rather than where List<String> fields is created ???– Bruce Martin
Nov 19 at 10:55
Why do you think the problem is in the
listed code rather than where List<String> fields is created ???– Bruce Martin
Nov 19 at 10:55
|
show 4 more comments
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53372680%2fstringbuilder-sometimes-results-in-mangled-lines%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Interesting, but how could we reproduce the issue? What does the
List<String>contain? How do you open/print the resulting file?– Konstantin Yovkov
Nov 19 at 10:35
1
Your
List<String> fieldsis presumably wrong.– Michael
Nov 19 at 10:36
2
It looks to me like multithreaded access gone wrong
– Milo Bem
Nov 19 at 10:40
1
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
Nov 19 at 10:41
2
Why do you think the problem is in the
listedcode rather than whereList<String> fieldsis created ???– Bruce Martin
Nov 19 at 10:55