JSON.stringify is very slow for large objects
I have a very big object in javascript (about 10MB).
And when I stringify it, it takes a long time, so I send it to backend and parse it to an object( actually nested objects with arrays), and that takes long time too but it's not our problem in this question.
The problem:
How can I make JSON.stringify faster, any ideas or alternatives, I need a javaScript solution, libraries I can use or ideas here.
What I've tried
I googled a lot and looks there is no better performance than JSON.stringify or my googling skills got rusty!
Result
I accept any suggestion that may solve me the long saving (sending to backend) in the request (I know its big request).
Code Sample of problem (details about problem)
Request URL:http://localhost:8081/systemName/controllerA/update.html;jsessionid=FB3848B6C0F4AD9873EA12DBE61E6008
Request Method:POST
Status Code:200 OK
Am sending a POST to backend and then in JAVA
request.getParameter("BigPostParameter")
and I read it to convert to object using
public boolean fromJSON(String string) {
if (string != null && !string.isEmpty()) {
ObjectMapper json = new ObjectMapper();
DateFormat dateFormat = new SimpleDateFormat(YYYY_MM_DD_T_HH_MM_SS_SSS_Z);
dateFormat.setTimeZone(TimeZone.getDefault());
json.setDateFormat(dateFormat);
json.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true);
WebObject object;
// Logger.getLogger("JSON Tracker").log(Level.SEVERE, "Start");
try {
object = json.readValue(string, this.getClass());
} catch (IOException ex) {
Logger.getLogger(JSON_ERROR).log(Level.SEVERE, "JSON Error: {0}", ex.getMessage());
return false;
}
// Logger.getLogger("JSON Tracker").log(Level.SEVERE, "END");
return this.setThis(object);
}
return false;
}
Like This
BigObject someObj = new BigObject();
someObj.fromJSON(request.getParameter("BigPostParameter"))
P.S : FYI this line object = json.readValue(string, this.getClass());
is also very very very slow.
Again to summarize
Problem in posting time (stringify) JavaScript bottle nick.
Another problem parsing that stringified into an object (using jackson), and mainly I have svg tags content in that stringified object as a style column, and other columns are strings, int mainly
javascript json performance
edited Dec 26 '18 at 5:47
priyanshi srivastava
938520
asked Aug 4 '17 at 19:01
shareefshareef
5,03474470
|
show 14 more comments
I have a very big object in javascript (about 10MB).
And when I stringify it, it takes a long time, so I send it to backend and parse it to an object( actually nested objects with arrays), and that takes long time too but it's not our problem in this question.
The problem:
How can I make JSON.stringify faster, any ideas or alternatives, I need a javaScript solution, libraries I can use or ideas here.
What I've tried
I googled a lot and looks there is no better performance than JSON.stringify or my googling skills got rusty!
Result
I accept any suggestion that may solve me the long saving (sending to backend) in the request (I know its big request).
Code Sample of problem (details about problem)
Request URL:http://localhost:8081/systemName/controllerA/update.html;jsessionid=FB3848B6C0F4AD9873EA12DBE61E6008
Request Method:POST
Status Code:200 OK
Am sending a POST to backend and then in JAVA
request.getParameter("BigPostParameter")
and I read it to convert to object using
public boolean fromJSON(String string) {
if (string != null && !string.isEmpty()) {
ObjectMapper json = new ObjectMapper();
DateFormat dateFormat = new SimpleDateFormat(YYYY_MM_DD_T_HH_MM_SS_SSS_Z);
dateFormat.setTimeZone(TimeZone.getDefault());
json.setDateFormat(dateFormat);
json.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true);
WebObject object;
// Logger.getLogger("JSON Tracker").log(Level.SEVERE, "Start");
try {
object = json.readValue(string, this.getClass());
} catch (IOException ex) {
Logger.getLogger(JSON_ERROR).log(Level.SEVERE, "JSON Error: {0}", ex.getMessage());
return false;
}
// Logger.getLogger("JSON Tracker").log(Level.SEVERE, "END");
return this.setThis(object);
}
return false;
}
Like This
BigObject someObj = new BigObject();
someObj.fromJSON(request.getParameter("BigPostParameter"))
P.S : FYI this line object = json.readValue(string, this.getClass());
is also very very very slow.
Again to summarize
Problem in posting time (stringify) JavaScript bottle nick.
Another problem parsing that stringified into an object (using jackson), and mainly I have svg tags content in that stringified object as a style column, and other columns are strings, int mainly
javascript json performance
edited Dec 26 '18 at 5:47
priyanshi srivastava
938520
asked Aug 4 '17 at 19:01
shareefshareef
5,03474470
1
How do you send it to a backend without converting it to JSON?
– csander
Aug 4 '17 at 19:03
JSON.stringify()is recursive. Why isJSON.stringify()call necessary? What is application and expected result?
– guest271314
Aug 4 '17 at 19:05
For serialization, you're going to be hard-pressed to find a faster alternative unless you look into designing or replicating a byte-encoded format. If you're hard-set on JSON format,JSON.stringify()is probably the fastest you'll get, though. There are other methods I know of that utilize streaming to be more memory efficient, but not faster.
– Patrick Roberts
Aug 4 '17 at 19:05
2
I'm inclined to say this is too broad, because the solution is to cache and/or break up the object, or both. There are no faster serializers: see github.com/kawanet/msgpack-lite
– Meirion Hughes
Aug 4 '17 at 19:43
1
@JohnWeisz Exactly that what we will do when we rewrite , it will be more dynamic for example after each change it will update some session tracking socket kind of . so we learn from our mistakes
– shareef
Aug 6 '17 at 14:04
|
show 14 more comments
I have a very big object in javascript (about 10MB).
And when I stringify it, it takes a long time, so I send it to backend and parse it to an object( actually nested objects with arrays), and that takes long time too but it's not our problem in this question.
The problem:
How can I make JSON.stringify faster, any ideas or alternatives, I need a javaScript solution, libraries I can use or ideas here.
What I've tried
I googled a lot and looks there is no better performance than JSON.stringify or my googling skills got rusty!
Result
I accept any suggestion that may solve me the long saving (sending to backend) in the request (I know its big request).
Code Sample of problem (details about problem)
Request URL:http://localhost:8081/systemName/controllerA/update.html;jsessionid=FB3848B6C0F4AD9873EA12DBE61E6008
Request Method:POST
Status Code:200 OK
Am sending a POST to backend and then in JAVA
request.getParameter("BigPostParameter")
and I read it to convert to object using
public boolean fromJSON(String string) {
if (string != null && !string.isEmpty()) {
ObjectMapper json = new ObjectMapper();
DateFormat dateFormat = new SimpleDateFormat(YYYY_MM_DD_T_HH_MM_SS_SSS_Z);
dateFormat.setTimeZone(TimeZone.getDefault());
json.setDateFormat(dateFormat);
json.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true);
WebObject object;
// Logger.getLogger("JSON Tracker").log(Level.SEVERE, "Start");
try {
object = json.readValue(string, this.getClass());
} catch (IOException ex) {
Logger.getLogger(JSON_ERROR).log(Level.SEVERE, "JSON Error: {0}", ex.getMessage());
return false;
}
// Logger.getLogger("JSON Tracker").log(Level.SEVERE, "END");
return this.setThis(object);
}
return false;
}
Like This
BigObject someObj = new BigObject();
someObj.fromJSON(request.getParameter("BigPostParameter"))
P.S : FYI this line object = json.readValue(string, this.getClass());
is also very very very slow.
Again to summarize
Problem in posting time (stringify) JavaScript bottle nick.
Another problem parsing that stringified into an object (using jackson), and mainly I have svg tags content in that stringified object as a style column, and other columns are strings, int mainly
javascript json performance
edited Dec 26 '18 at 5:47
priyanshi srivastava
938520
asked Aug 4 '17 at 19:01
shareefshareef
5,03474470
I have a very big object in javascript (about 10MB).
And when I stringify it, it takes a long time, so I send it to backend and parse it to an object( actually nested objects with arrays), and that takes long time too but it's not our problem in this question.
The problem:
How can I make JSON.stringify faster, any ideas or alternatives, I need a javaScript solution, libraries I can use or ideas here.
What I've tried
I googled a lot and looks there is no better performance than JSON.stringify or my googling skills got rusty!
Result
I accept any suggestion that may solve me the long saving (sending to backend) in the request (I know its big request).
Code Sample of problem (details about problem)
Request URL:http://localhost:8081/systemName/controllerA/update.html;jsessionid=FB3848B6C0F4AD9873EA12DBE61E6008
Request Method:POST
Status Code:200 OK
Am sending a POST to backend and then in JAVA
request.getParameter("BigPostParameter")
and I read it to convert to object using
public boolean fromJSON(String string) {
if (string != null && !string.isEmpty()) {
ObjectMapper json = new ObjectMapper();
DateFormat dateFormat = new SimpleDateFormat(YYYY_MM_DD_T_HH_MM_SS_SSS_Z);
dateFormat.setTimeZone(TimeZone.getDefault());
json.setDateFormat(dateFormat);
json.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true);
WebObject object;
// Logger.getLogger("JSON Tracker").log(Level.SEVERE, "Start");
try {
object = json.readValue(string, this.getClass());
} catch (IOException ex) {
Logger.getLogger(JSON_ERROR).log(Level.SEVERE, "JSON Error: {0}", ex.getMessage());
return false;
}
// Logger.getLogger("JSON Tracker").log(Level.SEVERE, "END");
return this.setThis(object);
}
return false;
}
Like This
BigObject someObj = new BigObject();
someObj.fromJSON(request.getParameter("BigPostParameter"))
P.S : FYI this line object = json.readValue(string, this.getClass());
is also very very very slow.
Again to summarize
Problem in posting time (stringify) JavaScript bottle nick.
Another problem parsing that stringified into an object (using jackson), and mainly I have svg tags content in that stringified object as a style column, and other columns are strings, int mainly
javascript json performance
javascript json performance
edited Dec 26 '18 at 5:47
priyanshi srivastava
938520
asked Aug 4 '17 at 19:01
shareefshareef
5,03474470
edited Dec 26 '18 at 5:47
priyanshi srivastava
938520
asked Aug 4 '17 at 19:01
shareefshareef
5,03474470
edited Dec 26 '18 at 5:47
priyanshi srivastava
938520
edited Dec 26 '18 at 5:47
priyanshi srivastava
938520
edited Dec 26 '18 at 5:47
priyanshi srivastava
938520
938520
asked Aug 4 '17 at 19:01
shareefshareef
5,03474470
asked Aug 4 '17 at 19:01
shareefshareef
5,03474470
asked Aug 4 '17 at 19:01
shareefshareef
5,03474470
5,03474470
1
How do you send it to a backend without converting it to JSON?
– csander
Aug 4 '17 at 19:03
JSON.stringify()is recursive. Why isJSON.stringify()call necessary? What is application and expected result?
– guest271314
Aug 4 '17 at 19:05
For serialization, you're going to be hard-pressed to find a faster alternative unless you look into designing or replicating a byte-encoded format. If you're hard-set on JSON format,JSON.stringify()is probably the fastest you'll get, though. There are other methods I know of that utilize streaming to be more memory efficient, but not faster.
– Patrick Roberts
Aug 4 '17 at 19:05
2
I'm inclined to say this is too broad, because the solution is to cache and/or break up the object, or both. There are no faster serializers: see github.com/kawanet/msgpack-lite
– Meirion Hughes
Aug 4 '17 at 19:43
1
@JohnWeisz Exactly that what we will do when we rewrite , it will be more dynamic for example after each change it will update some session tracking socket kind of . so we learn from our mistakes
– shareef
Aug 6 '17 at 14:04
|
show 14 more comments
1
How do you send it to a backend without converting it to JSON?
– csander
Aug 4 '17 at 19:03
JSON.stringify()is recursive. Why isJSON.stringify()call necessary? What is application and expected result?
– guest271314
Aug 4 '17 at 19:05
For serialization, you're going to be hard-pressed to find a faster alternative unless you look into designing or replicating a byte-encoded format. If you're hard-set on JSON format,JSON.stringify()is probably the fastest you'll get, though. There are other methods I know of that utilize streaming to be more memory efficient, but not faster.
– Patrick Roberts
Aug 4 '17 at 19:05
2
I'm inclined to say this is too broad, because the solution is to cache and/or break up the object, or both. There are no faster serializers: see github.com/kawanet/msgpack-lite
– Meirion Hughes
Aug 4 '17 at 19:43
1
@JohnWeisz Exactly that what we will do when we rewrite , it will be more dynamic for example after each change it will update some session tracking socket kind of . so we learn from our mistakes
– shareef
Aug 6 '17 at 14:04
1
1
How do you send it to a backend without converting it to JSON?
– csander
Aug 4 '17 at 19:03
How do you send it to a backend without converting it to JSON?
– csander
Aug 4 '17 at 19:03
JSON.stringify() is recursive. Why is JSON.stringify() call necessary? What is application and expected result?– guest271314
Aug 4 '17 at 19:05
JSON.stringify() is recursive. Why is JSON.stringify() call necessary? What is application and expected result?– guest271314
Aug 4 '17 at 19:05
For serialization, you're going to be hard-pressed to find a faster alternative unless you look into designing or replicating a byte-encoded format. If you're hard-set on JSON format,
JSON.stringify() is probably the fastest you'll get, though. There are other methods I know of that utilize streaming to be more memory efficient, but not faster.– Patrick Roberts
Aug 4 '17 at 19:05
For serialization, you're going to be hard-pressed to find a faster alternative unless you look into designing or replicating a byte-encoded format. If you're hard-set on JSON format,
JSON.stringify() is probably the fastest you'll get, though. There are other methods I know of that utilize streaming to be more memory efficient, but not faster.– Patrick Roberts
Aug 4 '17 at 19:05
2
2
I'm inclined to say this is too broad, because the solution is to cache and/or break up the object, or both. There are no faster serializers: see github.com/kawanet/msgpack-lite
– Meirion Hughes
Aug 4 '17 at 19:43
I'm inclined to say this is too broad, because the solution is to cache and/or break up the object, or both. There are no faster serializers: see github.com/kawanet/msgpack-lite
– Meirion Hughes
Aug 4 '17 at 19:43
1
1
@JohnWeisz Exactly that what we will do when we rewrite , it will be more dynamic for example after each change it will update some session tracking socket kind of . so we learn from our mistakes
– shareef
Aug 6 '17 at 14:04
@JohnWeisz Exactly that what we will do when we rewrite , it will be more dynamic for example after each change it will update some session tracking socket kind of . so we learn from our mistakes
– shareef
Aug 6 '17 at 14:04
|
show 14 more comments
1 Answer
1
active
oldest
votes
As commenters said - there is no way to make parsing faster.
If the concern is that the app is blocked while it's stringifying/parsing then try to split data into separate objects, stringily them and assemble back into one object before saving on the server.
If loading time of the app is not a problem you could try to ad-hoc incremental change on top of the existing app.
- ... App loading
- Load map data
- Make full copy of the data
- ... End loading
- ... App working without changes
- ... When saving changes
- diff copy with changed data to get JSON diff
- send changes (much smaller then full data)
- ... On server
- apply JSON diff changes on the server to the full data stored on server
- save changed data
I used json-diff https://github.com/andreyvit/json-diff to calc changes, and there are few analogs.
answered Jan 31 at 9:30
Wlodzislav K.Wlodzislav K.
30217
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f45513821%2fjson-stringify-is-very-slow-for-large-objects%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
As commenters said - there is no way to make parsing faster.
If the concern is that the app is blocked while it's stringifying/parsing then try to split data into separate objects, stringily them and assemble back into one object before saving on the server.
If loading time of the app is not a problem you could try to ad-hoc incremental change on top of the existing app.
- ... App loading
- Load map data
- Make full copy of the data
- ... End loading
- ... App working without changes
- ... When saving changes
- diff copy with changed data to get JSON diff
- send changes (much smaller then full data)
- ... On server
- apply JSON diff changes on the server to the full data stored on server
- save changed data
I used json-diff https://github.com/andreyvit/json-diff to calc changes, and there are few analogs.
answered Jan 31 at 9:30
Wlodzislav K.Wlodzislav K.
30217
add a comment |
As commenters said - there is no way to make parsing faster.
If the concern is that the app is blocked while it's stringifying/parsing then try to split data into separate objects, stringily them and assemble back into one object before saving on the server.
If loading time of the app is not a problem you could try to ad-hoc incremental change on top of the existing app.
- ... App loading
- Load map data
- Make full copy of the data
- ... End loading
- ... App working without changes
- ... When saving changes
- diff copy with changed data to get JSON diff
- send changes (much smaller then full data)
- ... On server
- apply JSON diff changes on the server to the full data stored on server
- save changed data
I used json-diff https://github.com/andreyvit/json-diff to calc changes, and there are few analogs.
answered Jan 31 at 9:30
Wlodzislav K.Wlodzislav K.
30217
add a comment |
As commenters said - there is no way to make parsing faster.
If the concern is that the app is blocked while it's stringifying/parsing then try to split data into separate objects, stringily them and assemble back into one object before saving on the server.
If loading time of the app is not a problem you could try to ad-hoc incremental change on top of the existing app.
- ... App loading
- Load map data
- Make full copy of the data
- ... End loading
- ... App working without changes
- ... When saving changes
- diff copy with changed data to get JSON diff
- send changes (much smaller then full data)
- ... On server
- apply JSON diff changes on the server to the full data stored on server
- save changed data
I used json-diff https://github.com/andreyvit/json-diff to calc changes, and there are few analogs.
answered Jan 31 at 9:30
Wlodzislav K.Wlodzislav K.
30217
As commenters said - there is no way to make parsing faster.
If the concern is that the app is blocked while it's stringifying/parsing then try to split data into separate objects, stringily them and assemble back into one object before saving on the server.
If loading time of the app is not a problem you could try to ad-hoc incremental change on top of the existing app.
- ... App loading
- Load map data
- Make full copy of the data
- ... End loading
- ... App working without changes
- ... When saving changes
- diff copy with changed data to get JSON diff
- send changes (much smaller then full data)
- ... On server
- apply JSON diff changes on the server to the full data stored on server
- save changed data
I used json-diff https://github.com/andreyvit/json-diff to calc changes, and there are few analogs.
answered Jan 31 at 9:30
Wlodzislav K.Wlodzislav K.
30217
answered Jan 31 at 9:30
Wlodzislav K.Wlodzislav K.
30217
answered Jan 31 at 9:30
Wlodzislav K.Wlodzislav K.
30217
answered Jan 31 at 9:30
Wlodzislav K.Wlodzislav K.
30217
30217
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f45513821%2fjson-stringify-is-very-slow-for-large-objects%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
How do you send it to a backend without converting it to JSON?
– csander
Aug 4 '17 at 19:03
JSON.stringify()is recursive. Why isJSON.stringify()call necessary? What is application and expected result?– guest271314
Aug 4 '17 at 19:05
For serialization, you're going to be hard-pressed to find a faster alternative unless you look into designing or replicating a byte-encoded format. If you're hard-set on JSON format,
JSON.stringify()is probably the fastest you'll get, though. There are other methods I know of that utilize streaming to be more memory efficient, but not faster.– Patrick Roberts
Aug 4 '17 at 19:05
2
I'm inclined to say this is too broad, because the solution is to cache and/or break up the object, or both. There are no faster serializers: see github.com/kawanet/msgpack-lite
– Meirion Hughes
Aug 4 '17 at 19:43
1
@JohnWeisz Exactly that what we will do when we rewrite , it will be more dynamic for example after each change it will update some session tracking socket kind of . so we learn from our mistakes
– shareef
Aug 6 '17 at 14:04