Are Node.js Buffers as JSON a portable storage format?
If I create Node.js Buffer containing the bytes of a binary file like a jpg -image, convert it to JSON, can I transport binary content in this way to other machines and have the images viewable on those other machines too?
In other words can I fill a Buffer on one machine with bytes of an image-file, and transport the buffer as JSON to another machine, then restore the image there by simply writing the same buffer to a file with the same name?
Would it work between platforms say Linux Windows and Mac? Does "endiannes" become an issue?
Would TypedArrays be a better solution?
node.js encoding buffer portability
asked Nov 25 '18 at 20:48
Panu LogicPanu Logic
676513
add a comment |
If I create Node.js Buffer containing the bytes of a binary file like a jpg -image, convert it to JSON, can I transport binary content in this way to other machines and have the images viewable on those other machines too?
In other words can I fill a Buffer on one machine with bytes of an image-file, and transport the buffer as JSON to another machine, then restore the image there by simply writing the same buffer to a file with the same name?
Would it work between platforms say Linux Windows and Mac? Does "endiannes" become an issue?
Would TypedArrays be a better solution?
node.js encoding buffer portability
asked Nov 25 '18 at 20:48
Panu LogicPanu Logic
676513
1
If it's just an "image" such as a JPG as stated, then diffrerent OS implementations and processor architectures do not care about such things. The worry would be executable content, where it does make a difference. As noted, the simplest solution is generally just to send the file, and common transfer protocols handle the how level stuff like byte order for you. For more complex data structures, then a standardized binary format in network byte order is generally preferred.
– Neil Lunn
Nov 25 '18 at 22:39
add a comment |
If I create Node.js Buffer containing the bytes of a binary file like a jpg -image, convert it to JSON, can I transport binary content in this way to other machines and have the images viewable on those other machines too?
In other words can I fill a Buffer on one machine with bytes of an image-file, and transport the buffer as JSON to another machine, then restore the image there by simply writing the same buffer to a file with the same name?
Would it work between platforms say Linux Windows and Mac? Does "endiannes" become an issue?
Would TypedArrays be a better solution?
node.js encoding buffer portability
asked Nov 25 '18 at 20:48
Panu LogicPanu Logic
676513
If I create Node.js Buffer containing the bytes of a binary file like a jpg -image, convert it to JSON, can I transport binary content in this way to other machines and have the images viewable on those other machines too?
In other words can I fill a Buffer on one machine with bytes of an image-file, and transport the buffer as JSON to another machine, then restore the image there by simply writing the same buffer to a file with the same name?
Would it work between platforms say Linux Windows and Mac? Does "endiannes" become an issue?
Would TypedArrays be a better solution?
node.js encoding buffer portability
node.js encoding buffer portability
asked Nov 25 '18 at 20:48
Panu LogicPanu Logic
676513
asked Nov 25 '18 at 20:48
Panu LogicPanu Logic
676513
asked Nov 25 '18 at 20:48
Panu LogicPanu Logic
676513
asked Nov 25 '18 at 20:48
Panu LogicPanu Logic
676513
asked Nov 25 '18 at 20:48
Panu LogicPanu Logic
676513
676513
1
If it's just an "image" such as a JPG as stated, then diffrerent OS implementations and processor architectures do not care about such things. The worry would be executable content, where it does make a difference. As noted, the simplest solution is generally just to send the file, and common transfer protocols handle the how level stuff like byte order for you. For more complex data structures, then a standardized binary format in network byte order is generally preferred.
– Neil Lunn
Nov 25 '18 at 22:39
add a comment |
1
If it's just an "image" such as a JPG as stated, then diffrerent OS implementations and processor architectures do not care about such things. The worry would be executable content, where it does make a difference. As noted, the simplest solution is generally just to send the file, and common transfer protocols handle the how level stuff like byte order for you. For more complex data structures, then a standardized binary format in network byte order is generally preferred.
– Neil Lunn
Nov 25 '18 at 22:39
1
1
If it's just an "image" such as a JPG as stated, then diffrerent OS implementations and processor architectures do not care about such things. The worry would be executable content, where it does make a difference. As noted, the simplest solution is generally just to send the file, and common transfer protocols handle the how level stuff like byte order for you. For more complex data structures, then a standardized binary format in network byte order is generally preferred.
– Neil Lunn
Nov 25 '18 at 22:39
If it's just an "image" such as a JPG as stated, then diffrerent OS implementations and processor architectures do not care about such things. The worry would be executable content, where it does make a difference. As noted, the simplest solution is generally just to send the file, and common transfer protocols handle the how level stuff like byte order for you. For more complex data structures, then a standardized binary format in network byte order is generally preferred.
– Neil Lunn
Nov 25 '18 at 22:39
add a comment |
1 Answer
1
active
oldest
votes
JSON isn't useful for transferring binary data... at least, not efficiently. You would have to base64-encode the data before putting it in JSON, which increases its size by 33% and adds an extra layer of processing on each end.
There is another standard serialization format you can use called CBOR. It's binary in nature and supports a byte string. There are libraries for many languages.
answered Nov 25 '18 at 20:55
BradBrad
116k28236396
Thanks for the link on CBOR. What if efficiency is not a concern? Say I need to do it just for a few small images, occasionally? My concern is more portability of the images and other binary content, than efficiency
– Panu Logic
Nov 25 '18 at 21:04
1
@PanuLogic The best thing to do is just to send the file itself. If you can't do that and must send some sort of structured data, CBOR is incredibly well supported. It's been standardized for years (tools.ietf.org/html/rfc7049), and Wikipedia lists many implementations (en.wikipedia.org/wiki/CBOR#Implementations).
– Brad
Nov 25 '18 at 21:06
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53471818%2fare-node-js-buffers-as-json-a-portable-storage-format%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
JSON isn't useful for transferring binary data... at least, not efficiently. You would have to base64-encode the data before putting it in JSON, which increases its size by 33% and adds an extra layer of processing on each end.
There is another standard serialization format you can use called CBOR. It's binary in nature and supports a byte string. There are libraries for many languages.
answered Nov 25 '18 at 20:55
BradBrad
116k28236396
Thanks for the link on CBOR. What if efficiency is not a concern? Say I need to do it just for a few small images, occasionally? My concern is more portability of the images and other binary content, than efficiency
– Panu Logic
Nov 25 '18 at 21:04
1
@PanuLogic The best thing to do is just to send the file itself. If you can't do that and must send some sort of structured data, CBOR is incredibly well supported. It's been standardized for years (tools.ietf.org/html/rfc7049), and Wikipedia lists many implementations (en.wikipedia.org/wiki/CBOR#Implementations).
– Brad
Nov 25 '18 at 21:06
add a comment |
JSON isn't useful for transferring binary data... at least, not efficiently. You would have to base64-encode the data before putting it in JSON, which increases its size by 33% and adds an extra layer of processing on each end.
There is another standard serialization format you can use called CBOR. It's binary in nature and supports a byte string. There are libraries for many languages.
answered Nov 25 '18 at 20:55
BradBrad
116k28236396
Thanks for the link on CBOR. What if efficiency is not a concern? Say I need to do it just for a few small images, occasionally? My concern is more portability of the images and other binary content, than efficiency
– Panu Logic
Nov 25 '18 at 21:04
1
@PanuLogic The best thing to do is just to send the file itself. If you can't do that and must send some sort of structured data, CBOR is incredibly well supported. It's been standardized for years (tools.ietf.org/html/rfc7049), and Wikipedia lists many implementations (en.wikipedia.org/wiki/CBOR#Implementations).
– Brad
Nov 25 '18 at 21:06
add a comment |
JSON isn't useful for transferring binary data... at least, not efficiently. You would have to base64-encode the data before putting it in JSON, which increases its size by 33% and adds an extra layer of processing on each end.
There is another standard serialization format you can use called CBOR. It's binary in nature and supports a byte string. There are libraries for many languages.
answered Nov 25 '18 at 20:55
BradBrad
116k28236396
JSON isn't useful for transferring binary data... at least, not efficiently. You would have to base64-encode the data before putting it in JSON, which increases its size by 33% and adds an extra layer of processing on each end.
There is another standard serialization format you can use called CBOR. It's binary in nature and supports a byte string. There are libraries for many languages.
answered Nov 25 '18 at 20:55
BradBrad
116k28236396
answered Nov 25 '18 at 20:55
BradBrad
116k28236396
answered Nov 25 '18 at 20:55
BradBrad
116k28236396
answered Nov 25 '18 at 20:55
BradBrad
116k28236396
116k28236396
Thanks for the link on CBOR. What if efficiency is not a concern? Say I need to do it just for a few small images, occasionally? My concern is more portability of the images and other binary content, than efficiency
– Panu Logic
Nov 25 '18 at 21:04
1
@PanuLogic The best thing to do is just to send the file itself. If you can't do that and must send some sort of structured data, CBOR is incredibly well supported. It's been standardized for years (tools.ietf.org/html/rfc7049), and Wikipedia lists many implementations (en.wikipedia.org/wiki/CBOR#Implementations).
– Brad
Nov 25 '18 at 21:06
add a comment |
Thanks for the link on CBOR. What if efficiency is not a concern? Say I need to do it just for a few small images, occasionally? My concern is more portability of the images and other binary content, than efficiency
– Panu Logic
Nov 25 '18 at 21:04
1
@PanuLogic The best thing to do is just to send the file itself. If you can't do that and must send some sort of structured data, CBOR is incredibly well supported. It's been standardized for years (tools.ietf.org/html/rfc7049), and Wikipedia lists many implementations (en.wikipedia.org/wiki/CBOR#Implementations).
– Brad
Nov 25 '18 at 21:06
Thanks for the link on CBOR. What if efficiency is not a concern? Say I need to do it just for a few small images, occasionally? My concern is more portability of the images and other binary content, than efficiency
– Panu Logic
Nov 25 '18 at 21:04
Thanks for the link on CBOR. What if efficiency is not a concern? Say I need to do it just for a few small images, occasionally? My concern is more portability of the images and other binary content, than efficiency
– Panu Logic
Nov 25 '18 at 21:04
1
1
@PanuLogic The best thing to do is just to send the file itself. If you can't do that and must send some sort of structured data, CBOR is incredibly well supported. It's been standardized for years (tools.ietf.org/html/rfc7049), and Wikipedia lists many implementations (en.wikipedia.org/wiki/CBOR#Implementations).
– Brad
Nov 25 '18 at 21:06
@PanuLogic The best thing to do is just to send the file itself. If you can't do that and must send some sort of structured data, CBOR is incredibly well supported. It's been standardized for years (tools.ietf.org/html/rfc7049), and Wikipedia lists many implementations (en.wikipedia.org/wiki/CBOR#Implementations).
– Brad
Nov 25 '18 at 21:06
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53471818%2fare-node-js-buffers-as-json-a-portable-storage-format%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
If it's just an "image" such as a JPG as stated, then diffrerent OS implementations and processor architectures do not care about such things. The worry would be executable content, where it does make a difference. As noted, the simplest solution is generally just to send the file, and common transfer protocols handle the how level stuff like byte order for you. For more complex data structures, then a standardized binary format in network byte order is generally preferred.
– Neil Lunn
Nov 25 '18 at 22:39