API Gateway stream large size content in response
Is there any way I can stream content in API response which is backed by AWS API gateway. My content can be very large size and i want to stream it to the requestor. At present i see there is a limit of 10Mb payload size on API Gateway.
I also generate the data at runtime when i get the request on my EC2 machine and as soon as some data is generated i want to start streaming it to the requestor.
Is it possible? How?
amazon-web-services aws-api-gateway
asked Nov 21 '18 at 11:58
iAviatoriAviator
260114
add a comment |
Is there any way I can stream content in API response which is backed by AWS API gateway. My content can be very large size and i want to stream it to the requestor. At present i see there is a limit of 10Mb payload size on API Gateway.
I also generate the data at runtime when i get the request on my EC2 machine and as soon as some data is generated i want to start streaming it to the requestor.
Is it possible? How?
amazon-web-services aws-api-gateway
asked Nov 21 '18 at 11:58
iAviatoriAviator
260114
add a comment |
Is there any way I can stream content in API response which is backed by AWS API gateway. My content can be very large size and i want to stream it to the requestor. At present i see there is a limit of 10Mb payload size on API Gateway.
I also generate the data at runtime when i get the request on my EC2 machine and as soon as some data is generated i want to start streaming it to the requestor.
Is it possible? How?
amazon-web-services aws-api-gateway
asked Nov 21 '18 at 11:58
iAviatoriAviator
260114
Is there any way I can stream content in API response which is backed by AWS API gateway. My content can be very large size and i want to stream it to the requestor. At present i see there is a limit of 10Mb payload size on API Gateway.
I also generate the data at runtime when i get the request on my EC2 machine and as soon as some data is generated i want to start streaming it to the requestor.
Is it possible? How?
amazon-web-services aws-api-gateway
amazon-web-services aws-api-gateway
asked Nov 21 '18 at 11:58
iAviatoriAviator
260114
asked Nov 21 '18 at 11:58
iAviatoriAviator
260114
asked Nov 21 '18 at 11:58
iAviatoriAviator
260114
asked Nov 21 '18 at 11:58
iAviatoriAviator
260114
asked Nov 21 '18 at 11:58
iAviatoriAviator
260114
260114
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
As you've seen, API Gateway has hard limits response sizes. This is because it's designed for quick and transactional use-cases. (API Gateway will also not keep a connection open longer than 30 seconds, so if you're streaming a file that takes longer than this to download, you'd be in trouble too.)
For these cases you might consider a different pattern, like:
- Have your EC2 machine upload the result to S3 and have API Gateway return a pre-signed url to download the response from S3. This would stream the download, but would have to wait for the EC2 -> S3 upload to complete first.
- Use Elastic Beanstalk, that way you would be in control of the server and able to keep your connections open for as long as you wanted, and send as much data as you want.
answered Nov 23 '18 at 10:52
thomasmichaelwallacethomasmichaelwallace
2,5351817
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53411571%2fapi-gateway-stream-large-size-content-in-response%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
As you've seen, API Gateway has hard limits response sizes. This is because it's designed for quick and transactional use-cases. (API Gateway will also not keep a connection open longer than 30 seconds, so if you're streaming a file that takes longer than this to download, you'd be in trouble too.)
For these cases you might consider a different pattern, like:
- Have your EC2 machine upload the result to S3 and have API Gateway return a pre-signed url to download the response from S3. This would stream the download, but would have to wait for the EC2 -> S3 upload to complete first.
- Use Elastic Beanstalk, that way you would be in control of the server and able to keep your connections open for as long as you wanted, and send as much data as you want.
answered Nov 23 '18 at 10:52
thomasmichaelwallacethomasmichaelwallace
2,5351817
add a comment |
As you've seen, API Gateway has hard limits response sizes. This is because it's designed for quick and transactional use-cases. (API Gateway will also not keep a connection open longer than 30 seconds, so if you're streaming a file that takes longer than this to download, you'd be in trouble too.)
For these cases you might consider a different pattern, like:
- Have your EC2 machine upload the result to S3 and have API Gateway return a pre-signed url to download the response from S3. This would stream the download, but would have to wait for the EC2 -> S3 upload to complete first.
- Use Elastic Beanstalk, that way you would be in control of the server and able to keep your connections open for as long as you wanted, and send as much data as you want.
answered Nov 23 '18 at 10:52
thomasmichaelwallacethomasmichaelwallace
2,5351817
add a comment |
As you've seen, API Gateway has hard limits response sizes. This is because it's designed for quick and transactional use-cases. (API Gateway will also not keep a connection open longer than 30 seconds, so if you're streaming a file that takes longer than this to download, you'd be in trouble too.)
For these cases you might consider a different pattern, like:
- Have your EC2 machine upload the result to S3 and have API Gateway return a pre-signed url to download the response from S3. This would stream the download, but would have to wait for the EC2 -> S3 upload to complete first.
- Use Elastic Beanstalk, that way you would be in control of the server and able to keep your connections open for as long as you wanted, and send as much data as you want.
answered Nov 23 '18 at 10:52
thomasmichaelwallacethomasmichaelwallace
2,5351817
As you've seen, API Gateway has hard limits response sizes. This is because it's designed for quick and transactional use-cases. (API Gateway will also not keep a connection open longer than 30 seconds, so if you're streaming a file that takes longer than this to download, you'd be in trouble too.)
For these cases you might consider a different pattern, like:
- Have your EC2 machine upload the result to S3 and have API Gateway return a pre-signed url to download the response from S3. This would stream the download, but would have to wait for the EC2 -> S3 upload to complete first.
- Use Elastic Beanstalk, that way you would be in control of the server and able to keep your connections open for as long as you wanted, and send as much data as you want.
answered Nov 23 '18 at 10:52
thomasmichaelwallacethomasmichaelwallace
2,5351817
answered Nov 23 '18 at 10:52
thomasmichaelwallacethomasmichaelwallace
2,5351817
answered Nov 23 '18 at 10:52
thomasmichaelwallacethomasmichaelwallace
2,5351817
answered Nov 23 '18 at 10:52
thomasmichaelwallacethomasmichaelwallace
2,5351817
2,5351817
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53411571%2fapi-gateway-stream-large-size-content-in-response%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
