AWS API Gateway file upload fails
I have a .NET Core website and a .NET Core Web API. On the website you can upload an image file which then gets posted to the API. The API then uploads this image to AWS S3 for storage.
The website is hosted on an EC2 instance with IIS and the API is an AWS Serverless Lambda application (https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/lambda-build-test-severless-app.html)
This is working, however I am encountering issues when the file size is large. For example, a 1 MB image will upload without any problems but a 5MB file will fail, seemingly before it even reaches the API code. I say this because I have logging on the first line of my Controller method and nothing is logged for the larger file. I am unable to provide much more information unfortunately because this is only happening in the live environment, when I run locally it works fine, regardless of file size.
I know the payload size limit for API Gateway is 10MB as shown here - https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html. Also, the default timeout for API Gateway requests is 29000ms and it does not hang for nearly that long when I try to upload an image so I don't think it's hitting this timeout.
amazon-web-services asp.net-core aws-lambda aws-api-gateway
asked Nov 20 '18 at 22:50
Andy Furniss
1,1711822
add a comment |
I have a .NET Core website and a .NET Core Web API. On the website you can upload an image file which then gets posted to the API. The API then uploads this image to AWS S3 for storage.
The website is hosted on an EC2 instance with IIS and the API is an AWS Serverless Lambda application (https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/lambda-build-test-severless-app.html)
This is working, however I am encountering issues when the file size is large. For example, a 1 MB image will upload without any problems but a 5MB file will fail, seemingly before it even reaches the API code. I say this because I have logging on the first line of my Controller method and nothing is logged for the larger file. I am unable to provide much more information unfortunately because this is only happening in the live environment, when I run locally it works fine, regardless of file size.
I know the payload size limit for API Gateway is 10MB as shown here - https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html. Also, the default timeout for API Gateway requests is 29000ms and it does not hang for nearly that long when I try to upload an image so I don't think it's hitting this timeout.
amazon-web-services asp.net-core aws-lambda aws-api-gateway
asked Nov 20 '18 at 22:50
Andy Furniss
1,1711822
1
You should check your logs to see what part of the stack are or are not being invoked. I'd start with Lambda, as I suspect your hitting the synchronous invocation payload limit of 6MB there. If you are sending a binary file, remember that API Gateway will convert that to a base64 file when forwarding to Lambda, and that will increase the size of the file by approx. a third.
– K Mo
Nov 21 '18 at 8:19
Ahh, I was not aware of this limit. I think I will just add a size limit of around 2MB to avoid this in future. Thank you very much for your help.
– Andy Furniss
Nov 21 '18 at 10:39
add a comment |
I have a .NET Core website and a .NET Core Web API. On the website you can upload an image file which then gets posted to the API. The API then uploads this image to AWS S3 for storage.
The website is hosted on an EC2 instance with IIS and the API is an AWS Serverless Lambda application (https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/lambda-build-test-severless-app.html)
This is working, however I am encountering issues when the file size is large. For example, a 1 MB image will upload without any problems but a 5MB file will fail, seemingly before it even reaches the API code. I say this because I have logging on the first line of my Controller method and nothing is logged for the larger file. I am unable to provide much more information unfortunately because this is only happening in the live environment, when I run locally it works fine, regardless of file size.
I know the payload size limit for API Gateway is 10MB as shown here - https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html. Also, the default timeout for API Gateway requests is 29000ms and it does not hang for nearly that long when I try to upload an image so I don't think it's hitting this timeout.
amazon-web-services asp.net-core aws-lambda aws-api-gateway
asked Nov 20 '18 at 22:50
Andy Furniss
1,1711822
I have a .NET Core website and a .NET Core Web API. On the website you can upload an image file which then gets posted to the API. The API then uploads this image to AWS S3 for storage.
The website is hosted on an EC2 instance with IIS and the API is an AWS Serverless Lambda application (https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/lambda-build-test-severless-app.html)
This is working, however I am encountering issues when the file size is large. For example, a 1 MB image will upload without any problems but a 5MB file will fail, seemingly before it even reaches the API code. I say this because I have logging on the first line of my Controller method and nothing is logged for the larger file. I am unable to provide much more information unfortunately because this is only happening in the live environment, when I run locally it works fine, regardless of file size.
I know the payload size limit for API Gateway is 10MB as shown here - https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html. Also, the default timeout for API Gateway requests is 29000ms and it does not hang for nearly that long when I try to upload an image so I don't think it's hitting this timeout.
amazon-web-services asp.net-core aws-lambda aws-api-gateway
amazon-web-services asp.net-core aws-lambda aws-api-gateway
asked Nov 20 '18 at 22:50
Andy Furniss
1,1711822
asked Nov 20 '18 at 22:50
Andy Furniss
1,1711822
asked Nov 20 '18 at 22:50
Andy Furniss
1,1711822
asked Nov 20 '18 at 22:50
Andy Furniss
1,1711822
asked Nov 20 '18 at 22:50
Andy Furniss
1,1711822
1,1711822
1
You should check your logs to see what part of the stack are or are not being invoked. I'd start with Lambda, as I suspect your hitting the synchronous invocation payload limit of 6MB there. If you are sending a binary file, remember that API Gateway will convert that to a base64 file when forwarding to Lambda, and that will increase the size of the file by approx. a third.
– K Mo
Nov 21 '18 at 8:19
Ahh, I was not aware of this limit. I think I will just add a size limit of around 2MB to avoid this in future. Thank you very much for your help.
– Andy Furniss
Nov 21 '18 at 10:39
add a comment |
1
You should check your logs to see what part of the stack are or are not being invoked. I'd start with Lambda, as I suspect your hitting the synchronous invocation payload limit of 6MB there. If you are sending a binary file, remember that API Gateway will convert that to a base64 file when forwarding to Lambda, and that will increase the size of the file by approx. a third.
– K Mo
Nov 21 '18 at 8:19
Ahh, I was not aware of this limit. I think I will just add a size limit of around 2MB to avoid this in future. Thank you very much for your help.
– Andy Furniss
Nov 21 '18 at 10:39
1
1
You should check your logs to see what part of the stack are or are not being invoked. I'd start with Lambda, as I suspect your hitting the synchronous invocation payload limit of 6MB there. If you are sending a binary file, remember that API Gateway will convert that to a base64 file when forwarding to Lambda, and that will increase the size of the file by approx. a third.
– K Mo
Nov 21 '18 at 8:19
You should check your logs to see what part of the stack are or are not being invoked. I'd start with Lambda, as I suspect your hitting the synchronous invocation payload limit of 6MB there. If you are sending a binary file, remember that API Gateway will convert that to a base64 file when forwarding to Lambda, and that will increase the size of the file by approx. a third.
– K Mo
Nov 21 '18 at 8:19
Ahh, I was not aware of this limit. I think I will just add a size limit of around 2MB to avoid this in future. Thank you very much for your help.
– Andy Furniss
Nov 21 '18 at 10:39
Ahh, I was not aware of this limit. I think I will just add a size limit of around 2MB to avoid this in future. Thank you very much for your help.
– Andy Furniss
Nov 21 '18 at 10:39
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53402772%2faws-api-gateway-file-upload-fails%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53402772%2faws-api-gateway-file-upload-fails%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
You should check your logs to see what part of the stack are or are not being invoked. I'd start with Lambda, as I suspect your hitting the synchronous invocation payload limit of 6MB there. If you are sending a binary file, remember that API Gateway will convert that to a base64 file when forwarding to Lambda, and that will increase the size of the file by approx. a third.
– K Mo
Nov 21 '18 at 8:19
Ahh, I was not aware of this limit. I think I will just add a size limit of around 2MB to avoid this in future. Thank you very much for your help.
– Andy Furniss
Nov 21 '18 at 10:39