Response time between Elasticsearch and Apache on Google Cloud lagspikes
I have an Elasticsearch server with data in it and a Compute engine with a LAMP stack running on it.
Both servers:
Elastic. europe-west1-d - Compute Engine / 6 vCPU's, 34 GB Mem
Lamp. europe-west4-c - Compute Engine / 4 vCPU's, 23 GB Mem
Both servers never reach anywhere near 20% of their resource limits and they are super fast most of the time.
But sometimes, for a few minutes the connection between the two slows down to 42 second requests.
- The website shows in 200 ms, but the products that come from Elastic search using an AJAX requests keeps loading for 42 seconds
- If I go to Kibana the same query, or different queries take 35ms
So I cant find anything wrong on any of the servers resource wise, but could it be a Firewall problem, or maybe a DNS / datacenter thing. Does any1 have any clue as to what could be causing this? Feel free to pitch even the most farfetched ideas.
elasticsearch networking google-cloud-platform google-compute-engine
asked Nov 22 '18 at 9:03
Hans WassinkHans Wassink
1,36431838
add a comment |
I have an Elasticsearch server with data in it and a Compute engine with a LAMP stack running on it.
Both servers:
Elastic. europe-west1-d - Compute Engine / 6 vCPU's, 34 GB Mem
Lamp. europe-west4-c - Compute Engine / 4 vCPU's, 23 GB Mem
Both servers never reach anywhere near 20% of their resource limits and they are super fast most of the time.
But sometimes, for a few minutes the connection between the two slows down to 42 second requests.
- The website shows in 200 ms, but the products that come from Elastic search using an AJAX requests keeps loading for 42 seconds
- If I go to Kibana the same query, or different queries take 35ms
So I cant find anything wrong on any of the servers resource wise, but could it be a Firewall problem, or maybe a DNS / datacenter thing. Does any1 have any clue as to what could be causing this? Feel free to pitch even the most farfetched ideas.
elasticsearch networking google-cloud-platform google-compute-engine
asked Nov 22 '18 at 9:03
Hans WassinkHans Wassink
1,36431838
I don't think this could be caused by any firewall. Firewall won't slow down the connection, will make it either work or not work.You can do MTR to see where the high latency is taking place in the traffic path. For more troubleshooting, you can also do iperf test to measure latency and the effective packet throughput between VMs.
– Md Zubayer
Nov 22 '18 at 22:27
add a comment |
I have an Elasticsearch server with data in it and a Compute engine with a LAMP stack running on it.
Both servers:
Elastic. europe-west1-d - Compute Engine / 6 vCPU's, 34 GB Mem
Lamp. europe-west4-c - Compute Engine / 4 vCPU's, 23 GB Mem
Both servers never reach anywhere near 20% of their resource limits and they are super fast most of the time.
But sometimes, for a few minutes the connection between the two slows down to 42 second requests.
- The website shows in 200 ms, but the products that come from Elastic search using an AJAX requests keeps loading for 42 seconds
- If I go to Kibana the same query, or different queries take 35ms
So I cant find anything wrong on any of the servers resource wise, but could it be a Firewall problem, or maybe a DNS / datacenter thing. Does any1 have any clue as to what could be causing this? Feel free to pitch even the most farfetched ideas.
elasticsearch networking google-cloud-platform google-compute-engine
asked Nov 22 '18 at 9:03
Hans WassinkHans Wassink
1,36431838
I have an Elasticsearch server with data in it and a Compute engine with a LAMP stack running on it.
Both servers:
Elastic. europe-west1-d - Compute Engine / 6 vCPU's, 34 GB Mem
Lamp. europe-west4-c - Compute Engine / 4 vCPU's, 23 GB Mem
Both servers never reach anywhere near 20% of their resource limits and they are super fast most of the time.
But sometimes, for a few minutes the connection between the two slows down to 42 second requests.
- The website shows in 200 ms, but the products that come from Elastic search using an AJAX requests keeps loading for 42 seconds
- If I go to Kibana the same query, or different queries take 35ms
So I cant find anything wrong on any of the servers resource wise, but could it be a Firewall problem, or maybe a DNS / datacenter thing. Does any1 have any clue as to what could be causing this? Feel free to pitch even the most farfetched ideas.
elasticsearch networking google-cloud-platform google-compute-engine
elasticsearch networking google-cloud-platform google-compute-engine
asked Nov 22 '18 at 9:03
Hans WassinkHans Wassink
1,36431838
asked Nov 22 '18 at 9:03
Hans WassinkHans Wassink
1,36431838
asked Nov 22 '18 at 9:03
Hans WassinkHans Wassink
1,36431838
asked Nov 22 '18 at 9:03
Hans WassinkHans Wassink
1,36431838
asked Nov 22 '18 at 9:03
Hans WassinkHans Wassink
1,36431838
1,36431838
I don't think this could be caused by any firewall. Firewall won't slow down the connection, will make it either work or not work.You can do MTR to see where the high latency is taking place in the traffic path. For more troubleshooting, you can also do iperf test to measure latency and the effective packet throughput between VMs.
– Md Zubayer
Nov 22 '18 at 22:27
add a comment |
I don't think this could be caused by any firewall. Firewall won't slow down the connection, will make it either work or not work.You can do MTR to see where the high latency is taking place in the traffic path. For more troubleshooting, you can also do iperf test to measure latency and the effective packet throughput between VMs.
– Md Zubayer
Nov 22 '18 at 22:27
I don't think this could be caused by any firewall. Firewall won't slow down the connection, will make it either work or not work.You can do MTR to see where the high latency is taking place in the traffic path. For more troubleshooting, you can also do iperf test to measure latency and the effective packet throughput between VMs.
– Md Zubayer
Nov 22 '18 at 22:27
I don't think this could be caused by any firewall. Firewall won't slow down the connection, will make it either work or not work.You can do MTR to see where the high latency is taking place in the traffic path. For more troubleshooting, you can also do iperf test to measure latency and the effective packet throughput between VMs.
– Md Zubayer
Nov 22 '18 at 22:27
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53427221%2fresponse-time-between-elasticsearch-and-apache-on-google-cloud-lagspikes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53427221%2fresponse-time-between-elasticsearch-and-apache-on-google-cloud-lagspikes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I don't think this could be caused by any firewall. Firewall won't slow down the connection, will make it either work or not work.You can do MTR to see where the high latency is taking place in the traffic path. For more troubleshooting, you can also do iperf test to measure latency and the effective packet throughput between VMs.
– Md Zubayer
Nov 22 '18 at 22:27