How to create rolling logs for Filebeat within a docker container
I'm new to log4j2 and the elastic stack.
I have a filebeat docker container that doesn't work exactly how I want and now I want to take a look at the logs. But when I do docker-compose logs I get a lot debug messages and json objects. It's unreadable how much there is.
How can I create a log4j2 properties setup to create some rolling log files. Maybe put the old logs into a monthly based folder or something? and where do I put this log4j2.properties file?
docker logging log4j2 elastic-stack filebeat
asked Oct 17 '18 at 12:32
JeggyJeggy
262521
add a comment |
I'm new to log4j2 and the elastic stack.
I have a filebeat docker container that doesn't work exactly how I want and now I want to take a look at the logs. But when I do docker-compose logs I get a lot debug messages and json objects. It's unreadable how much there is.
How can I create a log4j2 properties setup to create some rolling log files. Maybe put the old logs into a monthly based folder or something? and where do I put this log4j2.properties file?
docker logging log4j2 elastic-stack filebeat
asked Oct 17 '18 at 12:32
JeggyJeggy
262521
Did you manage to try out the answer below or confirm a solution of your own?
– Alexandre Juma
Nov 30 '18 at 10:20
add a comment |
I'm new to log4j2 and the elastic stack.
I have a filebeat docker container that doesn't work exactly how I want and now I want to take a look at the logs. But when I do docker-compose logs I get a lot debug messages and json objects. It's unreadable how much there is.
How can I create a log4j2 properties setup to create some rolling log files. Maybe put the old logs into a monthly based folder or something? and where do I put this log4j2.properties file?
docker logging log4j2 elastic-stack filebeat
asked Oct 17 '18 at 12:32
JeggyJeggy
262521
I'm new to log4j2 and the elastic stack.
I have a filebeat docker container that doesn't work exactly how I want and now I want to take a look at the logs. But when I do docker-compose logs I get a lot debug messages and json objects. It's unreadable how much there is.
How can I create a log4j2 properties setup to create some rolling log files. Maybe put the old logs into a monthly based folder or something? and where do I put this log4j2.properties file?
docker logging log4j2 elastic-stack filebeat
docker logging log4j2 elastic-stack filebeat
asked Oct 17 '18 at 12:32
JeggyJeggy
262521
asked Oct 17 '18 at 12:32
JeggyJeggy
262521
edited Oct 17 '18 at 12:40
Jeggy
asked Oct 17 '18 at 12:32
JeggyJeggy
262521
asked Oct 17 '18 at 12:32
JeggyJeggy
262521
asked Oct 17 '18 at 12:32
JeggyJeggy
262521
262521
Did you manage to try out the answer below or confirm a solution of your own?
– Alexandre Juma
Nov 30 '18 at 10:20
add a comment |
Did you manage to try out the answer below or confirm a solution of your own?
– Alexandre Juma
Nov 30 '18 at 10:20
Did you manage to try out the answer below or confirm a solution of your own?
– Alexandre Juma
Nov 30 '18 at 10:20
Did you manage to try out the answer below or confirm a solution of your own?
– Alexandre Juma
Nov 30 '18 at 10:20
add a comment |
1 Answer
1
active
oldest
votes
It's generating a lot of logs because you're running docker-compose logs, which will get the logs for all containers in your docker compose file.
What you want is probably:
docker logs <name-of-filebeat-container>. The name of the filebeat container can be found doing adocker ps.
docker compose logs <name-of-filebeat-service>. The name of the service can be found on your docker-composer.yml file.
Regarding the JSON outputs, you can query your Docker engine default logging driver with:
# docker info | grep 'Logging Driver'
Logging Driver: json-file
If your container have a different Logging Driver you can check with:
docker inspect -f '{{.HostConfig.LogConfig.Type}}' <name-or-id-of-the-container>
You can find all log drivers in this link
To run containers with a different log-driver you can do:
- With docker run:
docker run -it --log-driver <log-driver> alpine ash
With docker-compose:
`logging:
driver: syslog
options:
syslog-address: "tcp://192.168.0.42:123"`
Regarding your log rotation questio, I'd say the easyest way is to configure the logging driver with the syslog driver, configure it to your local machine (or your syslog server) and then logrotate the files.
You can find several logrotate articles for Linux (which I assume you're using), for example this one
answered Nov 22 '18 at 10:21
Alexandre JumaAlexandre Juma
544214
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52854955%2fhow-to-create-rolling-logs-for-filebeat-within-a-docker-container%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
It's generating a lot of logs because you're running docker-compose logs, which will get the logs for all containers in your docker compose file.
What you want is probably:
docker logs <name-of-filebeat-container>. The name of the filebeat container can be found doing adocker ps.
docker compose logs <name-of-filebeat-service>. The name of the service can be found on your docker-composer.yml file.
Regarding the JSON outputs, you can query your Docker engine default logging driver with:
# docker info | grep 'Logging Driver'
Logging Driver: json-file
If your container have a different Logging Driver you can check with:
docker inspect -f '{{.HostConfig.LogConfig.Type}}' <name-or-id-of-the-container>
You can find all log drivers in this link
To run containers with a different log-driver you can do:
- With docker run:
docker run -it --log-driver <log-driver> alpine ash
With docker-compose:
`logging:
driver: syslog
options:
syslog-address: "tcp://192.168.0.42:123"`
Regarding your log rotation questio, I'd say the easyest way is to configure the logging driver with the syslog driver, configure it to your local machine (or your syslog server) and then logrotate the files.
You can find several logrotate articles for Linux (which I assume you're using), for example this one
answered Nov 22 '18 at 10:21
Alexandre JumaAlexandre Juma
544214
add a comment |
It's generating a lot of logs because you're running docker-compose logs, which will get the logs for all containers in your docker compose file.
What you want is probably:
docker logs <name-of-filebeat-container>. The name of the filebeat container can be found doing adocker ps.
docker compose logs <name-of-filebeat-service>. The name of the service can be found on your docker-composer.yml file.
Regarding the JSON outputs, you can query your Docker engine default logging driver with:
# docker info | grep 'Logging Driver'
Logging Driver: json-file
If your container have a different Logging Driver you can check with:
docker inspect -f '{{.HostConfig.LogConfig.Type}}' <name-or-id-of-the-container>
You can find all log drivers in this link
To run containers with a different log-driver you can do:
- With docker run:
docker run -it --log-driver <log-driver> alpine ash
With docker-compose:
`logging:
driver: syslog
options:
syslog-address: "tcp://192.168.0.42:123"`
Regarding your log rotation questio, I'd say the easyest way is to configure the logging driver with the syslog driver, configure it to your local machine (or your syslog server) and then logrotate the files.
You can find several logrotate articles for Linux (which I assume you're using), for example this one
answered Nov 22 '18 at 10:21
Alexandre JumaAlexandre Juma
544214
add a comment |
It's generating a lot of logs because you're running docker-compose logs, which will get the logs for all containers in your docker compose file.
What you want is probably:
docker logs <name-of-filebeat-container>. The name of the filebeat container can be found doing adocker ps.
docker compose logs <name-of-filebeat-service>. The name of the service can be found on your docker-composer.yml file.
Regarding the JSON outputs, you can query your Docker engine default logging driver with:
# docker info | grep 'Logging Driver'
Logging Driver: json-file
If your container have a different Logging Driver you can check with:
docker inspect -f '{{.HostConfig.LogConfig.Type}}' <name-or-id-of-the-container>
You can find all log drivers in this link
To run containers with a different log-driver you can do:
- With docker run:
docker run -it --log-driver <log-driver> alpine ash
With docker-compose:
`logging:
driver: syslog
options:
syslog-address: "tcp://192.168.0.42:123"`
Regarding your log rotation questio, I'd say the easyest way is to configure the logging driver with the syslog driver, configure it to your local machine (or your syslog server) and then logrotate the files.
You can find several logrotate articles for Linux (which I assume you're using), for example this one
answered Nov 22 '18 at 10:21
Alexandre JumaAlexandre Juma
544214
It's generating a lot of logs because you're running docker-compose logs, which will get the logs for all containers in your docker compose file.
What you want is probably:
docker logs <name-of-filebeat-container>. The name of the filebeat container can be found doing adocker ps.
docker compose logs <name-of-filebeat-service>. The name of the service can be found on your docker-composer.yml file.
Regarding the JSON outputs, you can query your Docker engine default logging driver with:
# docker info | grep 'Logging Driver'
Logging Driver: json-file
If your container have a different Logging Driver you can check with:
docker inspect -f '{{.HostConfig.LogConfig.Type}}' <name-or-id-of-the-container>
You can find all log drivers in this link
To run containers with a different log-driver you can do:
- With docker run:
docker run -it --log-driver <log-driver> alpine ash
With docker-compose:
`logging:
driver: syslog
options:
syslog-address: "tcp://192.168.0.42:123"`
Regarding your log rotation questio, I'd say the easyest way is to configure the logging driver with the syslog driver, configure it to your local machine (or your syslog server) and then logrotate the files.
You can find several logrotate articles for Linux (which I assume you're using), for example this one
answered Nov 22 '18 at 10:21
Alexandre JumaAlexandre Juma
544214
answered Nov 22 '18 at 10:21
Alexandre JumaAlexandre Juma
544214
answered Nov 22 '18 at 10:21
Alexandre JumaAlexandre Juma
544214
answered Nov 22 '18 at 10:21
Alexandre JumaAlexandre Juma
544214
544214
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52854955%2fhow-to-create-rolling-logs-for-filebeat-within-a-docker-container%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Did you manage to try out the answer below or confirm a solution of your own?
– Alexandre Juma
Nov 30 '18 at 10:20