MariaDB Out of memory
-1
My database crash with out of memory error ~1-2 times per day. Last crash was at 17:30 in logs and at 19:30 in metrics (different timezones).
Errors in mariadb.log:
181120 16:09:00 [Warning] IP address '185.156.177.144' could not be resolved: Name or service not known
181120 16:35:40 [Warning] IP address '58.221.58.248' could not be resolved: Name or service not known
181120 17:30:38 mysqld_safe Number of processes running now: 0
181120 17:30:38 mysqld_safe mysqld restarted
181120 17:30:41 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release.
181120 17:30:41 [Note] /usr/libexec/mysqld (mysqld 5.5.52-MariaDB) starting as process 16636 ...
181120 17:30:41 [Warning] Changed limits: max_open_files: 1024 max_connections: 70 table_cache: 472
181120 17:30:41 InnoDB: The InnoDB memory heap is disabled
181120 17:30:41 InnoDB: Mutexes and rw_locks use GCC atomic builtins
181120 17:30:41 InnoDB: Compressed tables use zlib 1.2.7
181120 17:30:41 InnoDB: Using Linux native AIO
181120 17:30:41 InnoDB: Initializing buffer pool, size = 150.0M
181120 17:30:42 InnoDB: Completed initialization of buffer pool
181120 17:30:42 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
181120 17:30:42 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
181120 17:30:46 InnoDB: Waiting for the background threads to start
181120 17:30:47 Percona XtraDB (http://www.percona.com) 5.5.49-MariaDB-38.0 started; log sequence number 18741266161
181120 17:30:47 [Note] Plugin 'FEEDBACK' is disabled.
181120 17:30:48 [Note] Server socket created on IP: '0.0.0.0'.
181120 17:30:49 [Note] Event Scheduler: Loaded 0 events
181120 17:30:49 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.52-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server
181120 17:30:53 [ERROR] mysqld: Table './admin_wikis/wp_posts' is marked as crashed and should be repaired
181120 17:30:53 [Warning] Checking table: './admin_wikis/wp_posts'
181120 17:31:58 mysqld_safe Number of processes running now: 0
181120 17:31:58 mysqld_safe mysqld restarted
181120 17:32:01 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release.
181120 17:32:01 [Note] /usr/libexec/mysqld (mysqld 5.5.52-MariaDB) starting as process 16899 ...
181120 17:32:01 [Warning] Changed limits: max_open_files: 1024 max_connections: 70 table_cache: 472
181120 17:32:01 InnoDB: The InnoDB memory heap is disabled
181120 17:32:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins
181120 17:32:01 InnoDB: Compressed tables use zlib 1.2.7
181120 17:32:01 InnoDB: Using Linux native AIO
181120 17:32:01 InnoDB: Initializing buffer pool, size = 150.0M
InnoDB: mmap(161447936 bytes) failed; errno 12
181120 17:32:01 InnoDB: Completed initialization of buffer pool
181120 17:32:01 InnoDB: Fatal error: cannot allocate memory for the buffer pool
181120 17:32:01 [ERROR] Plugin 'InnoDB' init function returned error.
181120 17:32:01 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 128917504 bytes)
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 96681984 bytes)
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 72499200 bytes)
181120 17:32:01 [Note] Plugin 'FEEDBACK' is disabled.
181120 17:32:02 [ERROR] Unknown/unsupported storage engine: InnoDB
181120 17:32:02 [ERROR] Aborting
181120 17:32:02 [Note] /usr/libexec/mysqld: Shutdown complete
There is nothing interesing in apache log in this time. Event requests count
$ sudo grep -r "GET /" /etc/httpd/logs/domains/ | cut -d' ' -f4 | sed 's/[//;s/]//;s/"//' | cut -d':' -f1-2 | sort | uniq -c
459 19/Nov/2018:03
426 19/Nov/2018:04
239 19/Nov/2018:05
350 19/Nov/2018:06
381 19/Nov/2018:07
415 19/Nov/2018:08
778 19/Nov/2018:09
500 19/Nov/2018:10
450 19/Nov/2018:11
633 19/Nov/2018:12
458 19/Nov/2018:13
527 19/Nov/2018:14
713 19/Nov/2018:15
654 19/Nov/2018:16
573 19/Nov/2018:17
413 19/Nov/2018:18
471 19/Nov/2018:19
499 19/Nov/2018:20
661 19/Nov/2018:21
452 19/Nov/2018:22
773 19/Nov/2018:23
934 20/Nov/2018:00
295 20/Nov/2018:01
369 20/Nov/2018:02
441 20/Nov/2018:03
384 20/Nov/2018:04
927 20/Nov/2018:05
524 20/Nov/2018:06
972 20/Nov/2018:07
612 20/Nov/2018:08
609 20/Nov/2018:09
677 20/Nov/2018:10
753 20/Nov/2018:11
615 20/Nov/2018:12
717 20/Nov/2018:13
474 20/Nov/2018:14
973 20/Nov/2018:15
510 20/Nov/2018:16
534 20/Nov/2018:17
415 20/Nov/2018:18
90 20/Nov/2018:19
I get some metrics:
Processes before crash:
Processes in the moment of crash:
Memory and CPU usage:
Total RAM 3.6G. Does someone have any ideas?
mysql apache out-of-memory mariadb administration
asked Nov 20 '18 at 19:29
P. Dmitry
320311
add a comment |
-1
My database crash with out of memory error ~1-2 times per day. Last crash was at 17:30 in logs and at 19:30 in metrics (different timezones).
Errors in mariadb.log:
181120 16:09:00 [Warning] IP address '185.156.177.144' could not be resolved: Name or service not known
181120 16:35:40 [Warning] IP address '58.221.58.248' could not be resolved: Name or service not known
181120 17:30:38 mysqld_safe Number of processes running now: 0
181120 17:30:38 mysqld_safe mysqld restarted
181120 17:30:41 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release.
181120 17:30:41 [Note] /usr/libexec/mysqld (mysqld 5.5.52-MariaDB) starting as process 16636 ...
181120 17:30:41 [Warning] Changed limits: max_open_files: 1024 max_connections: 70 table_cache: 472
181120 17:30:41 InnoDB: The InnoDB memory heap is disabled
181120 17:30:41 InnoDB: Mutexes and rw_locks use GCC atomic builtins
181120 17:30:41 InnoDB: Compressed tables use zlib 1.2.7
181120 17:30:41 InnoDB: Using Linux native AIO
181120 17:30:41 InnoDB: Initializing buffer pool, size = 150.0M
181120 17:30:42 InnoDB: Completed initialization of buffer pool
181120 17:30:42 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
181120 17:30:42 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
181120 17:30:46 InnoDB: Waiting for the background threads to start
181120 17:30:47 Percona XtraDB (http://www.percona.com) 5.5.49-MariaDB-38.0 started; log sequence number 18741266161
181120 17:30:47 [Note] Plugin 'FEEDBACK' is disabled.
181120 17:30:48 [Note] Server socket created on IP: '0.0.0.0'.
181120 17:30:49 [Note] Event Scheduler: Loaded 0 events
181120 17:30:49 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.52-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server
181120 17:30:53 [ERROR] mysqld: Table './admin_wikis/wp_posts' is marked as crashed and should be repaired
181120 17:30:53 [Warning] Checking table: './admin_wikis/wp_posts'
181120 17:31:58 mysqld_safe Number of processes running now: 0
181120 17:31:58 mysqld_safe mysqld restarted
181120 17:32:01 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release.
181120 17:32:01 [Note] /usr/libexec/mysqld (mysqld 5.5.52-MariaDB) starting as process 16899 ...
181120 17:32:01 [Warning] Changed limits: max_open_files: 1024 max_connections: 70 table_cache: 472
181120 17:32:01 InnoDB: The InnoDB memory heap is disabled
181120 17:32:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins
181120 17:32:01 InnoDB: Compressed tables use zlib 1.2.7
181120 17:32:01 InnoDB: Using Linux native AIO
181120 17:32:01 InnoDB: Initializing buffer pool, size = 150.0M
InnoDB: mmap(161447936 bytes) failed; errno 12
181120 17:32:01 InnoDB: Completed initialization of buffer pool
181120 17:32:01 InnoDB: Fatal error: cannot allocate memory for the buffer pool
181120 17:32:01 [ERROR] Plugin 'InnoDB' init function returned error.
181120 17:32:01 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 128917504 bytes)
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 96681984 bytes)
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 72499200 bytes)
181120 17:32:01 [Note] Plugin 'FEEDBACK' is disabled.
181120 17:32:02 [ERROR] Unknown/unsupported storage engine: InnoDB
181120 17:32:02 [ERROR] Aborting
181120 17:32:02 [Note] /usr/libexec/mysqld: Shutdown complete
There is nothing interesing in apache log in this time. Event requests count
$ sudo grep -r "GET /" /etc/httpd/logs/domains/ | cut -d' ' -f4 | sed 's/[//;s/]//;s/"//' | cut -d':' -f1-2 | sort | uniq -c
459 19/Nov/2018:03
426 19/Nov/2018:04
239 19/Nov/2018:05
350 19/Nov/2018:06
381 19/Nov/2018:07
415 19/Nov/2018:08
778 19/Nov/2018:09
500 19/Nov/2018:10
450 19/Nov/2018:11
633 19/Nov/2018:12
458 19/Nov/2018:13
527 19/Nov/2018:14
713 19/Nov/2018:15
654 19/Nov/2018:16
573 19/Nov/2018:17
413 19/Nov/2018:18
471 19/Nov/2018:19
499 19/Nov/2018:20
661 19/Nov/2018:21
452 19/Nov/2018:22
773 19/Nov/2018:23
934 20/Nov/2018:00
295 20/Nov/2018:01
369 20/Nov/2018:02
441 20/Nov/2018:03
384 20/Nov/2018:04
927 20/Nov/2018:05
524 20/Nov/2018:06
972 20/Nov/2018:07
612 20/Nov/2018:08
609 20/Nov/2018:09
677 20/Nov/2018:10
753 20/Nov/2018:11
615 20/Nov/2018:12
717 20/Nov/2018:13
474 20/Nov/2018:14
973 20/Nov/2018:15
510 20/Nov/2018:16
534 20/Nov/2018:17
415 20/Nov/2018:18
90 20/Nov/2018:19
I get some metrics:
Processes before crash:
Processes in the moment of crash:
Memory and CPU usage:
Total RAM 3.6G. Does someone have any ideas?
mysql apache out-of-memory mariadb administration
asked Nov 20 '18 at 19:29
P. Dmitry
320311
Additional information request, please. Post on pastebin.com or here. OS in use, version? A) complete (not edited) my.cnf or my.ini Text results of: B) SHOW GLOBAL STATUS; after minimum 24 hours UPTIME C) SHOW GLOBAL VARIABLES; D) complete MySQLTuner report AND Optional very helpful information, if available includes - htop OR top OR mytop for most active apps, ulimit -a for a linux/unix list of limits, iostat -xm 5 3 when system is busy for an idea of IOPS by device, df -h for a linux/unix free space list by device, for server tuning analysis.
– Wilson Hauck
Nov 24 '18 at 20:27
add a comment |
-1
-1
-1
My database crash with out of memory error ~1-2 times per day. Last crash was at 17:30 in logs and at 19:30 in metrics (different timezones).
Errors in mariadb.log:
181120 16:09:00 [Warning] IP address '185.156.177.144' could not be resolved: Name or service not known
181120 16:35:40 [Warning] IP address '58.221.58.248' could not be resolved: Name or service not known
181120 17:30:38 mysqld_safe Number of processes running now: 0
181120 17:30:38 mysqld_safe mysqld restarted
181120 17:30:41 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release.
181120 17:30:41 [Note] /usr/libexec/mysqld (mysqld 5.5.52-MariaDB) starting as process 16636 ...
181120 17:30:41 [Warning] Changed limits: max_open_files: 1024 max_connections: 70 table_cache: 472
181120 17:30:41 InnoDB: The InnoDB memory heap is disabled
181120 17:30:41 InnoDB: Mutexes and rw_locks use GCC atomic builtins
181120 17:30:41 InnoDB: Compressed tables use zlib 1.2.7
181120 17:30:41 InnoDB: Using Linux native AIO
181120 17:30:41 InnoDB: Initializing buffer pool, size = 150.0M
181120 17:30:42 InnoDB: Completed initialization of buffer pool
181120 17:30:42 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
181120 17:30:42 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
181120 17:30:46 InnoDB: Waiting for the background threads to start
181120 17:30:47 Percona XtraDB (http://www.percona.com) 5.5.49-MariaDB-38.0 started; log sequence number 18741266161
181120 17:30:47 [Note] Plugin 'FEEDBACK' is disabled.
181120 17:30:48 [Note] Server socket created on IP: '0.0.0.0'.
181120 17:30:49 [Note] Event Scheduler: Loaded 0 events
181120 17:30:49 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.52-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server
181120 17:30:53 [ERROR] mysqld: Table './admin_wikis/wp_posts' is marked as crashed and should be repaired
181120 17:30:53 [Warning] Checking table: './admin_wikis/wp_posts'
181120 17:31:58 mysqld_safe Number of processes running now: 0
181120 17:31:58 mysqld_safe mysqld restarted
181120 17:32:01 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release.
181120 17:32:01 [Note] /usr/libexec/mysqld (mysqld 5.5.52-MariaDB) starting as process 16899 ...
181120 17:32:01 [Warning] Changed limits: max_open_files: 1024 max_connections: 70 table_cache: 472
181120 17:32:01 InnoDB: The InnoDB memory heap is disabled
181120 17:32:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins
181120 17:32:01 InnoDB: Compressed tables use zlib 1.2.7
181120 17:32:01 InnoDB: Using Linux native AIO
181120 17:32:01 InnoDB: Initializing buffer pool, size = 150.0M
InnoDB: mmap(161447936 bytes) failed; errno 12
181120 17:32:01 InnoDB: Completed initialization of buffer pool
181120 17:32:01 InnoDB: Fatal error: cannot allocate memory for the buffer pool
181120 17:32:01 [ERROR] Plugin 'InnoDB' init function returned error.
181120 17:32:01 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 128917504 bytes)
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 96681984 bytes)
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 72499200 bytes)
181120 17:32:01 [Note] Plugin 'FEEDBACK' is disabled.
181120 17:32:02 [ERROR] Unknown/unsupported storage engine: InnoDB
181120 17:32:02 [ERROR] Aborting
181120 17:32:02 [Note] /usr/libexec/mysqld: Shutdown complete
There is nothing interesing in apache log in this time. Event requests count
$ sudo grep -r "GET /" /etc/httpd/logs/domains/ | cut -d' ' -f4 | sed 's/[//;s/]//;s/"//' | cut -d':' -f1-2 | sort | uniq -c
459 19/Nov/2018:03
426 19/Nov/2018:04
239 19/Nov/2018:05
350 19/Nov/2018:06
381 19/Nov/2018:07
415 19/Nov/2018:08
778 19/Nov/2018:09
500 19/Nov/2018:10
450 19/Nov/2018:11
633 19/Nov/2018:12
458 19/Nov/2018:13
527 19/Nov/2018:14
713 19/Nov/2018:15
654 19/Nov/2018:16
573 19/Nov/2018:17
413 19/Nov/2018:18
471 19/Nov/2018:19
499 19/Nov/2018:20
661 19/Nov/2018:21
452 19/Nov/2018:22
773 19/Nov/2018:23
934 20/Nov/2018:00
295 20/Nov/2018:01
369 20/Nov/2018:02
441 20/Nov/2018:03
384 20/Nov/2018:04
927 20/Nov/2018:05
524 20/Nov/2018:06
972 20/Nov/2018:07
612 20/Nov/2018:08
609 20/Nov/2018:09
677 20/Nov/2018:10
753 20/Nov/2018:11
615 20/Nov/2018:12
717 20/Nov/2018:13
474 20/Nov/2018:14
973 20/Nov/2018:15
510 20/Nov/2018:16
534 20/Nov/2018:17
415 20/Nov/2018:18
90 20/Nov/2018:19
I get some metrics:
Processes before crash:
Processes in the moment of crash:
Memory and CPU usage:
Total RAM 3.6G. Does someone have any ideas?
mysql apache out-of-memory mariadb administration
asked Nov 20 '18 at 19:29
P. Dmitry
320311
My database crash with out of memory error ~1-2 times per day. Last crash was at 17:30 in logs and at 19:30 in metrics (different timezones).
Errors in mariadb.log:
181120 16:09:00 [Warning] IP address '185.156.177.144' could not be resolved: Name or service not known
181120 16:35:40 [Warning] IP address '58.221.58.248' could not be resolved: Name or service not known
181120 17:30:38 mysqld_safe Number of processes running now: 0
181120 17:30:38 mysqld_safe mysqld restarted
181120 17:30:41 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release.
181120 17:30:41 [Note] /usr/libexec/mysqld (mysqld 5.5.52-MariaDB) starting as process 16636 ...
181120 17:30:41 [Warning] Changed limits: max_open_files: 1024 max_connections: 70 table_cache: 472
181120 17:30:41 InnoDB: The InnoDB memory heap is disabled
181120 17:30:41 InnoDB: Mutexes and rw_locks use GCC atomic builtins
181120 17:30:41 InnoDB: Compressed tables use zlib 1.2.7
181120 17:30:41 InnoDB: Using Linux native AIO
181120 17:30:41 InnoDB: Initializing buffer pool, size = 150.0M
181120 17:30:42 InnoDB: Completed initialization of buffer pool
181120 17:30:42 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
181120 17:30:42 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
181120 17:30:46 InnoDB: Waiting for the background threads to start
181120 17:30:47 Percona XtraDB (http://www.percona.com) 5.5.49-MariaDB-38.0 started; log sequence number 18741266161
181120 17:30:47 [Note] Plugin 'FEEDBACK' is disabled.
181120 17:30:48 [Note] Server socket created on IP: '0.0.0.0'.
181120 17:30:49 [Note] Event Scheduler: Loaded 0 events
181120 17:30:49 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.52-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server
181120 17:30:53 [ERROR] mysqld: Table './admin_wikis/wp_posts' is marked as crashed and should be repaired
181120 17:30:53 [Warning] Checking table: './admin_wikis/wp_posts'
181120 17:31:58 mysqld_safe Number of processes running now: 0
181120 17:31:58 mysqld_safe mysqld restarted
181120 17:32:01 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release.
181120 17:32:01 [Note] /usr/libexec/mysqld (mysqld 5.5.52-MariaDB) starting as process 16899 ...
181120 17:32:01 [Warning] Changed limits: max_open_files: 1024 max_connections: 70 table_cache: 472
181120 17:32:01 InnoDB: The InnoDB memory heap is disabled
181120 17:32:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins
181120 17:32:01 InnoDB: Compressed tables use zlib 1.2.7
181120 17:32:01 InnoDB: Using Linux native AIO
181120 17:32:01 InnoDB: Initializing buffer pool, size = 150.0M
InnoDB: mmap(161447936 bytes) failed; errno 12
181120 17:32:01 InnoDB: Completed initialization of buffer pool
181120 17:32:01 InnoDB: Fatal error: cannot allocate memory for the buffer pool
181120 17:32:01 [ERROR] Plugin 'InnoDB' init function returned error.
181120 17:32:01 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 128917504 bytes)
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 96681984 bytes)
181120 17:32:01 [ERROR] mysqld: Out of memory (Needed 72499200 bytes)
181120 17:32:01 [Note] Plugin 'FEEDBACK' is disabled.
181120 17:32:02 [ERROR] Unknown/unsupported storage engine: InnoDB
181120 17:32:02 [ERROR] Aborting
181120 17:32:02 [Note] /usr/libexec/mysqld: Shutdown complete
There is nothing interesing in apache log in this time. Event requests count
$ sudo grep -r "GET /" /etc/httpd/logs/domains/ | cut -d' ' -f4 | sed 's/[//;s/]//;s/"//' | cut -d':' -f1-2 | sort | uniq -c
459 19/Nov/2018:03
426 19/Nov/2018:04
239 19/Nov/2018:05
350 19/Nov/2018:06
381 19/Nov/2018:07
415 19/Nov/2018:08
778 19/Nov/2018:09
500 19/Nov/2018:10
450 19/Nov/2018:11
633 19/Nov/2018:12
458 19/Nov/2018:13
527 19/Nov/2018:14
713 19/Nov/2018:15
654 19/Nov/2018:16
573 19/Nov/2018:17
413 19/Nov/2018:18
471 19/Nov/2018:19
499 19/Nov/2018:20
661 19/Nov/2018:21
452 19/Nov/2018:22
773 19/Nov/2018:23
934 20/Nov/2018:00
295 20/Nov/2018:01
369 20/Nov/2018:02
441 20/Nov/2018:03
384 20/Nov/2018:04
927 20/Nov/2018:05
524 20/Nov/2018:06
972 20/Nov/2018:07
612 20/Nov/2018:08
609 20/Nov/2018:09
677 20/Nov/2018:10
753 20/Nov/2018:11
615 20/Nov/2018:12
717 20/Nov/2018:13
474 20/Nov/2018:14
973 20/Nov/2018:15
510 20/Nov/2018:16
534 20/Nov/2018:17
415 20/Nov/2018:18
90 20/Nov/2018:19
I get some metrics:
Processes before crash:
Processes in the moment of crash:
Memory and CPU usage:
Total RAM 3.6G. Does someone have any ideas?
mysql apache out-of-memory mariadb administration
mysql apache out-of-memory mariadb administration
asked Nov 20 '18 at 19:29
P. Dmitry
320311
asked Nov 20 '18 at 19:29
P. Dmitry
320311
edited Nov 21 '18 at 9:15
asked Nov 20 '18 at 19:29
P. Dmitry
320311
asked Nov 20 '18 at 19:29
P. Dmitry
320311
asked Nov 20 '18 at 19:29
P. Dmitry
320311
320311
Additional information request, please. Post on pastebin.com or here. OS in use, version? A) complete (not edited) my.cnf or my.ini Text results of: B) SHOW GLOBAL STATUS; after minimum 24 hours UPTIME C) SHOW GLOBAL VARIABLES; D) complete MySQLTuner report AND Optional very helpful information, if available includes - htop OR top OR mytop for most active apps, ulimit -a for a linux/unix list of limits, iostat -xm 5 3 when system is busy for an idea of IOPS by device, df -h for a linux/unix free space list by device, for server tuning analysis.
– Wilson Hauck
Nov 24 '18 at 20:27
add a comment |
Additional information request, please. Post on pastebin.com or here. OS in use, version? A) complete (not edited) my.cnf or my.ini Text results of: B) SHOW GLOBAL STATUS; after minimum 24 hours UPTIME C) SHOW GLOBAL VARIABLES; D) complete MySQLTuner report AND Optional very helpful information, if available includes - htop OR top OR mytop for most active apps, ulimit -a for a linux/unix list of limits, iostat -xm 5 3 when system is busy for an idea of IOPS by device, df -h for a linux/unix free space list by device, for server tuning analysis.
– Wilson Hauck
Nov 24 '18 at 20:27
Additional information request, please. Post on pastebin.com or here. OS in use, version? A) complete (not edited) my.cnf or my.ini Text results of: B) SHOW GLOBAL STATUS; after minimum 24 hours UPTIME C) SHOW GLOBAL VARIABLES; D) complete MySQLTuner report AND Optional very helpful information, if available includes - htop OR top OR mytop for most active apps, ulimit -a for a linux/unix list of limits, iostat -xm 5 3 when system is busy for an idea of IOPS by device, df -h for a linux/unix free space list by device, for server tuning analysis.
– Wilson Hauck
Nov 24 '18 at 20:27
Additional information request, please. Post on pastebin.com or here. OS in use, version? A) complete (not edited) my.cnf or my.ini Text results of: B) SHOW GLOBAL STATUS; after minimum 24 hours UPTIME C) SHOW GLOBAL VARIABLES; D) complete MySQLTuner report AND Optional very helpful information, if available includes - htop OR top OR mytop for most active apps, ulimit -a for a linux/unix list of limits, iostat -xm 5 3 when system is busy for an idea of IOPS by device, df -h for a linux/unix free space list by device, for server tuning analysis.
– Wilson Hauck
Nov 24 '18 at 20:27
add a comment |
1 Answer
1
active
oldest
votes
2
httpdis the largest memory user; decrease its settings, especially the number of 'children' it is willing to spawn. (Under 10 should suffice for most small installations.)max_connections: 70 table_cache: 472-- Cut each of those in half; this will help a little with memory allocation.Next time the CPU hits 100%, do
SHOW FULL PROCESSLIST;in MySQL. That may give you a clue of what runaway queries you have. Then we can focus on them. If that fails, turn on the slowlog; it will catch some of them.Consider moving to an 8GB server. (Is this a cloud server?)
answered Nov 20 '18 at 20:43
Rick James
66.1k55897
Thanks. Will try today. But I think CPU and memory load at "Processes in the moment of crash" looks very strange
– P. Dmitry
Nov 21 '18 at 9:07
"Under 10 should suffice" ha ha ha ha ha!
– symcbean
Nov 21 '18 at 9:42
@symcbean - If you need more than N (eg, 10), then the threads will stumble over each other and pile up. Sure, they will all be "running", but in reality they will mostly be stalled waiting for CPU/swap/IO/MySQL/etc. Throughput will hit some limit, and latency will go through the roof. And, guess what, you get graphs like those.
– Rick James
Nov 21 '18 at 17:45
Have you ever been in a store where there were so many people that it was hard to move your cart around? Some stores are wise to that (or maybe threatened by the Fire Marshall) and nave you wait at the door, letting people in gradually.
– Rick James
Nov 21 '18 at 17:48
stdlib.net/~colmmacc/Apachecon-EU2005/… (I have personally ran LAMP boxes with much lower specs handling 500+ concurrent connections)
– symcbean
Nov 22 '18 at 10:33
|
show 1 more comment
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400214%2fmariadb-out-of-memory%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
2
httpdis the largest memory user; decrease its settings, especially the number of 'children' it is willing to spawn. (Under 10 should suffice for most small installations.)max_connections: 70 table_cache: 472-- Cut each of those in half; this will help a little with memory allocation.Next time the CPU hits 100%, do
SHOW FULL PROCESSLIST;in MySQL. That may give you a clue of what runaway queries you have. Then we can focus on them. If that fails, turn on the slowlog; it will catch some of them.Consider moving to an 8GB server. (Is this a cloud server?)
answered Nov 20 '18 at 20:43
Rick James
66.1k55897
Thanks. Will try today. But I think CPU and memory load at "Processes in the moment of crash" looks very strange
– P. Dmitry
Nov 21 '18 at 9:07
"Under 10 should suffice" ha ha ha ha ha!
– symcbean
Nov 21 '18 at 9:42
@symcbean - If you need more than N (eg, 10), then the threads will stumble over each other and pile up. Sure, they will all be "running", but in reality they will mostly be stalled waiting for CPU/swap/IO/MySQL/etc. Throughput will hit some limit, and latency will go through the roof. And, guess what, you get graphs like those.
– Rick James
Nov 21 '18 at 17:45
Have you ever been in a store where there were so many people that it was hard to move your cart around? Some stores are wise to that (or maybe threatened by the Fire Marshall) and nave you wait at the door, letting people in gradually.
– Rick James
Nov 21 '18 at 17:48
stdlib.net/~colmmacc/Apachecon-EU2005/… (I have personally ran LAMP boxes with much lower specs handling 500+ concurrent connections)
– symcbean
Nov 22 '18 at 10:33
|
show 1 more comment
2
httpdis the largest memory user; decrease its settings, especially the number of 'children' it is willing to spawn. (Under 10 should suffice for most small installations.)max_connections: 70 table_cache: 472-- Cut each of those in half; this will help a little with memory allocation.Next time the CPU hits 100%, do
SHOW FULL PROCESSLIST;in MySQL. That may give you a clue of what runaway queries you have. Then we can focus on them. If that fails, turn on the slowlog; it will catch some of them.Consider moving to an 8GB server. (Is this a cloud server?)
answered Nov 20 '18 at 20:43
Rick James
66.1k55897
Thanks. Will try today. But I think CPU and memory load at "Processes in the moment of crash" looks very strange
– P. Dmitry
Nov 21 '18 at 9:07
"Under 10 should suffice" ha ha ha ha ha!
– symcbean
Nov 21 '18 at 9:42
@symcbean - If you need more than N (eg, 10), then the threads will stumble over each other and pile up. Sure, they will all be "running", but in reality they will mostly be stalled waiting for CPU/swap/IO/MySQL/etc. Throughput will hit some limit, and latency will go through the roof. And, guess what, you get graphs like those.
– Rick James
Nov 21 '18 at 17:45
Have you ever been in a store where there were so many people that it was hard to move your cart around? Some stores are wise to that (or maybe threatened by the Fire Marshall) and nave you wait at the door, letting people in gradually.
– Rick James
Nov 21 '18 at 17:48
stdlib.net/~colmmacc/Apachecon-EU2005/… (I have personally ran LAMP boxes with much lower specs handling 500+ concurrent connections)
– symcbean
Nov 22 '18 at 10:33
|
show 1 more comment
2
2
2
httpdis the largest memory user; decrease its settings, especially the number of 'children' it is willing to spawn. (Under 10 should suffice for most small installations.)max_connections: 70 table_cache: 472-- Cut each of those in half; this will help a little with memory allocation.Next time the CPU hits 100%, do
SHOW FULL PROCESSLIST;in MySQL. That may give you a clue of what runaway queries you have. Then we can focus on them. If that fails, turn on the slowlog; it will catch some of them.Consider moving to an 8GB server. (Is this a cloud server?)
answered Nov 20 '18 at 20:43
Rick James
66.1k55897
httpdis the largest memory user; decrease its settings, especially the number of 'children' it is willing to spawn. (Under 10 should suffice for most small installations.)max_connections: 70 table_cache: 472-- Cut each of those in half; this will help a little with memory allocation.Next time the CPU hits 100%, do
SHOW FULL PROCESSLIST;in MySQL. That may give you a clue of what runaway queries you have. Then we can focus on them. If that fails, turn on the slowlog; it will catch some of them.Consider moving to an 8GB server. (Is this a cloud server?)
answered Nov 20 '18 at 20:43
Rick James
66.1k55897
answered Nov 20 '18 at 20:43
Rick James
66.1k55897
answered Nov 20 '18 at 20:43
Rick James
66.1k55897
answered Nov 20 '18 at 20:43
Rick James
66.1k55897
66.1k55897
Thanks. Will try today. But I think CPU and memory load at "Processes in the moment of crash" looks very strange
– P. Dmitry
Nov 21 '18 at 9:07
"Under 10 should suffice" ha ha ha ha ha!
– symcbean
Nov 21 '18 at 9:42
@symcbean - If you need more than N (eg, 10), then the threads will stumble over each other and pile up. Sure, they will all be "running", but in reality they will mostly be stalled waiting for CPU/swap/IO/MySQL/etc. Throughput will hit some limit, and latency will go through the roof. And, guess what, you get graphs like those.
– Rick James
Nov 21 '18 at 17:45
Have you ever been in a store where there were so many people that it was hard to move your cart around? Some stores are wise to that (or maybe threatened by the Fire Marshall) and nave you wait at the door, letting people in gradually.
– Rick James
Nov 21 '18 at 17:48
stdlib.net/~colmmacc/Apachecon-EU2005/… (I have personally ran LAMP boxes with much lower specs handling 500+ concurrent connections)
– symcbean
Nov 22 '18 at 10:33
|
show 1 more comment
Thanks. Will try today. But I think CPU and memory load at "Processes in the moment of crash" looks very strange
– P. Dmitry
Nov 21 '18 at 9:07
"Under 10 should suffice" ha ha ha ha ha!
– symcbean
Nov 21 '18 at 9:42
@symcbean - If you need more than N (eg, 10), then the threads will stumble over each other and pile up. Sure, they will all be "running", but in reality they will mostly be stalled waiting for CPU/swap/IO/MySQL/etc. Throughput will hit some limit, and latency will go through the roof. And, guess what, you get graphs like those.
– Rick James
Nov 21 '18 at 17:45
Have you ever been in a store where there were so many people that it was hard to move your cart around? Some stores are wise to that (or maybe threatened by the Fire Marshall) and nave you wait at the door, letting people in gradually.
– Rick James
Nov 21 '18 at 17:48
stdlib.net/~colmmacc/Apachecon-EU2005/… (I have personally ran LAMP boxes with much lower specs handling 500+ concurrent connections)
– symcbean
Nov 22 '18 at 10:33
Thanks. Will try today. But I think CPU and memory load at "Processes in the moment of crash" looks very strange
– P. Dmitry
Nov 21 '18 at 9:07
Thanks. Will try today. But I think CPU and memory load at "Processes in the moment of crash" looks very strange
– P. Dmitry
Nov 21 '18 at 9:07
"Under 10 should suffice" ha ha ha ha ha!
– symcbean
Nov 21 '18 at 9:42
"Under 10 should suffice" ha ha ha ha ha!
– symcbean
Nov 21 '18 at 9:42
@symcbean - If you need more than N (eg, 10), then the threads will stumble over each other and pile up. Sure, they will all be "running", but in reality they will mostly be stalled waiting for CPU/swap/IO/MySQL/etc. Throughput will hit some limit, and latency will go through the roof. And, guess what, you get graphs like those.
– Rick James
Nov 21 '18 at 17:45
@symcbean - If you need more than N (eg, 10), then the threads will stumble over each other and pile up. Sure, they will all be "running", but in reality they will mostly be stalled waiting for CPU/swap/IO/MySQL/etc. Throughput will hit some limit, and latency will go through the roof. And, guess what, you get graphs like those.
– Rick James
Nov 21 '18 at 17:45
Have you ever been in a store where there were so many people that it was hard to move your cart around? Some stores are wise to that (or maybe threatened by the Fire Marshall) and nave you wait at the door, letting people in gradually.
– Rick James
Nov 21 '18 at 17:48
Have you ever been in a store where there were so many people that it was hard to move your cart around? Some stores are wise to that (or maybe threatened by the Fire Marshall) and nave you wait at the door, letting people in gradually.
– Rick James
Nov 21 '18 at 17:48
stdlib.net/~colmmacc/Apachecon-EU2005/… (I have personally ran LAMP boxes with much lower specs handling 500+ concurrent connections)
– symcbean
Nov 22 '18 at 10:33
stdlib.net/~colmmacc/Apachecon-EU2005/… (I have personally ran LAMP boxes with much lower specs handling 500+ concurrent connections)
– symcbean
Nov 22 '18 at 10:33
|
show 1 more comment
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400214%2fmariadb-out-of-memory%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Additional information request, please. Post on pastebin.com or here. OS in use, version? A) complete (not edited) my.cnf or my.ini Text results of: B) SHOW GLOBAL STATUS; after minimum 24 hours UPTIME C) SHOW GLOBAL VARIABLES; D) complete MySQLTuner report AND Optional very helpful information, if available includes - htop OR top OR mytop for most active apps, ulimit -a for a linux/unix list of limits, iostat -xm 5 3 when system is busy for an idea of IOPS by device, df -h for a linux/unix free space list by device, for server tuning analysis.
– Wilson Hauck
Nov 24 '18 at 20:27