Never display connection errors manually!
MySQLi will generate a warning if it is unable to open the connection to MySQL. This warning tells you all you need to know including the error code, error message, and the place in the code where it happened. Checking for the error manually will not give you any more information.
If you can’t see the warning and your connection cannot be created, it might mean that your PHP is not configured to show them. In that case, you must check the error log file on your server. If you do not know where that is, use phpinfo()
to get that information and search for error_log
. It will tell you where the error log file is located.
If there is no warning in the error logs, it could mean that your PHP has error reporting silenced (either completely or just warnings). Check your PHP configuration.
In the production environment these settings should be maintained:
error_reporting
must beE_ALL
log_errors
must beOn
display_errors
must beOff
In the development environment these settings should be maintained:
error_reporting
must beE_ALL
log_errors
must beOn
display_errors
must beOn
As you can see in the error message your database username and password have been revealed to the end user. This is sensitive information that you do not want to show anyone. In fact, a normal user would not understand this cryptic message. This is why display_errors
must always be switched off in the production environment. Logging the errors on the server is safe.
Warnings vs. Exceptions
Warnings do not stop the script. If a warning is emitted the script will keep on executing until it encounters a fatal error. In most cases, you would want to throw an exception to stop the script. Do not use die/exit
! If mysqli connection cannot be made, an exception should be thrown and if it is unhandled it will bubble up and stop the script with a fatal error. You can configure mysqli to throw exceptions automatically. This is invaluable because all mysqli functions can fail for many reasons and they will not inform you about any problems unless you check for errors manually every single one of them. Use the following line before opening the connection:
mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);
$mysqli = new mysqli('localhost', 'my_user', 'my_password', 'my_db');
Do not catch the exceptions unless you really know what to do with them! One possible use case is described in How to connect properly using mysqli
Can mysqli_error()
show any connection-related problems?
No. mysqli_error($conn)
expects that the mysqli connection was successful. $conn
must be a valid mysqli connection otherwise you would get this error message:
Warning: mysqli_error(): Couldn’t fetch mysqli in C:…
Neither $conn->error
nor mysqli_error($conn)
can display any connection-related errors!
Related: Should I manually check for errors when calling “mysqli_stmt_prepare”?
MySQL is a widely used open source relational database management system (RDMS) owned by Oracle. It has over the years been the default choice for web-based applications and still remains popular in comparison to other database engines.
Read Also: How to Install Latest MySQL on RHEL/CentOS and Fedora
MySQL was designed and optimized for web applications – it forms an integral part of major web-based applications such as Facebook, Twitter, Wikipedia, YouTube, and many others.
Is your site or web application powered by MySQL? In this detailed article, we will explain how to troubleshoot problems and common errors in MySQL database server. We will describe how to determine the causes of the problems and what to do to solve them.
1. Can’t Connect to Local MySQL Server
One of the common client to server connection errors in MySQL is “ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var/run/mysqld/mysqld.sock’ (2)”.
This error indicates that there is no MySQL server (mysqld)
running on the host system or that you have specified a wrong Unix socket file name or TCP/IP port when trying to connect to the server.
Ensure that the server is running by checking a process named mysqld
on your database server host using the ps command and grep command together as shown.
$ ps xa | grep mysqld | grep -v mysqld
If the above commands show no output, then the database server isn’t running. Therefore the client can’t connect to it. To start the server, run the following systemctl command.
$ sudo systemctl start mysql #Debian/Ubuntu $ sudo systemctl start mysqld #RHEL/CentOS/Fedora
To verify the MySQL service status, use the following command.
$ sudo systemctl status mysql #Debian/Ubuntu $ sudo systemctl status mysqld #RHEL/CentOS/Fedora
From the output of the above command, the MySQL service has failed. In such a case, you can try to restart it and check its status once more.
$ sudo systemctl restart mysql $ sudo systemctl status mysql
In addition, if the server is running as shown by the following command, but you still see the above error, you should also verify that the TCP/IP port is blocked by a firewall or any port blocking service.
$ ps xa | grep mysqld | grep -v mysqld
To find the port the server is listening on, use the netstat command as shown.
$ sudo netstat -tlpn | grep "mysql"
2. Can’t Connect to MySQL Server
Another commonly encountered connection error is “(2003) Can’t connect to MySQL server on ‘server’ (10061)”, which means that the network connection has been refused.
Here, start by checking that there is a MySQL server running on the system as shown above. Also ensure that the server has network connections enabled and that the network port you are using to connect is the one configured on the server.
Other common errors you are likely to encounter when you try to connect to the MySQL server are:
ERROR 2003: Can't connect to MySQL server on 'host_name' (111) ERROR 2002: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (111)
These errors indicate that the server might be running, however, you are trying to connect using a TCP/IP port, named pipe, or Unix socket file different from the one on which the server is listening.
3. Access Denied Errors in MySQL
In MySQL, a user account is defined in terms of a username and the client host or hosts from which the user can connect to the server. In addition, an account may also have authentication credentials such as a password.
Although there are many different causes of “Access denied” errors, one of the common causes is relating to the MySQL accounts that the server permits client programs to use when connecting. It indicates that username specified in the connection does not have privileges to access the database.
MySQL allows the creation of accounts that enable client users to connect to the server and access data managed by the server. In this regard, if you encounter an access denied error, check if the user account is allowed to connect to the server via the client program you are using, and possibly the host from which the connection is coming from.
You can see what privileges a given account has by running the SHOW GRANTS
command as shown.
> SHOW GRANTS FOR 'tecmint'@'localhost';
You can grant privileges to a particular user on specific database to the remote ip address using the following commands in the MySQL shell.
> grant all privileges on *.test_db to 'tecmint'@'192.168.0.100'; > flush privileges;
Furthermore, access denied errors can also result from problems with connecting to MySQL, refer to the previously explained errors.
4. Lost Connection to MySQL Server
You may encounter this error due to one of the following reasons: poor network connectivity, connection timeout or a problem with BLOB values that are larger than max_allowed_packet. In case of a network connection problem, ensure that you have a good network connection especially if you are accessing a remote database server.
If it is a connection timeout problem, particularly when MySQL is trying to use an initial connection to the server, increase the value of the connect_timeout parameter. But in case of BLOB values that are larger than max_allowed_packet, you need to set a higher value for the max_allowed_packet in your /etc/my.cnf configuration file under [mysqld]
or [client]
section as shown.
[mysqld] connect_timeout=100 max_allowed_packet=500M
If the MySQL configuration file is not accessible for you, then you can set this value using the following command in the MySQL shell.
> SET GLOBAL connect_timeout=100; > SET GLOBAL max_allowed_packet=524288000;
5. Too Many MySQL Connections
In case a MySQL client encounters the “too many connections” error, it means that all available connections are in use by other clients. The number of connections (default is 151) is controlled by the max_connections
system variable; you can remedy the problem by increasing its value to permit more connections in your /etc/my.cnf configuration file.
[mysqld] max_connections=1000
6. Out of Memory MySQL
In case you run a query using the MySQL client program and encounter the error in question, it means that MySQL does not have enough memory to store the entire query result.
The first step is to ensure that the query is correct, if it is, then do the following:
- if you are using MySQL client directly, start it with
--quick switch
, to disable cached results or - if you are using the MyODBC driver, the configuration user interface (UI) has an advanced tab for flags. Check “Do not cache result“.
Another great tool is, MySQL Tuner – a useful script that will connect to a running MySQL server and gives suggestions for how it can be configured for higher performance.
$ sudo apt-get install mysqltuner #Debian/Ubuntu $ sudo yum install mysqltuner #RHEL/CentOS/Fedora $ mysqltuner
For MySQL optimization and performance tuning tips, read our article: 15 Useful MySQL/MariaDB Performance Tuning and Optimization Tips.
7. MySQL Keeps Crashing
If you encounter this problem, you should try to find out whether the problem is that the MySQL server dies or whether its the client with an issue. Note that many server crashes are caused by corrupted data files or index files.
You can check the server status to establish for how long it has been up and running.
$ sudo systemctl status mysql #Debian/Ubuntu $ sudo systemctl status mysqld #RHEL/CentOS/Fedora
Alternatively, run the following mysqladmin command to find uptime of MySQL server.
$ sudo mysqladmin version -p
Other solutions include but not limited to stopping the MySQL server and enabling debugging, then start the service again. You can try to make a test case that can be used to repeat the problem. In addition, open an additional terminal window and run the following command to display MySQL process statistics while you run your other queries:
$ sudo mysqladmin -i 5 status OR $ sudo mysqladmin -i 5 -r status
The Bottom Line: Determining What Is Causing a Problem or an Error
Although we have looked at some common MySQL problems and errors and also provided ways to troubleshoot and solve them, the most important thing with diagnosing an error is understanding what it means (in terms of what is causing it).
So how can you determine this? The following points will guide you on how to ascertain what is exactly causing a problem:
- The first and most important step is to look into the MySQL logs which are stored in the directory
/var/log/mysql/
. You can use command line utilities such as tail to read through the log files. - If MySQL service fails to start, check its status using systemctl or use the journetctl (with the
-xe
flag) command under systemd to examine the problem. - You can also examine system log file such as
/var/log/messages
or similar for reasons for your problem. - Try using tools such as Mytop, glances, top, ps, or htop to check which program is taking all CPU or is locking the machine or to inspect whether you are running out of memory, disk space, file descriptors, or some other important resource.
- Assuming that problem is some runaway process, you can always try to kill it (using the pkill or kill utility) so that MySQL works normally.
- Supposing that the mysqld server is causing problems, you can run the command:
mysqladmin -u root ping
ormysqladmin -u root processlist
to get any response from it. - If the problem is with your client program while trying to connect to the MySQL server, check why it is not working fine, try to get any output from it for troubleshooting purposes.
You might also like to read these following MySQL related articles:
- Learn MySQL / MariaDB for Beginners – Part 1
- How to Monitor MySQL/MariaDB Databases using Netdata on CentOS 7
- How to Transfer All MySQL Databases From Old to New Server
- Mytop – A Useful Tool for Monitoring MySQL/MariaDB Performance in Linux
- 12 MySQL/MariaDB Security Best Practices for Linux
For more information, consult the MySQL Reference manual concerning Problems and Common Errors, it comprehensively lists common problems and error messages that you may encounter while using MySQL, including the ones we have discussed above and more.
- PHP
MySQLi
Error Functions - Conclusion
MySQLi
is a PHP function used to access the MySQL database server. You can use this extension if you have MySQL version 4.1.13 or above.
There are various MySQLi
functions that you can use to perform different functions in PHP. In this article, we will learn MySQLi
error functions.
We will also see how and where to use code examples and observe the outputs. For this tutorial, we will use MySQL version 8.0.27 and PHP version 7.4.1.
PHP MySQLi
Error Functions
In this tutorial, we will learn about the following PHP MySQLi
error functions:
mysqli_error()
mysqli_errno()
mysqli_error_list()
mysqli_connect_error()
mysqli_connect_errno()
All these functions can be used in object-oriented style and procedural style. Let’s understand both syntax using the mysqli_error()
function.
Syntax of mysqli_error()
Function in Object Oriented Style
Syntax of mysqli_error()
Function in Procedural Style
string mysqli_error ( mysqli $link )
mysqli_error()
Function in MySQL
This function is used to output the last error description for the most recent function call (if there is any). It is used when you want to know whether the SQL
query has an error or not.
mysqli_error()
returns the error description and empty string if there is no error. See the following example.
Example Code Using Procedural Style:
<?php
$host = "localhost";
$username = "root";
$password = "";
$database = "person";
$connection = mysqli_connect($host, $username, $password, $database)
or die("Connection Failed");
$sql = "SELECT * FROM teacher";
$result = mysqli_query($connection, $sql);
$error_message = mysqli_error($connection);
if($error_message == ""){
echo "No error related to SQL query.";
}else{
echo "Query Failed: ".$error_message;
}
mysqli_close($connection);
?>
The code given above tries to make the connection using $host
, $username
, $password
, $database
variables and save this connection into the $connection
variable.
mysqli_error()
function will take this connection variable $connection
as a parameter and check if there is any error caused by the recent MySQLi
function call which is mysqli_query($connection, $sql)
here.
Output:
Now, change the table name in the SQL
query from teacher
to person
and observe the output given below.
Output:
We, as a developer, can easily understand that there is no person
table in the person
database (this is what it means in the above error).
Keep the table name changed and replace the line $error_message = mysqli_error($connection);
with $error_message = $connection->error;
to practice and understand the object oriented style using MySQLi
error function.
mysqli_errno()
Function in MySQL
mysqli_errno()
works the same as mysqli_error()
does, but it will return the error code instead of the error description.
Write the following code to practice and understand. You may have noticed that we use a procedural style to practice this function.
<?php
$host = "localhost";
$username = "root";
$password = "";
$database = "person";
$connection = mysqli_connect($host, $username, $password, $database)
or die("Connection Failed");
$sql = "SELECT * FROM person";
$result = mysqli_query($connection, $sql);
$error_message = mysqli_errno($connection);
if($error_message == ""){
echo "No error related to SQL query.";
}else{
echo "Query Failed: ".$error_message;
}
mysqli_close($connection);
?>
The code given above will show the following output where you will see a number as an error code.
Output:
The question is, why do we use this function to show the numbers only? Because if you want to print a user-friendly error message (custom message), you can use this error code in if-else
statements.
See the following code and its output below.
<?php
$host = "localhost";
$username = "root";
$password = "";
$database = "person";
$connection = mysqli_connect($host, $username, $password, $database)
or die("Connection Failed");
$sql = "SELECT * FROM person";
$result = mysqli_query($connection, $sql);
$error_message = mysqli_errno($connection);
if($error_message == 1146){
echo "You are trying to read the data from a table which doesn't exist in your database "."'".$database."'";
}
mysqli_close($connection);
?>
Output:
mysqli_error_list()
Function in MySQL
This function is very useful for knowing the error code, SQL state, and error description because this function returns an array containing all the necessary information.
Example Code:
<?php
$host = "localhost";
$username = "root";
$password = "";
$database = "person";
$connection = mysqli_connect($host, $username, $password, $database)
or die("Connection Failed");
$sql = "SELECT * FROM person";
$result = mysqli_query($connection, $sql);
print_r(mysqli_error_list($connection));
mysqli_close($connection);
?>
Output:
mysqli_connect_error()
Function in MySQL
mysqli_connect_error()
returns the error description from the last connection if there is any. Although, the die()
function also tell about the unsuccessful connection but mysqli_connect_error()
returns the error that we can understand easily.
Write the following code first, see its output, and then we’ll compare it with the output produced by mysqli_connect_error()
.
<?php
$host = "localhost";
$username = "root";
$password = "";
$database = "person";
$connection = mysqli_connect($host, $username, $password, $database)
or die("Connection Failed");
$sql = "SELECT * FROM person";
$result = mysqli_query($connection, $sql);
$error_message = mysqli_error($connection);
if($error_message != ""){
echo "Query Failed: ".$error_message;
}
mysqli_close($connection);
?>
Output:
See the output given above; you can see that the error we can understand is somewhere in the middle.
Imagine, if you have 2 or 3 errors, it would not be easy to find out. Now, use the mysqli_connect_error()
and see the difference using the following code and output.
<?php
$host = "localhost";
$username = "newroot";
$password = "";
$database = "person";
$connection = mysqli_connect($host, $username, $password, $database)
or die("Connection Failed: ".mysqli_connect_error());
$sql = "SELECT * FROM teacher";
$result = mysqli_query($connection, $sql);
$error_message = mysqli_error($connection);
if($error_message != ""){
echo "SQL Query Failed: ".$error_message;
}
mysqli_close($connection);
?>
Output:
The above output clearly says that there is no user named newroot
, which does not allow you to access the database.
mysqli_connect_errno()
Function in MySQL
This function behaves like mysqli_connect_error()
but displays the error code rather than the error message. We can use this error code to write custom error messages.
Example Code:
<?php
$host = "localhost";
$username = "newroot";
$password = "";
$database = "person";
$connection = mysqli_connect($host, $username, $password, $database)
or die("Connection Failed: ".mysqli_connect_errno());
$sql = "SELECT * FROM teacher";
$result = mysqli_query($connection, $sql);
$error_message = mysqli_error($connection);
if($error_message != ""){
echo "SQL Query Failed: ".$error_message;
}
mysqli_close($connection);
?>
Output:
Conclusion
Considering all the discussion and examples, we have concluded two main categories. The first category shows the errors about SQL queries and the other about database connections.
Depending on the project needs, we can print the error message or the error code in each category.
MySQL — система управления базами данных (СУБД) с открытым исходным кодом от компании Oracle. Она была разработана и оптимизирована специально для работы веб-приложений. MySQL является неотъемлемой частью таких веб-сервисов, как Facebook, Twitter, Wikipedia, YouTube и многих других.
Эта статья расскажет, как определять, с чем связаны частые ошибки на сервере MySQL, и устранять их.
Не удаётся подключиться к локальному серверу
Одной из распространённых ошибок подключения клиента к серверу является «ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var/run/mysqld/mysqld.sock’ (2)».
Эта ошибка означает, что на хосте не запущен сервер MySQL (mysqld
) или вы указали неправильное имя файла сокета Unix или порт TCP/IP при попытке подключения.
Убедитесь, что сервер работает. Проверьте процесс с именем mysqld
на хосте сервера, используя команды ps или grep, как показано ниже.
$ ps xa | grep mysqld | grep -v mysqld
Если эти команды не показывают выходных данных, то сервер БД не работает. Поэтому клиент не может подключиться к нему. Чтобы запустить сервер, выполните команду systemctl.
$ sudo systemctl start mysql #Debian/Ubuntu
$ sudo systemctl start mysqld #RHEL/CentOS/Fedora
Чтобы проверить состояние службы MySQL, используйте следующую команду:
$ sudo systemctl status mysql #Debian/Ubuntu
$ sudo systemctl status mysqld #RHEL/CentOS/Fedora
Если в результате выполнения команды произошла ошибка службы MySQL, вы можете попробовать перезапустить службу и ещё раз проверить её состояние.
$ sudo systemctl restart mysql
$ sudo systemctl status mysql
Если сервер работает (как показано) и вы по-прежнему видите эту ошибку, вам следует проверить, не заблокирован ли порт TCP/IP брандмауэром или любой другой службой блокировки портов.
Для поиска порта, который прослушивается сервером, используйте команду netstat
.
$ sudo netstat -tlpn | grep "mysql"
Ещё одна похожая и часто встречающаяся ошибка подключения — «(2003) Can’t connect to MySQL server on ‘server’ (10061)». Это означает, что в сетевом соединении было отказано.
Следует проверить, работает ли в системе сервер MySQL (смотрите выше) и на тот ли порт вы подключаетесь (как найти порт, можно посмотреть выше).
Похожие частые ошибки, с которыми вы можете столкнуться при попытке подключиться к серверу MySQL:
ERROR 2003: Cannot connect to MySQL server on 'host_name' (111)
ERROR 2002: Cannot connect to local MySQL server through socket '/tmp/mysql.sock' (111)
Ошибки запрета доступа в MySQL
В MySQL учётная запись (УЗ) определяется именем пользователя и клиентским хостом, с которого пользователь может подключиться. УЗ может также иметь данные для аутентификации (например, пароль).
Причин для запрета доступа может быть много. Одна из них связана с учётными записями MySQL, которые сервер разрешает использовать клиентским программам при подключении. Это означает, что имя пользователя, указанное в соединении, может не иметь прав доступа к базе данных.
В MySQL есть возможность создавать учётные записи, позволяющие пользователям клиентских программ подключаться к серверу и получать доступ к данным. Поэтому при ошибке доступа проверьте разрешение УЗ на подключение к серверу через клиентскую программу.
Увидеть разрешённые привилегии учётной записи можно, выполнив в консоли команду SHOW GRANTS
Входим в консоль (пример для Unix, для Windows консоль можно найти в стартовом меню):
В консоли вводим команду:
> SHOW GRANTS FOR 'tecmint'@'localhost';
Дать привилегии конкретному пользователю в БД по IP-адресу можно, используя следующие команды:
> grant all privileges on *.test_db to 'tecmint'@'192.168.0.100';
> flush privileges;
Ошибки запрещённого доступа могут также возникнуть из-за проблем с подключением к MySQL (см. выше).
Потеря соединения с сервером MySQL
С этой ошибкой можно столкнуться по одной из следующих причин:
- плохое сетевое соединение;
- истекло время ожидания соединения;
- размер BLOB больше, чем
max_allowed_packet
.
В первом случае убедитесь, что у вас стабильное сетевое подключение (особенно, если подключаетесь удалённо).
Если проблема с тайм-аутом соединения (особенно при первоначальном соединении MySQL с сервером), увеличьте значение параметра connect_timeout
.
В случае с размером BLOB нужно установить более высокое значение для max_allowed_packet
в файле конфигурации /etc/my.cnf
в разделах [mysqld]
или [client]
как показано ниже.
[mysqld]
connect_timeout=100
max_allowed_packet=500M
Если файл конфигурации недоступен, это значение можно установить с помощью следующей команды.
> SET GLOBAL connect_timeout=100;
> SET GLOBAL max_allowed_packet=524288000;
Слишком много подключений
Эта ошибка означает, что все доступные соединения используются клиентскими программами. Количество соединений (по умолчанию 151) контролируется системной переменной max_connections
. Устранить проблему можно, увеличив значение переменной в файле конфигурации /etc/my.cnf
.
[mysqld]
max_connections=1000
Недостаточно памяти
Если такая ошибка возникла, это может означать, что в MySQL недостаточно памяти для хранения всего результата запроса.
Сначала нужно убедиться, что запрос правильный. Если это так, то нужно выполнить одно из следующих действий:
- если клиент MySQL используется напрямую, запустите его с ключом
--quick switch
, чтобы отключить кешированные результаты; - если вы используете драйвер MyODBC, пользовательский интерфейс (UI) имеет расширенную вкладку с опциями. Отметьте галочкой «Do not cache result» (не кешировать результат).
Также может помочь MySQL Tuner. Это полезный скрипт, который подключается к работающему серверу MySQL и даёт рекомендации по настройке для более высокой производительности.
$ sudo apt-get install mysqltuner #Debian/Ubuntu
$ sudo yum install mysqltuner #RHEL/CentOS/Fedora
$ mysqltuner
MySQL продолжает «падать»
Если такая проблема возникает, необходимо выяснить, заключается она в сервере или в клиенте. Обратите внимание, что многие сбои сервера вызваны повреждёнными файлами данных или индексными файлами.
Вы можете проверить состояние сервера, чтобы определить, как долго он работал.
$ sudo systemctl status mysql #Debian/Ubuntu
$ sudo systemctl status mysqld #RHEL/CentOS/Fedora
Чтобы узнать время безотказной работы сервера, запустите команду mysqladmin
.
$ sudo mysqladmin version -p
Кроме того, можно остановить сервер, сделать отладку MySQL и снова запустить службу. Для отображения статистики процессов MySQL во время выполнения других процессов откройте окно командной строки и введите следующее:
$ sudo mysqladmin -i 5 status
Или
$ sudo mysqladmin -i 5 -r status
Заключение
Самое важное при диагностике — понять, что именно вызвало ошибку. Следующие шаги помогут вам в этом:
- Первый и самый важный шаг — просмотреть журналы MySQL, которые хранятся в каталоге
/var/log/mysql/
. Вы можете использовать утилиты командной строки вродеtail
для чтения файлов журнала. - Если служба MySQL не запускается, проверьте её состояние с помощью
systemctl
. Или используйте командуjournalctl
(с флагом-xe
) в systemd. - Вы также можете проверить файл системного журнала (например,
/var/log/messages
) на предмет обнаружения ошибок. - Попробуйте использовать такие инструменты, как Mytop, glances, top, ps или htop, чтобы проверить, какая программа использует весь ресурс процессора или блокирует машину. Они также помогут определить нехватку памяти, дискового пространства, файловых дескрипторов или какого-либо другого важного ресурса.
- Если проблема в каком-либо процессе, можно попытаться его принудительно остановить, а затем запустить (при необходимости).
- Если вы уверены, что проблемы именно на стороне сервера, можете выполнить команды:
mysqladmin -u root ping
илиmysqladmin -u root processlist
, чтобы получить от него ответ. - Если при подключении проблема не связана с сервером, проверьте, нормально ли работает клиент. Попробуйте получить какие-либо его выходные данные для устранения неполадок.
Перевод статьи «Useful Tips to Troubleshoot Common Errors in MySQL»
If you are experiencing database connection errors, then look no further for troubleshooting assistance! This guide details the common causes of the dreaded Database connection error, how to find the root cause, and what to do to fix it. The commands in this article work with MySQL or MariaDB on CentOS 6 or 7 cPanel servers.
I know that this is a frustrating time, and you are rushed to fix your website, so I’ve listed below the most common causes for the Database Connection Error in order of prevalence to help you find and fix this as quickly as possible (don’t worry, we will discuss how to troubleshoot each of these in detail):
- Out of Memory errors are killing MySQL
- Disk space is exceeded and no further writes can be made to the database.
- Max_connections setting is exceeded.
- MySQL is failing due to a syntax error, is corrupted, or has crashed tables.
- Invalid or mismatched credentials in the site’s database configuration file as compared to those stored in cPanel/MySQL.
- PHP was upgraded, but the site isn’t yet compatible with PHP 7 (still uses the deprecated function mysql_connect).
- There are missing PHP Modules to handle SQL connections.
The first thing you should ask yourself when you have this error is, “Is the error persistent or intermittent?”. Second, “Is this occurring for all sites or just one?”.
If the error is intermittent, check for the following:
- OOMs
- Max_connections being exceeded
If the error is persistent but occurs for all sites, check the following:
- Disk space exceeded
- Whether MySQL is running, has crashed tables, or has InnoDB corruption.
- here are missing PHP Modules to handle SQL connections (rarely seen issue).
If the error is persistent, but only occurs for one site, check the following:
- Invalid or mismatched credentials in the site’s database configuration file as compared to those stored in cPanel/MySQL.
- PHP has been upgraded, but the site isn’t yet compatible with PHP 7 (still uses the deprecated function mysql_connect).
This article discusses in detail how to deal with the following variations of errors you may see logged in different locations in your server:
- Max Connections Exceeded
- main process exited, code=killed, status=9/KILL
- Out of memory in UB ####: OOM Killed process PID (service) score 0 vm:#####mb, rss:#####mb, swap:0kb
- Killed process PID (service) score ### or sacrifice child
- mmap() failed: [12] Cannot allocate memory
- service invoked oom-killer
- OOM killer in rage, # tasks killed
- [ERROR] mysqld: Table ‘./database_name/wp_revslider_css’ is marked as crashed and should be repaired
- myisamchk: error: Can’t create new tempfile: ‘./DATABASE_NAME/TABLE_NAME.TMD’
- The system failed to open the session file “/var/cpanel/sessions/raw/root:########” because of an error: Disk quota exceeded at /usr/local/cpanel/Cpanel/Session.pm line 277.
Out of Memory Errors
If the error is intermittent, you should check for OOMs, or Out of Memory errors. To check for OOMs, you can run the following command:
grep -i 'kill|oom' /var/log/messages
You may see output with errors similar to these:
If this is the case, then you should immediately check your domlogs (domain access logs) for abuse, such as WordPress login brute forcing, XMLRPC abuse, or DoS attacks. Please don’t hesitate to ask KnownHost support for help with this if you are experiencing Out of Memory errors!
If no abuse is occurring, you may want to check the output of sar to determine how long they have been occurring so that you can determine whether you are running your server with insufficient RAM. Sar only stores the last 30 days of information, but will at least give you an overview of RAM usage over the last month. Here is how to check the current day’s RAM usage with sar:
sar -r
The following command can be used to check the RAM usage on the prior day dated the 13th:
sar -f /var/log/sa/sa13
You can use the above command to check any day of the last 30-31 days by replacing ’13’ with the date that you want to check. If ‘dd’ represents the day, then the following is a generalization:
sar -f /var/log/sa/sadd
Just replace ‘dd’ with the day of the month that you want to review RAM usage.
Also, you can use the following command to check for the total number of OOMs that have occurred on your VPS (this command doesn’t work for non-virtualized containers):
grep oom /proc/user_beancounters
The last number listed is the count of OOMs. If no OOMs have occurred, then your output will look like this:
If OOMs have occurred, as in the case of the following server which has had 126K, you will see the number of OOMs that have occurred:
If OOMs are occurring too frequently to be logged via /var/log/messages, then you will want to watch this number. This does sometimes occur where OOMs are happening but are not being logged on the server, though the fail count is logged to /proc/user_beancounters.
If you do find abuse, depending on the type of abuse occurring, you can take several actions. You can hide the wp-login.php, disable access to xmlrpc.php file, block ‘bad’ bots, block IPs from which the abuse is originating from (if the abuse is originating from one to few IPs), and consider a service such as Cloudflare that offers protection from DoS attacks and will help to reduce your RAM usage. These would be resolutions to the most common types of abuse we see.
If you’ve found no abuse occurring, you may consider adding more RAM or disabling RAM-intensive services they may not be needed or in use (cPanel-Solr and Clamd are not required and are known to require a lot of RAM).
You may also check for multiple instances of the same service running. A recent bug in cPanel causes many OOMs due to multiple instances of tailwatchd being instantiated repeatedly. If you find something like this occurring, chances are that cPanel is aware and have released a patch, so you can stop those services, remove them from the Service Manager for long enough to run an update, and then add the service back to the Service Manager once the update completes. If the issue persists, please open a support request with the KnownHost support team.
Exceeded Disk Space
To find out if the disk space is exceeded, run the following command via SSH as root:
df -h
You will see something similar to this showing that you have reached 100% disk space utilization, or are within MB of doing so:
Or try to log into WHM. The following error is typically shown when you have exceeded your disk space and try to access WHM:
When your disk space is exceeded, you can only correct it by accessing via SSH. If you are uncomfortable with SSH, open a support request with KH support setting the priority Critical, and we will work quickly to find what is consuming space and get you back online.
You can use the following command to find what directory beneath root is using the most disk space:
du -Hhsc --exclude virtfs /*
You will get output similar to this:
For this particular server, you can see that 22G is in /backup, and 11G in /home. If you wanted to look further into what is in /home, you can do so as follows:
du -Hhsc --exclude virtfs /home/*
If you find that most of the disk space under /home is under one account, let’s say, account ‘cpuser’ you could then look under that account as follows:
du -Hhsc --exclude virtfs /home/cpuser/*
And then, if you find that the disk space under that account is mostly due to mail, you could find what email account is responsible with the following:
du -Hhsc --exclude virtfs /home/cpuser/mail/*
Thus, you can get an idea of what is consuming the most space by examining the largest directories on the server using the du command.
Note: The reason for the –exclude virtfs tag above is so that virtfs output is not included. The reason is that virtfs will show a lot of space being consumed, however, space is not actually consumed. The /home/virtfs consists of bind mounts. and does not actually take up disk space, even though it will show up as used disk space when you use du. Do not use rm or try to delete the /home/virtfs directory as doing so will also delete the content that it is mounted to and potentially render your server non-functional! If you want to remove these mounts, see the following link:
documentation.cpanel.net/display/68Docs/VirtFS+-+Jailed+Shell
Max Connections
To find out whether MySQL’s configured max_connections are exceeded, you can compare the output of the following commands:
mysql -e "SHOW STATUS" | grep -E 'Max|Upt'
grep "max_con" /etc/my.cnf
You’ll see output similar to the following:
If the value of Max_used_connections from the first command is the value of max_connections + 1 or more, then the max_connections for MySQL have been exceeded. For example, if the Max_used_connections is 51 but the max_connections is 50, then you did exceed your max_connections. As long as traffic looks legitimate and you have sufficient RAM to support the increase, you can increase this setting in the MySQL configuration file by doing the following:
1) edit /etc/my.cnf and change the setting as desired, but do so in small increments as not to exhaust your RAM 2) restart MySQL
This can be accomplished with the following commands:
nano /etc/my.cnf
Ctrl X to exit. You will be prompted to confirm the changes and the file name. Finally, restart MySQL:
service mysql restart
PHP
Did you recently upgrade PHP to version 7? If so, you will want to check the site for the function mysql_connect. This function has been deprecated, and the site software will need to upgraded to be compatible with PHP 7, ( use mysqli_connect instead). You can run the following search via SSH in your site’s document root to check:
grep -Rli mysql_connect.
Regarding Missing modules, this is rarely an issue. However, these are the modules often use to establish a database connection (varies depends on what EasyApache/CustomBuild and PHP versions you are using):
- mysqli
- mysqlnd
- pdo_mysql
- pdo_sqlite
- sqlite3
Database Credentials Mismatch
Another possible cause is that there could be invalid or mismatched credentials in the site’s database configuration file as compared to those stored in the panel/MySQL. This often occurs when a migration is done and you were previously using the old, less secure, MySQL password hashing. cPanel will change the passwords to a secure, random password using the new password hashing format to provide you with utmost security, however, you must update the passwords in cPanel to match those as present in your configuration files after this is done.
The following lists the locations (path is relative to the installation’s root) of the configuration files that stores the database credentials for several popular CMSs:
- WordPress – wp-config.php
- Drupal – sites/default/settings.php
- Joomla – configuration.php
- Magento – app/etc/local.xml
- OpenCart – admin/config.php
- phpBB – config.php
- Prestashop – app/config/parameters.php
- WHMCS – configuration.php
- Laravel – config/database.php
Once you retrieve the database name, username, and password from these files, you will need to log into cPanel for the domain and update the database user’s password.
- Log into CPanel at https://yourdomain.tld/cpanel
- Navigate to ‘MySQL Databases’
- Scroll down to the bottom of the page where you see the list of users
- Locate the key icon next to each user.
- Click the key icon to reset the user’s password to the password used in the configuration file.
Now you can clear your browser cache and test whether or not this has resolved your Database Connection Error.
MySQL Failure, Corruption or Crashed MyIsam Tables
Finally, the worst possible issue… MySQL failure/corruption or crashed tables. The worst case scenario isn’t that bad thanks to KnownHost and their external backup system. If MySQL InnoDB corruption cannot be repaired, we can try to restore the server to a previous backup, which may not have as much corruption, and then be successful in repairing the corruption.
To check for these errors, the first thing that must be done is to check to see if MySQL is running:
service mysql status
Then you will want to check the MySQL logs for errors as shown below:
tail -400 /var/lib/mysql/`hostname`.err
The command will show the last 400 errors. You can adjust the number of errors to show as you see fit. If your error log is in a different location you will want to adjust the command accordingly.
This should give you a clue as to what the issue is. If you are still unsure and MySQL was running in the previous check, you can tail the MySQL error logs while attempting to restart MySQL in another ssh session and this should give you more clues as to why these errors are occurring. Tail the log with this command:
tail -f /var/lib/mysql/`hostname`.err
And, in another terminal, restart MySQL with the following command:
service mysql restart
Syntax Error
If a syntax error is present in the MySQL configuration file, an error will be logged detailing the exact error. You can then edit /etc/my.cnf to comment out or place the symbol # in front of the statement with the syntax error to disable it. Then, try to start MySQL:
service mysql start
If there is no apparent syntax error and the error log fills with a lot of 0’s, .’s, and it just seems that MySQL is incoherent and drunkenly logging to the error log, then it is likely that InnoDB corruption is to blame. The logs may explicitly reference an article from MySQL about InnoDB Forced Recovery as well. If this is the case, InnoDB Forced Recovery must be done.
InnoDB Forced Recovery
MySQL may or may not be running. It may be crashing immediately upon restart and logging errors that do not make much sense. It may also be referencing a link to a MySQL article about recovering InnoDB.
IMPORTANT Make sure you have sufficient RAM and OOMs are not occurring before you recover InnoDB! Also, make sure you remove MySQL from the Service Manager before you perform any of the recovery steps below, and then be certain to reverse this once you have recovered InnoDB! You can toggle ‘monitored’ on and off using a WHM API command. The command below turns it off (replace the ‘0’ with ‘1’ for ‘monitored’ to re-enable):
whmapi1 configureservice service=mysql enabled=1 monitored=0
The first thing you’ll want to do is open a second terminal and watch the MySQL error log as you force InnoDB recovery. You can watch the error log using the following command:
tail -f /var/lib/mysql/`hostname`.err
Watching the error log while performing the steps below will allow you to determine whether InnoDB corruption has been repaired or not because you will no longer see the error that indicates InnoDB corruption is present and MySQL will be working again.
If MySQL is stopped, you may also want to run the following before you start just to make sure that no crashed MyISAM tables prevent your mysqldumps from completing:
myisamchk -rf /var/lib/mysql/*/*
To perform Forced InnoDB Recovery, you must do the following:
- edit the /etc/my.cnf file and add the following: innodb_force_recovery=1
- try to start MySQL if it is not yet running
If MySQL fails to start, try proceeding to innodb_force_recovery=2. You can go all the way to 6, but you want to start with the lowest possible value. The higher the value of innodb_force_recovery, the more data loss likely to occur.
If MySQL starts, then proceed with the following steps:
mysqldump --all-databases > /root/support/alldbs.sql
Now, stop MySQL:
service mysql stop
While stopped, do the following:
mkdir -p /root/support/mysql-bak/
mv /var/lib/mysql/* /root/support/mysql-bak/
cp -pr /root/support/mysql-bak/mysql/ /var/lib/mysql/
Now, edit /etc/my.cnf and comment out (place the symbol # in front of) innodb_force_recovery=1.
service mysql start
mysql < /root/support/alldbs.sql
If MySQL still fails, try repeating Innodb forced recovery with a higher setting. If you reach 6 without success, you may want to request that the support team restores /var/lib/mysql from a previous date when the corruption is hopefully not as bad, and then re-attempt InnoDB forced recovery.
If you see neither of the errors above (syntax errors nor InnoDB corruption), you can check for crashed MyISAM tables.
Crashed MyIsam Tables
To check for crashed MyISAM tables, run the following command, and if errors are present, check that the date for which they are present is, in fact, the current date (cPanel will detect and automatically fix crashed tables for you in most cases, which is why you would need to check the date of the errors):
grep -i crash /var/lib/mysql/`hostname`.err
You will see errors such as this:
[ERROR] mysqld: Table './database_name/wp_revslider_css' is marked as crashed and should be repaired
Fortunately, repairing MyISAM tables is actually pretty easy. You can do so by running the following commands:
mysql
use DATABASE;
repair table TABLENAME;
exit;
IMPORTANT Make sure you have sufficient RAM and OOMs are not occurring before you repair your tables!
You could even use the following alternate means to repair the table as well:
myisamchk --recover TABLENAME
Or the following to repair the database:
mysqlcheck --repair --databases DATABASE
Note that myisamchk can be run while MySQL is not running, whereas mysqlcheck and repair table can only be run while MySQL is running.
You will replace DATABASE with the database’s name, and TABLENAME with the table name. You get these directly from the error logged above, which is in the format:
yymmdd hh:mm:ss [ERROR] mysqld: Table ‘./database_name/table_name’ is marked as crashed and should be repaired
Here is what this type of repair looks like:
Occasionally, this repair will fail due to a temporary table. The error will read something similar to this:
myisamchk: error: Can't create new tempfile: './DATABASE_NAME/TABLE_NAME.TMD'
Here is what this error looks like when you encounter it after attempting to repair a table via the MySQL CLI:
Note that ‘TMD’ will actually be capitalized in your server, whereas the actual ‘/DATABASE_NAME/TABLE_NAME’ will be lowercase.
If this occurs, you will need to remove the temporary table. I prefer to rename it to preserve the copy, which can be done via the following command:
mv /var/lib/mysql/DATABASE/TABLENAME.TMD /var/lib/mysql/DATABASE/TABLENAME.TMD.bak
You should now be able to run the repair successfully.
If you have multiple crashed tables, you can use the following command to check, analyze, optimize, and repair all tables (some of these functions only work with certain storage engines, such as repair with MyISAM):
mysqlcheck -Aa; mysqlcheck -Ace; mysqlcheck -Arv; mysqlcheck -Ao
You may want to watch the error log in a separate terminal while running this command:
tail -f /var/lib/mysql/`hostname`.err
If your databases are very large, then you may want to use the screen utility to run the command above in to make sure that it can complete even if you log out:
screen
Conclusion
Keep frequent MySQL backups! This is very important. KnownHost does take full VPS backups for our VPS customers every other day and offers awesome backup solutions to our dedicated server clients. There are some limitations to our backups that can be overcome with cPanel backups, so cPanel backups are certainly advised.
Obviously, we wouldn’t want you to deal with these issues alone, especially if you are unfamiliar with SSH. We are here to help. If you encounter any of these issues, please submit a support request and we’ll be happy to assist. Regardless of the case, we will do our best to have your server functioning and online as soon as possible. We are here to help!