I'm having a lot of problems with MariaDB.
I created a InnoDB table few days ago and I started to add records... after a crash the table got corrupted and I lost close to all. I started again and now a power failure and i lost all again.
After 2 errors, I added a full backup of the database each hour with mysqldump. Now i was able to return to the previous point. I got a third error, all the with exception 0x80000003. This time the 3rd time it was without any freeze or power off...., i must add that all times i have reinstalled mysql ( i had problems to reset and delete the databases/tables with the error even changing innodb_force_recovery = 1 to 6 ). This hard disk was a 512gb SSD.
Reading the comments in different pages I tried with a different Hard Disk, now a normal 2TB HDD, i got the last backup, and I inserted all rows....
It worked fine the first minutes, but without inserting new rows.. just exploring the table with the 30+ millions of rows it crashed again with same exception....
I have lost more than 15 days already with this problem, it doesnt make any sense at all. I have tried all kind of different my.cfg files. Last time it was just my-innodb-heavy-4G.ini
Last time it failed just exploring ,but all previous times it failed inserting masively (usually i have 100 threads doing selects and inserts).
I'm a bit tired of this, i dont remember this kind of stability problems with mysql 2-3 years ago I was doing something similar.
Table structure that is giving all error is:
-- MySQL dump 10.16 Distrib 10.1.13-MariaDB, for Win32 (AMD64)
--
-- Host: localhost Database: info
-- ------------------------------------------------------
-- Server version 10.1.13-MariaDB
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Table structure for table `urls`
--
DROP TABLE IF EXISTS `urls`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `urls` (
`ID` int(11) NOT NULL AUTO_INCREMENT,
`Host` varchar(64) COLLATE utf8_bin NOT NULL,
`Path` varchar(512) COLLATE utf8_bin NOT NULL,
`Code` smallint(5) unsigned DEFAULT NULL,
`UpdatedOn` timestamp NULL DEFAULT NULL,
`NextUpdate` datetime DEFAULT NULL,
`UrlType` tinyint(3) unsigned NOT NULL DEFAULT '0',
`LastParse` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`ID`),
KEY `Host` (`Host`),
KEY `Path` (`Path`(255)),
KEY `UrlType` (`UrlType`),
KEY `Code` (`Code`),
KEY `LastParse` (`LastParse`),
KEY `UpdatedOn` (`UpdatedOn`),
KEY `NextUpdate` (`NextUpdate`)
) ENGINE=InnoDB AUTO_INCREMENT=30181676 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
/*!40101 SET character_set_client = @saved_cs_client */;
I will show the last config:
# Example MySQL config file for small systems.
#
# This is for a system with little memory (<= 64M) where MySQL is only used
# from time to time and it's important that the mysqld daemon
# doesn't use much resources.
#
# You can copy this file to
# E:/xampp/mysql/bin/my.cnf to set global options,
# mysql-data-dir/my.cnf to set server-specific options (in this
# installation this directory is E:/xampp/mysql/data) or
# ~/.my.cnf to set user-specific options.
#
# In this file, you can use all long options that a program supports.
# If you want to know which options a program supports, run the program
# with the "--help" option.
# The following options will be passed to all MySQL clients
[client]
# password = your_password
port = 3306
socket = "E:/xampp/mysql/mysql.sock"
# Here follows entries for some specific programs
# The MySQL server
[mysqld]
port= 3306
socket = "E:/xampp/mysql/mysql.sock"
basedir = "E:/xampp/mysql"
tmpdir = "E:/xampp/tmp"
datadir = "E:/xampp/mysql/data"
pid_file = "mysql.pid"
# enable-named-pipe
key_buffer = 16M
max_allowed_packet = 1M
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
log_error = "mysql_error.log"
# Change here for bind listening
# bind-address="127.0.0.1"
# bind-address = ::1 # for ipv6
# Where do all the plugins live
plugin_dir = "E:/xampp/mysql/lib/plugin/"
# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (via the "enable-named-pipe" option) will render mysqld useless!
#
# commented in by lampp security
#skip-networking
#skip-federated
# Replication Master Server (default)
# binary logging is required for replication
# log-bin deactivated by default since XAMPP 1.4.11
#log-bin=mysql-bin
# required unique id between 1 and 2^32 - 1
# defaults to 1 if master-host is not set
# but will not function as a master if omitted
server-id = 1
# Replication Slave (comment out master section to use this)
#
# To configure this host as a replication slave, you can choose between
# two methods :
#
# 1) Use the CHANGE MASTER TO command (fully described in our manual) -
# the syntax is:
#
# CHANGE MASTER TO MASTER_HOST=<host>, MASTER_PORT=<port>,
# MASTER_USER=<user>, MASTER_PASSWORD=<password> ;
#
# where you replace <host>, <user>, <password> by quoted strings and
# <port> by the master's port number (3306 by default).
#
# Example:
#
# CHANGE MASTER TO MASTER_HOST='125.564.12.1', MASTER_PORT=3306,
# MASTER_USER='joe', MASTER_PASSWORD='secret';
#
# OR
#
# 2) Set the variables below. However, in case you choose this method, then
# start replication for the first time (even unsuccessfully, for example
# if you mistyped the password in master-password and the slave fails to
# connect), the slave will create a master.info file, and any later
# change in this file to the variables' values below will be ignored and
# overridden by the content of the master.info file, unless you shutdown
# the slave server, delete master.info and restart the slaver server.
# For that reason, you may want to leave the lines below untouched
# (commented) and instead use CHANGE MASTER TO (see above)
#
# required unique id between 2 and 2^32 - 1
# (and different from the master)
# defaults to 2 if master-host is set
# but will not function as a slave if omitted
#server-id = 2
#
# The replication master for this slave - required
#master-host = <hostname>
#
# The username the slave will use for authentication when connecting
# to the master - required
#master-user = <username>
#
# The password the slave will authenticate with when connecting to
# the master - required
#master-password = <password>
#
# The port the master is listening on.
# optional - defaults to 3306
#master-port = <port>
#
# binary logging - not required for slaves, but recommended
#log-bin=mysql-bin
# Point the following paths to different dedicated disks
#tmpdir = "E:/xampp/tmp"
#log-update = /path-to-dedicated-directory/hostname
# Uncomment the following if you are using BDB tables
#bdb_cache_size = 4M
#bdb_max_lock = 10000
# Comment the following if you are using InnoDB tables
#skip-innodb
innodb_data_home_dir = "E:/xampp/mysql/data"
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = "E:/xampp/mysql/data"
#innodb_log_arch_dir = "E:/xampp/mysql/data"
## You can set .._buffer_pool_size up to 50 - 80 %
## of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 16M
innodb_additional_mem_pool_size = 2M
## Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 5M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50
## UTF 8 Settings
#init-connect=\'SET NAMES utf8\'
#collation_server=utf8_unicode_ci
#character_set_server=utf8
#skip-character-set-client-handshake
#character_sets-dir="E:/xampp/mysql/share/charsets"
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates
[isamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
And this is the log:
2016-07-14 21:40:52 3528 [Note] Using unique option prefix 'myisam_recover' is error-prone and can break in the future. Please use the full name 'myisam-recover-options' instead.
2016-07-14 21:40:52 dc8 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.
2016-07-14 21:40:52 3528 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2016-07-14 21:40:52 3528 [Note] InnoDB: The InnoDB memory heap is disabled
2016-07-14 21:40:52 3528 [Note] InnoDB: Mutexes and rw_locks use Windows interlocked functions
2016-07-14 21:40:52 3528 [Note] InnoDB: Memory barrier is not used
2016-07-14 21:40:52 3528 [Note] InnoDB: Compressed tables use zlib 1.2.3
2016-07-14 21:40:52 3528 [Note] InnoDB: Using generic crc32 instructions
2016-07-14 21:40:52 3528 [Note] InnoDB: Initializing buffer pool, size = 2.0G
2016-07-14 21:40:52 3528 [Note] InnoDB: Completed initialization of buffer pool
2016-07-14 21:40:52 3528 [Note] InnoDB: Highest supported file format is Barracuda.
2016-07-14 21:40:52 3528 [Note] InnoDB: The log sequence numbers 10385392078 and 10385392078 in ibdata files do not match the log sequence number 10385392118 in the ib_logfiles!
2016-07-14 21:40:52 3528 [Note] InnoDB: Database was not shutdown normally!
2016-07-14 21:40:52 3528 [Note] InnoDB: Starting crash recovery.
2016-07-14 21:40:52 3528 [Note] InnoDB: Reading tablespace information from the .ibd files...
2016-07-14 21:40:52 3528 [Note] InnoDB: Restoring possible half-written data pages
2016-07-14 21:40:52 3528 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Last MySQL binlog file position 0 393530316, file name .\mysql-bin.000005
2016-07-14 21:40:52 3528 [Note] InnoDB: 128 rollback segment(s) are active.
2016-07-14 21:40:52 3528 [Note] InnoDB: Waiting for purge to start
2016-07-14 21:40:52 3528 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 10385392118
2016-07-14 21:40:53 3052 [Note] InnoDB: Dumping buffer pool(s) not yet started
2016-07-14 21:40:53 3528 [Note] Plugin 'FEEDBACK' is disabled.
2016-07-14 21:40:53 3528 [Note] Recovering after a crash using mysql-bin
2016-07-14 21:40:53 3528 [Note] Starting crash recovery...
2016-07-14 21:40:53 3528 [Note] Crash recovery finished.
2016-07-14 21:40:53 3528 [Note] Server socket created on IP: '::'.
2016-07-14 21:40:53 3528 [Note] E:\xampp\mysql\bin\mysqld.exe: ready for connections.
Version: '10.1.13-MariaDB' socket: '' port: 3306 mariadb.org binary distribution
InnoDB: Error: trying to access page number 513921 in space 37,
InnoDB: space name cookpad@002ecom/urls,
InnoDB: which is outside the tablespace bounds.
InnoDB: Byte offset 0, len 16384, i/o type 10.
InnoDB: If you get this error at mysqld startup, please check that
InnoDB: your my.cnf matches the ibdata files that you have in the
InnoDB: MySQL server.
2016-07-14 21:41:07 25e4 InnoDB: Assertion failure in thread 9700 in file fil0fil.cc line 5866
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
160714 21:41:07 [ERROR] mysqld got exception 0x80000003 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Server version: 10.1.13-MariaDB
key_buffer_size=33554432
read_buffer_size=2097152
max_used_connections=2
max_threads=1001
thread_count=2
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1895694 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0x4c722730
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
mysqld.exe!my_parameter_handler()
mysqld.exe!my_mb_ctype_mb()
mysqld.exe!?get_ctx@MDL_ticket@@QBEPAVMDL_context@@XZ()
mysqld.exe!?get_ctx@MDL_ticket@@QBEPAVMDL_context@@XZ()
mysqld.exe!?functype@Item_func_dyncol_create@@UBE?AW4Functype@Item_func@@XZ()
mysqld.exe!?get_ctx@MDL_ticket@@QBEPAVMDL_context@@XZ()
mysqld.exe!??0Global_read_lock@@QAE@XZ()
mysqld.exe!??0Global_read_lock@@QAE@XZ()
mysqld.exe!?get_trg_event_map@Update_rows_log_event@@UAEEXZ()
mysqld.exe!?get_trg_event_map@Update_rows_log_event@@UAEEXZ()
mysqld.exe!?ha_open@handler@@QAEHPAUTABLE@@PBDHI@Z()
mysqld.exe!?open_table_from_share@@YA?AW4open_frm_error@@PAVTHD@@PAUTABLE_SHARE@@PBDIIIPAUTABLE@@_N@Z()
mysqld.exe!?open_table@@YA_NPAVTHD@@PAUTABLE_LIST@@PAVOpen_table_context@@@Z()
mysqld.exe!?recover_from_failed_open@Open_table_context@@QAE_NXZ()
mysqld.exe!?open_tables@@YA_NPAVTHD@@ABUDDL_options_st@@PAPAUTABLE_LIST@@PAIIPAVPrelocking_strategy@@@Z()
mysqld.exe!?open_normal_and_derived_tables@@YA_NPAVTHD@@PAUTABLE_LIST@@II@Z()
mysqld.exe!?send_data@select_result_text_buffer@@UAEHAAV?$List@VItem@@@@@Z()
mysqld.exe!?get_all_tables@@YAHPAVTHD@@PAUTABLE_LIST@@PAVItem@@@Z()
mysqld.exe!?get_schema_tables_result@@YA_NPAVJOIN@@W4enum_schema_table_state@@@Z()
mysqld.exe!?exec_inner@JOIN@@QAEXXZ()
mysqld.exe!?exec@JOIN@@QAEXXZ()
mysqld.exe!?handle_select@@YA_NPAVTHD@@PAULEX@@PAVselect_result@@K@Z()
mysqld.exe!??0Table_scope_and_contents_source_st@@QAE@ABU0@@Z()
mysqld.exe!?mysql_execute_command@@YAHPAVTHD@@@Z()
mysqld.exe!?mysql_parse@@YAXPAVTHD@@PADIPAVParser_state@@@Z()
mysqld.exe!?dispatch_command@@YA_NW4enum_server_command@@PAVTHD@@PADI@Z()
mysqld.exe!?do_command@@YA_NPAVTHD@@@Z()
mysqld.exe!?threadpool_process_request@@YAHPAVTHD@@@Z()
mysqld.exe!?tp_end@@YAXXZ()
KERNEL32.DLL!SetUserGeoID()
ntdll.dll!TpSimpleTryPost()
ntdll.dll!EtwNotificationRegister()
KERNEL32.DLL!BaseThreadInitThunk()
ntdll.dll!RtlUnicodeStringToInteger()
ntdll.dll!RtlUnicodeStringToInteger()
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x4c780ec8): SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` AS `Name`,
`TABLE_TYPE` AS `TABLE_TYPE`,
`ENGINE` AS `Engine`,
`ENGINE` AS `Type`,
`VERSION` AS `Version`,
`ROW_FORMAT` AS `Row_format`,
`TABLE_ROWS` AS `Rows`,
`AVG_ROW_LENGTH` AS `Avg_row_length`,
`DATA_LENGTH` AS `Data_length`,
`MAX_DATA_LENGTH` AS `Max_data_length`,
`INDEX_LENGTH` AS `Index_length`,
`DATA_FREE` AS `Data_free`,
`AUTO_INCREMENT` AS `Auto_increment`,
`CREATE_TIME` AS `Create_time`,
`UPDATE_TIME` AS `Update_time`,
`CHECK_TIME` AS `Check_time`,
`TABLE_COLLATION` AS `Collation`,
`CHECKSUM` AS `Checksum`,
`CREATE_OPTIONS` AS `Create_options`,
`TABLE_COMMENT` AS `Comment`
FROM `information_schema`.`TABLES` t
WHERE `TABLE_SCHEMA`
IN ('cookpad.com')
ORDER BY Name ASC LIMIT 250 OFFSET 0
Connection ID (thread ID): 7
Status: NOT_KILLED
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Last time with innodb_force_recovery = 4 i was not able to even read the database, and with innodb_force_recovery = 5 and 6 the table.
With no inserts at all after the import, and just a fresh install in a new disk it failed. Anyways ALL times i got the same error, exactly the same error
Any tip to solve this stability problem? I though that i was the freeze of my computer, after the power off, then the hard disk or maybe too many inserts in parallel, and with 4th error.. really i dont get it, maybe configuration or mysql/mariadb is not good enough to work with millions of records.
As note I use it from PHP and just a thread launcher that i coded do the parallel HTTP calls to apache
User contributions licensed under CC BY-SA 3.0