Hello!
We've had great feedback from our customers on our beta product which incorporates metabase. However, there was a mySql crash on 3/20, right around the time we started using metabase (the day after, actually).
Here are the relevant versions:
Metabase:
You're on version v0.33.2
Built on 2019-09-05
MySql:
version innodb_version 5.7.25
The mysql log:
https://sicks3bucket.s3.amazonaws.com/sick_an_mysql.err
Here's the metabase log:
https://sicks3bucket.s3.amazonaws.com/metabase_log.txt
Only two queries were run:
Query 1:
select date, system_name, device_name, valid_read_rate from
(SELECT left(from_unixtime(c.capture_time/1000),10) as date, a.name as 'system_name', b.name as 'device_name', c.read_count, c.valid_object_count,
if(valid_object_count = 0,'00.00%',concat(format(read_count/valid_object_count*100,2),'%')) as valid_read_rate, read_count/valid_object_count*100 as valid_read_rate_unformatted
FROM sick_il_services.system_config a
join sick_il_services.system_device b
on a.id = b.system_id
join sick_il_aap.daily_device_read_rate c
on b.id = c.device_id
where 1=1
[[AND c.capture_time >= unix_timestamp(concat({{start_date}}, ' 00:00:00.000')) * 1000 ]]
[[AND c.capture_time <= unix_timestamp(concat({{end_date}}, ' 23:59:59.999')) * 1000]]
[[AND a.name = '{{system_name}}']]
group by c.capture_time,a.name, b.name
order by c.capture_time, a.name, b.name) as sub
where sub.valid_read_rate_unformatted < {{expected_read_rate_percent}}
Query 2:
SELECT from_unixtime(capture_time/1000), group_concat(distinct `daily_system_read_rate`.`system_id`) AS `System Ids`,
`daily_system_read_rate`.`condition_level` AS `condition_level`,
`daily_system_read_rate`.`condition_name` AS `condition_name`,
sum(`daily_system_read_rate`.`read_count`) AS `read_count`,
sum(`daily_system_read_rate`.`valid_object_count`) AS `valid_object_count`,
((sum(`daily_system_read_rate`.`read_count`) * 100) / CASE WHEN sum(`daily_system_read_rate`.`valid_object_count`) = 0 THEN NULL ELSE sum(`daily_system_read_rate`.`valid_object_count`) END) AS `read_rate`
FROM `daily_system_read_rate`
Where condition_id != -1
AND `daily_system_read_rate`.`capture_time` >= unix_timestamp(concat({{start_date}}, ' 00:00:00.000')) * 1000
[[AND `daily_system_read_rate`.`capture_time` <= unix_timestamp(concat({{end_date}}, ' 23:59:59.999')) * 1000]]
[[AND {{system_id}}]]
[[AND {{condition_name}}]]
GROUP BY capture_time, `daily_system_read_rate`.`condition_level`, `daily_system_read_rate`.`condition_name`
ORDER BY capture_time, `daily_system_read_rate`.`condition_level` ASC, `daily_system_read_rate`.`condition_name` ASC
The error message from the MySql log where it crashed is as follows:
2021-03-20T11:50:22.345980-07:00 5196223 [Note] Aborted connection 5196223 to db: 'sick_il_services' user: 'sick_ap_reporting' host: 'localhost' (Got timeout reading communication packets)
22:45:44 UTC - mysqld got exception 0xc00000fd ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.
key_buffer_size=49152
read_buffer_size=2048000
max_used_connections=75
max_threads=200
thread_count=40
connection_count=40
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1221685 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Nothing after that until the restart on 3/25...
The error code matches exactly matches this SO post, which resolved the error by removing the sub-query.
(I can't post the link, but here's the message)
Im trying to get the 10 best matches of a fulltext search using fulltext score.
Following query works perfectly on my 10M lines MyISAM table :
SELECT index, MATCH(villes) AGAINST('montreuil') as score FROM ( SELECT index, villes FROM table WHERE MATCH(villes) AGAINST('montreuil' IN BOOLEAN MODE) ) tHowever, when I get an error when trying to retrieve to order scores :
SELECT index FROM ( *** PREVIOUS QUERY *** ) t2 ORDER BY t2.scoreError is : "Lost connection to mysql server" in mysql workbench after 4 seconds.
So far I've tried:
- using phpmyadmin => same error
- removing the "order by" => same error
- extending the mysql buffer memory => same error
- limiting the number of the subquery results to 10 (using "limit 10") => weird error "cannot find a fulltext index"
The mysql log is unclear to me :
10:10:58 UTC - mysqld got exception 0xc00000fd ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. Attempting to collect some information that could help diagnose the problem. As this is a crash and something is definitely wrong, the information collection process might fail. key_buffer_size=67108864 read_buffer_size=8388608 max_used_connections=3 max_threads=151 thread_count=2 connection_count=2 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2541349 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x21ee0a9b580 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... 2019-04-16T10:11:12.359438Z 0 [Warning] 'NO_AUTO_CREATE_USER' sql mode was not set.EDIT
SHOW CREATE TABLE :
'CREATE TABLE `table` ( `index` int(11) NOT NULL, `villes` text, PRIMARY KEY (`index`), FULLTEXT KEY `villes` (`villes`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8'EXPLAIN :
type: fulltext possible keys: villes key: villes ref: const rows: 1 filtered: 100.00 extra: using where, using filesort
And here's the fix:
I couldnt find a direct solution to this problem. I read some posts explaining that sorting a subquery using fulltext might cause some memory issues.
However a workaround was to create a temporary table with the result of the subquery and sort it which works perfectly.
Is this simply a matter of restructuring the query to remove the subquery? I put in in for ease of use, but if it's going to crash mysql, obviously it's useless. Is this a metabase problem or a mysql/subquery problem?
Thanks!