Clickhouse container shuts down randomly

I’m on a Ubuntu 18.04.5 VServer. My Plausible installation pretty often shows me the 500 error.
Sometimes after minutes, sometimes after days.

The reason is that the Clickhouse container shuts down. All I get from docker-compose logs is:

plausible_1            | 08:31:01.229 [error] Clickhousex.Protocol (#PID<0.4252.0>) disconnected: ** (Clickhousex.Error) closed

Docker has access to all 8G memory of my machine and when I look at top, clickhouse-server has a max of 10% CPU and 10% MEM.

I tried exec -iting into the clickhouse container and look at the logs, but I cannot do that when the container is down. And after restart the logs are empty.

Can someone help me to get better logs and analyze them, so I can find out if the issue is with Clickhouse, with Docker or with Ubuntu?

Okay, I finally found a way to look into clickhouse-server.err.log.

2020.11.19 10:56:17.125972 [ 101 ] {} <Fatal> BaseDaemon: ########################################
2020.11.19 10:56:17.126097 [ 101 ] {} <Fatal> BaseDaemon: (version 20.11.3.3 (official build), build id: C88CD350740ED614) (from thread 90) (query_id: 5778912e-c592-4890-b2bf-7b29e7d2936a) Received signal Segmentation fault (11)
2020.11.19 10:56:17.126177 [ 101 ] {} <Fatal> BaseDaemon: Address: 0x10a8edf000 Access: read. Address not mapped to object.
2020.11.19 10:56:17.126229 [ 101 ] {} <Fatal> BaseDaemon: Stack trace: 0x7cb3bcd 0x7f2bc06e53c0
2020.11.19 10:56:17.126319 [ 101 ] {} <Fatal> BaseDaemon: 0. ? @ 0x7cb3bcd in /usr/bin/clickhouse
2020.11.19 10:56:17.126391 [ 101 ] {} <Fatal> BaseDaemon: 1. ? @ 0x153c0 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so

So I guess it’s a clickhouse bug? This is the respective section from clickhouse-server.log

2020.11.19 10:56:17.123637 [ 90 ] {} <Trace> HTTPHandler-factory: HTTP Request for HTTPHandler-factory. Method: POST, Address: 172.18.0.5:51515, User-Agent: hackney/1.15.2, Length: 19, Content Type: text/plain, Transfer Encoding: identity
2020.11.19 10:56:17.123723 [ 90 ] {} <Trace> DynamicQueryHandler: Request URI: /?database=plausible_events_db&query=SELECT+1
2020.11.19 10:56:17.124031 [ 90 ] {5778912e-c592-4890-b2bf-7b29e7d2936a} <Debug> executeQuery: (from 172.18.0.5:51515) SELECT 1 FORMAT JSONCompact
2020.11.19 10:56:17.124250 [ 90 ] {5778912e-c592-4890-b2bf-7b29e7d2936a} <Trace> ContextAccess (default): Access granted: SELECT(dummy) ON system.one
2020.11.19 10:56:17.124312 [ 90 ] {5778912e-c592-4890-b2bf-7b29e7d2936a} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
2020.11.19 10:56:17.125103 [ 45 ] {} <Trace> BaseDaemon: Received signal 11
2020.11.19 10:56:17.125972 [ 101 ] {} <Fatal> BaseDaemon: ########################################
2020.11.19 10:56:17.126097 [ 101 ] {} <Fatal> BaseDaemon: (version 20.11.3.3 (official build), build id: C88CD350740ED614) (from thread 90) (query_id: 5778912e-c592-4890-b2bf-7b29e7d2936a) Received signal Segmentation fault (11)
2020.11.19 10:56:17.126177 [ 101 ] {} <Fatal> BaseDaemon: Address: 0x10a8edf000 Access: read. Address not mapped to object.
2020.11.19 10:56:17.126229 [ 101 ] {} <Fatal> BaseDaemon: Stack trace: 0x7cb3bcd 0x7f2bc06e53c0
2020.11.19 10:56:17.126319 [ 101 ] {} <Fatal> BaseDaemon: 0. ? @ 0x7cb3bcd in /usr/bin/clickhouse
2020.11.19 10:56:17.126391 [ 101 ] {} <Fatal> BaseDaemon: 1. ? @ 0x153c0 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
2020.11.19 10:56:17.126476 [ 101 ] {} <Information> SentryWriter: Not sending crash report

Update: I opened this issue with ClickHouse on github, hoping for more insight into the problem: https://github.com/ClickHouse/ClickHouse/issues/17222