Multiple Instances of Java & Excessive RAM and CPU Usage

Get Help
1

Hello,
I hope this topic is in the right place.
Initially program was being run via PM2 but it made our server clog up too much(to a point where killing the processes via ssh took like 30 minutes.)
Later we trasferred into systemd
still, in the journalctl logs it appears to be restarted because of cpu usage, i guess?
Are we missing something?

metabase.sh

#!/bin/bash                                                                     
cd·/home/olcay·||·exit·1                                                        
                                                                                
echo·"Starting·Metabase..."                                                     
java·-jar·metabase.jar·>>·metabase_startup.log·2>&1                             
                                                                                
export·MB_JETTY_SSL="true"                                                      
export·MB_JETTY_SSL_PORT="3443"                                                 
export·MB_JETTY_SSL_KEYSTORE="/home/olcay/Turuncu/ssls/gw_ssls/keystore.jks"       
export·MB_JETTY_SSL_KEYSTORE_PASSWORD="" # removed before posting there but it is there.                            
                                                                                
LOG_FILE="/home/olcay/metabase_startup.log"                                     
                                                                                
nohup·java·-jar·metabase.jar·>>·"$LOG_FILE"·2>&1·&                              
                                                                                
echo·$!·>·/home/olcay/metabase.pid                                              
                                                                                
echo·"Metabase·started·with·PID·$(cat·/home/olcay/metabase.pid).·Logs·available·at·$LOG_FILE."

/etc/systemd/system/metabase.service

[Unit]
Description=Metabase Service
After=network.target

[Service]
User=olcay
ExecStart=/usr/bin/bash /home/olcay/metabase.sh
Type=simple
Restart=always
WorkingDirectory=/home/olcay/

[Install]
WantedBy=multi-user.target

journalctl logs(it goes like this forever)

Oct 26 11:46:26 nodejsgateway systemd[1]: Stopped Metabase Service.
Oct 26 11:46:26 nodejsgateway systemd[1]: metabase.service: Consumed 35.353s CPU time.
Oct 26 11:46:26 nodejsgateway systemd[1]: Started Metabase Service.
Oct 26 11:46:26 nodejsgateway bash[2856480]: Starting Metabase...
Oct 26 11:46:39 nodejsgateway bash[2856480]: Metabase started with PID 2856521. Logs available at /home/olcay/metabase_startup.log.
Oct 26 11:46:39 nodejsgateway systemd[1]: metabase.service: Deactivated successfully.
Oct 26 11:46:39 nodejsgateway systemd[1]: metabase.service: Consumed 32.368s CPU time.
Oct 26 11:46:39 nodejsgateway systemd[1]: metabase.service: Scheduled restart job, restart counter is at 5566.
Oct 26 11:46:39 nodejsgateway systemd[1]: Stopped Metabase Service.
Oct 26 11:46:39 nodejsgateway systemd[1]: metabase.service: Consumed 32.368s CPU time.
Oct 26 11:46:39 nodejsgateway systemd[1]: Started Metabase Service.
Oct 26 11:46:39 nodejsgateway bash[2856524]: Starting Metabase...
Oct 26 11:46:52 nodejsgateway bash[2856524]: Metabase started with PID 2856563. Logs available at /home/olcay/metabase_startup.log.
Oct 26 11:46:52 nodejsgateway systemd[1]: metabase.service: Deactivated successfully.
Oct 26 11:46:52 nodejsgateway systemd[1]: metabase.service: Consumed 33.646s CPU time.
Oct 26 11:46:52 nodejsgateway systemd[1]: metabase.service: Scheduled restart job, restart counter is at 5567.
Oct 26 11:46:52 nodejsgateway systemd[1]: Stopped Metabase Service.
Oct 26 11:46:52 nodejsgateway systemd[1]: metabase.service: Consumed 33.646s CPU time.
Oct 26 11:46:52 nodejsgateway systemd[1]: Started Metabase Service.
Oct 26 11:46:52 nodejsgateway bash[2856566]: Starting Metabase...
Oct 26 11:47:05 nodejsgateway bash[2856566]: Metabase started with PID 2856611. Logs available at /home/olcay/metabase_startup.log.
Oct 26 11:47:05 nodejsgateway systemd[1]: metabase.service: Deactivated successfully.
Oct 26 11:47:05 nodejsgateway systemd[1]: metabase.service: Consumed 33.695s CPU time.
Oct 26 11:47:06 nodejsgateway systemd[1]: metabase.service: Scheduled restart job, restart counter is at 5568.
Oct 26 11:47:06 nodejsgateway systemd[1]: Stopped Metabase Service.
Oct 26 11:47:06 nodejsgateway systemd[1]: metabase.service: Consumed 33.695s CPU time.
Oct 26 11:47:06 nodejsgateway systemd[1]: Started Metabase Service.
Oct 26 11:47:06 nodejsgateway bash[2856614]: Starting Metabase...
Oct 26 11:47:19 nodejsgateway bash[2856614]: Metabase started with PID 2856653. Logs available at /home/olcay/metabase_startup.log.
Oct 26 11:47:19 nodejsgateway systemd[1]: metabase.service: Deactivated successfully.
Oct 26 11:47:19 nodejsgateway systemd[1]: metabase.service: Consumed 31.474s CPU time.
Oct 26 11:47:19 nodejsgateway systemd[1]: metabase.service: Scheduled restart job, restart counter is at 5569.
Oct 26 11:47:19 nodejsgateway systemd[1]: Stopped Metabase Service.
Oct 26 11:47:19 nodejsgateway systemd[1]: metabase.service: Consumed 31.474s CPU time.
Oct 26 11:47:19 nodejsgateway systemd[1]: Started Metabase Service.
Oct 26 11:47:19 nodejsgateway bash[2856655]: Starting Metabase...
Oct 26 11:47:32 nodejsgateway bash[2856655]: Metabase started with PID 2856697. Logs available at /home/olcay/metabase_startup.log.
Oct 26 11:47:32 nodejsgateway systemd[1]: metabase.service: Deactivated successfully.
Oct 26 11:47:32 nodejsgateway systemd[1]: metabase.service: Consumed 33.173s CPU time.
Oct 26 11:47:32 nodejsgateway systemd[1]: metabase.service: Scheduled restart job, restart counter is at 5570.
Oct 26 11:47:32 nodejsgateway systemd[1]: Stopped Metabase Service.
lines 1-45

olcay@nodejsgateway:~$ ll | grep metabase
-rwxrwxr-x  1 olcay olcay  166 Oct 17 11:55 metabase_database_backup.sh*
-rw-rw-r--  1 olcay olcay 9.7M Oct 27 11:47 metabase.db.mv.db
-rw-rw-r--  1 olcay olcay  27K Sep 27 10:46 metabase.db.trace.db
-rw-rw-r--  1 olcay olcay 371M Sep 24 21:54 metabase.jar
-rw-rw-r--  1 olcay olcay    8 Oct 27 10:35 metabase.pid
-rwxrwxr-x  1 olcay olcay  378 Oct 17 10:49 __metabase.sh*
-rwxrwxr-x  1 olcay olcay  536 Oct 17 11:03 metabase.sh*
-rw-rw-r--  1 olcay olcay 310M Oct 27 11:41 metabase_startup.log
olcay@nodejsgateway:~$

~/metabase_startup.log

ly.invokeStatic(core.clj:667)
	at clojure.core$apply.invoke(core.clj:662)
	at metabase.bootstrap$_main.invokeStatic(bootstrap.clj:31)
	at metabase.bootstrap$_main.doInvoke(bootstrap.clj:28)
	at clojure.lang.RestFn.invoke(RestFn.java:397)
	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at metabase.bootstrap.main(Unknown Source)
Caused by: java.net.BindException: Address already in use
	at java.base/sun.nio.ch.Net.bind0(Native Method)
	at java.base/sun.nio.ch.Net.bind(Net.java:459)
	at java.base/sun.nio.ch.Net.bind(Net.java:448)
	at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
	at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:339)
	... 32 more
2024-10-27 10:35:28,798 INFO metabase.util :: Maximum memory available to JVM: 7.8 GB
2024-10-27 10:35:30,603 INFO util.encryption :: Saved credentials encryption is DISABLED for this Metabase instance. 🔓 
 For more information, see https://metabase.com/docs/latest/operations-guide/encrypting-database-details-at-rest.html
2024-10-27 10:35:31,143 WARN db.env :: WARNING: Using Metabase with an H2 application database is not recommended for production deployments. For production deployments, we highly recommend using Postgres, MySQL, or MariaDB instead. If you decide to continue to use H2, please be sure to back up the database file regularly. For more information, see https://metabase.com/docs/latest/operations-guide/migrating-from-h2.html
2024-10-27 10:35:35,245 INFO driver.impl :: Registered abstract driver :sql  🚚
2024-10-27 10:35:35,277 INFO driver.impl :: Registered abstract driver :sql-jdbc (parents: [:sql]) 🚚
2024-10-27 10:35:35,282 INFO metabase.util :: Load driver :sql-jdbc took 15.8 ms
2024-10-27 10:35:35,282 INFO driver.impl :: Registered driver :h2 (parents: [:sql-jdbc]) 🚚
2024-10-27 10:35:35,513 INFO driver.impl :: Registered driver :mysql (parents: [:sql-jdbc]) 🚚
2024-10-27 10:35:35,544 INFO driver.impl :: Registered driver :postgres (parents: [:sql-jdbc]) 🚚
2024-10-27 10:35:36,928 INFO metabase.core :: 
Metabase v0.50.27 (8b9a8fc) 

Copyright © 2024 Metabase, Inc. 

Metabase Enterprise Edition extensions are NOT PRESENT.
2024-10-27 10:35:36,936 INFO metabase.core :: Starting Metabase in STANDALONE mode
2024-10-27 10:35:36,972 INFO metabase.server :: Launching Embedded Jetty Webserver with config:
 {:port 3000}

2024-10-27 10:35:36,993 ERROR metabase.core :: Metabase Initialization FAILED
java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:3000
	at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:344)
	at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:304)
	at org.eclipse.jetty.server.Server.lambda$doStart$0(Server.java:402)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
	at org.eclipse.jetty.server.Server.doStart(Server.java:398)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
	at metabase.server$start_web_server_BANG_.invokeStatic(server.clj:120)
	at metabase.server$start_web_server_BANG_.invoke(server.clj:106)
	at metabase.core$start_normally.invokeStatic(core.clj:181)
	at metabase.core$start_normally.invoke(core.clj:177)
	at metabase.core$entrypoint.invokeStatic(core.clj:216)
	at metabase.core$entrypoint.doInvoke(core.clj:210)
	at clojure.lang.RestFn.invoke(RestFn.java:397)
	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$apply.invoke(core.clj:662)
	at metabase.bootstrap$_main.invokeStatic(bootstrap.clj:31)
	at metabase.bootstrap$_main.doInvoke(bootstrap.clj:28)
	at clojure.lang.RestFn.invoke(RestFn.java:397)
	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at metabase.bootstrap.main(Unknown Source)
Caused by: java.net.BindException: Address already in use
	at java.base/sun.nio.ch.Net.bind0(Native Method)
	at java.base/sun.nio.ch.Net.bind(Net.java:459)
	at java.base/sun.nio.ch.Net.bind(Net.java:448)
	at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
	at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:339)
	... 32 more
2

There might be some moments where Metabase uses the cpu heavily, please read our documentation about using Metabase in production

3

Hi,
I read the doc you mentioned, there is only one user(barely does anything)
I dont think problem i mentioned is due to server's hardware

ejsgateway:~$ sudo lshw -short
[sudo] password for olcay: 
H/W path              Device      Class      Description
========================================================
                                  system     VMware Virtual Platform
/0                                bus        440BX Desktop Reference Platform
/0/0                              memory     86KiB BIOS
/0/1                              processor  Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz
/0/1/0                            memory     16KiB L1 cache
/0/1/1                            memory     16KiB L1 cache
/0/2                              processor  Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz
/0/5                              processor  Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz
/0/5/1d                           memory     16KiB L1 cache
/0/6                              processor  Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz
/0/6/1e                           memory     16KiB L1 cache
/0/7                              processor  CPU
/0/7/1f                           memory     16KiB L1 cache
/0/8                              processor  CPU
/0/8/20                           memory     16KiB L1 cache
/0/9                              processor  CPU
/0/9/21                           memory     16KiB L1 cache
/0/a                              processor  CPU
/0/a/22                           memory     16KiB L1 cache
/0/3a                             memory     32GiB System Memory
/0/3a/0                           memory     16GiB DIMM DRAM EDO
/0/3a/1                           memory     16GiB DIMM DRAM EDO
/0/3a/2                           memory     DIMM DRAM [empty]
/0/3a/3                           memory     DIMM DRAM [empty]
/0/3a/4                           memory     DIMM DRA

### Metabase application server size

For every 20 concurrent people using your Metabase, Metabase will need 1 CPU and 1GB of RAM. These system recommendations apply whether you’re running Metabase as a JAR or as a Docker container.

### Metabase application database server size

The application database is probably the most important component of the entire architecture: it’s the single point of failure, and the faster the app db can return queries to the Metabase application server, the better. As a starting point, allocate 1 CPU core and 2GB of RAM the server running your application database. As a general rule, for every 40 concurrent people using your Metabase, a PostgreSQL application database will need 1 CPU core and 1 GB of RAM.