Hardware and Storage Sizing for Production

Get Help
1

Hi Team,

I would like to understand what is required HW Sizing and Application DB Sizing required for following requirements:

  1. No of Concurrent Queries - 50
  2. Query Size - Avg 1 Gb / Query
  3. Will be hosted using JAR on RHEL
    Note: As per architecture Metabase will connect with existing TD or HIVE to fetch data.

Do let me know if any other parameter needed to come up with accurate sizing.

Also, Is it recommended to run on a single instance or multiple instance ?

Cheers,
JK

2

Hi @JKotha
It’s a difficult question to answer. It depends on how Metabase is used, if you have cache, how fast queries completes, and multiple other things. The best solution is to start low and then scale up.

  1. What does that mean? That there will be 50 queries running actively at any time? That must mean you have thousands of users or long-running-queries.
  2. Are you returning 1 GigaByte of data in every query? That doesn’t sound correct.
  3. You should probably run multiple instances in horizontal scaling, sharing the same application database, so the users can be load balanced.
3

Hi Flamber,

Thanks for your your response. However, there should be a way to size Hardware requirements based on some assumptions.

I am proposing Metabase for one of our Telecom customer and hence data could be huge. For them to procure HW from respective OEM I need to provide them HW Sizing based on certain assumption.

So there should be a way to size Metabase based on certain sizing parameters.

Is there any benchmark available for Metabase? What should be the sizing for Application DB ? (In my case I am proposing MySQL DB)

Thanks,
Jay

4

Hi Flamber,

Thanks for your your response. However, there should be a way to size Hardware requirements based on some assumptions.

I am proposing Metabase for one of our Telecom customer and hence data could be huge. For them to procure HW from respective OEM I need to provide them HW Sizing based on certain assumption.

So there should be a way to size Metabase based on certain sizing parameters.

Is there any benchmark available for Metabase? What should be the sizing for Application DB ? (In my case I am proposing MySQL DB)

Thanks,
Jay

5

@JKotha
But it’s like asking the HW requirements for a Windows server, but without providing all the details, then it’s a difficult question to answer.

Don’t they have any spare HW or VM, where they could do some testing?

If you’re really meaning that the database will return a result of 1 GigaByte of data on each query, then I don’t think Metabase is the right tool.

I’m sure it would work with 4GB RAM and then scale up depending on usage.

I would probably recommend Postgres for the backend. And the amount of RAM would depend on how much data your datasources have, since it would define how much information is stored during sync+scan.

6

@flamber, my client is looking for fixed price contract. and would like to get details about HW upfront based on Metabase implementation somewhere else.

I understand 1 Gb / query may not be realistic however since this is client’s requirement I cannot completely ignore it.

But thanks for your input. For now I will propose 16 GB RAM & 8 Core CPU for Metabase application & 10 Gb storage for application DB. Hope that will address their requirement.