VVV
December 19, 2018, 5:17pm
1
Hey guys
I found that Metabase provides a driver for connecting to SparkSQL databases,
But unfortunately, this SparkSQL driver doesn't work with my hiveserver.
logs like this:
12-19 09:11:57 DEBUG metabase.middleware :: POST /api/database 400 (5 s) (0 DB calls).
{:valid false, :dbname "Timed out after 5000 milliseconds.", :message "Timed out after 5000 milliseconds."}
12-19 09:13:08 ERROR metabase.driver :: Failed to connect to database: java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://172.0.0.11:8080/test: Invalid status 72
i have seen many of these discussions and issues on github,
Hi, guys!
I’m testing the new Spark SQL driver but I’m not sure how to connect Metabase to my cluster. My Metabase is running on the same machine of my Spark Master.
I’ve tried
Host: localhost
Host: local[*]
Host: spark://localhost
Always on port 7077, but the server is returning me this error:
05-02 17:48:25 ERROR metabase.driver :: Failed to connect to database: java.lang.NoClassDefFoundError: org/apache/h
adoop/conf/Configuration
05-02 17:48:25 DEBUG metabase.middleware :: POST /api/set…
opened 10:36PM - 02 May 18 UTC
closed 07:27PM - 16 May 18 UTC
Type:Bug
Priority:P1
Hi, I'm testing the new Spark SQL driver in Metabase v0.29 and it's not working.… O filled the "new database" form with required info and got the error ```Couldn't connect to the database. Please check the connection details.```
When I check the logs the following info is displayed:
```sh
05-02 22:18:30 DEBUG metabase.middleware :: GET /api/user/current 200 (7 ms) (1 DB calls). Jetty threads: 8/50 (4 busy, 6 idle, 0 queued)
05-02 22:18:31 DEBUG metabase.middleware :: GET /api/setting 200 (2 ms) (0 DB calls). Jetty threads: 8/50 (4 busy, 6 idle, 0 queued)
05-02 22:18:50 ERROR metabase.driver :: Failed to connect to database: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
```
It seems that we're missing some hadoop-commons files in the build (reference: https://github.com/metabase/metabase/issues/2157#issuecomment-386065796). Not sure if it's important to highlight, but I'm trying to connect to a remote Spark (I can access it using other tools, but I'm not able to to this using my Metabase).
* Operating System: Ubuntu 17;10
* Database: Spark SQL
* Metabase version: 0.29.0-RC1 and 0.29.0
* Metabase hosting environment: Docker in my local machine
* Metabase internal Database: PostgreSQL
opened 01:46PM - 16 Mar 16 UTC
Database/
Priority:P2
Type:New Feature
Hi,
Hive / Apache Drill support would be great !
Best regards,
Damien
… ⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍 comment**
metabase:master
← wjoel:spark-sql
opened 04:01PM - 30 Mar 17 UTC
Only tested with the Spark Thrift Server and Drill #2157, but might work with ot… her Hive compatible SQL engines (Hive itself)
###### TODO
- [x] Sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform)
(unless it's a tiny documentation change).
- [x] Add tests? If so, I might need some help. (edit: 2208 tests pass)
opened 12:11AM - 12 May 18 UTC
closed 05:08PM - 26 Jun 19 UTC
Type:Bug
Priority:P2
Hi,
I'm using a Metabase version with SparkSQL driver and it's working correc… tly (I described what I did in https://github.com/metabase/metabase/issues/7528#issuecomment-387095716 ). I was able to add a new connection to my Hive instance and run queries in it.
However, Metabase can't read my Database Schema. Everytime I press the button `Sync database schema now` I receive the following error:
```sh
05-11 21:01:42 INFO sync.util :: STARTING: Sync metadata for sparksql Database 2 'teste'
05-11 21:01:42 ERROR sync.util :: Error running sync step: Output of db-metadata does not match schema:
{:tables #{{:name (not (instance? java.lang.String nil))}}}
("sync.fetch_metadata$fn__39574$db_metadata__39579.invoke(fetch_metadata.clj:10)"
"sync.sync_metadata.tables$fn__40704$db_metadata__40709$fn__40710.invoke(tables.clj:127)"
"sync.sync_metadata.tables$fn__40704$db_metadata__40709.invoke(tables.clj:124)"
"sync.sync_metadata.tables$fn__40764$sync_tables_BANG___40769$fn__40770.invoke(tables.clj:144)"
"sync.sync_metadata.tables$fn__40764$sync_tables_BANG___40769.invoke(tables.clj:139)"
"sync.sync_metadata$fn__40794$sync_db_metadata_BANG___40799$fn__40800$fn__40801.invoke(sync_metadata.clj:26)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:124)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:122)"
"sync.util$do_with_error_handling.invoke(util.clj:119)"
"driver$fn__26280.invokeStatic(driver.clj:245)"
"driver$fn__26280.invoke(driver.clj:245)"
"driver$fn__26215$G__26022__26224.invoke(driver.clj:61)"
"sync.util$sync_in_context$fn__38140.invoke(util.clj:115)"
"sync.util$with_db_logging_disabled$fn__38137.invoke(util.clj:106)"
"sync.util$with_start_and_finish_logging$fn__38132.invoke(util.clj:92)"
"sync.util$with_sync_events$fn__38129.invoke(util.clj:75)"
"sync.util$with_duplicate_ops_prevented$fn__38120.invoke(util.clj:54)"
"sync.util$do_sync_operation.invokeStatic(util.clj:142)"
"sync.util$do_sync_operation.invoke(util.clj:139)"
"sync.sync_metadata$fn__40794$sync_db_metadata_BANG___40799$fn__40800.invoke(sync_metadata.clj:23)"
"sync.sync_metadata$fn__40794$sync_db_metadata_BANG___40799.invoke(sync_metadata.clj:20)"
"api.database$fn__41692$fn__41693$fn__41694.invoke(database.clj:510)")
05-11 21:01:42 INFO sync.util :: FINISHED: Sync metadata for sparksql Database 2 'teste' (77 ms)
```
I'm not sure if this bug is related to Metabase or to the driver version I'm using (it's not the latest one), but I believe this point is worth fixing because without the schema sync, Metabase is not able to display my Hive Tables in Admin->Data Model view and also it's not able to display tables in "Custom Question" and in "Native Query" views.
- Metabase version: v.0.29.0 (I compiled it with SparkSQL dependencies)
- Operating system: Ubuntu 17.10
- SparkSQL driver version: v0.13.1
... ... ...
but never have the solution for supporting the jdbc://hive(not 2),
I supposed that only jdbc:hive but not jdbc:hive2 would be worked.
so maybe the root cause is "org.apache.hive.jdbc",
im new in Clojure and failed to rebuild spark deps ,
how can i use "org.apache.hadoop.hive.jdbc" instead?
or can we build a new plugin for older hive,
or support two kind of protocol on the same driver like this:
any idea of this?
thx a lot!