Could not initialize class org.duckdb.DuckDBNative #1

tomsej · 2022-09-22T18:28:43Z

Hi, thank you for this wonderful driver. Unfortunately, I am unable to install it into the official Docker image. I have tried following Dockerfile:

FROM metabase/metabase:v0.43.0

RUN apk add libc6-compat

ADD https://github.com/AlexR2D2/metabase_duckdb_driver/releases/download/0.1.1/duckdb.metabase-driver.jar /plugins/
RUN chmod 744 /plugins/duckdb.metabase-driver.jar

But when I try to create a DuckDB database connection, I get the following error:

2022-09-22 20:20:00,334 ERROR api.database :: Cannot connect to Database
java.lang.Exception: Could not initialize class org.duckdb.DuckDBNative
	at metabase.driver.util$can_connect_with_details_QMARK_.invokeStatic(util.clj:55)
	at metabase.driver.util$can_connect_with_details_QMARK_.doInvoke(util.clj:39)
	at clojure.lang.RestFn.invoke(RestFn.java:442)
	at metabase.api.database$test_database_connection.invokeStatic(database.clj:514)
	at metabase.api.database$test_database_connection.doInvoke(database.clj:503)
	at clojure.lang.RestFn.invoke(RestFn.java:425)
	at metabase.api.database$fn__74666$test_connection_details__74671$fn__74672.invoke(database.clj:563)
	at metabase.api.database$fn__74666$test_connection_details__74671.invoke(database.clj:544)
	at metabase.api.database$fn__74698.invokeStatic(database.clj:580)
	at metabase.api.database$fn__74698.invoke(database.clj:566)
	at compojure.core$wrap_response$fn__28029.invoke(core.clj:160)
	at compojure.core$wrap_route_middleware$fn__28013.invoke(core.clj:132)
	at compojure.core$wrap_route_info$fn__28018.invoke(core.clj:139)
	at compojure.core$wrap_route_matches$fn__28022.invoke(core.clj:151)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041.invoke(core.clj:200)
	at metabase.server.middleware.auth$enforce_authentication$fn__66669.invoke(auth.clj:14)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041.invoke(core.clj:200)
	at compojure.core$make_context$handler__28069.invoke(core.clj:289)
	at compojure.core$make_context$fn__28073.invoke(core.clj:299)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at compojure.core$make_context$fn__28073.invoke(core.clj:300)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at compojure.core$make_context$fn__28073.invoke(core.clj:300)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at compojure.core$make_context$fn__28073.invoke(core.clj:300)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at compojure.core$make_context$fn__28073.invoke(core.clj:300)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at compojure.core$make_context$fn__28073.invoke(core.clj:300)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at compojure.core$make_context$fn__28073.invoke(core.clj:300)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at compojure.core$make_context$fn__28073.invoke(core.clj:300)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at metabase.api.routes$fn__78001$fn__78004.invoke(routes.clj:59)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041.invoke(core.clj:200)
	at clojure.lang.AFn.applyToHelper(AFn.java:160)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$apply.invoke(core.clj:662)
	at metabase.server.routes$fn__78149$fn__78150.doInvoke(routes.clj:57)
	at clojure.lang.RestFn.invoke(RestFn.java:436)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041.invoke(core.clj:200)
	at compojure.core$make_context$handler__28069.invoke(core.clj:289)
	at compojure.core$make_context$fn__28073.invoke(core.clj:299)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at compojure.core$wrap_route_matches$fn__28022.invoke(core.clj:153)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at compojure.core$wrap_route_matches$fn__28022.invoke(core.clj:153)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at compojure.core$wrap_route_matches$fn__28022.invoke(core.clj:153)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041$f__28042$respond_SINGLEQUOTE___28043.invoke(core.clj:197)
	at metabase.server.routes$fn__78137$fn__78139.invoke(routes.clj:41)
	at compojure.core$routes$fn__28041$f__28042.invoke(core.clj:198)
	at compojure.core$routes$fn__28041.invoke(core.clj:200)
	at metabase.server.middleware.exceptions$catch_uncaught_exceptions$fn__74985.invoke(exceptions.clj:98)
	at metabase.server.middleware.exceptions$catch_api_exceptions$fn__74982.invoke(exceptions.clj:86)
	at metabase.server.middleware.log$log_api_call$fn__78619$fn__78620$fn__78621.invoke(log.clj:211)
	at metabase.driver.sql_jdbc.execute.diagnostic$do_with_diagnostic_info.invokeStatic(diagnostic.clj:15)
	at metabase.driver.sql_jdbc.execute.diagnostic$do_with_diagnostic_info.invoke(diagnostic.clj:9)
	at metabase.server.middleware.log$log_api_call$fn__78619$fn__78620.invoke(log.clj:203)
	at toucan.db$_do_with_call_counting.invokeStatic(db.clj:216)
	at toucan.db$_do_with_call_counting.invoke(db.clj:209)
	at metabase.server.middleware.log$log_api_call$fn__78619.invoke(log.clj:202)
	at metabase.server.middleware.browser_cookie$ensure_browser_id_cookie$fn__81845.invoke(browser_cookie.clj:38)
	at metabase.server.middleware.security$add_security_headers$fn__60355.invoke(security.clj:148)
	at metabase.server.middleware.json$wrap_json_body$fn__80994.invoke(json.clj:63)
	at metabase.server.middleware.json$wrap_streamed_json_response$fn__81012.invoke(json.clj:99)
	at metabase.server.middleware.offset_paging$handle_paging$fn__60379.invoke(offset_paging.clj:42)
	at ring.middleware.keyword_params$wrap_keyword_params$fn__82112.invoke(keyword_params.clj:55)
	at ring.middleware.params$wrap_params$fn__82131.invoke(params.clj:77)
	at metabase.server.middleware.misc$maybe_set_site_url$fn__35170.invoke(misc.clj:59)
	at metabase.server.middleware.session$bind_current_user$fn__46635$fn__46636.invoke(session.clj:291)
	at metabase.server.middleware.session$do_with_current_user.invokeStatic(session.clj:270)
	at metabase.server.middleware.session$do_with_current_user.invoke(session.clj:259)
	at metabase.server.middleware.session$bind_current_user$fn__46635.invoke(session.clj:290)
	at metabase.server.middleware.session$wrap_current_user_info$fn__46617.invoke(session.clj:240)
	at metabase.server.middleware.session$wrap_session_id$fn__46601.invoke(session.clj:173)
	at metabase.server.middleware.auth$wrap_api_key$fn__66677.invoke(auth.clj:27)
	at ring.middleware.cookies$wrap_cookies$fn__82032.invoke(cookies.clj:216)
	at metabase.server.middleware.misc$add_content_type$fn__35153.invoke(misc.clj:27)
	at metabase.server.middleware.misc$disable_streaming_buffering$fn__35178.invoke(misc.clj:76)
	at ring.middleware.gzip$wrap_gzip$fn__82074.invoke(gzip.clj:86)
	at metabase.server.middleware.misc$bind_request$fn__35181.invoke(misc.clj:93)
	at metabase.server.middleware.ssl$redirect_to_https_middleware$fn__81861.invoke(ssl.clj:38)
	at metabase.server$async_proxy_handler$fn__78391.invoke(server.clj:73)
	at metabase.server.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a.handle(Unknown Source)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:516)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.duckdb.DuckDBNative
	at org.duckdb.DuckDBDatabase.<init>(DuckDBDatabase.java:22)
	at org.duckdb.DuckDBDriver.connect(DuckDBDriver.java:35)
	at metabase.plugins.jdbc_proxy$proxy_driver$reify__60163.connect(jdbc_proxy.clj:33)
	at java.sql/java.sql.DriverManager.getConnection(Unknown Source)
	at java.sql/java.sql.DriverManager.getConnection(Unknown Source)
	at clojure.java.jdbc$get_driver_connection.invokeStatic(jdbc.clj:271)
	at clojure.java.jdbc$get_driver_connection.invoke(jdbc.clj:250)
	at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:411)
	at clojure.java.jdbc$get_connection.invoke(jdbc.clj:274)
	at clojure.java.jdbc$db_query_with_resultset_STAR_.invokeStatic(jdbc.clj:1111)
	at clojure.java.jdbc$db_query_with_resultset_STAR_.invoke(jdbc.clj:1093)
	at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1182)
	at clojure.java.jdbc$query.invoke(jdbc.clj:1144)
	at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1160)
	at clojure.java.jdbc$query.invoke(jdbc.clj:1144)
	at metabase.driver.sql_jdbc.connection$can_connect_with_spec_QMARK_.invokeStatic(connection.clj:264)
	at metabase.driver.sql_jdbc.connection$can_connect_with_spec_QMARK_.invoke(connection.clj:261)
	at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invokeStatic(connection.clj:272)
	at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invoke(connection.clj:268)
	at metabase.driver.sql_jdbc$fn__83386.invokeStatic(sql_jdbc.clj:35)
	at metabase.driver.sql_jdbc$fn__83386.invoke(sql_jdbc.clj:33)
	at clojure.lang.MultiFn.invoke(MultiFn.java:234)
	at metabase.driver.util$can_connect_with_details_QMARK_$fn__32661.invoke(util.clj:51)
	at metabase.util$do_with_timeout$fn__6203.invoke(util.clj:353)
	at clojure.core$binding_conveyor_fn$fn__5772.invoke(core.clj:2034)
	at clojure.lang.AFn.call(AFn.java:18)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	... 1 more
2022-09-22 20:20:00,349 DEBUG middleware.log :: POST /api/database 400 7.6 ms (0 DB calls)
{:valid false,
 :dbname "Could not initialize class org.duckdb.DuckDBNative",
 :message "Could not initialize class org.duckdb.DuckDBNative"}

Not sure if the problem is the Alpine base image of Metabase.

The text was updated successfully, but these errors were encountered:

AlexR2D2 · 2022-09-27T08:04:00Z

Hello, thanks for feedback, i am working on issue from time to time :)

AlexR2D2 · 2022-10-05T18:07:07Z

Yes, the problem in the Alpine image of Metabase. The DuckDB embeds the icu lib. ICU requeres the glibc pthread lib (not the musl pthread lib installed in the Alpine by default). I tried to install different versions of the glibc library to the Metabase Alpine image but there is some issues with anyway.

Until 2.32-r0 version of the glibc, there is the std::system_error in the icu std::call_once call. I this this is the same error as in microsoft/ClearScript#331.

Starting from 2.33-r0 version of glibc there is "Error relocating /lib/ld-linux-x86-64.so.2: unsupported relocation type 37" error.

So, instead of digging into all of this issues i just created a Ubuntu based image of Metabase and added DuckDB plugin. Please see Docker section of README.md

peralmq · 2023-03-01T16:22:15Z

@AlexR2D2 is the script still working for you? For me it is stuck. Or.. how long should it take? I've been waiting for 2 hours and it seems stalled on the below line => [builder 4/4] RUN INTERACTIVE=false CI=true MB_EDITION=oss bin/bu 6773.7s.

And I really appreciate that the build script was created in the first place. I'm on a Mac M1 if that matters. But am using the export DOCKER_DEFAULT_PLATFORM=linux/amd64 environment variable.


 => => #       Create backend artifact /home/circleci/resources/i18n/tr.edn from /home/circleci/locales/tr.po                                                 
[+] Building 2859.0s (10/13)                                                    => CACHED [runner 1/4] FROM docker.io/library/eclipse-temurin:11-jre@sh  0.0s [+] Building 6777.0s (10/13)
 => [internal] load build definition from Dockerfile                      0.0s  => => transferring dockerfile: 1.91kB                                    0.0s  => [internal] load .dockerignore                                         0.0s
 => => transferring context: 274B                                         0.0s  => [internal] load metadata for docker.io/library/eclipse-temurin:11-jr  1.2s  => [internal] load metadata for docker.io/metabase/ci:java-11-clj-1.11.  1.2s
 => [internal] load build context                                         1.3sf => => transferring context: 111.86MB                                     1.3s  => [builder 1/4] FROM docker.io/metabase/ci:java-11-clj-1.11.0.1100.04-  0.0s
 => CACHED [runner 1/4] FROM docker.io/library/eclipse-temurin:11-jre@sh  0.0s
 => [runner 2/4] RUN apt-get update &&     apt-get install -y bash font  94.8s
 => CACHED [builder 2/4] WORKDIR /home/circleci                           0.0s
 => [builder 3/4] COPY --chown=circleci . .                               0.7s
 => [builder 4/4] RUN INTERACTIVE=false CI=true MB_EDITION=oss bin/bu  6773.7s
 => => #           Wrote 3602 messages.
 => => #       Create backend artifact /home/circleci/resources/i18n/tr.edn fr
 => => #         Create directory /home/circleci/resources/i18n if it does not
 => => #           /home/circleci/resources/i18n already exists.
 => => #         Delete /home/circleci/resources/i18n/tr.edn if exists
 => => #           Don't need to delete /home/circleci/resources/i18n/tr.edn,

AlexR2D2 · 2023-03-02T08:42:14Z

@peralmq yet it still works in my mac. It takes around 20 minutes to build from scratch (2,5 GHz Quad-Core Intel Core i7, 16 GB 1600 MHz DDR3). But there were a couple of times it hanging too.

I pushed my image to docker hub, so you can try the follow:

Change the file content metabase_duckdb_driver/docker/Dockerfile to look like this:

FROM elmidster/ubuntu_metabase:v1.44.6

COPY target/duckdb.metabase-driver.jar /plugins/
RUN chmod 744 /plugins/duckdb.metabase-driver.jar

Download plugin into target dir

metabase_duckdb_driver $ mkdir target
metabase_duckdb_driver $ wget -P target https://github.com/AlexR2D2/metabase_duckdb_driver/releases/download/0.1.6/duckdb.metabase-driver.jar

Build image

metabase_duckdb_driver $ docker build -t metabase_duckdb -f docker/Dockerfile .

peralmq · 2023-03-02T14:27:11Z

Thanks so much for publishing it to DockerHub, it works! 🙏

I skipped the separate downloading of the plugin and just did it all in the Dockerfile

FROM elmidster/ubuntu_metabase:v1.44.6

ADD https://github.com/AlexR2D2/metabase_duckdb_driver/releases/download/0.1.6/duckdb.metabase-driver.jar /plugins/
RUN chmod 744 /plugins/duckdb.metabase-driver.jar

yet it still works in my mac. It takes around 20 minutes to build from scratch (2,5 GHz Quad-Core Intel Core i7, 16 GB 1600 MHz DDR3). But there were a couple of times it hanging too.

OK, I tried and retried to build it on this M1 Pro and it always stalled on the same place.. must be something with the CPU architecture, I find it to be a source of difficulty still when building docker images.

gianluca-valentini · 2024-04-03T11:09:01Z

Hi, any news about this?
Hi, I have the same problem using DuckDB on Alpine 3.10.
java.lang.NoClassDefFoundError: Could not initialize class org.duckdb.DuckDBNative at org.duckdb.DuckDBConnection.newConnection(DuckDBConnection.java:48) ~[duckdb_jdbc-0.10.1.jar:?] at org.duckdb.DuckDBDriver.connect(DuckDBDriver.java:41) ~[duckdb_jdbc-0.10.1.jar:?] at java.sql.DriverManager.getConnection(DriverManager.java:677) ~[java.sql:?] at java.sql.DriverManager.getConnection(DriverManager.java:189) ~[java.sql:?]

Is there a workaround to fix it?
I'm using DuckDB 0.10.1 in memory
The code that create the error is

ByteBuffer nativeReference =
            DuckDBNative.duckdb_jdbc_startup(db_dir.getBytes(StandardCharsets.UTF_8), readOnly, properties);

and after this, I cannot understand what happen.

UPDATE
the problem is present when I run jar file using java -cp I hope that it can help

AlexR2D2 · 2024-04-03T11:46:15Z

Hi, did you try the native DuckDB binary in Alpine?

gianluca-valentini · 2024-04-03T13:37:49Z

Hi @AlexR2D2

Thank you for reaching out. I haven't tried the native DuckDB binary in Alpine yet. Could you provide me with more details on how to proceed?

Currently, I have a pod based on Alpine that needs to utilize DuckDB. The pod is launched using the command "java -cp ". Could you please guide me on how to test the DuckDB binary within this setup?

Thanks a lot!

Gianluca

AlexR2D2 · 2024-04-03T13:55:59Z

The idea is to check if the native DuckDB binary works in Alpine or not. You can check this using docker image of Alpine. Most likely the DuckDB will not work there. Because DuckDB embeds the icu lib requeres the glibc pthread lib (not the musl pthread lib installed in the Alpine by default). If so you can't get metabase_duckdb_driver working in Alpine

gianluca-valentini · 2024-04-03T15:40:16Z

@AlexR2D2
thanks a lot. It does not work on Alpine. I hope that DuckDB will fix it.
Changing pod SO with UBUNTU, it works fine

Thanks a lot for your help
Gianluca

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Could not initialize class org.duckdb.DuckDBNative #1

Could not initialize class org.duckdb.DuckDBNative #1

tomsej commented Sep 22, 2022 •

edited

Loading

AlexR2D2 commented Sep 27, 2022

AlexR2D2 commented Oct 5, 2022

peralmq commented Mar 1, 2023 •

edited

Loading

AlexR2D2 commented Mar 2, 2023 •

edited

Loading

peralmq commented Mar 2, 2023

gianluca-valentini commented Apr 3, 2024 •

edited

Loading

AlexR2D2 commented Apr 3, 2024

gianluca-valentini commented Apr 3, 2024

AlexR2D2 commented Apr 3, 2024

gianluca-valentini commented Apr 3, 2024

Could not initialize class org.duckdb.DuckDBNative #1

Could not initialize class org.duckdb.DuckDBNative #1

Comments

tomsej commented Sep 22, 2022 • edited Loading

AlexR2D2 commented Sep 27, 2022

AlexR2D2 commented Oct 5, 2022

peralmq commented Mar 1, 2023 • edited Loading

AlexR2D2 commented Mar 2, 2023 • edited Loading

peralmq commented Mar 2, 2023

gianluca-valentini commented Apr 3, 2024 • edited Loading

AlexR2D2 commented Apr 3, 2024

gianluca-valentini commented Apr 3, 2024

AlexR2D2 commented Apr 3, 2024

gianluca-valentini commented Apr 3, 2024

tomsej commented Sep 22, 2022 •

edited

Loading

peralmq commented Mar 1, 2023 •

edited

Loading

AlexR2D2 commented Mar 2, 2023 •

edited

Loading

gianluca-valentini commented Apr 3, 2024 •

edited

Loading