Can dbt load data to the data warehouse #295
Unanswered
woshimajintao
asked this question in
Q&A
Replies: 1 comment
-
dbt-duckdb can be used to load data into a DuckDB database via its plugin system, but it probably should not be used to do that, especially for the kinds of complex data loading scenarios required for e.g. a TPC benchmark. Loading data is generally the most operationally complex part of running a data pipeline (i.e., the place where the most number of things can go wrong) and so using one of the numerous tools dedicated to doing loading correctly is the best choice in all but the simplest data loading scenarios. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I want to use dbt+duckdb to implement the TPC-DI bencmark (ETL process).
https://www.vldb.org/pvldb/vol7/p1367-poess.pdf
I just know dbt can be used to transform data.And I would like to know can dbt load data to the data warehouse?
I hope to your reply.
Beta Was this translation helpful? Give feedback.
All reactions