!1565 add mariadb connector

Merge pull request !1565 from yuyao/master

hetu-docs/en/connector/mariadb.md

@@ -0,0 +1,97 @@
+The MariaDB connector allows querying and creating tables in external MariaDB databases. This can be used to join data between different systems such as MySQL and MariaDB or between two different MariaDB instances.
+
+## Configuration
+
+To configure the MariaDB connector, create a catalog properties file in `etc/catalog` in the openLooKeng installation directory, for example `maria.properties`, and mount the MariaDB connector as the maria directory. Create the file using the following and replace the connection properties according to the settings.
+
+```properties
+connector.name=maria
+connection-url=jdbc:mariadb://Deployed MairaDB ip:MariaDB service port (default 3306)
+connection-user=Your data source user
+connection-password=Your data source password
+```
+
+- Whether to enable the query push-down feature
+
+If you want to enable the connector push-down feature of MariaDB connector, you don't need to do anything, the MariaDB connector push-down feature is turned on by default. However, it can also be set as follows.
+
+```properties
+#true means push down is turned on, false means off.
+jdbc.pushdown-enabled=true
+```
+
+- Push down mode selection
+
+The default push-down mode of MariaDB connector is partial push-down, if you want to enable the full push-down function of MariaDB connector, you can set it as follows.
+
+```properties
+#FULL_PUSHDOWN, means all push down; BASE_PUSHDOWN, means partial push down, where partial push down means filter/aggregation/limit/topN/project which can be pushed down.
+jdbc.pushdown-module=FULL_PUSHDOWN
+```
+
+### External function registration
+
+The MariaDB connector supports registration of external functions.
+
+Configure the external function registration namespace `catalog.schema` that supports push-down. For example, in `etc/catalog/maria.properties` configure.
+
+```properties
+jdbc.pushdown.remotenamespace=mariafun.default
+```
+
+### External function push-down
+
+Push down external functions to MariaDB data sources for execution.
+
+Configure the external function registration namespace `catalog.schema` that supports push-down. For example, in `etc/catalog/maria.properties` configure.
+
+```properties
+jdbc.pushdown.remotenamespace=mariafun.default
+```
+
+You can declare that you support functions in multiple function namespaces, just use '|' split in the `jdbc.pushdown.remotenamespace` configuration item. For example
+
+```properties
+# indicates that the current Connector instance supports both mysqlfun1.default, mysqlfun2.default, and mysqlfun3.default function namespaces final function push down to the currently connected data source for execution.
+jdbc.pushdown.remotenamespace=mariafun1.default|mariafun2.default|mariafun3.default
+```
+
+### Multiple MariaDB servers
+
+As many catalogs as needed can be created, so if there are additional MariaDB servers, just add another properties file with a different name to `etc/catalog` (make sure it ends with `.properties`). For example, if the properties file is named `sales.properties`, openLooKeng will create a catalog named `sales` using the configured connector.
+
+## Querying MariaDB
+
+The MariaDB Connector provides a schema for each MariaDB database. Available MariaDB databases can be viewed by executing `SHOW SCHEMAS`
+
+```sql
+-- The catalog name for this case is called maria
+SHOW SCHEMAS FROM maria;
+```
+
+If you have a MariaDB database named `test`, you can view the tables in the database by executing `SHOW TABLES`
+
+```sql
+SHOW TABLES FROM maria.test;
+```
+
+You can view a list of columns in the `user` table in the `test` database using one of the following methods
+
+```sql
+DESCRIBE maria.test.user;
+SHOW COLUMNS FROM maria.test.user;
+```
+
+Finally, you can access the `user` table in the `test` database
+
+```sql
+SELECT * FROM maria.test.user;
+```
+
+If you use a different name for the directory properties file, use that directory name instead of `maria` in the above example.
+
+## MariaDB Connector Restrictions
+
+The following SQL statements are not supported at this time
+
+[DELETE](../sql/delete.md)、[GRANT](../sql/grant.md)、[REVOKE](../sql/revoke.md)、[SHOW GRANTS](../sql/show-grants.md)、[SHOW ROLES](../sql/show-roles.md)、[SHOW ROLE GRANTS](../sql/show-role-grants.md)

hetu-docs/en/index.md

@@ -86,6 +86,7 @@ headless: true
     - [Kafka Connector Tutorial]({{< relref "./docs/connector/kafka-tutorial.md" >}})
     - [Redis]({{< relref "./docs/connector/redis.md" >}})
     - [Local File]({{< relref "./docs/connector/localfile.md" >}})
+    - [MariaDB]({{< relref "./docs/connector/mariadb.md" >}})
     - [Memory]({{< relref "./docs/connector/memory.md" >}})
     - [MongoDB]({{< relref "./docs/connector/mongodb.md" >}})
     - [MySQL]({{< relref "./docs/connector/mysql.md" >}})

hetu-docs/zh/connector/mariadb.md

@@ -0,0 +1,99 @@
+# MariaDB连接器
+
+MariaDB连接器允许在外部MariaDB数据库中查询和创建表。这可用于在MySQL和MariaDB等不同系统之间或在两个不同的MariaDB实例之间联接数据。
+
+## 配置
+
+要配置MariaDB连接器，在openLooKeng的安装目录中`etc/catalog`中创建一个目录属性文件，例如`maria.properties`，将MariaDB连接器挂载为maria目录。使用以下内容创建文件，并根据设置替换连接属性：
+
+```properties
+connector.name=maria
+connection-url=jdbc:mariadb://部署了MairaDB的ip:MariaDB服务端口(默认3306)
+connection-user=您的数据源用户
+connection-password=您的数据源密码
+```
+
+- 是否开启查询下推功能
+
+如果要启用MariaDB连接器的连接器下推功能，不需要做任何操作，MariaDB连接器的下推功能默认是打开的。但也可以按如下设置：
+
+```properties
+#true表示打开下推，false表示关闭。
+jdbc.pushdown-enabled=true
+```
+
+- 下推模式选择
+
+MariaDB连接器的下推模式默认是部分下推的，如果要启用MariaDB连接器的全部下推功能，可以按如下设置：
+
+```properties
+#FULL_PUSHDOWN，表示全部下推；BASE_PUSHDOWN，表示部分下推，其中部分下推是指filter/aggregation/limit/topN/project这些可以下推。
+jdbc.pushdown-module=FULL_PUSHDOWN
+```
+
+### 外部函数注册
+
+MariaDB连接器支持注册外部函数。
+
+配置支持下推的外部函数注册命名空间`catalog.schema`。 例如在`etc/catalog/maria.properties`中配置：
+
+```Properties
+jdbc.pushdown.remotenamespace=mariafun.default
+```
+
+### 外部函数下推
+
+将外部函数下推到MariaDB数据源执行。
+
+配置支持下推的外部函数注册命名空间`catalog.schema`。 例如在`etc/catalog/maria.properties`中配置：
+
+```Properties
+jdbc.pushdown.remotenamespace=mariafun.default
+```
+
+可以声明自己支持多个函数命名空间中的函数，在`jdbc.pushdown.remotenamespace`配置项中使用'|'分割既可。例如：
+
+```Properties
+jdbc.pushdown.remotenamespace=mariafun1.default|mariafun2.default|mariafun3.default
+#表示当前Connector实例同时支持mysqlfun1.default、mysqlfun2.default、mysqlfun3.default三个函数命名空间最终的函数下推到当前连接的数据源中执行。
+```
+
+### 多个MariaDB服务器
+
+可以根据需要创建任意多的目录，因此，如果有额外的MariaDB服务器，只需添加另一个不同名称的属性文件到`etc/catalog`中（确保它以`.properties`结尾）。例如，如果将属性文件命名为`sales.properties`，openLooKeng将使用配置的连接器创建一个名为`sales`的目录。
+
+## 查询MySQL
+
+MariaDB连接器为每个MariaDB*数据库*提供一个模式。可通过执行`SHOW SCHEMAS`来查看可用MariaDB数据库：
+
+```sql
+-- 我们的catalog名称为maria
+SHOW SCHEMAS FROM maria;
+```
+
+如果有一个名为`test`的MySQL数据库，可以通过执行`SHOW TABLES`查看数据库中的表：
+
+```sql
+SHOW TABLES FROM maria.test;
+```
+
+可以使用以下方法之一查看`test`数据库中`user`表中的列的列表：
+
+```sql
+DESCRIBE maria.test.user;
+SHOW COLUMNS FROM maria.test.user;
+```
+
+最后，可以访问`test`数据库中`user`的表：
+
+```sql
+SELECT * FROM maria.test.user;
+```
+
+如果对目录属性文件使用不同的名称，请使用该目录名称，而不要使用上述示例中的`maria`。
+
+## MariaDB连接器限制
+
+暂不支持以下SQL语句：
+
+[DELETE](../sql/delete.md)、[GRANT](../sql/grant.md)、[REVOKE](../sql/revoke.md)、[SHOW GRANTS](../sql/show-grants.md)、[SHOW ROLES](../sql/show-roles.md)、[SHOW ROLE GRANTS](../sql/show-role-grants.md)

hetu-docs/zh/index.md

@@ -86,6 +86,7 @@ headless: true
     - [Kafka连接器教程]({{< relref "./docs/connector/kafka-tutorial.md" >}})
     - [Redis]({{< relref "./docs/connector/redis.md" >}})
     - [本地文件]({{< relref "./docs/connector/localfile.md" >}})
+    - [MariaDB]({{< relref "./docs/connector/mariadb.md" >}})
     - [内存]({{< relref "./docs/connector/memory.md" >}})
     - [MongoDB]({{< relref "./docs/connector/mongodb.md" >}})
     - [MySQL]({{< relref "./docs/connector/mysql.md" >}})

hetu-mariadb/pom.xml

@@ -0,0 +1,169 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+
+    <parent>
+        <groupId>io.hetu.core</groupId>
+        <artifactId>presto-root</artifactId>
+        <version>1.8.0-SNAPSHOT</version>
+    </parent>
+
+    <artifactId>hetu-mariadb</artifactId>
+    <description>Hetu - MariaDB Connector</description>
+    <packaging>hetu-plugin</packaging>
+
+    <properties>
+        <air.main.basedir>${project.parent.basedir}</air.main.basedir>
+    </properties>
+
+    <dependencies>
+        <dependency>
+            <groupId>io.hetu.core</groupId>
+            <artifactId>presto-base-jdbc</artifactId>
+        </dependency>
+
+        <dependency>
+            <groupId>io.airlift</groupId>
+            <artifactId>configuration</artifactId>
+        </dependency>
+
+        <dependency>
+            <groupId>com.google.guava</groupId>
+            <artifactId>guava</artifactId>
+        </dependency>
+
+        <dependency>
+            <groupId>com.google.inject</groupId>
+            <artifactId>guice</artifactId>
+        </dependency>
+
+        <dependency>
+            <groupId>javax.validation</groupId>
+            <artifactId>validation-api</artifactId>
+        </dependency>
+
+        <dependency>
+            <groupId>javax.inject</groupId>
+            <artifactId>javax.inject</artifactId>
+        </dependency>
+
+        <dependency>
+            <groupId>com.fasterxml.jackson.core</groupId>
+            <artifactId>jackson-core</artifactId>
+        </dependency>
+
+        <dependency>
+            <groupId>com.fasterxml.jackson.core</groupId>
+            <artifactId>jackson-databind</artifactId>
+        </dependency>
+
+        <dependency>
+            <groupId>io.airlift</groupId>
+            <artifactId>json</artifactId>
+        </dependency>
+
+        <!-- Presto SPI -->
+        <dependency>
+            <groupId>io.hetu.core</groupId>
+            <artifactId>presto-spi</artifactId>
+            <scope>provided</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>io.airlift</groupId>
+            <artifactId>slice</artifactId>
+            <scope>provided</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>io.airlift</groupId>
+            <artifactId>units</artifactId>
+            <scope>provided</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>com.fasterxml.jackson.core</groupId>
+            <artifactId>jackson-annotations</artifactId>
+            <scope>provided</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>org.openjdk.jol</groupId>
+            <artifactId>jol-core</artifactId>
+            <scope>provided</scope>
+        </dependency>
+
+        <!-- for testing -->
+        <dependency>
+            <groupId>org.testng</groupId>
+            <artifactId>testng</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>org.assertj</groupId>
+            <artifactId>assertj-core</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>io.airlift</groupId>
+            <artifactId>testing</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>io.hetu.core</groupId>
+            <artifactId>presto-main</artifactId>
+            <exclusions>
+                <exclusion>
+                    <groupId>org.bouncycastle</groupId>
+                    <artifactId>bcprov-jdk15on</artifactId>
+                </exclusion>
+            </exclusions>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>io.hetu.core</groupId>
+            <artifactId>presto-tpch</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>io.airlift.tpch</groupId>
+            <artifactId>tpch</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>io.hetu.core</groupId>
+            <artifactId>presto-tests</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>io.airlift</groupId>
+            <artifactId>testing-mysql-server</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>io.hetu.core</groupId>
+            <artifactId>presto-testing-docker</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>org.jetbrains</groupId>
+            <artifactId>annotations</artifactId>
+            <scope>test</scope>
+        </dependency>
+        <!-- MariaDB Driver -->
+        <dependency>
+            <groupId>org.mariadb.jdbc</groupId>
+            <artifactId>mariadb-java-client</artifactId>
+            <version>2.7.3</version>
+        </dependency>
+    </dependencies>
+</project>