Skip to content

Getting Started

Kaleb Houck edited this page Aug 19, 2021 · 29 revisions

In order to get DeepLynx set up in a local environment for development and testing purposes, please follow these steps and requirements:

Requirements

  • node.js 8.x, 10.x, 12.x, 14.x and 15.x
  • Typescript ^3.5.x
  • npm ^6.x
  • Docker ^18.x - optional - for ease of use in development
  • Docker Compose ^1.x.x - optional - for ease of use in development
  • Private RSA key. This is used for encryption of sensitive data. If you need help on generating a private key, we recommend using openssl to do so. Here is a tutorial.

Data Source Requirements

  • Required - PostgreSQL ^11.x
  • Redis ^6.x - required only if not using the in-memory cache system. We highly recommend that you use Redis over in-memory in a production environment, especially if you're planning on running DeepLynx in a sharded environment.

Steps

  1. NodeJS must be installed. You can find the download for your platform here: https://nodejs.org/en/download/ note - Newer versions of Node may be incompatible with some of the following commands. The most recent version tested that works fully is 13.10.1.

  2. Clone the DeepLynx repository.

  3. Run npm upgrade && npm install in your local DeepLynx directory.

image image

  1. Copy and rename .env-sample to .env.

  2. Update .env file. See the readme or comments in the file itself for details.If you are not using Docker, ensure that you update the ENCRYPTION_KEY_PATH environment variable in .env to reflect the absolute path of a RSA private key.

  3. Run npm run build:dev to build the internal modules. image

  4. optional - If you would like to use Docker rather than a dedicated PostgreSQL database, please follow these steps:

    • Ensure Docker is installed. You can find the download here: https://www.docker.com/products/docker-desktop.
    • Run npm run docker:postgres:build to create a docker image containing a Postgres data source.
    • Mac users may need to create the directory to mount to the docker container at /private/var/lib/docker/basedata. If this directory does not exist, please create it (you may need to use sudo as in sudo mkdir /private/var/lib/docker/basedata).
    • Verify that image is properly created. See below. image
    • Run npm run docker:postgres:run to run the created docker image (For Mac users, there is an alternative command npm run mac:docker:postgres:run).
  5. Run npm run migrate to create the database and schema within a PostgreSQL database configured in the .env file.
    image

  6. Run npm run watch or npm run start to start the application. See the readme for additional details and available commands.
    image

Installation Gotchas

  • When running npm run migrate you should see some kind of output regardless of success or failure. If you see no output, but the program exits without errors do not assume the migrate ran correctly. Check your node version as silent errors of this type are generally present when running an unsupported node version.

  • If you're planning on uploading actual files you must have the FILE_STORAGE_METHOD set. If you're using the filesystem storage method, insure that you have also set FILESYSTEM_STORAGE_DIRECTORY and that it points to a directory that the DL instance has read/write permissions to.

  • You must provide a valid RSA private key to DeepLynx so that it can store encrypted information. DeepLynx will only error on those operations if you're missing the key, so you might not catch that until later.

  • If you are having difficulties authenticating in the Admin Web GUI - insure that you've setup the OAuth Application correctly and that the Admin Web GUI is configured correctly. Find more information here.

Verifying DeepLynx is working correctly

Here are a few simple ways you can verify that your installation of Deep Lynx is working correctly.

  • Send a simple GET request to your DeepLynx instance's health check endpoint. This is located at {host}/health and should return a 200 OK HTTP status response if DeepLynx is up and running correctly. (You can find where Deep Lynx is exposing it's HTTP server by checking the ROOT_ADDRESS and SERVER_PORT environment variables - by default it should be localhost:8090).
  • Verify that the database specified with the CORE_DB_CONNECTION_STRING and DB_NAME exists and that it has been populated with multiple different schemas.
  • Navigate to your DeepLynx's login page (default is http://localhost:8090/). If you had your environment variables instruct DeepLynx to create a default User, attempt to login with said User.

Tools like Postman can be used for verifying HTTP response/requests and TablePlus or PgAdmin can be used when verifying database structure or values.

Next Steps - Create Your Ontology

Now that DeepLynx is up and running, you must first create an ontology before you can begin storing data. Simply put, ontologies are a formal way to describe taxonomies and classification networks. In DeepLynx creating an ontology is easy.

DeepLynx Wiki

Sections marked with ! are in progress.

Building DeepLynx

DeepLynx Overview

Getting Started

Building From Source

Admin Web App


Deploying DeepLynx


Integrating with DeepLynx


Using DeepLynx

Ontology

Data Ingestion

Timeseries Data

Manual Path
Automated Path
File/Blob Storage

Data Querying

Event System

Data Targets


Developing DeepLynx

Developer Overview

Project Structure and Patterns

Data Access Layer

Development Process

Current Proposals

Clone this wiki locally