Skip to content
/ jina Public
forked from jina-ai/serve

An easier way to build neural search in the cloud

License

Notifications You must be signed in to change notification settings

lucalila/jina

Β 
Β 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Jina banner

Jina Jina Jina Jina Docs We are hiring tweet button Python 3.7 3.8 PyPI Docker Docker Image Version (latest semver) CI CD Release Cycle Release CD API Schema codecov

English β€’ ζ—₯本θͺž β€’ franΓ§ais β€’ Deutsch β€’ Русский язык β€’ δΈ­ζ–‡

Website β€’ Docs β€’ Examples β€’ Newsletter β€’ Dashboard (beta) β€’ Twitter β€’ We are Hiring

Want to build a search system backed by deep learning? You've come to the right place!

Jina is the easiest way to build neural search in the cloud. It provides an one-stop solution for multi-/cross-modality search. Jina has long-term support from a full-time, venture-backed team.

⏱️ Time Saver - Bootstrapping an AI-powered search system with Jina takes just minutes, and saves engineers months of time!

🧠 First-class AI models - Jina is a new design pattern for neural search systems, offering first-class support for state-of-the-art AI models.

🌌 Universal Search - Large-scale indexing and querying data of any kind on multiple platforms. Video, image, long/short text, music, source code, you name it!

πŸš€ Production Ready - Cloud-native features come out-of-the-box, e.g. containerization, microservice, distributing, scaling, sharding, async IO, REST, gRPC.

Contents

Install

Install from PyPi

On Linux/MacOS with Python >= 3.7:

pip install jina

To install Jina with extra dependencies, or install on Raspberry Pi please refer to the documentation.

...or Run in a Docker Container

We provide a universal Docker image (only 80MB!) that supports multiple architectures (including x64, x86, arm-64/v7/v6). Simply run:

docker run jinaai/jina --help

Jina "Hello, World!" πŸ‘‹πŸŒ

As a starter, you can try out our "Hello, World" - a simple demo of image neural search for Fashion-MNIST. No extra dependencies needed, just run:

jina hello-world

...or even easier for Docker users, no install required:

docker run -v "$(pwd)/j:/j" jinaai/jina hello-world --workdir /j && open j/hello-world.html  # replace "open" with "xdg-open" on Linux
Click here to see console output

hello world console output

The Docker image downloads the Fashion-MNIST training and test dataset and tells Jina to index 60,000 images from the training set. Then it randomly samples images from the test set as queries and asks Jina to retrieve relevant results. The whole process takes about 1 minute, and eventually opens a webpage and shows results like this:

Jina banner

The implementation behind it is as simple as can be:

Python API or use YAML spec or use Dashboard
from jina.flow import Flow

f = (Flow()
        .add(uses='encoder.yml', parallel=2)
        .add(uses='indexer.yml', shards=2, 
             separated_workspace=True))

with f:
    f.index(fashion_mnist, batch_size=1024)
!Flow
pods:
  encode:
    uses: encoder.yml
    parallel: 2
  index:
    uses: indexer.yml
    shards: 2
    separated_workspace: true

Flow in Dashboard

All the big words you can name: computer vision, neural IR, microservice, message queue, elastic, replicas & shards. They all happened in just one minute!

Adding Parallelism and Sharding

from jina.flow import Flow

f = (Flow().add(uses='encoder.yml', parallel=2)
           .add(uses='indexer.yml', shards=2, separated_workspace=True))
from jina.flow import Flow

f = Flow().add(uses='encoder.yml', host='192.168.0.99')
from jina.flow import Flow

f = (Flow().add(uses='jinahub/cnn-encode:0.1')
           .add(uses='jinahub/faiss-index:0.2', host='192.168.0.99'))

Concatenating Embeddings

from jina.flow import Flow

f = (Flow().add(name='eb1', uses='BiTImageEncoder')
           .add(name='eb2', uses='KerasImageEncoder', needs='gateway')
           .join(needs=['eb1', 'eb2'], uses='_concat'))
from jina.flow import Flow

f = Flow(port_expose=45678, rest_api=True)

with f:
    f.block()

Intrigued? Play with different options:

jina hello-world --help

Be sure to continue with our Jina 101 Guide - to understand all key concepts of Jina in 3 minutes!

Build your own Project

pip install cookiecutter && cookiecutter gh:jina-ai/cookiecutter-jina

With Cookiecutter you can easily create a Jina project from templates with one terminal command. This creates a Python entrypoint, YAML configs and a Dockerfile. You can start from there.

Tutorials

Jina 101 Concept Illustration Book, Copyright by Jina AI Limited Β Β  Β Β English β€’ ζ—₯本θͺž β€’ franΓ§ais β€’ PortuguΓͺs β€’ Deutsch β€’ Русский язык β€’ δΈ­ζ–‡ β€’ عربية
TutorialsLevel
Orchestrate Pods to work together: sequentially and in parallel; locally and remotely

🐣

Use Jina's input and output functions

🐣

Monitor workflows and get insights with Jina's dashboard

🐣

Extract feature vector data using any deep learning representation

🐣

Search South Park scripts and practice with Flows and Pods

🐣

Search images, define your own executors, and run them in Docker

🐣

Increase performance using prefetching and sharding

πŸ•Š

Run a Flow remotely and connect from a local client

πŸ•Š

Run Jina on remote instances and distribute your workflow

πŸ•Š

Implement your own ideas as Jina plugins

πŸ•Š

Solve complex dependencies easily with Docker containers

πŸ•Š

Search Pokemon with SOTA visual representation!

πŸš€

Documentation

The best way to learn Jina in depth is to read our documentation. Documentation is built on every push, merge, and release of the master branch.

Are you a "Doc"-star? Join us! We welcome all kinds of improvements on the documentation.

Documentation for older versions is archived here.

Contributing

We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to your active involvement.

Contributors ✨

All Contributors

Community

  • Slack workspace - a communication platform for developers to discuss Jina
  • Youtube channel - subscribe to the latest video tutorials, release demos, webinars and presentations.
  • LinkedIn - get to know Jina AI as a company and find job opportunities
  • Twitter Follow - follow us and interact with using hashtag #JinaSearch
  • Company - know more about our company and how we are fully committed to open-source!

Join Us

Jina is an open-source project. We are hiring full-stack developers, evangelists, and PMs to build the next neural search ecosystem in open source.

Open Governance

GitHub milestones lay out the path to Jina's future improvements.

As a part of open governance model, we host the Engineering All Hands of Jina AI in public. This Zoom meeting recurs monthly on the second Tuesday, 14:00-15:30 (CET). Everyone can join in via the following calendar invite.

The meeting will also be live-streamed and later published to our Youtube channel. Make sure to subscribe it if you want to watch the replay.

License

Copyright (c) 2020 Jina AI Limited. All rights reserved.

Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

About

An easier way to build neural search in the cloud

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 95.3%
  • HTML 2.8%
  • Other 1.9%