Skip to content

codelibs/fess-webapp-multimodal

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Fess for Multimodal Search

Java CI with Maven

Overview

This is a multimodal-search plugin for Fess, enabling the crawling and indexing of various media formats such as text, images, audio, and video (support for audio and video will be available in future updates).

Download

See Maven Repository.

Installation

See Plugin of Administration guide.

Usage

After installing the plugin, follow these steps to configure and use it:

  1. Start Fess: Launch Fess and log in to the administration console.

  2. Configure System Properties: Add the following properties to the general settings under system properties:

    fess.multimodal.content.field=content_vector
    fess.multimodal.content.dimension=512
    fess.multimodal.content.method=hnsw
    fess.multimodal.content.engine=lucene
    fess.multimodal.content.space_type=cosinesimil
    fess.multimodal.min_score=0.5
    
  3. Update Settings: Navigate to the scheduler page and execu te Config Reloader.

  4. Re-indexing: Navigate to the maintenance page and execute re-indexing.

  5. Setup CLIP as Service: For image embedding, use CLIP as Service. Start the CLIP API using Docker:

    git clone https://github.com/codelibs/fess-webapp-multimodal.git
    cd fess-webapp-multimodal/docker
    docker compose up -d

    This will make the CLIP API accessible at localhost:51000.

  6. Crawl Directories: Crawl directories containing image files.

  7. Test Data: If you need test data, you can download the Open Images Dataset. For example:

    pip install fiftyone
    fiftyone zoo datasets load open-images-v7 --split validation --kwargs max_samples=1000 -d fiftyone

    This will download 1000 images into the fiftyone directory.

Contributing

We welcome contributions to enhance the functionality of this plugin. Please fork the repository and submit pull requests.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages