Welcome to the ♻️🌎 Eco Sage 🌎♻️ hackathon project repository! Our goal is to create an innovative recycling app that simplifies the recycling process by providing clear instructions on how to recycle various objects. The app takes an input image of an object, uses advanced image processing and language models, and outputs detailed recycling instructions based on a comprehensive knowledge database.
In this hackathon project, we are building a recycling app that consists of three main components: image segmentation, image captioning, and recycling instructions generation. The web app is launched using Streamlit.
The first step of our pipeline involves image segmentation. We utilize Fast Segment Anything Model (FastSAM) CNN model to analyze the input image and identify objects within it. The model will then crop these objects to provide a clear view of each item.
For more information about FastSAM, visit the documentation.
Once the objects are segmented, we proceed to generate descriptions for each of them using the BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation model. It takes the segmented image and provides a caption describing the contents of an image.
For more information about BLIP, visit the documentation.
The final step of our pipeline involves generating recycling instructions based on the object descriptions and our comprehensive knowledge database. We utilize OpenAI's Language Model to create detailed and specific instructions on how to recycle each identified object. The knowledge database contains links to the City of Calgary's recycling guidelines What goes where?.
For more information about OpenAI, visit the documentation.
For more information about City of Calgary guidelined, visit the What goes where?
- Clone this repository using
git clone https://github.com/altaml-hackathon/hackathon-aug-2023-happy.git - Go to the root folder and create your conda environment using
make setup-environment - Copy the secrets file using
cp secrets.env.template .envif working in bash, orcopy secrets.env.template .envif working in CMD or PowerShell and replace the contents of the file with your credentials. - Open another terminal. Then, launch the web app using
cd deployment_components\backend\knowledge_basefrom the root folder and runuvicorn main:app --reload. - Upload an image of
.png,.jpegor.jpgformat in the app and enjoy the recycling instructions! 😊
Since this is a proof of concept project during a hackathon, it has several areas of improvement and scalability! As such, we encourage contributions to improve the ♻️🌎 Eco Sage 🌎♻️ app! To make you contribution, follow these steps:
- Fork this repository to your GitHub account.
- Create a new branch for your feature using
git checkout -b <feature>/<your-feature-name>. - Commit your changes using
git commit -m 'Add some feature'. - Push to the branch using
git push origin <feature>/<your-feature-name>. - Open a pull request, and our team will review your contribution. 🌱🌍
Our beautiful team: @fatemehyb, @ameeny, @timurkz, @hannayurchyk, @ACCHarles and @StarrySkyrs! ❤️
Thank you for joining us in this important endeavor to promote recycling and sustainability!
🌿🌟