Model Runtime #1858
Merged
Model Runtime #1858
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
crazywoola
reviewed
Jan 2, 2024
|
Nice work!! It's possible add fine tuned models right now? from openAI? |
2 tasks
Suspect IssuesThis pull request was deployed and Sentry observed the following issues:
Did you find this useful? React with a 👍 or 👎 |
yep |
HuberyHuV1
pushed a commit
to HuberyHuV1/dify
that referenced
this pull request
Jul 22, 2024
Co-authored-by: StyleZhang <[email protected]> Co-authored-by: Garfield Dai <[email protected]> Co-authored-by: chenhe <[email protected]> Co-authored-by: jyong <[email protected]> Co-authored-by: Joel <[email protected]> Co-authored-by: Yeuoly <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🎉🎉 Dify's Version 0.4 is out now.
We've made some serious under-the-hood changes to how the Model Runtime works, making it more straightforward for our specific needs, and paving the way for smoother model expansions and more robust production use.
What's Changed:
Model Runtime Rework: We've moved away from LangChain, simplifying the model layer. Now, expanding models is as easy as setting up the model provider in the backend with a bit of YAML.
For more details, see: https://github.com/langgenius/dify/blob/feat/model-runtime/api/core/model_runtime/README.md
App Generation Update: Replacing the old
Redis Pubsubqueue withthreading.Queuefor a more reliable, performant, and straightforward workflow.Model Providers Upgraded: Support for both preset and custom models, ideal for adding OpenAI
fine-tuned modelsor fitting into various MaaS platforms. Plus, you can now check out supported models without any initial configuration.Context Size Definition: Introduced distinct
context sizesettings, separate fromMax Tokens, to handle the different limits and sizes in models like OpenAI'sGPT-4 Turbo.Flexible Model Parameters: Customize your model's behavior with easily adjustable parameters through YAML.
GPT-2 Tokenizer Files: Now cached within Dify's codebase, making builds quicker and solving issues related to acquiring tokenizer files in offline source deployments.
Model List Display: The App now displays all supported preset models, including details on any that aren't available and how to configure them.
New Model Additions: Including Google's
Gemini ProandGemini Pro Visionmodels (Vision requires an image input), Azure OpenAI'sGPT-4V, and support forOpenAI-API-compatibleproviders.Expanded Inference Support:
Xorbit Inferencenow includes chat mode models, and there's a wider range of models supporting Agent inference.Updates & Fixes: We've updated other model providers to be in sync with the latest version APIs and features, and squashed a series of minor bugs for a smoother experience.
Catch you in the code,
The Dify Team 🛠️