feat: support LLM process document file

[hjlarry]

Summary

Tip

Close issue syntax: Fixes #<issue number> or Resolves #<issue number>, see documentation for more details.

Currently, lots of LLM( gemini, sonnet ...) can directly process document, and make user's chat context based on these documents. This PR aimed to support this feature in a dify agent app. For the chatflow app, maybe this PR can resolve.

ChangeList

Backend

• add DocumentPromptMessageContent and more model features, copied from feat: Allow using file variables directly in the LLM node and support more file types. #10679
• support gemini series models receive document
• add video feature to support video models, document feature to support document models

Frontend

• add a switch button to control whether allow upload document
• for not support video feature's LLM, open vision config will not allow upload video
• chat page whether display file upload button depends on file.allowed_file_types has any value

remaining issues

• the vision settings is strange, Resolution only affect Image type files, Upload Method and Upload Limit affect all type files.

Screenshots

Checklist

Important

Please review the checklist below before submitting your pull request.

• This change requires a documentation update, included: Dify Document
• I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• I've updated the documentation accordingly.
• I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

[laipz8200]

Thank you for this awesome contribution! Some changes in #10679 are still in processing, I'll review this PR after #10679 is merged.

[laipz8200]

the vision settings is strange, Resolution only affect Image type files, Upload Method and Upload Limit affect all type files.

I think this resolution config should be removed in the future.

[laipz8200]

Hi @hjlarry! #10679 is merged, could you please sync the code with the main branch?

[hjlarry]

Hi @hjlarry! #10679 is merged, could you please sync the code with the main branch?

Done :)

[laipz8200]

Screen.Recording.2024-11-22.at.5.56.05.PM.mov

[hjlarry]

Screen.Recording.2024-11-22.at.5.56.05.PM.mov

seems the icon has been overwrite by the merge action, please try again