You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
I want the option to use Go to access Groq hosted models, due to their greater inference speed.
Describe the solution you'd like
I would like to suggest this project do two three things:
Use Groq's OpenAI Compatibility for minimal changes
Provide a list of Groq models for easy usage
Add basic documentation since this more of a side-feature with huge value than a main case.
Describe alternatives you've considered
This could be a new/different library, but I think it makes sense more sense to build on the work done by this project.
OpenAI Compatibility
Groq's APIs are designed to be compatible with OpenAI's, with the goal of making it easy to leverage Groq in applications you may have already built.
Supported Models
GroqCloud currently supports the following models: LLaMA2-70b
Developer: Meta
Model Name: LLaMA2-70b-chat
Context Window: 4,096 tokens
API String: llama2-70b-4096 Mixtral-8x7b
Developer: Mistral
Model Name: Mixtral-8x7b-Instruct-v0.1
Context Window: 32,768 tokens
API String: mixtral-8x7b-32768 Gemma-7b-it
Developer: Google
Model Name: Gemma-7b-it
Context Window: 8,192 tokens
API String: gemma-7b-it
I'll submit a PR that outlines this change shortly.
Is your feature request related to a problem? Please describe.
I want the option to use Go to access Groq hosted models, due to their greater inference speed.
Describe the solution you'd like
I would like to suggest this project do
twothree things:Describe alternatives you've considered
This could be a new/different library, but I think it makes sense more sense to build on the work done by this project.
Additional context
I'll submit a PR that outlines this change shortly.
EDIT FOR PR LINK: #683
The text was updated successfully, but these errors were encountered: