Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

better formatted table of 'lmdeploy list' #2289

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 16 additions & 4 deletions lmdeploy/cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,10 +156,22 @@ def convert(args):
def list(args):
"""List the supported model names."""
from lmdeploy.model import MODELS
model_names = list(MODELS.module_dict.keys())
model_names.sort()
print('The supported chat template names are:')
print('\n'.join(model_names))
builtin_templates = list(MODELS.module_dict.keys())
builtin_templates.sort()
builtin_templates.remove('base')
builtin_templates.remove('llama')
print('The built-in chat templates are:')
from texttable import Texttable
table = Texttable()
# model path, model type, model arch, chat_template
table.add_row(
['chat_template', 'representative', 'model_type', 'architecture'])
for name in builtin_templates:
chat_template = MODELS.get(name)()
if hasattr(chat_template, 'meta'):
model_path, model_type, architecture = chat_template.meta()
table.add_row([name, model_path, model_type, architecture])
print(table.draw())

@staticmethod
def check_env(args):
Expand Down
10 changes: 10 additions & 0 deletions lmdeploy/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,11 @@ def match(cls, model_path: str) -> Optional[str]:
if 'cogvlm' in path and 'cogvlm2' not in path:
return 'cogvlm'

def meta(self):
"""Return (model_repo_id, model_type, architecture) of a model who uses
this chat template."""
return 'THUDM/cogvlm-chat-hf', 'MLLM', 'CogVLMForCausalLM'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

要不直接建一个大表放文档里?如果用代码的话,一个对话模板对应的 huggingface repo id 会很多。

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

也是一个方法



@MODELS.register_module(name='cogvlm2')
class CogVLM2(CogVLM):
Expand All @@ -284,6 +289,11 @@ def match(cls, model_path: str) -> Optional[str]:
if 'cogvlm2' in path:
return 'cogvlm2'

def meta(self):
"""Return typical (model_path, model_type, model_architecture) of a
model who uses this chat template."""
return 'TTHUDM/cogvlm2-llama3-chat-19B', 'MLLM', 'CogVLMForCausalLM'


@MODELS.register_module(name='wizardlm')
@MODELS.register_module(name='vicuna')
Expand Down
1 change: 1 addition & 0 deletions requirements/runtime.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ pynvml
safetensors
sentencepiece
shortuuid
texttable
tiktoken
torch<=2.3.1,>=2.0.0
torchvision<=0.18.1,>=0.15.0
Expand Down
Loading