Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

profile throughput without new threads #2826

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

profile throughput without new threads #2826

wants to merge 12 commits into from

Commits on Nov 27, 2024

  1. profile throughput without threads

    @grimoire
    grimoire committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    4a18be7 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2024

  1. optimize main loop

    @grimoire
    grimoire committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    31afcf2 View commit details
    Browse the repository at this point in the history

  2. fix torch.event

    @grimoire
    grimoire committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    88ad4dc View commit details
    Browse the repository at this point in the history

  3. fix python>3.11

    @grimoire
    grimoire committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    9585aef View commit details
    Browse the repository at this point in the history

  4. optimize tp

    @grimoire
    grimoire committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    3ea4aa8 View commit details
    Browse the repository at this point in the history

  5. reduce cudagraph copy

    @grimoire
    grimoire committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    549c6c6 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2024

  1. optimize fill kv cache

    @grimoire
    grimoire committed Nov 29, 2024
    Configuration menu
    Copy the full SHA
    3df2e49 View commit details
    Browse the repository at this point in the history

  2. optimize silu and mul

    @grimoire
    grimoire committed Nov 29, 2024
    Configuration menu
    Copy the full SHA
    037cac6 View commit details
    Browse the repository at this point in the history

  3. optimize apply rotary

    @grimoire
    grimoire committed Nov 29, 2024
    Configuration menu
    Copy the full SHA
    4247295 View commit details
    Browse the repository at this point in the history

  4. remove executor

    @grimoire
    grimoire committed Nov 29, 2024
    Configuration menu
    Copy the full SHA
    aa25512 View commit details
    Browse the repository at this point in the history

Commits on Dec 2, 2024

  1. remove kernel

    @grimoire
    grimoire committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    4ea3127 View commit details
    Browse the repository at this point in the history

  2. remove num_heads==1

    @grimoire
    grimoire committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    b269793 View commit details
    Browse the repository at this point in the history