Skip to content

想问一下在A800上测试的吞吐量,换算到推理速度的话有多少tokens/s? #138

@HJT9328

Description

@HJT9328

Required prerequisites

Questions

7B模型实现了A800 上单卡吞吐的情况下实现了 70tokens/s 比较怀疑,

Checklist

  • I have provided all relevant and necessary information above.
  • I have chosen a suitable title for this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions