Add docker cmdline options to handle long context#515
Add docker cmdline options to handle long context#515nngokhale wants to merge 1 commit intovllm-project:v0.10.2_nextfrom
Conversation
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
nngokhale
force-pushed
the
plugin-cd-0.10.2_next_exp
branch
from
7877ef4 to
be6eae1
Compare
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
nngokhale
force-pushed
the
plugin-cd-0.10.2_next_exp
branch
from
be6eae1 to
8620b73
Compare
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
nngokhale
force-pushed
the
plugin-cd-0.10.2_next_exp
branch
from
8620b73 to
f290c83
Compare
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
nngokhale
force-pushed
the
plugin-cd-0.10.2_next_exp
branch
from
f290c83 to
660dcee
Compare
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
Signed-off-by: Neelesh Gokhale <neelesh.gokhale@intel.com>
nngokhale
force-pushed
the
plugin-cd-0.10.2_next_exp
branch
from
660dcee to
6fbb8e5
Compare
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
nngokhale
requested review from
mgawarkiewicz-intel,
piotrbocian and
wpyszka
as code owners
|
It's too late in the release to push it to 0.10.2. |
2 participants
Add exponential bucketing support, tested with exp 32k and linear 4k
Change multi modal data set
Adds VLLM_PROMPT_CTX_BUCKET_STEP capability
Removes profiler memory field, fsdpa field