Skip to content

Commit

Permalink
Fix (batch-size) after rebase.
Browse files Browse the repository at this point in the history
  • Loading branch information
srogmann committed Nov 23, 2024
1 parent c600af7 commit 60d6b1e
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion Llama3.java
Original file line number Diff line number Diff line change
Expand Up @@ -365,7 +365,7 @@ else if ("user".equals(role)) {
}
if (httpSession == null) {
// We build a new HTTP-session.
final Llama.State state = model.createNewState();
final Llama.State state = model.createNewState(BATCH_SIZE);
sessionKey = "SESS-" + reqCounter.get() + "-" + UUID.randomUUID().toString();
exchange.getResponseHeaders().add("Set-Cookie", "LLAMA_SESS_ID=" + sessionKey);

Expand Down

0 comments on commit 60d6b1e

Please sign in to comment.