Skip to content

Commit ab1728d

Browse files
committed
common : add llama.vim preset for Qwen2.5Coder1.5B
This commit adds a preset for llama.vim to use the default Qwen 2.5 Coder 1.5B model. The motivation for this change is to make it easier to start a server suitable to be used with the llama.vim plugin. For example, the server can be started with the following command: ```console $ llama.vim --vim-qwen2.5-Coder-1.5B-default ``` Refs: ggml-org#10932
1 parent 63ac128 commit ab1728d

File tree

1 file changed

+16
-0
lines changed

1 file changed

+16
-0
lines changed

common/arg.cpp

+16
Original file line numberDiff line numberDiff line change
@@ -2501,5 +2501,21 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
25012501
}
25022502
).set_examples({LLAMA_EXAMPLE_EMBEDDING, LLAMA_EXAMPLE_SERVER}));
25032503

2504+
add_opt(common_arg(
2505+
{"--vim-qwen2.5-Coder-1.5B-default"},
2506+
string_format("use default Qwen 2.5 Coder 1.5B (note: can download weights from the internet)"),
2507+
[](common_params & params) {
2508+
params.hf_repo = "ggml-org/Qwen2.5-Coder-1.5B-Q8_0-GGUF";
2509+
params.hf_file = "qwen2.5-coder-1.5b-q8_0.gguf";
2510+
params.port = 8012;
2511+
params.n_gpu_layers = 99;
2512+
params.flash_attn = true;
2513+
params.n_ubatch = 1024;
2514+
params.n_batch = 1024;
2515+
params.n_ctx = 0;
2516+
params.n_cache_reuse = 256;
2517+
}
2518+
).set_examples({LLAMA_EXAMPLE_SERVER}));
2519+
25042520
return ctx_arg;
25052521
}

0 commit comments

Comments
 (0)