llama.cpp b9442 adds tokenizer support for jina-embeddings-v2-base-zh with a whitespace tokenizer.
$ git clone --branch b9442 https://github.com/ggml-org/llama.cpp.git # already have the repo? check out this version: $ git checkout b9442
- ›Adds whitespace tokenizer support for
jina-embeddings-v2-base-zh, withlowercasedefaulting totrue.