Skip to content

Changes default cache embedding model #326

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 24, 2025
Merged

Changes default cache embedding model #326

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1a1e75d
changes default cache embedding model
justin-cechmanek Apr 23, 2025
f2240a7
widens threshold in notebook for new embedding default model
justin-cechmanek Apr 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/user_guide/03_llmcache.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@
"outputs": [],
"source": [
"# Widen the semantic distance threshold\n",
"llmcache.set_threshold(0.3)"
"llmcache.set_threshold(0.5)"
]
},
{
Expand Down
27 changes: 26 additions & 1 deletion redisvl/extensions/cache/llm/semantic.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,19 @@ def __init__(
if dtype:
vectorizer_kwargs.update(dtype=dtype)

# raise a warning to inform users we changed the default model
# remove this warning in future releases
logger.warning(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the warning below in the case of pre-existing index AND overwrite=True, I don't think we actually need this one here right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that a mismatch won't happen in that case, but there still is a behavior change when we switch default models. It's why I had to update the tests because the embedding distances changed.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If someone goes from 0.5 to 0.6 and never specifies a vectorizer, and runs the same script start to finish - writes/reads/clears - they may get different results.

"The default vectorizer has changed from `sentence-transformers/all-mpnet-base-v2` "
"to `redis/langcache-embed-v1` in version 0.6.0 of RedisVL. "
"For more information about this model, please refer to https://arxiv.org/abs/2504.02268 "
"or visit https://huggingface.co/redis/langcache-embed-v1. "
"To continue using the old vectorizer, please specify it explicitly in the constructor as: "
"vectorizer=HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2')"
)

self._vectorizer = HFTextVectorizer(
model="sentence-transformers/all-mpnet-base-v2",
model="redis/langcache-embed-v1",
**vectorizer_kwargs,
)

Expand Down Expand Up @@ -147,6 +158,20 @@ def __init__(
# Check for existing cache index and handle schema mismatch
self.overwrite = overwrite
if not self.overwrite and self._index.exists():

if not vectorizer:
# user hasn't specified a vectorizer and an index already exists they're not overwriting
# raise a warning to inform users we changed the default embedding model
# remove this warning in future releases
logger.warning(
"The default vectorizer has changed from `sentence-transformers/all-mpnet-base-v2` "
"to `redis/langcache-embed-v1` in version 0.6.0 of RedisVL. "
"For more information about this model, please refer to https://arxiv.org/abs/2504.02268 "
"or visit https://huggingface.co/redis/langcache-embed-v1. "
"To continue using the old vectorizer, please specify it explicitly in the constructor as: "
"vectorizer=HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2')"
)

existing_index = SearchIndex.from_existing(
name, redis_client=self._index.client
)
Expand Down
16 changes: 12 additions & 4 deletions tests/integration/test_llmcache.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

@pytest.fixture
def vectorizer():
return HFTextVectorizer("sentence-transformers/all-mpnet-base-v2")
return HFTextVectorizer("redis/langcache-embed-v1")


@pytest.fixture
Expand Down Expand Up @@ -720,12 +720,17 @@ def test_cache_filtering(cache_with_filters):
# test we can pass a list of tags
combined_filter = filter_1 | filter_2 | filter_3
results = cache_with_filters.check(
"test prompt 1", filter_expression=combined_filter, num_results=5
"test prompt 1",
filter_expression=combined_filter,
num_results=5,
distance_threshold=0.5,
)
assert len(results) == 3

# test that default tag param searches full cache
results = cache_with_filters.check("test prompt 1", num_results=5)
results = cache_with_filters.check(
"test prompt 1", num_results=5, distance_threshold=0.6
)
assert len(results) == 4

# test no results are returned if we pass a nonexistant tag
Expand Down Expand Up @@ -784,7 +789,10 @@ def test_complex_filters(cache_with_filters):
# test we can do range filters on inserted_at and updated_at fields
range_filter = Num("inserted_at") < current_timestamp
results = cache_with_filters.check(
"prompt 1", filter_expression=range_filter, num_results=5
"prompt 1",
filter_expression=range_filter,
num_results=5,
distance_threshold=0.5,
)
assert len(results) == 2

Expand Down