Skip to content

Add LangCache API integration for semantic caching #321

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 14 commits into from
Closed
564 changes: 246 additions & 318 deletions poetry.lock

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ mistralai = { version = ">=1.0.0", optional = true }
voyageai = { version = ">=0.2.2", optional = true }
ranx = { version = "^0.3.0", python=">=3.10", optional = true }
boto3 = {version = "1.36.0", optional = true, extras = ["bedrock"]}
langcache = {path = "vendor/langcache-0.0.4-py3-none-any.whl"}

[tool.poetry.extras]
openai = ["openai"]
Expand Down
3 changes: 2 additions & 1 deletion redisvl/extensions/cache/embeddings/embeddings.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@

from redisvl.extensions.cache.base import BaseCache
from redisvl.extensions.cache.embeddings.schema import CacheEntry
from redisvl.redis.utils import convert_bytes, hashify
from redisvl.redis.utils import convert_bytes
from redisvl.utils.utils import hashify


class EmbeddingsCache(BaseCache):
Expand Down
9 changes: 8 additions & 1 deletion redisvl/extensions/cache/llm/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,18 @@
This module provides LLM cache implementations for RedisVL.
"""

from redisvl.extensions.cache.llm.langcache_api import LangCache
from redisvl.extensions.cache.llm.schema import (
CacheEntry,
CacheHit,
SemanticCacheIndexSchema,
)
from redisvl.extensions.cache.llm.semantic import SemanticCache

__all__ = ["SemanticCache", "CacheEntry", "CacheHit", "SemanticCacheIndexSchema"]
__all__ = [
"SemanticCache",
"CacheEntry",
"CacheHit",
"SemanticCacheIndexSchema",
"LangCache",
]
Loading
Loading