1df1b1eaa0
# Which issue(s) this PR fixes Related to https://github.com/grafana/oncall-private/issues/2363 Addresses this issue that arises when using `cache.get_many`/`cache.set_many` operations with a Redis Cluster: ```python3 File "/usr/local/lib/python3.11/site-packages/redis/cluster.py", line 1006, in determine_slot raise RedisClusterException( redis.exceptions.RedisClusterException: MGET - all keys must map to the same key slot ``` From the Redis Cluster [docs](https://redis.io/docs/reference/cluster-spec/#hash-tags), this can be addressed with this 👇 . Basically this will ensure that keys in multi-key operations will resolve to the same hash slot (read: node): > Hash tags > There is an exception for the computation of the hash slot that is used in order to implement hash tags. Hash tags are a way to ensure that multiple keys are allocated in the same hash slot. This is used in order to implement multi-key operations in Redis Cluster. > > To implement hash tags, the hash slot for a key is computed in a slightly different way in certain conditions. If the key contains a "{...}" pattern only the substring between { and } is hashed in order to obtain the hash slot. However since it is possible that there are multiple occurrences of { or } the algorithm is well specified by the following rules: > > IF the key contains a { character. > AND IF there is a } character to the right of {. > AND IF there are one or more characters between the first occurrence of { and the first occurrence of }. > Then instead of hashing the key, only what is between the first occurrence of { and the following first occurrence of } is hashed. ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [ ] Documentation added (or `pr:no public docs` PR label added if not required) - [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not required)
51 lines
2.3 KiB
Python
51 lines
2.3 KiB
Python
from django.test import override_settings
|
|
|
|
from common.cache import ensure_cache_key_allocates_to_the_same_hash_slot
|
|
|
|
PATTERN = "schedule_oncall_users"
|
|
NON_EXISTENT_PATTERN = "nmzxcnvmzxcv"
|
|
NUM_CACHE_KEYS = 5
|
|
SINGLE_CACHE_KEY = f"{PATTERN}_0"
|
|
CACHE_KEYS = [f"{PATTERN}_{pk}" for pk in range(NUM_CACHE_KEYS)]
|
|
SET_MANY_CACHE_KEYS_DICT = {k: "foo" for k in CACHE_KEYS}
|
|
|
|
|
|
def test_ensure_cache_key_allocates_to_the_same_hash_slot() -> None:
|
|
def _convert_key(key: str) -> str:
|
|
return key.replace(PATTERN, f"{{{PATTERN}}}")
|
|
|
|
# when USE_REDIS_CLUSTER is False the method should just return the cache keys
|
|
with override_settings(USE_REDIS_CLUSTER=False):
|
|
assert ensure_cache_key_allocates_to_the_same_hash_slot(SINGLE_CACHE_KEY, PATTERN) == SINGLE_CACHE_KEY
|
|
assert ensure_cache_key_allocates_to_the_same_hash_slot(CACHE_KEYS, PATTERN) == CACHE_KEYS
|
|
assert (
|
|
ensure_cache_key_allocates_to_the_same_hash_slot(SET_MANY_CACHE_KEYS_DICT, PATTERN)
|
|
== SET_MANY_CACHE_KEYS_DICT
|
|
)
|
|
|
|
# when USE_REDIS_CLUSTER is True the method should wrap the specified pattern within the cache keys in curly brackets
|
|
with override_settings(USE_REDIS_CLUSTER=True):
|
|
# works with a single str cache key
|
|
assert ensure_cache_key_allocates_to_the_same_hash_slot(SINGLE_CACHE_KEY, PATTERN) == _convert_key(
|
|
SINGLE_CACHE_KEY
|
|
)
|
|
|
|
# works with a list (useful for cache.get_many operations)
|
|
assert ensure_cache_key_allocates_to_the_same_hash_slot(CACHE_KEYS, PATTERN) == [
|
|
_convert_key(k) for k in CACHE_KEYS
|
|
]
|
|
|
|
# works with a dict (useful for cache.set_many operations)
|
|
assert ensure_cache_key_allocates_to_the_same_hash_slot(SET_MANY_CACHE_KEYS_DICT, PATTERN) == {
|
|
_convert_key(k): v for k, v in SET_MANY_CACHE_KEYS_DICT.items()
|
|
}
|
|
|
|
# if the pattern doesn't exist, we don't wrap it in brackets
|
|
assert (
|
|
ensure_cache_key_allocates_to_the_same_hash_slot(SINGLE_CACHE_KEY, NON_EXISTENT_PATTERN) == SINGLE_CACHE_KEY
|
|
)
|
|
assert ensure_cache_key_allocates_to_the_same_hash_slot(CACHE_KEYS, NON_EXISTENT_PATTERN) == CACHE_KEYS
|
|
assert (
|
|
ensure_cache_key_allocates_to_the_same_hash_slot(SET_MANY_CACHE_KEYS_DICT, NON_EXISTENT_PATTERN)
|
|
== SET_MANY_CACHE_KEYS_DICT
|
|
)
|