最新国产好看的视频,伊人天堂AV在线,国产Aaaaaa视频,蜜臀视频在线观看一区,人妻av色图,密臀久久久精品影片,青青视频免费观看毛片,久草在线观看视,国产三级精品色情在线

Redis中key過期策略的實現(xiàn)

 更新時間:2024年09月26日 09:23:31   作者:smileNicky  
Key的過期機制是Redis保持高可用性的重要策略,過期策略分為惰性過期和定期過期,惰性過期在每次訪問key時檢查是否過期,定期過期則由serverCron方法定時清理過期key,本文就來詳細的介紹一下,感興趣的可以了解一下

為什么要有過期策略?

Redis是一個內(nèi)存型的數(shù)據(jù)庫,數(shù)據(jù)是放在內(nèi)存里的,但是內(nèi)存也是有大小的,所以,需要配置redis占用的最大內(nèi)存,主要通過maxmemory配置

maxmomory <bytes>  # redis占用的最大內(nèi)存

官網(wǎng):https://redis.io/docs/manual/eviction/ 介紹

For example, to configure a memory limit of 100 megabytes, you can use the following directive inside the redis.conf file:

maxmomory 100mb

Setting maxmemory to zero results into no memory limits. This is the default behavior for 64 bit systems, while 32 bit systems use an implicit memory limit of 3GB.

翻譯一下,大致意思是如果配置為0,那么模式最大內(nèi)存大小就是電腦的內(nèi)存,如果是32bit隱式大小就是3G。

如果我們不淘汰過期的key數(shù)據(jù),堆積到一定程度,就會占滿內(nèi)存,滿了,就不能再放數(shù)據(jù),所以我們需要key過期機制,去刪除過期的數(shù)據(jù),保證redis的高可用。

什么是Redis key過期策略?

我們知道redis有一個特性,redis中的數(shù)據(jù),我們都是可以設(shè)置過期時間的,如果時間到了,這個數(shù)據(jù)就會從Redis中移除。那么redis key的過期策略就是我們怎么將redis中的過期數(shù)據(jù)移除。

key的惰性過期策略

惰性過期,就是在redis里面,在每次訪問操作key的時候,才判斷這個key是否過期了,如果過期了就刪除數(shù)據(jù)。redis中主要是通過db.c的expireIfNeeded方法去判斷,調(diào)用到相關(guān)命令時才會去調(diào)用,平時不會去判斷是否過期

查看一下源碼,expireIfNeeded方法,在db.c源碼,基于Redis6.0

int expireIfNeeded(redisDb *db, robj *key) {
    if (!keyIsExpired(db,key)) return 0;

    /* If we are running in the context of a slave, instead of
     * evicting the expired key from the database, we return ASAP:
     * the slave key expiration is controlled by the master that will
     * send us synthesized DEL operations for expired keys.
     *
     * Still we try to return the right information to the caller,
     * that is, 0 if we think the key should be still valid, 1 if
     * we think the key is expired at this time. */
    // 如果有配置masterhost,說明是從節(jié)點,那么不執(zhí)行key刪除操作
    if (server.masterhost != NULL) return 1;

    /* Delete the key */
    server.stat_expiredkeys++;
    propagateExpire(db,key,server.lazyfree_lazy_expire);
    notifyKeyspaceEvent(NOTIFY_EXPIRED,
        "expired",key,db->id);
    // 判斷l(xiāng)azyfree_lazy_expire是否開啟,開啟執(zhí)行異步刪除,不開啟執(zhí)行同步刪除,4.0之后新增的功能,默認是關(guān)閉
    int retval = server.lazyfree_lazy_expire ? dbAsyncDelete(db,key) :
                                               dbSyncDelete(db,key);
    if (retval) signalModifiedKey(NULL,db,key);
    return retval;
}

惰性刪除策略可以節(jié)省CPU資源,因為只需要訪問key的時候才去判斷是否過期,所以平時是沒啥CPU損耗的,但是如果沒有再次訪問,改過期的key就一直堆積在內(nèi)存里面,不會被清除,從而占用大量內(nèi)存空間,所以我們需要另外一種策略來配合使用,解決內(nèi)存占用問題,就是下面說的key定時過期策略。

key的定期過期策略

Redis中也提供了定期清除過期key的策略,在redis源碼里的server.c,里面有個serverCron方法,這個方法除了做Rehash以外,還會做很多其他的操作,比如

  • 清理數(shù)據(jù)庫中的過期鍵值對
  • 關(guān)閉和清理連接失效的客戶端
  • 嘗試進行持久化操作
  • 更新服務(wù)器的各類統(tǒng)計信息(時間、內(nèi)存占用、數(shù)據(jù)庫占用情況等)

Redis多久去清除過期的數(shù)據(jù),執(zhí)行頻率根據(jù)redis.conf里的配置hz

在這里插入圖片描述

然后實現(xiàn)流程大概是咋樣的?具體實現(xiàn)流程如下:

  • serverCron方法去執(zhí)行定時清理,執(zhí)行頻率redis.confhz參數(shù)配置,默認是10,也就是1s執(zhí)行10次,100ms執(zhí)行1次

  • 執(zhí)行清理的時候,去掃描所有設(shè)置了過期時間的key,不會去掃描所有的key

  • 根據(jù)桶的維度去掃描key,直到掃到20個key(可配)且最多取400個桶。假如第一個桶是15個key,沒有達到20個key,所以會繼續(xù)掃描第二個桶,第二個桶20個key,由于是以桶為維度進行掃描的,第二個桶會被全部掃描,所以總共掃描了35個key

  • 找到掃描的key里面過期的key,進行刪除操作

  • 判斷掃描的過期數(shù)據(jù)跟掃描總數(shù)的比例是否超過10%,是,繼續(xù)執(zhí)行3、4步;否,刪除完成。

執(zhí)行過程,畫一個流程圖:

在這里插入圖片描述

查看源碼,驗證一下,在redis源碼里的server.c有一個serverCron方法,里面有個databasesCron函數(shù)

/* Handle background operations on Redis databases. */
databasesCron();

同個類里,查看databasesCron函數(shù)

void databasesCron(void) {
    /* Expire keys by random sampling. Not required for slaves
     * as master will synthesize DELs for us. */
    if (server.active_expire_enabled) {
        if (iAmMaster()) { // 是否主服務(wù)器
            activeExpireCycle(ACTIVE_EXPIRE_CYCLE_SLOW);
        } else { // 從服務(wù)器
            expireSlaveKeys();
        }
    }

    /* Defrag keys gradually. */
    activeDefragCycle();

    /* Perform hash tables rehashing if needed, but only if there are no
     * other processes saving the DB on disk. Otherwise rehashing is bad
     * as will cause a lot of copy-on-write of memory pages. */
    if (!hasActiveChildProcess()) {
        /* We use global counters so if we stop the computation at a given
         * DB we'll be able to start from the successive in the next
         * cron loop iteration. */
        static unsigned int resize_db = 0;
        static unsigned int rehash_db = 0;
        int dbs_per_call = CRON_DBS_PER_CALL;
        int j;

        /* Don't test more DBs than we have. */
        if (dbs_per_call > server.dbnum) dbs_per_call = server.dbnum;

        /* Resize */
        for (j = 0; j < dbs_per_call; j++) {
            tryResizeHashTables(resize_db % server.dbnum);
            resize_db++;
        }

        /* Rehash */
        if (server.activerehashing) {
            for (j = 0; j < dbs_per_call; j++) {
                int work_done = incrementallyRehash(rehash_db);
                if (work_done) {
                    /* If the function did some work, stop here, we'll do
                     * more at the next cron loop. */
                    break;
                } else {
                    /* If this db didn't need rehash, we'll try the next one. */
                    rehash_db++;
                    rehash_db %= server.dbnum;
                }
            }
        }
    }
}

查看activeExpireCycle方法,在expire.c

void activeExpireCycle(int type) {
    /* Adjust the running parameters according to the configured expire
     * effort. The default effort is 1, and the maximum configurable effort
     * is 10. */
    unsigned long
    effort = server.active_expire_effort-1, /* Rescale from 0 to 9. */
    config_keys_per_loop = ACTIVE_EXPIRE_CYCLE_KEYS_PER_LOOP +
                           ACTIVE_EXPIRE_CYCLE_KEYS_PER_LOOP/4*effort,
    config_cycle_fast_duration = ACTIVE_EXPIRE_CYCLE_FAST_DURATION +
                                 ACTIVE_EXPIRE_CYCLE_FAST_DURATION/4*effort,
    config_cycle_slow_time_perc = ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC +
                                  2*effort,
    config_cycle_acceptable_stale = ACTIVE_EXPIRE_CYCLE_ACCEPTABLE_STALE-
                                    effort;

    /* This function has some global state in order to continue the work
     * incrementally across calls. */
    static unsigned int current_db = 0; /* Last DB tested. */
    static int timelimit_exit = 0;      /* Time limit hit in previous call? */
    static long long last_fast_cycle = 0; /* When last fast cycle ran. */

    int j, iteration = 0;
    int dbs_per_call = CRON_DBS_PER_CALL;
    long long start = ustime(), timelimit, elapsed;

    /* When clients are paused the dataset should be static not just from the
     * POV of clients not being able to write, but also from the POV of
     * expires and evictions of keys not being performed. */
    if (clientsArePaused()) return;

    if (type == ACTIVE_EXPIRE_CYCLE_FAST) {
        /* Don't start a fast cycle if the previous cycle did not exit
         * for time limit, unless the percentage of estimated stale keys is
         * too high. Also never repeat a fast cycle for the same period
         * as the fast cycle total duration itself. */
        if (!timelimit_exit &&
            server.stat_expired_stale_perc < config_cycle_acceptable_stale)
            return;

        if (start < last_fast_cycle + (long long)config_cycle_fast_duration*2)
            return;

        last_fast_cycle = start;
    }

    /* We usually should test CRON_DBS_PER_CALL per iteration, with
     * two exceptions:
     *
     * 1) Don't test more DBs than we have.
     * 2) If last time we hit the time limit, we want to scan all DBs
     * in this iteration, as there is work to do in some DB and we don't want
     * expired keys to use memory for too much time. */
    if (dbs_per_call > server.dbnum || timelimit_exit)
        dbs_per_call = server.dbnum;

    /* We can use at max 'config_cycle_slow_time_perc' percentage of CPU
     * time per iteration. Since this function gets called with a frequency of
     * server.hz times per second, the following is the max amount of
     * microseconds we can spend in this function. */
    timelimit = config_cycle_slow_time_perc*1000000/server.hz/100;
    timelimit_exit = 0;
    if (timelimit <= 0) timelimit = 1;

    if (type == ACTIVE_EXPIRE_CYCLE_FAST)
        timelimit = config_cycle_fast_duration; /* in microseconds. */

    /* Accumulate some global stats as we expire keys, to have some idea
     * about the number of keys that are already logically expired, but still
     * existing inside the database. */
    long total_sampled = 0;
    long total_expired = 0;

    for (j = 0; j < dbs_per_call && timelimit_exit == 0; j++) {
        /* Expired and checked in a single loop. */
        unsigned long expired, sampled;

        redisDb *db = server.db+(current_db % server.dbnum);

        /* Increment the DB now so we are sure if we run out of time
         * in the current DB we'll restart from the next. This allows to
         * distribute the time evenly across DBs. */
        current_db++;

        /* Continue to expire if at the end of the cycle there are still
         * a big percentage of keys to expire, compared to the number of keys
         * we scanned. The percentage, stored in config_cycle_acceptable_stale
         * is not fixed, but depends on the Redis configured "expire effort". */
        do {
            unsigned long num, slots;
            long long now, ttl_sum;
            int ttl_samples;
            iteration++;

            /* If there is nothing to expire try next DB ASAP. */
            if ((num = dictSize(db->expires)) == 0) {
                db->avg_ttl = 0;
                break;
            }
            slots = dictSlots(db->expires);
            now = mstime();

            /* When there are less than 1% filled slots, sampling the key
             * space is expensive, so stop here waiting for better times...
             * The dictionary will be resized asap. */
            if (num && slots > DICT_HT_INITIAL_SIZE &&
                (num*100/slots < 1)) break;

            /* The main collection cycle. Sample random keys among keys
             * with an expire set, checking for expired ones. */
            expired = 0;
            sampled = 0;
            ttl_sum = 0;
            ttl_samples = 0;
			// 最多那20個
            if (num > config_keys_per_loop)
                num = config_keys_per_loop;

            /* Here we access the low level representation of the hash table
             * for speed concerns: this makes this code coupled with dict.c,
             * but it hardly changed in ten years.
             *
             * Note that certain places of the hash table may be empty,
             * so we want also a stop condition about the number of
             * buckets that we scanned. However scanning for free buckets
             * is very fast: we are in the cache line scanning a sequential
             * array of NULL pointers, so we can scan a lot more buckets
             * than keys in the same time. */
            long max_buckets = num*20;
            long checked_buckets = 0;
			// 如果拿到的key數(shù)量大于20 或者 checked_buckets大于400,跳出循環(huán)
            while (sampled < num && checked_buckets < max_buckets) {
                for (int table = 0; table < 2; table++) {
                    if (table == 1 && !dictIsRehashing(db->expires)) break;

                    unsigned long idx = db->expires_cursor;
                    idx &= db->expires->ht[table].sizemask;
                    // 根據(jù)index拿到hash桶
                    dictEntry *de = db->expires->ht[table].table[idx];
                    long long ttl;

                    /* Scan the current bucket of the current table. */
                    checked_buckets++;
                    // 循環(huán)hash桶里的key
                    while(de) {
                        /* Get the next entry now since this entry may get
                         * deleted. */
                        dictEntry *e = de;
                        de = de->next;

                        ttl = dictGetSignedIntegerVal(e)-now;
                        if (activeExpireCycleTryExpire(db,e,now)) expired++;
                        if (ttl > 0) {
                            /* We want the average TTL of keys yet
                             * not expired. */
                            ttl_sum += ttl;
                            ttl_samples++;
                        }
                        sampled++;
                    }
                }
                db->expires_cursor++;
            }
            total_expired += expired;
            total_sampled += sampled;

            /* Update the average TTL stats for this database. */
            if (ttl_samples) {
                long long avg_ttl = ttl_sum/ttl_samples;

                /* Do a simple running average with a few samples.
                 * We just use the current estimate with a weight of 2%
                 * and the previous estimate with a weight of 98%. */
                if (db->avg_ttl == 0) db->avg_ttl = avg_ttl;
                db->avg_ttl = (db->avg_ttl/50)*49 + (avg_ttl/50);
            }

            /* We can't block forever here even if there are many keys to
             * expire. So after a given amount of milliseconds return to the
             * caller waiting for the other active expire cycle. */
            if ((iteration & 0xf) == 0) { /* check once every 16 iterations. */
                elapsed = ustime()-start;
                if (elapsed > timelimit) {
                    timelimit_exit = 1;
                    server.stat_expired_time_cap_reached_count++;
                    break;
                }
            }
            /* We don't repeat the cycle for the current database if there are
             * an acceptable amount of stale keys (logically expired but yet
             * not reclaimed). */
             // 比例超過10%,expired過期的key數(shù)量,sampled總的掃描數(shù)量
        } while (sampled == 0 ||
                 (expired*100/sampled) > config_cycle_acceptable_stale);
    }

    elapsed = ustime()-start;
    server.stat_expire_cycle_time_used += elapsed;
    latencyAddSampleIfNeeded("expire-cycle",elapsed/1000);

    /* Update our estimate of keys existing but yet to be expired.
     * Running average with this sample accounting for 5%. */
    double current_perc;
    if (total_sampled) {
        current_perc = (double)total_expired/total_sampled;
    } else
        current_perc = 0;
    server.stat_expired_stale_perc = (current_perc*0.05)+
                                     (server.stat_expired_stale_perc*0.95);
}

到此這篇關(guān)于Redis中key過期策略的實現(xiàn)的文章就介紹到這了,更多相關(guān)Redis key過期內(nèi)容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家! 

相關(guān)文章

  • redis使用watch秒殺搶購實現(xiàn)思路

    redis使用watch秒殺搶購實現(xiàn)思路

    這篇文章主要為大家詳細介紹了redis使用watch秒殺搶購的實現(xiàn)思路,文中示例代碼介紹的非常詳細,具有一定的參考價值,感興趣的小伙伴們可以參考一下
    2018-02-02
  • 使用Docker部署Redis并配置持久化與密碼保護的詳細步驟

    使用Docker部署Redis并配置持久化與密碼保護的詳細步驟

    本文將詳細介紹如何使用 Docker 部署 Redis,并通過 redis.conf 配置文件實現(xiàn)數(shù)據(jù)持久化和密碼保護,適合在生產(chǎn)環(huán)境中使用,文章通過代碼示例講解的非常詳細,需要的朋友可以參考下
    2025-03-03
  • MyBatis緩存和二級緩存整合Redis的解決方案

    MyBatis緩存和二級緩存整合Redis的解決方案

    這篇文章主要介紹了MyBatis緩存和二級緩存整合Redis,將MyBatis緩存和二級緩存整合Redis,可以提高查詢效率,同時也能保證數(shù)據(jù)的可靠性和一致性,需要的朋友可以參考下
    2023-07-07
  • 如何保證Redis與數(shù)據(jù)庫的數(shù)據(jù)一致性

    如何保證Redis與數(shù)據(jù)庫的數(shù)據(jù)一致性

    這篇文章主要介紹了如何保證Redis與數(shù)據(jù)庫的數(shù)據(jù)一致性,文中舉了兩個場景例子介紹的非常詳細,需要的朋友可以參考下
    2023-05-05
  • Redis 數(shù)據(jù)類型Streams詳解

    Redis 數(shù)據(jù)類型Streams詳解

    Redis Streams是Redis 5.0新增的數(shù)據(jù)類型,提供了一種日志結(jié)構(gòu)化數(shù)據(jù)存儲方式,這種類型適合用于構(gòu)建消息隊列、事件日志和處理時間序列數(shù)據(jù)的應(yīng)用,本文介紹Redis 數(shù)據(jù)類型Streams相關(guān)知識,感興趣的朋友一起看看吧
    2024-10-10
  • 詳細分析Redis集群故障

    詳細分析Redis集群故障

    這篇文章主要介紹了詳細分析Redis集群故障的相關(guān)內(nèi)容,具有一定的參考價值,這里分享給大家,供需要的朋友參考。
    2017-10-10
  • 基于redis集群設(shè)置密碼的實例

    基于redis集群設(shè)置密碼的實例

    今天小編就為大家分享一篇基于redis集群設(shè)置密碼的實例,具有很好的參考價值,希望對大家有所幫助。一起跟隨小編過來看看吧
    2018-05-05
  • Redis存儲經(jīng)緯度信息的實現(xiàn)

    Redis存儲經(jīng)緯度信息的實現(xiàn)

    在一些向用戶提供天氣信息的業(yè)務(wù)場景中,我們通常會通過前端獲取用戶經(jīng)緯度信息,傳遞給后端作為參數(shù)進行外部天氣接口調(diào)用,本文就來詳細的介紹一下Redis存儲經(jīng)緯度信息的實現(xiàn),感興趣的可以了解一下
    2025-10-10
  • redis yml配置的用法小結(jié)

    redis yml配置的用法小結(jié)

    RedisYML配置是Redis的一種配置文件格式,,對Redis的配置進行統(tǒng)一管理,本文就來介紹了redis yml配置的用法小結(jié),具有一定的參考價值,感興趣的可以了解一下
    2024-02-02
  • 如何在SpringBoot中使用Redis實現(xiàn)分布式鎖

    如何在SpringBoot中使用Redis實現(xiàn)分布式鎖

    這篇文章主要介紹了如何在SpringBoot中使用Redis實現(xiàn)分布式鎖,在實際開發(fā)中有可能會遇到多個線程同時訪問同一個共享變量,那么上鎖就很重要了,需要的朋友可以參考下
    2023-03-03

最新評論

虞城县| 泰顺县| 博客| 商城县| 邵阳县| 遂宁市| 楚雄市| 长垣县| 商城县| 沙河市| 吐鲁番市| 武平县| 蒙城县| 印江| 安顺市| 定远县| 嵩明县| 溧阳市| 收藏| 大冶市| 顺平县| 临西县| 南丹县| 和林格尔县| 若尔盖县| 驻马店市| 棋牌| 邵武市| 峨眉山市| 大宁县| 菏泽市| 电白县| 平乐县| 时尚| 搜索| 上虞市| 安义县| 临朐县| 尤溪县| 蛟河市| 高碑店市|