nologcz/headline
2
1
Fork 0
mirror of https://git.nolog.cz/NoLog.cz/headline.git synced 2025-01-31 03:43:35 +01:00
Code Issues Projects Releases Packages Wiki Activity

add expire value to redis keys (7 days)

Browse source
This commit is contained in:
mdivecky 2022-08-27 13:45:25 +02:00
parent e61fc639db
commit f485239bf3
2 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
README.md
View file

@ -25,5 +25,4 @@ After first start, you have to wait for ~5mins for the "processor" to create fir
* Collect creation time of orig/new article, write it to permanent storage (sqlite3 for now) and display it. * Collect creation time of orig/new article, write it to permanent storage (sqlite3 for now) and display it.
* Write better readme and little more docs. * Write better readme and little more docs.
* Create view with some more info and stats (list of feeds, articles in redis, etc.) * Create view with some more info and stats (list of feeds, articles in redis, etc.)
* Create a routine to clear old articles from Redis (otherwise it will just fill up the disk space at some point...)
* IDEA: Figure out how to monitor changes in article description (maybe just compare hashes?) and how to present them. (Right now, the code can store descriptions in redis, but nothing else) * IDEA: Figure out how to monitor changes in article description (maybe just compare hashes?) and how to present them. (Right now, the code can store descriptions in redis, but nothing else)

3
processor/app.py
View file

@ -19,6 +19,7 @@ from diff_match_patch import diff_match_patch
# Teď budeme kontrolovat jen titulky, ale postupně můžeme přidat i description článku # Teď budeme kontrolovat jen titulky, ale postupně můžeme přidat i description článku
CONFIG_FILE = "../data/config.yaml" CONFIG_FILE = "../data/config.yaml"
REDIS_ARTICLE_EXPIRE_SEC = 604800
config = confuse.Configuration('headline', __name__) config = confuse.Configuration('headline', __name__)
config.set_file(CONFIG_FILE) config.set_file(CONFIG_FILE)
@ -44,7 +45,7 @@ article_count = 0
def write_article(article, rc): def write_article(article, rc):
rval = json.dumps(article['content']) rval = json.dumps(article['content'])
rc.set(article['rss_id'], rval) rc.set(article['rss_id'], rval, ex=REDIS_ARTICLE_EXPIRE_SEC)
def process_diff(old, new, rss_id): def process_diff(old, new, rss_id):
diff = dmp.diff_main(old['title'], new['title']) diff = dmp.diff_main(old['title'], new['title'])