-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
We have our prune_logs task here:
https://github.com/maykinmedia/django-log-outgoing-requests/blob/main/log_outgoing_requests/tasks.py
This works if you have unlimited resources, however when running this for a very large database (eg the task hasn't been run before) and in a resource-restricted environment (k8s resource limits) the following can occur:
In [1]: from log_outgoing_requests.models import *
In [2]: OutgoingRequestsLog.objects.first()
Out[2]: <OutgoingRequestsLog: api.twitter.com at 2024-02-07 12:10:00.254395+00:00>
In [3]: OutgoingRequestsLog.objects.count()
Out[3]: 697150
In [4]: from datetime import timedelta
In [5]: now = timezone.now()
In [6]: settings.LOG_OUTGOING_REQUESTS_MAX_AGE
Out[6]: 30
In [7]: OutgoingRequestsLog.objects.filter(timestamp__lt=now - timedelta(180)).delete()
Geëlimineerd
I'd recommend to perform this delete in batches (it wasn't a problem when I did this in batches of ~60000 logs)
Metadata
Metadata
Assignees
Labels
No labels