Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite the MQ sender code to make it resilient towards nonresponsive MQ brokers #474

Open
oyvindhagberg opened this issue Nov 30, 2022 · 2 comments
Labels
bug Something isn't working enhancement New feature or request

Comments

@oyvindhagberg
Copy link
Contributor

The original comment: #473 (comment)

Some downsides to this implementation:
if the message broker is having performance issues, it will add latency to all model changes / API requests
best case: the connection is retried ten times
worst case: the connection hangs, waiting for a timeout, ten times
if the message broker is down, messages are silently discarded

Suggestion:
add an intermediate Notification model for serialized message data
change the signals to create Notification objects
add a long-running process (via a management command) that:
selects the N oldest notifications
publishes them to the message broker
deletes the notification if successful
sleeps, repeats
With this, an auto-incrementing primary key could be used as the message ID.

@terjekv terjekv added bug Something isn't working enhancement New feature or request labels May 28, 2023
@terjekv
Copy link
Collaborator

terjekv commented May 28, 2023

@ponas / @oyvindhagberg, is this a candidate for a celery queue of some sort? It'd require redis though. Probably easier to do it ourselves.

@terjekv
Copy link
Collaborator

terjekv commented May 29, 2023

https://django-q2.readthedocs.io/en/master/index.html with https://django-q2.readthedocs.io/en/master/configure.html#orm-configuration might be a viable option. The real question is if the load impact on our DB is a problem. I severely doubt it will be, but yeah.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants