Project

General

Profile

Task #8079

Story #8061: develop queue-based processing system for the CN

prototype durable task processing for d1_index_processor

Added by Rob Nahf over 4 years ago. Updated over 4 years ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
d1_indexer
Target version:
-
Start date:
2017-04-24
Due date:
% Done:

30%

Milestone:
None
Product Version:
*
Story Points:
Sprint:

Description

The requirement is publisher confirmation when submitting task to a queue, and consumer acknowledgements when pulling from a queue. Don't acknowledge consumption until you get publisher confirmation from the outputQueue, or else you risk dropping the task.

It would be good to generalize this pattern to isolate the safe-hand off, but RabbitMQ Java client makes it a bit tricky, since it implements consumers as callbacks - methods that are called under the hood when a message is consumed.

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)