Hi,
We have multiple akka stream flows deployed and in our model, each of the stages in the stream is using mapAsyncUnordered and .async. This is by design for now in our higher level framework, that allows users to define their stream. I wanted to provide a way to detect backpressure within such a defined stream.
I wrote the following piece of code to create a InstrumentedStage that I attach to every stage in the stream using the via operator, so that it runs on the same actor as the stage underneath.
In this instrumentation, i am trying to calculate the upstream and downstream latencies for a stage and then signal backpressure whenever downstream latency is greater than upstream latency. I am using micrometer timers so that i can monitor the heatmap in grafana before i start to use this.
For now I am unable to see backpressure, mostly the upstream latencies are much higher than downstream in the heatmap.
I wanted to see given we are using mapAsyncUnordered and async, is this the right logic to do this, or do I have to record the id of every event flowing through onPush and onPull and then calculate the latencies according to those. Appreciate any help and insights into this.
Are there examples of custom stages when using a combination of mapAsyncUnordered and async that I can look at as well.
Thanks
Mohnish
2 posts - 2 participants