Heartbeat interval is growing too large when sending large payload between nodes

@weicheng113 wrote:

Hi,

My akka cluster is processing messages from a Queue. For each message, I will do a calculation, which will result in an big Update payload being sent to its correponding persistent actor(which can be sitting in different machine). Message processing is done within an akka stream, which control maximal parallel messages at a time. The Update payload can be quite large, up to 50MB. I observed akka.remote.default-remote-dispatcher having high cpu and “Heartbeat interval is growing too large” message started to appear. I was reading this article - https://petabridge.com/blog/large-messages-and-sockets-in-akkadotnet/. As Heartbeat message is sent through akka.remote.default-remote-dispatcher also, I think this big Update payload is a problem. I need an akka stream to control and limit the parallel message processing. Do you have a recommendation of design for such system? Thanks.

====Update
I am currently trying out artery and its artery.advanced.idle-cpu-level=1(it seems this option no longer exists? only artery.advanced.aeron.idle-cpu-level = 1 exists?). But I am still receiving “heartbeat interval is growing too large for address”

akka.remote {
  artery {
    enabled = on
    transport = tcp
    large-message-destinations = [
      "/user/myActor1"
    ]
    advanced {
      aeron.idle-cpu-level = 1
      idle-cpu-level = 1
      maximum-large-frame-size = 50 MiB
      large-buffer-pool-size = 5
    }
  }

Cheng

Posts: 1

Participants: 1

Read full topic