[go: up one dir, main page]
Scribd
Upload
430 views6 pages

Kafka Cluster Sizing Calculator

The document summarizes messaging metrics for a 3 node Kafka cluster. It reports an average message size of 8.5KB with 600 messages per second and a retention period of 7 days. The cluster has a total storage of 104MB replicated across 3 nodes. Network utilization is 60% with a bandwidth of 1Gbps. There are 900 partitions distributed across brokers with a target throughput of 1536Mbps requiring 31 partitions per broker.

Uploaded by

JhonnyMoreno
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as XLSX, PDF, TXT or read online on Scribd
430 views6 pages

Kafka Cluster Sizing Calculator

The document summarizes messaging metrics for a 3 node Kafka cluster. It reports an average message size of 8.5KB with 600 messages per second and a retention period of 7 days. The cluster has a total storage of 104MB replicated across 3 nodes. Network utilization is 60% with a bandwidth of 1Gbps. There are 900 partitions distributed across brokers with a target throughput of 1536Mbps requiring 31 partitions per broker.

Uploaded by

JhonnyMoreno
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as XLSX, PDF, TXT or read online on Scribd
You are on page 1/ 6

Message sizing Number of Nodes

Avg size (bytes) 0 bytes Total Broker Storage


Avg size (Kb) 0 Kb Disk utilization
Avg Size (Mb) 0 Mb
Number of Nodes
Messages per day 18000000 messages/day
Messages per sec 600 msg/sec

Retention 1 day
Replication Factor 3

Storage (retención y rep 0 Mb


0 Gb 0

Number of Nodes (Total 3

Number of Partitions
(care about latency)
Partitions

For a Single partition (based on thoughpu


Write Througput
Read Througput
Target Througput

Number of partitions
Nodes Network Utilization

2048 Gb Incoming byterate


100% Number of Subscribers
Network bandwidth (bytes/sec)
3 Network bandwidth (Mb/sec)
Network bandwidth (Gb/sec)

Disk bandwidth (Mb/sec)


Disk bandwidth (bytes/sec)

Max net thoughput (Gb)


Max net thoughput (Mb)

Number of Nodes

Partitions

900 partitions / broker

tion (based on thoughput)


60 Mb/sec
50 Mb/sec
0 Mb/sec

0 partitions / broker
0 bytes/sec
100 clients
1073741824 bytes/sec
1048576 Mb/sec
1 Gb/sec

550 Mb/sec
576716800 bytes/sec

0 Gb/sec
0 Mb/sec

0
Message sizing Number of Nodes

Avg size (bytes) 8590 bytes Total Broker Storage


Avg size (Kb) 8.388671875 Kb Disk utilization
Avg Size (Mb) 0.00819206238 Mb
Number of Nodes
Messages per day 600 messages/day
Messages per sec 125 msg/sec

Retention 7 day
Replication Factor 3

Storage 104 Mb
1 Gb

Number of Nodes (Total 3

Number of Partitions
(care about latency)
Partitions

For a Single partition (based on thoughpu


Write Througput
Read Througput
Target Througput

Number of partitions
Nodes Network Utilization

5120 Gb Incoming byterate 1073750


60% Number of Subscribers 28
Network bandwidth (bytes/ 1073741824
3 Network bandwidth (Mb/se 1024
Network bandwidth (Gb/se 1

Disk bandwidth (Mb/sec) 125


Disk bandwidth (bytes/sec) 131072000

Max net thoughput (Gb) 1.5


Max net thoughput (Mb) 1536

Number of Nodes 1

Partitions

900 partitions / broker

tion (based on thoughput)


60 Mb/sec
50 Mb/sec
1536 Mb/sec

31 partitions / broker
bytes/sec
clients
bytes/sec
Mb/sec
Gb/sec

Mb/sec
bytes/sec

Gb/sec
Mb/sec

You might also like