I will explain this with an example :
Consider you have a data of 30 TB in JSON/ or any other format -
If you directly store this data into Kafka Topic, you will need to pay according to the space you are consuming (if using the cloud)
Advice to use AVRO format in Kafka topic to compress the data
Research shows that we can save up to 91% of space using Avro
Example :
Data : 30 TB
After Avro conversion : 2.7 TB (up to 91% )
For this, you might need to convert JSON data to Avro data, which is complex but doable.
