In a world of text log files and JSON over HTTP, do not forget about the absolute treasure trove of incredibly valuable binary data that can be captured and preprocessed.
To name just a few examples :
- proprietary industry protocols such as MATIP in Aviation or ISO8583 in Payments Processing
- media files , images/audio/video
- compressed data
- encrypted data
- binary application dumps
A few years ago I wrote a really cool free and open source Splunk App called Protocol Data Inputs (PDI), in use by many customers today, that allows you to capture any data (binary or text) and then preprocess it for textual indexing in Splunk.
Check out some of the content in these blogs and presentations, I hope you can get some inspiration from them and start looking around for what binary data you have to unleash!