Export Mordor Data¶
You can create your own mordor datasets like the json files available in this repo. The mordor style to do this is by exporting data from a kafka broker and writinng it to a JSON file while executing the simulated test.
In consumer mode, Kafkacat reads messages from a topic and prints them to standard output (stdout). You can also redirect it to a file (i.e. JSON) This means that you can save all the data collected right before you start a simulated test from a Kafka broker. You can stop the consumption when you are done performing the simulated test. You can just grab the logs from this repo and re-play them as if they were being ingested in real-time.
- Kafka Broker : A distributed publish-subscribe messaging system that is designed to be fast, scalable, fault-tolerant, and durable (
Installed by HELK).
- Kafkacat : A generic non-JVM producer and consumer for Apache Kafka >=0.8, think of it as a netcat for Kafka.
Install Kafkacat following the instructions from the official Kafkacat repo.
- If you are using a debian-based system, make sure you install the latest Kafkacat deb package.
- I recommend at least Ubuntu 18.04. You can check its Kafkacat deb package version and compare it with the latest one in the Kafkacat GitHub repo.
- You can also install it from source following the Quick Build instructions.
Consume data being produced from a kafka broker with the following flags:
Topic in the Kafka Broker to consume the data from
Offset to start consuming from (i.e. end)
$ kafkacat -b <HELK IP>:9092 -t winlogbeat -C -o end > empire_dcsync_$(date +%F%H%M%S).json
That’s it! You now can share that dataset with the community!