A piece of CAKe

[GCA]

I don't know about you but I find pretty difficult to generate realistic traffic for testing. While existing tools such as the replayfilegen, perftestsyslog or bleep are nice I often feel they lack of flexibility. That's why I decided to write my own tool. CAKe (standing for CEF Army Knife experiment) is a python script that I define as a versatile CEF manipulation and generation tool.

CAKe can:

• replay existing CEF events or generate partially random CEF events based on a customizable template
• Modify timestamps of your CEF events. You can decide the timestamp(s) you want to modify and 4 modes exist:
  • realtime: you define the EPS you want and the timestamps are generated to simulate realtime events
  • customtime : you define a number of events and a time period (can be something like Now-1h:Now:100). Timestamps are created to spread your events equally all over this period.
  • keeptimestamp: keep existing timestamp
  • no timestamp: remove existing timestamp
• generate different output for your new cef events
  • CEF files: events are simply stored in files.
  • CSV file: allows to get a human readable format for CEF events
  • Syslog: CEF events can be sent to a syslog connector in TCP or UDP
  • display: CEF events are directly displayed in your terminal
• Modify CEF events content in multiple ways:
  • sanitization: it's possible to remove IPs or to randomize them. It's possible to remove CEF fields if the fieldname or the value contains a given string
  • extract: allows only keeping some CEF fields from a given CEF event
  • select/unselect: allows to keep only or remove CEF events containing a given pattern
  • add: allows adding some extra CEF fields
  • fix: a string search and replace function in the CEF event

If you are interested you can find a lot more details in the documentation. If you want to give it a try, please help yourself and have a piece of CAKe.

All your comments are more than welcome

[Michel Beaudry]

Quite a bit of work you've put in this program, we'll give a try.

Good job Gaetan and thanks for sharing.

[jdefrain1]

Wow, this look awesome! thanks for sharing.

[jdefrain1]

What's the highest ePS you have ever achieved?  In your doc you mentioned 10,000 ePS using syslogd.

[karl.bergstrom1]

I´m setting up a new lab environment and this is just what I was looking for.

Excellent work Gaetan and thanks for sharing this.

[GCA]

I'm not too sure about the exact value, I can just tell you it will depend on many parameters but if you just replay an existing cef file and if you don't select any other output at the same time you should be able to go higher than 10000 eps. Just be careful if you use udp because CAKe could generate an higher eps than what your syslog server is able to handle. If you conduct some performance testing with CAKe let us know your result.

[GCA]

I have conducted a few stress tests and I have been able to reach 30.000 eps for syslog/udp but it seems to be the limit on a standard workstation for a single instance of CAKe.  I also tried to run several instances in parallel on the same workstation. I could reach 25.000 eps per instance on 2 instances or 20.000 eps on 3 instances. This means I could reach a total of 60.000 eps on a standard workstation which is not that bad.

[jdefrain1]

So CAke sends 30,000 eps via UDP/Syslog to an ArcSight Syslog connector which then sends the data to ESM?

[GCA]

Well, this is not what I said. I just focused on the client side and was able to send 30.000 EPS with Cake in udp but I don't know if a syslog connector will be able to deal with that. There are some experts in high throughput connectors on the forum (jbur for instance) who should probably be able to tell us what's the maximum EPS a syslog connector can handle.

That being said, if your goal is to reach the highest EPS possible, you can just setup multiple syslog connectors and run one (or several) instance of CAKe for each connector.

[dbarry1]

HI Gaetan:

Forgive me, but I'm trying to get started with what looks like an excellent tool!    I've never used python before, so I may be coming from ignorance.

I installed Python 3.4.1 for Windows with all of its' optional defaults.   I placed your script and template in the directory below.  .py is properly assoicated with the python executable.  (My environment is Windows 7.)

However, I recieve the errors as seen below:

C:\Users\pc04888\Documents\CAKE>CAKe.py -g -n 100 -i CEFtemplate.txt -o CEFout

  File "C:\Users\pc04888\Documents\CAKE\CAKe.py", line 536

    print""

          ^

SyntaxError: invalid syntax

C:\Users\pc04888\Documents\CAKE>CAKE.py

  File "C:\Users\pc04888\Documents\CAKE\CAKe.py", line 536

    print""

          ^

SyntaxError: invalid syntax

C:\Users\pc04888\Documents\CAKE>py

Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:38:22) [MSC v.1600 32 bit (Intel)] on win32

Type "help", "copyright", "credits" or "license" for more information.

>>> ^Z

I hope this is enough to suggest a remedy.    Looking forward to getting to use your code!

d.

[dbarry1]

I'm going to answer my own question, so others may benefit.

In my case, I needed to install and use the Python 2.x series of code  (Presently 2.7.8), not the 3.x version.

Cheers,

d.

[GCA]

Yes indeed, python 3.x came with some syntax changes and the print function is one of those which makes the script incompatible with the new python releases. Feel free to provide feedback on the blog or via pm.

Cheers,

Gaetan