Best mechanism to obfuscate fields at point of collection to a specific mapping table
I am looking for a design pattern to obfuscate certain fields, as close to point of collection as possible, to a specific value so ideally at the Connector I would use a mapping table where I could map the username (for example) to another specified value.
I know there is the "Fields to Obfuscate" feature within the Connector which will produce the MD5 hash of the fields specified however we have a requirement to map to a specific set of mappings.
If you are looking for posts that talk about the built in function please see:
There is still an outstanding question regarding the MD5 hashing itself:
Any help on this topic gratefully received. I have considered placing a custom Connector in the transmission path to perform this function:
SmartConnector --> Custom Connector performing mapping --> Transmission to Logger (compressed)
but wonderer if there is another way?
I am having the same issue. I need to replace real user names with a custom mapping to anonymized names. This mapping ensures reversing the process is possible only for people with access to the mapping list.
Did you ever solve this or did you build a custom connector? If yes, did you use a FlexConnector or some home brew perl/sed/whatever script to mangle the data before forwarding to the connector? Additionally, I am not sure how to self-mangle usernames in case of the Windows Unified connector.
Any hints are welcome
I'm thinking that you could use map files to map your custom value into a field, based on the known value of the first field. However,
- this method doesn't destroy or obfuscate the known value in the first field. Perhaps combining the obfuscation function with a map file would solve the problem, but it would depend on which function is performed first, both are at the connector.
- If this method works, then there would be no need for a FlexConnector (Custom Connector)
- I haven't tried this yet, but I wonder if you could use subsequent map files to then replace the value of the first known good field, with a dummy value.
I'm willing to assist in the iterations here if you are interested.
Could use map files as Aaron suggested.
Option 1: for known usernames to obfuscated
Option 2: For any usernames to obfuscated
Make sure to include conditions to replace usernames in specific events. The above will replace all usernames(dst) from all device logs to xxxx