Character encoding in plugin development

 
I'm having issues with character encoding on plugin development. The scandinavian characters we are using in our code to search certain strings in events are not working because those characters are messed up in Sentinel. This is how they are in debugger:

if (!username) {
username = this.findUserName(this.RXMap.EVENT_DESCRIPTION, "N�yt� k�ytt�j� ");
}


What that should be is this:

if (!username) {
username = this.findUserName(this.RXMap.EVENT_DESCRIPTION, "Näytä käyttäjä ");
}


So if the event contains that text "Näytä käyttäjä " nothing happens in the Sentinel parsing. I've tried to modify the file encoding (UTF8 or ISO 8859-1) in notepad but can't get the wanted result. I also noticed that my new SDK install behaves the same way. I'm running Win10 workstation and i also tried to modify the code in Server 2012 R2 environment in case my computer is doing this erratic behaviour. I also tried to upload the plugin from my own computer to Sentinel and from that 2012 R2 server to see if locally installed Java -version could affect it when running Control Center. Any ideas what i should try next?
  • 0  
    On 07/10/2018 01:44 AM, jarivaahtera wrote:
    >
    > I'm having issues with character encoding on plugin development. The
    > scandinavian characters we are using in our code to search certain
    > strings in events are not working because those characters are messed up
    > in Sentinel. This is how they are in debugger:
    >
    >
    > Code:
    > --------------------
    > if (!username) {
    > username = this.findUserName(this.RXMap.EVENT_DESCRIPTION, "N�yt� k�ytt�j� ");
    > }
    >
    > --------------------


    Which version of Sentinel are you using, exactly, and with which type of
    event source?

    I understand this is likely a completely custom collector, but for which
    application? Could you share the full code?

    > What that should be is this:
    >
    >
    > Code:
    > --------------------
    > if (!username) {
    > username = this.findUserName(this.RXMap.EVENT_DESCRIPTION, "N�yt� k�ytt�j� ");
    > }
    > --------------------
    >
    >
    > So if the event contains that text "N�yt� k�ytt�j� " nothing happens in
    > the Sentinel parsing. I've tried to modify the file encoding (UTF8 or


    I am guessing that "nothing happens" is inaccurate as well as imprecise.
    Please elaborate on "nothing", as if you really get no collector behavior
    at all, then the problem is not with the collector but probably with the
    connector, or the Collector Manager (CM) service itself.

    It seems likely that you mean the username variable is not set properly,
    or the function does not return as you expect so the username is munged.
    Since this conditional is checking for username before setting it, knowing
    what code impacts the username variable before this point could definitely
    be useful.

    Could you help us understand why you are hard-coding a person's name into
    the collector code? I can think of a couple reasons, but it seems like
    something unusual considering that looks like a regular person's name and
    not something like 'admin' or 'root' or whatever.

    Could you post a connector dump of the original event?

    Have you tried using the Unicode escape sequences to pass in the non-ASCII
    characters? e.g. '\u00a9' and the like?

    If the entire event fails you should get a line indicating the general
    failure in the CM log files, but you could make output more verbose by
    adding debugging messages and using the debugger directly. As is, I think
    I need more information to guess at the root cause, including at least the
    new collector (your function, etc.) and some sample data for duplication.

    --
    Good luck.

    If you find this post helpful and are logged into the web interface,
    show your appreciation and click on the star below.

    If you want to send me a private message, please let me know in the
    forum as I do not use the web interface often.
  • 0  
    jarivaahtera;2483713 wrote:
    I'm having issues with character encoding on plugin development. The scandinavian characters we are using in our code to search certain strings in events are not working because those characters are messed up in Sentinel. This is how they are in debugger:

    if (!username) {
    username = this.findUserName(this.RXMap.EVENT_DESCRIPTION, "N�yt� k�ytt�j� ");
    }


    What that should be is this:

    if (!username) {
    username = this.findUserName(this.RXMap.EVENT_DESCRIPTION, "Näytä käyttäjä ");
    }


    So if the event contains that text "Näytä käyttäjä " nothing happens in the Sentinel parsing. I've tried to modify the file encoding (UTF8 or ISO 8859-1) in notepad but can't get the wanted result. I also noticed that my new SDK install behaves the same way. I'm running Win10 workstation and i also tried to modify the code in Server 2012 R2 environment in case my computer is doing this erratic behaviour. I also tried to upload the plugin from my own computer to Sentinel and from that 2012 R2 server to see if locally installed Java -version could affect it when running Control Center. Any ideas what i should try next?


    This should be a helpful article:

    www.google.com/search characters in javascript
  • 0   in reply to brandon-langley
    Thanks for the tips! It worked when i replaced those special characters with unicode characters. We have some plugins that have those special characters in the code same way as i tried to use them in the code on my first post in this thread. They work as they should be working and mine doesn't. Though those working plugins are not made by us, so there can be some differences in SDK environments or something.
  • 0   in reply to jarivaahtera
    jarivaahtera;2483829 wrote:
    Thanks for the tips! It worked when i replaced those special characters with unicode characters. We have some plugins that have those special characters in the code same way as i tried to use them in the code on my first post in this thread. They work as they should be working and mine doesn't. Though those working plugins are not made by us, so there can be some differences in SDK environments or something.


    It's one of those unfortunate cases where the unicode escapes are the written rule, but some older versions of Java/JS/etc may have permitted poor practices. Glad to see that it works, but do beware if those plugins get updated or on future versions of Sentinel, that you may see those plugins fail to work for this reason.