Welcome Serena Central users! CLICK HERE
The migration of the Serena Central community is currently underway. Be sure to read THIS MESSAGE to get your new login set up to access your account.
Knowledge Partner
Knowledge Partner
237 views

Interesting Split Token behavior

A colleague was using a | symbol (pipe) as a delimiter in a string.
Then used the Split token upon it.

This rule demonstrates the craziness of what happens...

<rule>
<description>Split Token funniness</description>
<conditions>
<and/>
</conditions>
<actions>
<do-set-local-variable name="VALUES" scope="policy">
<arg-string>
<token-text xml:space="preserve">This|That|The|Other</token-text>
</arg-string>
</do-set-local-variable>
<do-set-local-variable name="NODES-WRONG" scope="policy">
<arg-node-set>
<token-split delimiter="|">
<token-local-variable name="VALUES"/>
</token-split>
</arg-node-set>
</do-set-local-variable>
<do-set-local-variable name="NODES-RIGHT" scope="policy">
<arg-node-set>
<token-split delimiter="\|">
<token-local-variable name="VALUES"/>
</token-split>
</arg-node-set>
</do-set-local-variable>
</actions>
</rule>

Run it through Simulator and you get:


Toolkit Rule Loopback : Action:
do-set-local-variable("NODES-WRONG",scope="policy",arg-node-set(token-split("|",token-local-variable("VALUES")))).
Toolkit Rule Loopback :
arg-node-set(token-split("|",token-local-variable("VALUES")))
Toolkit Rule Loopback :
token-split("|",token-local-variable("VALUES"))
Toolkit Rule Loopback :
token-split("|",token-local-variable("VALUES"))
Toolkit Rule Loopback : token-local-variable("VALUES")
Toolkit Rule Loopback : Token Value: "This|That|The|Other".
Toolkit Rule Loopback : Arg Value: "This|That|The|Other".
Toolkit Rule Loopback : Token Value:
{"","T","h","i","s","|","T","h","a","t","|","T","h","e","|","O","t","h","e","r",""}.
Toolkit Rule Loopback : Arg Value:
{"","T","h","i","s","|","T","h","a","t","|","T","h","e","|","O","t","h","e","r",""}.
Toolkit Rule Loopback : Action:
do-set-local-variable("NODES-RIGHT",scope="policy",arg-node-set(token-split("\|",token-local-variable("VALUES")))).
Toolkit Rule Loopback :
arg-node-set(token-split("\|",token-local-variable("VALUES")))
Toolkit Rule Loopback :
token-split("\|",token-local-variable("VALUES"))
Toolkit Rule Loopback :
token-split("\|",token-local-variable("VALUES"))
Toolkit Rule Loopback : token-local-variable("VALUES")
Toolkit Rule Loopback : Token Value: "This|That|The|Other".
Toolkit Rule Loopback : Arg Value: "This|That|The|Other".
Toolkit Rule Loopback : Token Value: {"This","That","The","Other"}.
Toolkit Rule Loopback : Arg Value: {"This","That","The","Other"}.

Look at what happens when you use just a pipe symbol as the delimiter in
the Split token!

Use \| and it works properly. That field is NOT called out as a Regex
field! And predictably, [|] (The Regex set, consisting of just the pipe
symbol) works properly as well.

Now I am trying to think of a fun delimiter to do in Regex...
Labels (1)
0 Likes
1 Reply
Anonymous_User Absent Member.
Absent Member.

Re: Interesting Split Token behavior

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The DTD says token-split uses a regex for the delimiter.

http://developer.novell.com/documentation/dirxml/dirxmlbk/ref/dirxmlscript/token-split.html

This is also how Perl handles it, interestingly enough. I assume this
is because the pipe is an OR operator and so when you do not escape it
the regex tries to match (a single character since you are in a regex
and did not use any quantifiers) and every single byte matches. Perl
example below:

ab@mybox0:~/Desktop> perl
$str='asdf|qwer|zxcv|wert|sdfg';
@ary = split(/|/, $str);
foreach $key (keys(@ary)){
print 'value: ' . $ary[$key] . "\n";
}
value: a
value: s
value: d
value: f
value: |
value: q
value: w
value: e
value: r
value: |
value: z
value: x
value: c
value: v
value: |
value: w
value: e
value: r
value: t
value: |
value: s
value: d
value: f
value: g

Good luck.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJO+pljAAoJEF+XTK08PnB5dDIQAJvg7BfLdtEAA8y4/2uH+/LS
Nj4JLOJy26souZUVVj3rDn0X3+HvxhvD/RhESW1E5RSuRiD+KYEI+lX9WpzncMnN
7ZueGW9YnvGBeEA9kchww9whKiMrb0HW5fsiKwEagqMO0e9BdFLLnmIDH4sKKBjh
G/Fzn1e1wo/6lFFijEURxDFYLderZdGN+KOA6qPU8fK6Xxam76R5RLmZiDFvBrim
Fy8x1BEVT5zBXYZgFg0Z/NAJmyfjqqoiz0PeaNZuESCynnhrpnUXDlb0cKna5+dq
Lq1nbvbbDFqrqSAuV3GH7KADbXju5I3j7VZwei46tQ88ofHNfuCygEqAEq7VAP2Y
NLpOCzFhXA0vFi+V1tn+53v56TEPAZKWGygn5179vlZD1XHd9FcuUhc60dhySz2S
mo226JGkXK5JwkqEzIKj7VVCXwoai3+GnZbJR9m/KkzTSHkdJc7ea5/bpRuq8tFZ
ikf+n5sy74iW01FABLtQ9TQXnO6rqdF6Qa/SlYyFVaJcev9AYXD0Sk2BZGI3Cbp0
P1Twf4hZHs7EEdXdl8XYA5mBxWoFxASl6F2btQ0Vie2RQGBWp5H0vSFyHTZmQfWl
reFPKcgx1NnoYkKNfc4J0yHmbYL0XRppMt5vP5tv/XcssUMT1WNL/MkLJXHeREz0
I9t4QYLjkfj+fO/+0nUt
=IzIJ
-----END PGP SIGNATURE-----
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.