The generated CSV report should meet Global Specifications when double-quote(s) are in row log

Idea ID 2770762

The generated CSV report should meet Global Specifications when double-quote(s) are in row log

0 Votes

When the row log into ArcSight includes double-quotes, they are often not properly handled.
The generated CSV file does not escape the double-quotes violating CVS file format specification.


The exported CSV files in such cases are erroneous and can't be interchange with other systems making them unusable.

 

Below are some cases we observed with double-quotes in row log:

Row log                    ⇒  Arcsight's data          ⇒   csv Result
----------------------------------------------------------------------- ----
abcd                          ⇒ abcd                               ⇒ abcd OK
\"abcd\"                    ⇒ "abcd"                            ⇒ "abcd" ▲("""abcd""" is true)
a\"bc\"d                    ⇒ a"bc"d                            ⇒ "a""bc""d" OK
\"a\"bc\"d\"              ⇒ "a"bc"d"                         ⇒ "a"bc"d" NG (") isn't escaped
ab\"cd                       ⇒ ab"cd                             ⇒ "ab""cd" OK
\"ab\"cd\"                 ⇒ "ab"cd"                          ⇒ "ab"cd" NG (") isn't escaped
a,b,c,d                       ⇒ a,b,c,d                             ⇒ "a,b,c,d" OK
\"a,b,c,d\"                 ⇒ "a,b,c,d"                          ⇒ ""a,b,c,d"", NG (") isn't escaped
a\rb\rc\rd                ⇒ a\rb\rc\rd                     ⇒ a<CR>b<CR>c<CR>d NG in spite of being <CR>, " is not enclosed
\"a\rb\rc\rd\"          ⇒ "a\rb\rc\rd"                   ⇒ "a<CR>b<CR>c<CR>d" ▲("""a<CR>b<CR>c<CR>d""" is true)
a\nb\nc\nd            ⇒ a\nb\nc\nd                    ⇒ "a<LF>b<LF>c<LF>d" OK
\"a\nb\nc\nd\"      ⇒ "a\nb\nc\nd"                 ⇒ ""a<LF>b<LF>c<LF>d"" NG (") isn't escaped
a\r\nb\r\nc\r\nd   ⇒ a\r\nb\r\nc\r\nd          ⇒ "a<CR><LF>b<CR><LF>c<CR><LF>d" OK
\"a\r\nb\r\nc\r\nd\" ⇒ "a\r\nb\r\nc\r\nd"  ⇒ ""a<CR><LF>b<CR><LF>c<CR><LF>d"" NG (") isn't escaped

Where:
※\r=<CR>=newline code
※\n=<LF>=newline code
※\r\n=<CR><LF>=newline code


Reference: RFC 4180, Common Format and MIME Type for Comma-Separated Values (CSV) Files
https://tools.ietf.org/html/rfc4180#page-3

We recommend the inclusion of an efficient module which will properly escape double-quotes in row log in accordance to definition 7 of RFC 4180.

4 Comments
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor
Status changed to: New Idea

Hi, On what versions of the products have you faced this issue? Also, was it on search export or reports?

Member..
Member..

Hi @kousalya.nagara 

Sorry for my late reply. We faced above issues on product version 6.11 (Linux system) during report export. 

Micro Focus Frequent Contributor
Micro Focus Frequent Contributor
Status changed to: New Idea

Do you mean ESM 6.11?

Member..
Member..

Yes.

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.