Knowledge Partner
Knowledge Partner
1497 views

Which DB to use?

When choosing between the HSQL default database and the recommended
MySQL database for the Analyzer store, is there a difference in string
length for attributes?

Latest Analyzer build 4.02.
Labels (1)
0 Likes
24 Replies
Anonymous_User Absent Member.
Absent Member.

Re: Which DB to use?

On 9/30/2014 3:45 PM, Geoffrey Carman wrote:
> When choosing between the HSQL default database and the recommended MySQL database for the Analyzer store, is there a
> difference in string length for attributes?
>
> Latest Analyzer build 4.02.


I noticed recently that when you switch to MySQL the default string length is 255 characters. This is despite the fact that
MySQL has for some time now supported up to 65,535. I found I had to wait until the tables are created and then hurriedly
change the table DDL to increase the length during the import. This is problematic as you still do get data loss.

The HSQL database however is too slow to be useful on larger data sets.

Is there a solution for this?

--
-----------------------------------------------------------------------
Will Schneider
Knowledge Partner http://forums.netiq.com

If you find this post helpful, please click on the star below.
0 Likes
Knowledge Partner
Knowledge Partner

Re: Which DB to use?

On 9/30/2014 4:52 PM, Will Schneider wrote:
> On 9/30/2014 3:45 PM, Geoffrey Carman wrote:
>> When choosing between the HSQL default database and the recommended
>> MySQL database for the Analyzer store, is there a
>> difference in string length for attributes?
>>
>> Latest Analyzer build 4.02.

>
> I noticed recently that when you switch to MySQL the default string
> length is 255 characters. This is despite the fact that MySQL has for
> some time now supported up to 65,535. I found I had to wait until the
> tables are created and then hurriedly change the table DDL to increase
> the length during the import. This is problematic as you still do get
> data loss.
>
> The HSQL database however is too slow to be useful on larger data sets.
>
> Is there a solution for this?


Have you had need for strings longer than 256?

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Which DB to use?

On 9/30/2014 3:58 PM, Geoffrey Carman wrote:
> Have you had need for strings longer than 256?


Yes. I had a custom attr in an Active Directory that was absurdly long because it had a comma delimited list of group names.

--
-----------------------------------------------------------------------
Will Schneider
Knowledge Partner http://forums.netiq.com

If you find this post helpful, please click on the star below.
0 Likes
Knowledge Partner
Knowledge Partner

Re: Which DB to use?

On 9/30/2014 5:01 PM, Will Schneider wrote:
> On 9/30/2014 3:58 PM, Geoffrey Carman wrote:
>> Have you had need for strings longer than 256?

>
> Yes. I had a custom attr in an Active Directory that was absurdly long
> because it had a comma delimited list of group names.


That is a stupid approach. Customers, can't live with them, can't live
without em.

Ok, I suppose DirXML-EntitlementRef could get pretty long as well as a
simpler example.

Was that to reimplement the memberOf functionality?

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Which DB to use?

On 9/30/2014 4:06 PM, Geoffrey Carman wrote:
> On 9/30/2014 5:01 PM, Will Schneider wrote:
>> On 9/30/2014 3:58 PM, Geoffrey Carman wrote:
>>> Have you had need for strings longer than 256?

>>
>> Yes. I had a custom attr in an Active Directory that was absurdly long
>> because it had a comma delimited list of group names.

>
> That is a stupid approach. Customers, can't live with them, can't live without em.
>
> Ok, I suppose DirXML-EntitlementRef could get pretty long as well as a simpler example.
>
> Was that to reimplement the memberOf functionality?
>


It was functionality to support a specific application. It's messy.

--
-----------------------------------------------------------------------
Will Schneider
Knowledge Partner http://forums.netiq.com

If you find this post helpful, please click on the star below.
0 Likes
Knowledge Partner
Knowledge Partner

Re: Which DB to use?

On 9/30/2014 5:10 PM, Will Schneider wrote:
> On 9/30/2014 4:06 PM, Geoffrey Carman wrote:
>> On 9/30/2014 5:01 PM, Will Schneider wrote:
>>> On 9/30/2014 3:58 PM, Geoffrey Carman wrote:
>>>> Have you had need for strings longer than 256?
>>>
>>> Yes. I had a custom attr in an Active Directory that was absurdly long
>>> because it had a comma delimited list of group names.

>>
>> That is a stupid approach. Customers, can't live with them, can't
>> live without em.
>>
>> Ok, I suppose DirXML-EntitlementRef could get pretty long as well as a
>> simpler example.
>>
>> Was that to reimplement the memberOf functionality?
>>

>
> It was functionality to support a specific application. It's messy.


I suppose there are other examples.

The specific app was looking for memberOf equivs, but not getting what
it needed from AD, so they decided to cheat on AD by concatting all the
group names into a string for the app.

Naughty.


0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Which DB to use?

On 9/30/2014 4:12 PM, Geoffrey Carman wrote:
> The specific app was looking for memberOf equivs, but not getting what it needed from AD, so they decided to cheat on AD by
> concatting all the group names into a string for the app.
>
> Naughty.


Sadly, it is using that attribute as a source to provision users into the actual groups.

--
-----------------------------------------------------------------------
Will Schneider
Knowledge Partner http://forums.netiq.com

If you find this post helpful, please click on the star below.
0 Likes
Knowledge Partner
Knowledge Partner

Re: Which DB to use?

On 9/30/2014 5:16 PM, Will Schneider wrote:
> On 9/30/2014 4:12 PM, Geoffrey Carman wrote:
>> The specific app was looking for memberOf equivs, but not getting what
>> it needed from AD, so they decided to cheat on AD by
>> concatting all the group names into a string for the app.
>>
>> Naughty.

>
> Sadly, it is using that attribute as a source to provision users into
> the actual groups.


Wild.

Do you recall Condrey Consulting showing off a product for AD systems
that would convert OU placement into Group membership? Not an IDM
thing, but for AD guys, who need groups not OU placements.

Similar sort of thing perhaps?


0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Which DB to use?

On 9/30/2014 4:17 PM, Geoffrey Carman wrote:
> On 9/30/2014 5:16 PM, Will Schneider wrote:
>> On 9/30/2014 4:12 PM, Geoffrey Carman wrote:
>>> The specific app was looking for memberOf equivs, but not getting what
>>> it needed from AD, so they decided to cheat on AD by
>>> concatting all the group names into a string for the app.
>>>
>>> Naughty.

>>
>> Sadly, it is using that attribute as a source to provision users into
>> the actual groups.

>
> Wild.
>
> Do you recall Condrey Consulting showing off a product for AD systems that would convert OU placement into Group membership?
> Not an IDM thing, but for AD guys, who need groups not OU placements.
>
> Similar sort of thing perhaps?
>
>

No, it was an IDM software package (owned by M$ I believe) that uses AD as its data store. These were actually group objects
representing users with custom attrs for the actual user attrs. Then that would replicate to an actual user object.
<puke>

--
-----------------------------------------------------------------------
Will Schneider
Knowledge Partner http://forums.netiq.com

If you find this post helpful, please click on the star below.
0 Likes
Knowledge Partner
Knowledge Partner

Re: Which DB to use?

On 9/30/2014 5:22 PM, Will Schneider wrote:
> On 9/30/2014 4:17 PM, Geoffrey Carman wrote:
>> On 9/30/2014 5:16 PM, Will Schneider wrote:
>>> On 9/30/2014 4:12 PM, Geoffrey Carman wrote:
>>>> The specific app was looking for memberOf equivs, but not getting what
>>>> it needed from AD, so they decided to cheat on AD by
>>>> concatting all the group names into a string for the app.
>>>>
>>>> Naughty.
>>>
>>> Sadly, it is using that attribute as a source to provision users into
>>> the actual groups.

>>
>> Wild.
>>
>> Do you recall Condrey Consulting showing off a product for AD systems
>> that would convert OU placement into Group membership?
>> Not an IDM thing, but for AD guys, who need groups not OU placements.
>>
>> Similar sort of thing perhaps?
>>
>>

> No, it was an IDM software package (owned by M$ I believe) that uses AD
> as its data store. These were actually group objects representing users
> with custom attrs for the actual user attrs. Then that would replicate
> to an actual user object.
> <puke>


FIM? ILM? MIIS? original project they bought and destroyed whose name
escapes me? (Forefront Identity Manager, Identity Lifecycle Manager,
Microsoft Identity Integration Server?)

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Which DB to use?

On 9/30/2014 4:28 PM, Geoffrey Carman wrote:
> On 9/30/2014 5:22 PM, Will Schneider wrote:
>> On 9/30/2014 4:17 PM, Geoffrey Carman wrote:
>>> On 9/30/2014 5:16 PM, Will Schneider wrote:
>>>> On 9/30/2014 4:12 PM, Geoffrey Carman wrote:
>>>>> The specific app was looking for memberOf equivs, but not getting what
>>>>> it needed from AD, so they decided to cheat on AD by
>>>>> concatting all the group names into a string for the app.
>>>>>
>>>>> Naughty.
>>>>
>>>> Sadly, it is using that attribute as a source to provision users into
>>>> the actual groups.
>>>
>>> Wild.
>>>
>>> Do you recall Condrey Consulting showing off a product for AD systems
>>> that would convert OU placement into Group membership?
>>> Not an IDM thing, but for AD guys, who need groups not OU placements.
>>>
>>> Similar sort of thing perhaps?
>>>
>>>

>> No, it was an IDM software package (owned by M$ I believe) that uses AD
>> as its data store. These were actually group objects representing users
>> with custom attrs for the actual user attrs. Then that would replicate
>> to an actual user object.
>> <puke>

>
> FIM? ILM? MIIS? original project they bought and destroyed whose name escapes me? (Forefront Identity Manager, Identity
> Lifecycle Manager, Microsoft Identity Integration Server?)
>


No, it is a point solution that is part of a completely different stack. I'll decline to name it but it is an interesting beast.

--
-----------------------------------------------------------------------
Will Schneider
Knowledge Partner http://forums.netiq.com

If you find this post helpful, please click on the star below.
0 Likes
Knowledge Partner
Knowledge Partner

Re: Which DB to use?

On 9/30/2014 5:44 PM, Will Schneider wrote:
> On 9/30/2014 4:28 PM, Geoffrey Carman wrote:
>> On 9/30/2014 5:22 PM, Will Schneider wrote:
>>> On 9/30/2014 4:17 PM, Geoffrey Carman wrote:
>>>> On 9/30/2014 5:16 PM, Will Schneider wrote:
>>>>> On 9/30/2014 4:12 PM, Geoffrey Carman wrote:
>>>>>> The specific app was looking for memberOf equivs, but not getting
>>>>>> what
>>>>>> it needed from AD, so they decided to cheat on AD by
>>>>>> concatting all the group names into a string for the app.
>>>>>>
>>>>>> Naughty.
>>>>>
>>>>> Sadly, it is using that attribute as a source to provision users into
>>>>> the actual groups.
>>>>
>>>> Wild.
>>>>
>>>> Do you recall Condrey Consulting showing off a product for AD systems
>>>> that would convert OU placement into Group membership?
>>>> Not an IDM thing, but for AD guys, who need groups not OU placements.
>>>>
>>>> Similar sort of thing perhaps?
>>>>
>>>>
>>> No, it was an IDM software package (owned by M$ I believe) that uses AD
>>> as its data store. These were actually group objects representing users
>>> with custom attrs for the actual user attrs. Then that would replicate
>>> to an actual user object.
>>> <puke>

>>
>> FIM? ILM? MIIS? original project they bought and destroyed whose name
>> escapes me? (Forefront Identity Manager, Identity
>> Lifecycle Manager, Microsoft Identity Integration Server?)
>>

>
> No, it is a point solution that is part of a completely different
> stack. I'll decline to name it but it is an interesting beast.



Hmm, now you have me wondering. Shall have to ask you privately about that.

But was there anything else useful in the different DB choice?


0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Which DB to use?

On 9/30/2014 4:49 PM, Geoffrey Carman wrote:
> But was there anything else useful in the different DB choice?


Massive performance difference.

Also the side effect is that you can use a myraid of tools to build your own views on the imported data, and then use those
views as a data source. Since the data is static this presents some interesting options for parsing and sorting further subsets.

--
-----------------------------------------------------------------------
Will Schneider
Knowledge Partner http://forums.netiq.com

If you find this post helpful, please click on the star below.
0 Likes
Knowledge Partner
Knowledge Partner

Re: Which DB to use?

On 9/30/2014 5:53 PM, Will Schneider wrote:
> On 9/30/2014 4:49 PM, Geoffrey Carman wrote:
>> But was there anything else useful in the different DB choice?

>
> Massive performance difference.
>
> Also the side effect is that you can use a myraid of tools to build your
> own views on the imported data, and then use those views as a data
> source. Since the data is static this presents some interesting options
> for parsing and sorting further subsets.


Any native tools for the HSQL DB?

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.