Filr constantly crashing

Large install, appliance specs below, won't run for more than a day or 2. I am pointing Filr at 4 SLES10/OES2 servers to sync and index about 3TB of data. Mostly PDF, Word, Excel documents. I have less than 100 users all provisions form LDAP sources (same SLES/OES servers). I have a rather vanilla file server/mapped drive setup with every user assigned a G drive that points to a "Groups" directory. In Filr, I have a netfolder for each Groups directory on each server which brings the total NetFolder count to 4. The sync schedule is set to kick of, at the NetFolder Server level, every other day in middle of the night. I am also asking the NetFolders to be indexed, which is the hwole point we're looking at Filr.

After initial setup of the large install everything is great for 24-48 hours. However, it will eventually fail when the '/' of the Filr Search appliance gets filled up with a 'core.dmp' file. The Filr appliance is suffering from a similar problem in that /vastorage/filr gets full but I cannot find the exact culprit just yet.

Filr Appliance
4 vCPU
12GB RAM
128GB Secondary HD

Filr DB Appliance
4 vCPU
8GB RAM
256GB Secondary HD

Filr Search Appliance
4 vCPU
12GB RAM
65GB Secondary HD
  • >>> jwhitson<jwhitson@no-mx.forums.novell.com> schrieb am 05.06.2013 um
    20:36
    in Nachricht <jwhitson.5wh621@no-mx.forums.novell.com>:

    > After initial setup of the large install everything is great for 24-48
    > hours. However, it will eventually fail when the '/' of the Filr Search
    > appliance gets filled up with a 'core.dmp' file. The Filr appliance is
    > suffering from a similar problem in that /vastorage/filr gets full but I
    > cannot find the exact culprit just yet.


    From my past experiences of full-text indexing of vast amounts of data
    (>10TB) i can only report that this is indeed quite fragile.

    * It takes quite some time.
    * There are always files which will cause the indexing to crash (especially
    office stuff, pdf is a little better)


    I did the following:

    Split the indexing jobs. Smaller, easier to handle (restarting, etc.)
    Cleanup tmp, log and core-dumps with cron jobs and collect statistics (by
    mail, sql, etc.) about the 'housekeeping'. This will allow you to identfy
    problem areas.

    As Vibe/Filr is just using lucene i do not see a benefit in asking Novell to
    much about indexing crashes, i would concentrate on stuff specifc to Filr
    (like famtd). I get the impression Novell is already looking into the many
    issues missed or ignored during beta testing.

    For your use-case i would not rely on JITS. If searching (and FINDING) stuff
    is important, JITS is a no-go.


    Georg


    > 65GB Secondary HD


    Why the extra GB? :-) SCNR

    Georg

  • >>> jwhitson<jwhitson@no-mx.forums.novell.com> schrieb am 05.06.2013 um
    20:36
    in Nachricht <jwhitson.5wh621@no-mx.forums.novell.com>:

    > After initial setup of the large install everything is great for 24-48
    > hours. However, it will eventually fail when the '/' of the Filr Search
    > appliance gets filled up with a 'core.dmp' file. The Filr appliance is
    > suffering from a similar problem in that /vastorage/filr gets full but I
    > cannot find the exact culprit just yet.


    From my past experiences of full-text indexing of vast amounts of data
    (>10TB) i can only report that this is indeed quite fragile.

    * It takes quite some time.
    * There are always files which will cause the indexing to crash (especially
    office stuff, pdf is a little better)


    I did the following:

    Split the indexing jobs. Smaller, easier to handle (restarting, etc.)
    Cleanup tmp, log and core-dumps with cron jobs and collect statistics (by
    mail, sql, etc.) about the 'housekeeping'. This will allow you to identfy
    problem areas.

    As Vibe/Filr is just using lucene i do not see a benefit in asking Novell to
    much about indexing crashes, i would concentrate on stuff specifc to Filr
    (like famtd). I get the impression Novell is already looking into the many
    issues missed or ignored during beta testing.

    For your use-case i would not rely on JITS. If searching (and FINDING) stuff
    is important, JITS is a no-go.


    Georg


    > 65GB Secondary HD


    Why the extra GB? :-) SCNR

    Georg

  • >>> jwhitson<jwhitson@no-mx.forums.novell.com> schrieb am 05.06.2013 um
    20:36
    in Nachricht <jwhitson.5wh621@no-mx.forums.novell.com>:

    > After initial setup of the large install everything is great for 24-48
    > hours. However, it will eventually fail when the '/' of the Filr Search
    > appliance gets filled up with a 'core.dmp' file. The Filr appliance is
    > suffering from a similar problem in that /vastorage/filr gets full but I
    > cannot find the exact culprit just yet.


    From my past experiences of full-text indexing of vast amounts of data
    (>10TB) i can only report that this is indeed quite fragile.

    * It takes quite some time.
    * There are always files which will cause the indexing to crash (especially
    office stuff, pdf is a little better)


    I did the following:

    Split the indexing jobs. Smaller, easier to handle (restarting, etc.)
    Cleanup tmp, log and core-dumps with cron jobs and collect statistics (by
    mail, sql, etc.) about the 'housekeeping'. This will allow you to identfy
    problem areas.

    As Vibe/Filr is just using lucene i do not see a benefit in asking Novell to
    much about indexing crashes, i would concentrate on stuff specifc to Filr
    (like famtd). I get the impression Novell is already looking into the many
    issues missed or ignored during beta testing.

    For your use-case i would not rely on JITS. If searching (and FINDING) stuff
    is important, JITS is a no-go.


    Georg


    > 65GB Secondary HD


    Why the extra GB? :-) SCNR

    Georg

  • Hi Georg,

    Thanks for the reply. I think your suggestions will be very useful.

    As for the 65GB, typo.

    Jason