eDir Transaction ID Finder

1 Likes
over 12 years ago

Avoiding eDirectory Meltdowns With the 'eDirectory Transaction ID Finder



This program finds Transaction IDs embedded in eDirectory 8.7.3.x / 8.8x roll-forward log files (as found in the nds.rfl directory).



If repairs are not run regularly on your eDirectory servers, it is important to know the range of transaction IDs that are currently being issued on each server, so that you can take the necessary preventative action when required.



eDirectory is based on the FLAIM database engine, which has a maximum transaction value of FFFFE000 (hex) and as such it is possible to 'run out' of transaction IDs if a repair is not run (which will reduce the last transaction ID counter).



If a server does run out of transaction IDs, only Novell Technical Services has the ability to correct the problem.



See: eDirectory "Transaction ID" Warning / error -618 database stops working and won't open.
http://www.novell.com/support/viewContent.do?externalId=7002658&sliceId=1



By using this program and comparing results, it is also possible to approximate how long it will be before a particular server runs out of transaction IDs, allowing you to plan accordingly if you have a 24x7 eDirectory server.



Running The Program



The program uses a copy of the roll-forward log from any 8.7.3.x or 8.8x database.



The roll-forward log files are normally located in:




  • NetWare - SYS:\_netware\nds.rfl

  • Linux - /var/opt/novell/eDirectory/data/nds.rfl

  • Windows - C:\Novell\NDS\DIBFiles\nds.rfl



In most cases, the roll-forward log will be called 00000001.LOG and should have a fairly recent timestamp.



In some cases, there may be no log files, the log file may be only 512 bytes in size (no transactions) or there may be multiple log files.



Generally, only one log file is in use at any time by one eDirectory instance.



There are multiple ways to identify the current roll-forward log file. If there is only one log file in the nds.rfl directory then this is the file 'in use'. If there are multiple files, identifying the file with the latest time stamp should indicate which file is being used. The dsbk command can also be used:




  • On NetWare: dsbk getconfig

    Depending on the eDir version, the results are displayed on the logger screen or on a separate 'dsbk' console screen.


  • On Linux: dsbk getconfig (if previously configured - if no /etc/dsbk.conf, will not work)

    The output from the command is sent to the default log file (ndsd.log).


  • On Windows: From control panel, run the 'Novell eDirectory services' applet. Highlight the dsbk.dlm line, enter the word 'getconfig' in the 'Startup Parameters' box, and then click on start. Nothing will appear to happen, but the config info should have been written to the output log file.


    The output is written to C:\Novell\NDS\backup.out



Via iManager - eDirectory Maintenance - Backup Configuration options.



If there are no log files, you may need to enable roll-forward logging before you can make use of this program (see eDirectory documentation).



NOTE: In most cases roll-forward log files appear to be created and updated even if roll forward logging is set to OFF.


NOTE: Although it can be OK to delete log files which are no longer used, do not do this unless you need the space. Be sure you do not delete or modify the 'current' roll-forward log file as it could have severe consequences.


Program Requirements:



The program has been tested on SLED10 and Windows XP, but should run on any OS that supports Perl 5.5.8 or later.



The program was tested with Perl 5.5.8 on Linux and Activeperl 5.10.1 on MS Windows.



Installation:



Make a new directory and place the roll.pl and allroll.pl programs in the directory. Acquire a copy of the the roll-forward log file from the server to be checked and place in the directory with the perl programs.



Execute the program using: perl roll.pl



Enter the name of the log file to be processed (case sensitive on Linux/Unix).




NOTES:

The program allroll.pl is included as a diagnostic utility which will extract ALL transaction IDs from a roll-forward log file regardless of age. The roll.pl program only references what it has identified as recent transactions and will not include transactions considered as 'old' data. Log files are actually re-used from the start, so the ordering of transaction IDs can appear to be inconsistent when viewing transactions from the whole file. The output file for the allroll.pl program is called 'All_eDir_Transactions.txt'.



When running dsrepair/ndsrepair in order to reduce the current transaction ID counter, please make sure you ONLY set the 2 options to 'yes' as indicated in TID#7002658. If any other combination of yes/no switches is used, the transaction ID will probably not be reduced.




Example Program Run / Output:



Name of roll-forward log file [Default=00000001.log]: 00000001.LOG

Processing!


Last Transaction ID identified: Hex [6392b4e5] : HexMax [FFFFe000]
Dec [1670558949] : DecMax [4294959104]

Number of Transactions left : Dec [2624400155] 61.10% remaining

Labels:

Collateral
Comment List
Anonymous
Parents
  • Ok, so Novell's stance seems to be to only run dsrepair if you're having a problem. Yet, with this transaction ID limit, you would think that periodic dsrepairs would be recommended by Novell (?).
  • Well, running dsrepair on a regular basis is the choice of some and used to be recommended by many in Novell. However, the eDirectory code is a lot better these days that it was years ago, so that recommendation isn't herd much these days.

    The fact that no-one really knew about the FLAIM transaction limit (except the developers) coupled with the fact that customers generally ran dsrepair 'at the drop of a hat', meant it was never an issue, as no one every came close to the 4.2bn limit.

    My own personal recommendation about running dsrepair has always been 'only run it if there's a reason to do so'.

    Well, now we know there is a reason to run dsrepair. How often? I'd say it depends on how 'busy' the server is as regards it's eDirectory database. In other words, just how many changes are being made to the database on that server over a given period?

    The amount of changes being made to a database will depend on a number of factors which will include:-

    1. Size of the eDirectory database (in terms of the number of objects held on any one server).
    2. Number of services/applications running on that server that 'touch' or make changes to objects in the database.
    3. Number of users authenticating to that server or other servers that have replicas containing copies of user objects in this database etc etc

    At a guess, I'd say running a repair once every 3 months (even on a busy server with a big database) should be more than adequate to avoid any risk of running out of transaction IDs.

    See this cool solution on dsrepair usage:-
    www.novell.com/.../15312.html
Comment
  • Well, running dsrepair on a regular basis is the choice of some and used to be recommended by many in Novell. However, the eDirectory code is a lot better these days that it was years ago, so that recommendation isn't herd much these days.

    The fact that no-one really knew about the FLAIM transaction limit (except the developers) coupled with the fact that customers generally ran dsrepair 'at the drop of a hat', meant it was never an issue, as no one every came close to the 4.2bn limit.

    My own personal recommendation about running dsrepair has always been 'only run it if there's a reason to do so'.

    Well, now we know there is a reason to run dsrepair. How often? I'd say it depends on how 'busy' the server is as regards it's eDirectory database. In other words, just how many changes are being made to the database on that server over a given period?

    The amount of changes being made to a database will depend on a number of factors which will include:-

    1. Size of the eDirectory database (in terms of the number of objects held on any one server).
    2. Number of services/applications running on that server that 'touch' or make changes to objects in the database.
    3. Number of users authenticating to that server or other servers that have replicas containing copies of user objects in this database etc etc

    At a guess, I'd say running a repair once every 3 months (even on a busy server with a big database) should be more than adequate to avoid any risk of running out of transaction IDs.

    See this cool solution on dsrepair usage:-
    www.novell.com/.../15312.html
Children
No Data
Related Discussions
Recommended