AccuRev Keep - Your Developers Moving Forward! AccuRev Admin’s Real Life Experiences // Backups of Replicas

Being a former AccuRev Admin for an AccuRev Master Server and a few AccuRev Replicas, one thing I coordinated was regular backups on all these machines. The process was simple, run “accurev backup _someFileName_” and once that was finished IT would then make a physical backup of the machine’s file system. This was all automated and worked just as expected.

 

HOWEVER, there was an AccuRev Replica in a remote location that was not under my control. The local IT people did not make regular backups of the AccuRev Replica as they knew the AccuRev Master Server was being backed up. As fate would have it, they had a disk failure. Compounding the problem, they never checked their backups to see if they were restorable, which they weren’t. Two lessoned learned so far: 1) Make regular backups of your AccuRev Replicas. 2) Test the backup by restoring it on another machine.

 

Now if the AccuRev Replica was on a fast network link along with the AccuRev Master Server, rebuilding it from scratch wouldn’t have been too bad. However the remote location was on a network link just faster than carrier pigeons and it took over a week to get all the data transferred and the AccuRev Replica fully functional from a developer’s perspective.

 

Following are the restoration actions taken once the disk was replaced: 1) install OS, 2) Ensure that NTP (http://www.ntp.org/)was working, 3) Ensure AccuRev Replica’s time (UTC) was the same as the AccuRev Master Server’s time (UTC).

 

At this point the rebuilding of the AccuRev Replica could commence. First thing was to install the AccuRev server software, configure the acserver.cnf file with the correct entries for MASTER_SERVER, LOCAL_SERVER, PORT and LOCAL_PORT. Then start up the AccuRev Replica.

 

With AccuRev now running on the AccuRev Replica we followed the steps from the AccuRev® Administrator’s Guide in the section  Replication of the AccuRev Repository:

  1. accurev login –n –H AccuRevMasterServerName:5050 acreplica
  2. accurev replica sync
  3. accurev login –n acreplica
  4. accurev mkdepot –p DepotName
  5. accurev replica sync
  6. Go To step 4 until the needed Depots were restored
  7. Re-enabled the crontab(1) entries that were configured to keep a set of known “active development” Dynamic Streams current. This was accomplished by having workspaces off the streams and via cron(8) execute a script that ran an “accurev update” within the workspaces. The “accurev update” will pull over the files from the associated Dynamic Stream from the AccuRev Master Server to the AccuRev Replica.

 

So this begs the question, “Does it matter when I make my backup of the AccuRev Replica?” and the answer is: “Yes.”

 

Backups on an AccuRev Replica must come before that of the AccuRev Master Server and here’s why. In the event where the AccuRev Master Server has to be restored from backup then all AccuRev Replicas will have to also be restored as their content would be in the future.

 

“Why you ask?” A simple example and answer is in order. The AccuRev Replica’s Meta data, stored in its database would be ahead in time with respect to the now restored AccuRev Master Server. Here’s a Transaction based time line of AccuRev Commands.

 

Transaction Id

AccuRev Principal

AccuRev Command

191

Frank

Keep

192

Jack

Promote

193

Brad

Add

194

ITadmin

accurev backup filename

195

Brad

Keep

196

Jack

Keep

197

Julie

Keep

 

At Transaction Id 198 the AccuRev Replica does an “accurev replica sync” so it now has the Meta data up to and including Transaction 197.

 

Some point in the future the AccuRev Master Server has to be restored. It will be restored to the point in time of the last “accurev backup filename” command which is Transaction Id # 194.  Now the AccuRev Replica is in the future with respect to the AccuRev Master Server, and no you can’t wait for it to catch up as all events post restore will not be in the same order. After the restore and the AccuRev Master Server is once again running the transaction set now looks like:

 

Transaction Id

AccuRev Principal

AccuRev Command

191

Frank

Keep

192

Jack

Promote

193

Brad

Add

194

ITadmin

accurev backup filename

195

Joe

Purge

196

Rich

Add

197

Jim

Promote

 

The AccuRev Replicas Transactions Id 195, 196 and 197 are now wrong with respect to the AccuRev Master Server, predicating that they must now be restored.

 

Additional Lessons Learned: 1) Backup your AccuRev Replicas prior to backing up your AccuRev Master Server. 2) If the AccuRev Replica is on a slow link having good backups is even more important than if the AccuRev Replica is on a fast link due to the amount of data that may have to be transferred and the time that will consume.