Running rsync from a clustered resource

[kat888]

We need to replace 2 NW rsync boxes. Does anyone have experience of running an rsync destination (ie central office) server on a clustered resource? The source servers are all at remote sites. The cluster is 64bit SLES10.3/OES2.3. Presumably rsyncd needs to be bound to the ipaddr of resource?

thanks all

Kat

[kjhurni]

Hi Kat,

I'm running something similar. Now, whether I did it *right* is a diff. story, but it works.

In OUR setup, we have a central office "software repository" that we have setup on a clustered NSS resource. We run rsyncd on the cluster node "hosts". The only downside is that you need to configure rsyncd "identically" on the cluster node that could run said resource, but all I've had to do is copy the rsynd.conf file to the other nodes.

Our regional servers (35+) have a simple shell script that we run with cron. the scripts run the "local" rsync to essentiall "pull" the files like:

rsync --ravz --delete SOMETHING:SOMETHING blah blah blah


(obviously it depends on what you want to accomplish).

We don't sync trustees, although you CAN do this (my friend Darren did a nice writeup on this).

I didn't adjust the cluster load script to stop/start rsync (we just leave it running on the nodes that can host the resource) since the rsyncd.conf file points to the clustered resource. rsyncd.conf I think controls which IP it binds to or something (I think)

If you need more info, I'll see if I can "scrub" my rsynd.conf file and post it.

[kat888]

kjhurni;2168791 wrote:
If you need more info, I'll see if I can "scrub" my rsynd.conf file and post it.

Thanks. The more information the merrier IMO. I'd really appreciate it if you would do that.

[kjhurni]

Here's a snippet. I didn't put the entire list of things we set up. Yeah, this may not be the most "secure" (no ssh, etc. we allow any hosts) but this is our own software that users can install anyway, we just put it on their "local" servers.



uid = root
#gid = users
gid = root

#read only = true
read only = false
use chroot = true
transfer logging = true
log format = %h %o %f %l %b
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
#hosts allow = trusted.hosts
slp refresh = 300

#[Example]
# path = /home/Example
# comment = An Example
# auth users = user
# secrets file = /etc/rsyncd.secrets


[DEPLOY-ARCGIS]
path = /media/nss/DATA1/deploy/ArcGis/
comment = CO deploy arcgis
read only = no
use chroot = no
timeout = 3600
transfer logging = yes

[kat888]

My question related more to specifying the --config for the virtual resource

rsync --address=10.10.10.10 --config=xxx.xxx.xxx.xxx:RSYNC/SYS/rsyncd.conf port=873 --daemon

or

rsync --address=10.10.10.10 --config=TESTCLUSTER_RSYNC_SERVER:RSYNC/SYS/rsyncd.conf port=873 --daemon


have tried a few syntax permutations of the SERVER/VOL/DIR and it doesn't like it.

[kjhurni]

Oh, okay.

So, let's say you have this setup:
SERVER1 = 10.10.1.10

and that's the server running rsyncd with that conf file I posted.
But let's say that ON that server, you have a clustered resource of:
RESOURCE1=10.10.1.15

Okay, so you have your "regional/remote" server

It would execute something like:

rsync -ravz --delete RESOURCE1::DEPLOY-VSE85i /media/nss/APPS1/deploy/vse85i

at least that's how I did it.

rsyncd on the "cluster" node (based upon MY .conf file) should listen/bind on all the IP's on that server (the physical node and the clustered resource).

does this help?

[kjhurni]

IF you send me a PM with your email, I'll email you the exact screenshots I did on the cluster/host node to you.

I don't think I put this article on the Cool Solutions/communities site.

[markusernst]

Hello kjhurni

Have seen, that you are working with ArcGis, too. At our company, the users like to work local on their W7 workstations, because it's slow to work directly on the network drive due of the huge data needs transferred.
They like to have synchronisation solution, which is multi-user compatible. Do you have a solution for that ? Would you share this knowledge ?

Thanks and regards
Markus

kjhurni;2169150 wrote:
Here's a snippet. I didn't put the entire list of things we set up. Yeah, this may not be the most "secure" (no ssh, etc. we allow any hosts) but this is our own software that users can install anyway, we just put it on their "local" servers.


[DEPLOY-ARCGIS]
path = /media/nss/DATA1/deploy/ArcGis/
comment = CO deploy arcgis
read only = no
use chroot = no
timeout = 3600
transfer logging = yes

[kjhurni]

Yes, we use nasty ESRI stuff.

We have two solutions in place. Our regional offices use Citrix via a Published ArcGIS application (the Citrix servers are in our main office). The ArcGIS data is stored on our Sun Solaris servers via SAMBA shares. That seems to provide adequate performance (I don't use the product, so I cannot tell you what "fast" means). Our users in the main office do not use Citrix, they simply map a drive to the SAMBA share and run the ArcGIS from their desktop and their "workspace" is on the SAMBA share.

If you send me a PM, I would be more than happy to forward your email to our GIS folks who would be more than happy to talk "turkey" with you. We don't have Windows 7 here yet, but I don't think that would affect performance much.

[markusernst]

kjhurni;2169884 wrote:
Yes, we use nasty ESRI stuff.


If you send me a PM, I would be more than happy to forward your email to our GIS folks who would be more than happy to talk "turkey" with you.

Hello

Here is my yahoo-address. markusernst#at#yahoo.com. I'll reply with my company email address to your email.

Thanks
Markus