Information researched, written and provided by Dan Christensen - Senior Software Engineer

Acronyms

• SDD: Software Distribution Directory

• C/S: Client/Server protocol used by GroupWise client to talk to POA usually over port 1677.

Motivation

Software distribution directories are remnants of times past when the primary method of network access was via shared volumes on a file server. In this world it made sense to have a single repository (or central repositories per geographic location) for software to be installed and referenced by users, admins and components of the GroupWise system.

Today, the world has faster network access and more local hard drive space. In this world it is much more common to download files directly to a local drive from a web server rather than access files from a shared network drive.

The following are the reasons engineering would like to eliminate the use of software distribution directories from GroupWise.

1. Unnecessary complexity when installing a new system. Before installing a new GroupWise system it requires the administrator to create a new software area and copy all of the install files to that area. The administrator also needs to ensure that the area that they installed the files to is shared on the network for certain admin and client features to work properly. Why is the administrator required to copy the setup files somewhere else when the files were already downloaded locally to run the install in the first place?

• Unnecessary complexity when updating an existing system. After installing a new version of the GroupWise agent and client software, an administrator may try to login with the new client and be greeted with an error message that the views are not installed properly for this version of the client. In order for a new client to access a given post office, the administrator must update the files in the software distribution directory and then update the view files on the post office as well. If certain files are required in order for the software to run properly why weren't they installed when the software was updated?

• Difficult to manage with a web based admin console. Updating a network software distribution directory with files downloaded on the admins local hard drive via a web browser is unfriendly and difficult. There are hundreds of megabytes that are included in a GroupWise installation and uploading these via a browser is not feasible. To even support backend file copies, the admin would have to setup special permissions on the network share to allow writes from the computer running the admin service while granting only read-only access to everyone else. Of course, this assumes that the software distribution directory is network accessible at all. On Linux, additional configuration is required to make a SDD consumable both by the agents and as a network share.

• No longer used as a central installation repository. Engineering assumes and believes that the SDD are used as a place for users or administrators to go to install software. Users will generally download software from a web browser rather than a network share. Admins will download from a web browser and/or use utilities like SCP to copy software around to servers where they need to be installed. In Linux environments, accessing software from a UNC path is possible but generally more trouble than it is worth.

• No longer an integral part of the GroupWise architecture. There are not very many GroupWise components that actually consume the software distribution directory. In fact, the primary remaining consumer is the windows client... and it only uses it in a limited fashion. All of the other files stored in the software distribution directory are largely wasted space.

Current Usage

The software distribution directory is consumed by three components/features in GroupWise:

1. Admin - Admin references the *.dc (and other supporting) files when creating a new system, domain, post office or gateway.

• Client Views - Post offices update their local cache of view files by copying them from the software distribution directory.

• Client Update - The client auto-update process currently references the software distribution directory to retrieve new installation components. This is by far the most complicated usage of the software distribution directory.

Admin

For Admin, engineering will make the following change. Install the *.dc files (and all other supporting files) alongside the admin components. This allows an admin install to be self-sufficient and include all of the files that it needs to perform its job. This choice has schedule/effort benefits.

Client Views

For Client views, engineering will install the client view files as part of the agent install. This will create an "ofviews" directory relative to the installation path of the agents (i.e. /opt/novell/groupwise/agents/ofviews). The engine code will need to be modified to retrieve view files from the relative path instead of the ofviews directory that is relative to the post office. Or, alternatively the engine could be modified to composite the two directory locations (i.e. try to retrieve from the post office location and fall back to the relative location). This would account for scenarios where the views had been customized by the administrator on the post office. Question: Are you using custom views?

Client Update

This is by far the most complicated use of the software distribution directory and as such requires a little bit of background on how the process currently works.

Key Files Involved

• SDD\client\update32.dll - Performs update of client by UNC access to the SDD

• SDD\client\gwipupdt.dll - Performs update of client by Client/Server access to SDD in conjunction with setupip.exe

• SDD\client\setup.cfg - INI file containing a section with auto-update settings

• SDD\client\software.inf - INF file containing the current version of the software

• SDD\client\win32\* - Contains all of the client installation files required to launch an installation over a UNC path.

• SDD\client\setupip.exe - SetupIP installation stub launches installation and downloads components from pre-configured website URL.

How it works now

1. The client update process begins with the administrator updating the software distribution directory with the new client files. Most importantly this includes the client\software.inf file which contains the build number of the software installed to the SDD. The client will use this build number to determine if the software in the SDD is newer than what it has installed.

• The administrator must increment the "bump" number on the software area record in the domain database. This is done from ConsoleOne in the System\Software Directory Management by clicking on the "Update" button.

• Every time that the client logs in, the POA gets the GUID of the software area associated with the post office that the user belongs to. The GUID is used to lookup the software area record from which the UNC path and bump number are retrieved. Both of these fields are returned by the server to the client.

• The client compares the bump number to a version in the windows registry. This registry contains the last number that the client processed to trigger an auto-upate check. If the bump number is newer than the number in the registry then the client will begin its auto-update check. Note that this only triggers a check for newer software and will only trigger an actual install if the software in the SDD is newer than what is installed locally.

• The client will take the UNC path returned from the server and attempt to load "update32.dll" from that path. This requires the client to have direct network access to the server hosting the SDD shared directory. If the client is able to access and load this DLL then it calls into the DLL to perform and update check and possibly and update:
  
  
  
  1. The routine in update32.dll will attempt to load the "setup.cfg" and "software.inf" directly from the UNC path. Note that on versions of windows newer than Windows XP that Windows prohibits loading INI/INF files from a network drive so the files must first be copied to a local temp directory before they are accessed.
  
  
• The build number of the software in the SDD is read from the software.inf and compared to the build number that was written to the registry when the client was installed. If the SDD software is newer, the update check proceeds, otherwise it returns without taking any further action.

• The auto-update settings are read from the setup.cfg to determine if the system is configured for auto-update, whether it is forced, and whether or not the user is allowed grace logins. Depending on the auto-update parameters the user may be prompted to update or an update may be forced.

• Finally, setup.exe is launched via UNC from the SDD\client\win32 directory. Note that on most versions of windows this will at best trigger a security warning, at worst will fail to launch due to settings regarding launching executable from a network drive.

• If the client fails to load and process "update32.dll", then it moves on to a second attempt to update over TCP/IP. The client will use the C/S protocol to retrieve the file "gwipupdt.dll" from the SDD into a temporary folder. Once downloaded the client loads the gwipupdt.dll and calls a function in the DLL to continue the update process.
  
  
  
  1. The update function uses a callback into the hosting client process to download additional files from the software distribution directory. These files are software.inf and setup.cfg.
  
  
• The same processing occurs as described in the UNC update path to process the software.inf and setup.cfg to determine if an update needs to occur.

• If an update needs to occur, then gwipupdt.dll uses the C/S protocol to download the setupip.exe stub from the SDD. Note that the admin must have generated the setupip.exe and placed it into the SDD in a separate step for this to work.

• Once launched setupip.exe will attach to one of five web servers specified by the administrator to download the client installation packages and start the install.

SetupIP troubles

Novell does not actually ship the file "setupip.exe". This is generated by an administrator at the customer site by running the WriteIP.exe utility. The write ip utility allows the administrator to define up to five URL's where the client installation can be downloaded. It takes these addresses and writes modifies a template executable to generate the actual setupip.exe. Not only is this an additional step for the administrator but the process itself is problematic. CopyIP.exe and SetupIp.exe are frequently flagged as viruses by many anti-virus products. This is because they are:

1. Writing to an executable file.

• Running code that connects to internet, downloads executable files and then runs them.

• Incapable of being added to an exclusion list because the dynamic nature of an auto-generated executable provides a constantly change hash or signature.

• Incapable of being signed by Novell since the executable is modified after it ships.

Engineering will make all of the following changes.

Change #1: Remove UNC Auto-Update

In order to eliminate the dependence on the software distribution directory, engineering must eliminate the portion of the auto-update process that attempts to access the SDD directly. This means getting rid of "update32.dll". Since the auto-update process automatically falls back to the gwipupdt.dll, the update32.dll portion of the auto-update is entirely redundant. Also, since the update32 portion of the update counts on direct access to a shared network drive and the ability to run software from the network share which is the less reliable of the two processes.

Change #2: Install Auto-Update Files Relative to Agent

Move the necessary auto-update files out of the software distribution directory and into a path relative to the agent (i.e. /opt/novell/groupwise/agents/client-update). One example, the administrator would only need to lay down these files if they were interested in supporting the auto-update functionality of the client. That would imply that these would be a separate *.rpm or *.msi file.

Minimally, this file set would include:

• gwipupdt.dll (and supporting language DLL's)

• software.inf

• setup.cfg (more about this later)

• setupip.exe and/or setupip.* (more about this later)

Note that this file set no longer includes the uncompressed client/win32/* set of files that comprise the normal client install image. This means that much less total space is required to support auto-update.

In order to support loading files out of this directory the engine code would have to be modified to look in this directory for software files instead of the software distribution directory.

Change #3: Move setup.cfg Auto-Update settings into Admin Console

In order to prevent the administrator from having to edit the auto-update settings in the setup.cfg that is installed with each agent (as per Proposal #2), The setup.cfg file will be eliminated altogether and the auto-update settings will move into the domain database where they can be modified through the admin console.

The auto-update settings include the following 4 settings:

1. Enabled: Yes|No

• ForceUpdate: Yes|No

• GraceLoginCount: 0-n

• PromptUntilUpdated: Yes|No

These settings will be added into the domain database and applied to System, Domain and Post Office records. In order to maintain backwards compatibility, these settings will be serialized into a setup.cfg file on demand when the setup.cfg file is requested from the server by the client.

Change #4: Support Update over IP without SetupIp.exe stub

The primary purpose of the setupip.exe stub is to allow administrators to define a location from which the GroupWise client installation can be downloaded. In the case of auto-update, this is not necessary. The connection to a valid repository for the client update files (i.e. the post office) has already been established. Therefore, engineering will modify gwipupdt.dll to directly download the compressed setup files and launch the install rather than downloading setupip.exe. This eliminates the need for the admin to generate a setupip.exe stub and specify an additional web server to host the setup files on.

In order to download the setup files from the POA, engineering will either use the C/S protocol to download the files from the auto-update directory (defined in Proposal #1) or download the files from the POA over HTTP. Note that downloading the files over HTTP is likely to be more efficient but would require HTTP access to the POA and changes to the POA to return the setup files when requested.

Change #5: Add admin setting for "Auto-Update URL"

Thus far, the assumption has been that every post office will host the auto-update files (or not support auto-update). Some administrators may prefer to host a single auto-update repository rather than installing the auto-update files at each post office. Engineering will add a new "Auto-Update URL" admin setting that allows the admin to define a location from which the setup files can be downloaded. This setting would be available on the system, domain and post office objects.

Per change #3, this new admin setting would get generated into the virtual setup.cfg that is returned to gwipupdt.dll. gwipupdt.dll would then look for this setting and if present, attempt to download the software from the specified URL. The update process would first attempt to download the software.inf to check the version of the software installed at the given URL and then would download the individual files.

The gwipupdt.dll update process would then look something like this:

• Download gwipupdt.dll from server using C/S protocol.

• Download (virtual) setup.cfg from server using C/S protocol.

• Check for AutoUpdateURL
  
  
  
  • If present, download software.inf from HTTP server
  
  
• If not present, download software.inf form server using C/S protocol.

• Check if update necessary and/or prompt user

• Download setup via HTTP or C/S depending on whether or not the AutoUpdateURL was set.

• Run setup.

Deprecate Software Directories in Admin

Once the dependency on software distribution directories is eliminated, engineering will remove support for certain operations in the admin console. Here are the following changes:

1. Continue to show existing software areas, but don't allow the admin to create any new software areas.

• Eliminate the requirement to create a software area before installing a new system.

• Eliminate the requirement to specify a software area when creating a new post office.

• Remove the functionality to "update" the files in a software distribution directory (this can still be done manually).

• Remove the functionality to submit a system maintenance task to update the views on a post office.

• Move the functionality of "bumping" the software directory from the software area to the post office.

These are all of the changes that will be made to facilitate this overall functionality change in the product. These changes will be part of Windermere.