ZENworks Patch Management - Baselines and testing

ZENworks Patch Management (ZPM) is a relatively easy product to architect and deploy, the main challenges are around best practices for using the product day to day. Questions that need to be answered include:

What is the best way to schedule my patch operations?
What is the best way to handle patches that require reboots?
What business processes should be in place to ensure effective patching?

Mandatory Baselines
Well, let's start with Mandatory Baselines. This functionality allows you to define a patch level, or list of vulnerabilities, that must be met by the targeted devices. ZPM automatically ensures that devices are patched to the desired level. I tend to avoid using Mandatory Baselines for production desktops as there is no easy way of scheduling when patching occurs. For example, if you install a product that back-revs a DLL and enables a previously patched vulnerability, on the next check-in the device could be patched again.

One approach is to use the "hours of operation" setting to prevent patching from occurring during normal working hours, this approach does minimise the impact on users but does have some drawbacks. Firstly, desktops and laptops are often not available outside of normal working hours, a lot of customers I visit encourage users to shutdown at the end of the day due to reduce energy consumption. More importantly, if I prevent patching during normal working hours, how do I deploy a patch in the event of a destructive virus?

Mandatory Baselines are useful in scenarios where you want to manage a standard build. Let's assume for a moment that you are using a standard desktop image that is deployed to multiple machines. Adding your source machine for the build to a mandatory baseline will allow the device to be patched automatically before you seal the image for distribution. In this scenario, availability of the device and the requirement for rebooting is not an issue.

Do you use mandatory baselines? What approach do you take?

Testing and documentation
As with any software delivery mechanism, testing and release management is critical to minimising the impact on your business. A summary for a typical approach to managing patch delivery is as follows:

  1. Identify new vulnerability

  • Analyse risk and exposure to the enterprise

  • Create deployment plan

  • Test deployment

  • Phase deployment to the enterprise

Each of these stages are normally documented to provide an audit trail.

Often I see customers deploying to test machines, then to the IT department and then phasing the deployment to the rest of the enterprise. What's your approach to testing and deploying patches?


How To-Best Practice
Comment List
  • "I tend to avoid using Mandatory Baselines for production desktops as there is no easy way of scheduling when patching occurs."

    I'm sorry, but this... bogglesome... lack of support for the #1 use case of patching is exactly why we stopped using ZPM and switched straight to WSUS. And since we went to WSUS, we haven't had any problems.

    Seriously, we tried to make ZPM do the simplest of our requirements, but it was a farce. Pushing out the monthly Microsoft service packs for Windows/Office/IE to all our desktop clients is exactly what we need, and exactly what ZPM refuses to do.

    I really, really tried to understand why ZPM's approach was "you only use Mandatory Baselines to update major service packs... which you will generally be updating on your regular image build and pushing out by Zen Imaging anyway, thus making ZPM irrelevant... for all the urgent stuff, you must do it manually". But I couldn't get my head around that kind of logic. Creating all those manual deployment jobs for every iteration of monthly MS updates turned into a full-time job, *plus*, it didn't provide any security whenever an unpatched desktop turned up on our campus. Which happens all the time, since we only update our images on a quarterly-to-yearly cycle, but our helpdesk's first response to software trouble is to restore any broken desktop to the current image, which of course will be several month's worth of patches behind.

    "Often I see customers deploying to test machines, then to the IT department and then phasing the deployment to the rest of the enterprise. What’s your approach to testing and deploying patches?"

    Yep, that's exactly what we do, which is why ZPM's lack of support for this use case boggles me. Day after Patch Tuesday, we have the latest monthly crop of MS fixes auto-installing on a tiny group of IT staff machines (one room full). A couple of days later, we push them to all our IT machines. A week later, they go to our beta test group (a couple of classroom suites and representative staff machines). A week later, it goes to a single building, finally to the entire campus.

    With WSUS, this is pretty easy: each week I just go through and check the logs for failed installs, and if all is well I go through each outstanding patch and tick the boxes to install to the next group of machines. An AutoIt3 script (Zen force run at daily startup) on each machine detects what group it's in based on room, building, machine time, and sets the WSUS group appropriately.

    And WSUS is *perfectly* capable, unlike ZPM, of scheduling patches to apply daily, at the hour of your choice, so we make it 6am. And it Just Works.

    Please, fix ZPM so it makes Mandatory Baselines as pain-free to use as WSUS!