Patching Windows with ZENworks: Part 2
Novell Cool Solutions: Feature
By David Lange
Digg This -
Updated: 3 Nov 2005
David responded to an OPEN CALL on this topic with such a well-detailed response, we thought it needed to be placed in a separate article. To read all the other suggestions on this topic, see this article.
I'm glad someone asked this question. I've done it so many times that I feel I could write a book on the subject. Perhaps I'll do that while I relax on the Hawaii cruise I'm earning with all my Novell Rewards points!
I've used ZENworks to push each and every MS patch to over 2000 workstations at 32 connected sites since becoming our ZENworks guy a couple of years ago. I know Novell recently purchased and is integrating a patch management product and I'm sure it works very well, but if you're already paying for ZENworks, it's difficult to justify paying again for a product that does almost exactly the same thing. (One of the Cool Guys will insert an advertisement box here to say ZPM is a very good product and you should check it out. And they're right, you should - if you don't already have or know how to use ZENworks for Desktops to do the same thing.) [Editor's Note: ZENworks Patch Management allows you to quickly determine what patches are right for your organization and automatically apply them across the enterprise. Available for Windows, NetWare, Macintosh, AIX, Solaris HP-UX and more, ZENworks Patch Management closes security holes before hacking, compliance or infringement issues jeopardize your infrastructure. It is a great product, and you should check it out.]
Let me preface this by saying that deploying patches with ZENworks is not a simple process. You don't just click on a button to deploy. However, it is a process and it is repeatable with predictable results. I currently have over 80 ZENworks applications in my tree devoted to patching Win2k and WinXP workstations. Since I delete the obsolete applications as Microsoft replaces them, I'd estimate that I've built over 200 patch applications in the last two years. Once you've built your first twenty it starts to get routine. I'm often able to copy an old ZENworks application and modify it slightly to push out a new patch. For simple patches I can sometimes be ready to test a ZENworks application in about 30 minutes.
Additional Note from David: NEW
It is worth mentioning as part of this topic that if you're going to deploy patches using ZENworks Workstation Management (or ZENworks Patch Managment), you should ensure that your workstations don't also try to get them from Windows Update. If you fail to do this, there is nothing stopping your users from getting the updates from the web site and applying them automatically. This can lead to unnecessary bandwidth usage if all your workstations try to get this month's 9 patches over the Internet at the same time.
Failing to prevent Windows Update also nullifies any attempt by administrators to pre-test or prevent the deployment of patches that may harm your other applications.
There are two ways to use ZEN to block Windows Update:
- Create a ZEN application to make a single registry change. Apply the
change to all users at every login:
"DisableWindowsUpdateAccess" = 1 (hex)
- Create a policy:
User Configuration\Administrative Templates\Windows Components\Windows
"Remove access to use all Windows Update features" = Enabled
Either of these settings will prevent the user from starting the Windows Update service. They can still navigate to the web site, but when WU tries to run it will announce that the service is disabled by policy.
DO YOUR HOMEWORK
Before you build your first ZENworks app for a Microsoft patch, there is some homework to do:
- Become very familiar with Microsoft's security bulletins - the technical versions. You can sign up to receive them by e-mail here. Here is an example bulletin.
- Download the full file versions of the security patch from the bulletin web site. They are near the top of each bulletin in the "Affected software" section. There is usually a version for each OS, and sometimes a different version of each patch for different versions of an installed product (like Internet Explorer or MS-Office patches).
- See if this update replaces another. Check the "Security Update Replacement" section near the top of the bulletin. If you have a ZENworks application for a now obsolete patch, you'll want to disable the old ZENworks application (but don't delete it yet - replaced patches often have similar requirements so you might want to copy the old ZENworks app to create your new one).
- Read through the security bulletin paying particular attention to the FAQ, Details, and Workaround sections. Determine if you really need this patch and prioritize your actions against other patches as appropriate.
- Before building a ZENworks application, test the patch on VMWare or another virtual environment to see if it affects any other critical applications you use. If you don't use a virtual environment for testing, test it on a real system. Find the "Security Update Information" section for each OS version you wish to patch. Record the following:
- Prerequisites. These will become your availability requirements in ZENworks.
- Available switches. You will use these for silent deployment.
- Verification using Registry Key. You will use this to prevent ZENworks from re-applying the patch.
Note that all of the above preliminary steps are required no matter what patch deployment method you use.
GO AHEAD, PUT ALL YOUR EGGS IN ONE BASKET
You should have a central file system for your patches. Depending on the number of systems at each site and the bandwidth to each site, you may decide to use a distribution system like ZENworks Server Management to deploy ZENworks application files to remote servers and then build your ZENworks apps to pull their files from the closest server. Before you get too complicated, you should be aware that the vast majority of Microsoft's patch files are less than 10 MB. Of the 105 patch files still in my system, only two are larger than 10MB and 2/3 of them are less than 1MB. I strongly recommend using only a central file system to deploy the majority of your patches. If you still have a bandwidth problem, try using a pre-deployment schedule to spread the deployment out over a day. Most central patch management products only support a central file server.
My central patch file system looks like this:
\\Server\Data\Apps\ (This directory is public read) In this directory I keep a sub for each patch. For example, the \MS05-039\ directory has \Log (a directory that will hold the log for this patch. I keep it separate because this directory needs to be writeable by the masses) MS05-039.lnk (a shortcut to the bulletin for this patch) Windows2000-KB899588-x86-ENU.EXE (The 2k patch file) WindowsXP-KB899588-x86-ENU.exe (The XP patch file)
BUILDING THE ZENworks APPLICATION
You'll need at least one ZENworks app for each possible patch file because each patch file has different system requirements. Most OS patches are very simple: If it's WinXP it gets the WinXP patch. However, some product patches, like IE or MS-Office, can have complicated requirements. You may have to dig deeper to determine which of your systems have certain versions of certain files to get a different patch file.
I keep all my ZENworks Microsoft patch applications in a single OU:
Here's how I built my WinXP patch file for MS05-039:
- Application Name: MS05-039_XP (Of course, the corresponding Win2k application is MS05-039_2k)
- Identification tab
- Title: MS05-039_XP
- Wait on force run: YES
- Determine force run order: YES
- Order: 139
I order my patch apps in the sequence they were issued. Critical patches start with 100. After a system has finished loading all the critical patches, it will then start running the "Important" patches, which start with 200, etc.
- Show progress: NO (I run all patches silently)
- Admin Note: I always put my name, the date, and a brief description of what this patch should do. You should do this for all your ZENworks applications.
- Distribution Options tab
- Registry: HKLM.Software.Your_Company_Name.Patches.MS05-039_Patch
- Install Date: %MONTH%/%DAY%/%YEAR%
- Installer: %LOGIN_NAME%
All my ZENworks applications put a record in the registry to indicate who ran what application when. You should do this too.
- Application Files: Copy the file from the server to the local hard drive.
This prevents each PC from holding the patch file open during execution
while other PCs try to do the same thing.
- Destination: C:\Your_Company_Name\Patches\MS05-039\ WindowsXP-KB899588-x86-ENU.exe Make sure you copy the right version)
- Item will: Copy if different (Just in case it got a bad copy, next time it will try again.) iii.
- Distribute always: YES
- Pre-Install Schedule: You may want to use this if you think you have a bandwidth problem.
- Distribute always: YES
- Prompt: NO
- Reboot: NEVER
- Registry: HKLM.Software.Your_Company_Name.Patches.MS05-039_Patch
- Run Options tab
- Path to file: C:\Your_Company_Name\Patches\MS05-039\ WindowsXP-KB899588-x86-ENU.exe (Tip: copy and paste this from above so you don't spell it wrong)
- Parameters: /norestart /quiet /overwriteoem /nobackup (I run silently and live dangerously! In most cases, delaying the restart will not result in a problem - it just prevents the patch from taking effect until the restart. If you've tested thoroughly, you can do this).
- Run once: NO (If something went wrong, I want the ZENworks app to try again).
- Force run as user: If you are using workstation association, you might need to check this.
- Environment: You may need to run as "secure user", depending on the policy restrictions you place on your users.
- Associations: For testing purposes, you should initially associate this to a test user - or maybe just yourself. After testing, you will come back and associate it to all users or all workstations.
- Availability tab
- System requirements
- OS Version: For XP use WinNT >= 5.1 and WinNT < 5.2 (2k is between 5.0 and 5.1)
- Registry: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Updates\Windows XP\SP3\KB899588 - NOT EXIST (Refer to the security bulletin's "Registry Key Verification" section for this OS. When the patch is applied it will set this registry, so the ZENworks app should only run if this key does not exist.
- Other: Depending on the requirements and pre-requisites of the bulletin, this can become complex. For most simple OS patches, all you need to do is identify the right OS and registry key.
- Schedule: I don't use this option, but you might want to schedule your patches to happen overnight by using ZENworks to Wake-On-LAN all your workstations at a certain time and then apply the patches overnight.
- System requirements
- Common tab
- Macros: Because I serve all my patch files from a central location, I don't use this. If you are using a distributed file system, you'll need to identify the variable that gets each workstation to the closest source.
- File rights: I use a Create/Write/Modify right to the log directory for my application so the reporting function (next) will work.
- Launch success/fail and distribution success/fail = Log to File (you could use log to database too).
- Location: \\Server\Data\Apps\MS05-039\Log\MS05-039log.csv
Reporting to a CSV (Comma Separated Value) file is a cool feature of ZENworks. You can check the log file any time to see how your application is progressing through your enterprise. Since it opens in Excel, you can easily sort and report the data.
TESTING THE ZENworks APPLICATION
Now you're ready to test your ZENworks application to make sure it does what you expect.
- For testing purposes, make sure the application is associated only to your test user (or test workstation) to appear on the desktop. This helps to confirm it will only run once.
- Log in as the test user on a workstation with the correct OS that has not yet been patched. After ZENworks finishes refreshing you should see the application icon on your desktop. Resist the urge to run it - just for a moment.
- Open the task manager (CTRL-ALT-DEL: Task List) and go to the "Processes" tab. Click twice on the CPU column to sort the list with the most CPU-active tasks on top.
- Run the ZENworks application by double-clicking on the desktop icon. You should see no indication on your screen that the application ran, and you should notice the patch executing briefly at the top of your Task Manager.
- After it finishes, refresh Application Explorer. The desktop icon should disappear. This means it detected the registry key indicating it already ran the patch. Now when you force-run the app you can be assured it will only run only once if it was successful.
- Confirm that the patch file was copied to the expected location on your
- Confirm that the registry was updated to indicate you ran the patch today
- Check the log file (\\Server\Data\Apps\MS05-039\Log\MS05-039log.csv) to confirm you can track everyone's progress applying the patch.
- Just to make sure you know what you're doing, you might want to log on to a workstation that has the wrong OS or otherwise doesn't meet the requirements. For example, if you're creating an app to deploy a WinXP patch, make sure it won't apply to a Win2k workstation.
DEPLOY TO THE MASSES
If all your tests were successful, you're ready to deploy. Change your association to your magic "ALL USERS" and/or "ALL WORKSTATIONS" group, or whoever your target is. Make the association "forced run" (not desktop or anything else if you don't want your users to see it).
Since you created a requirement for only this OS, and only if the registry key for the patch is not yet created, only workstations that should receive this version of the patch file will receive it. Even though a user is in the "ALL USERS" group, if the workstation they logged in to doesn't meet the other requirements, they won't receive this patch.
The next time ZENworks refreshes on each user's workstation, it will download and execute the patch silently on the workstations that need it.
Some of the cool features of this method:
- You have a central log which shows when every user or PC deployed the patch.
- Each PC has a record (in the registry) of running the ZENworks application which should have deployed the patch.
- Users or a technician can re-deploy the patch from the local hard drive if necessary.
- Once you have the method down, just copy an older WinXP ZENworks patch application to deploy another simple WinXP patch. Make a few changes to the new application and you're ready to go.
Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com