Patch Tuesday often results in many late nights testing and applying patches, as organizations face a continuous onslaught of vulnerabilities and patches that can adversely affect IT infrastructure. This preparation guide outlines best practices for preparing, remediating, testing and continually improving your patch management process. It is based on 1000s of customer engagements and is designed to help create and implement a coherent and repeatable patch management process. It is applicable to the full spectrum of high priority security patches released throughout the month, even beyond the monthly Patch Tuesday release from Microsoft.
Laying the Ground Work
1. Discover Assets - Identify all firmware and software on the network and categorize them by platform, department, etc.
2. Agent Maintenance - Ensure that all assets in the network have been fully installed with an automated patch solution. Install new patch management agents where required, if this task has not yet been fully automated with a group policy, login script, or other technique.
3. Classify Value and Risk - Determine which systems are most critical to protect based on the assets housed and/or the function it provides. Define level of risk by criticality of system and how prone it is to attack.
4. Establish Workflow and Groups - Determine ownership, permissions needed and responsibilities for threat identification, testing and remediation across security, IT and business units. Define correlating system groups.
5. Identify Test Groups - Build a representative sample set of each type of machines based on steps (2) and (3), in readiness for patch testing steps (11) and (14).
6. Staff Training - Train applicable staff on vulnerability monitoring and remediation techniques.
Before Patch Tuesday
7. Schedule Resources - Allocate IT resources for Patch Tuesday while also integrating additional patch release schedules from Adobe (starting Q2 2006), Apple (ad hoc), Oracle, and so forth.
8. Reserve Down-Time for Servers - Reserve time slots to be able to deploy patch updates to any mission critical servers within 72 hours of Patch Tuesday release.
9. Watch for Pre- Announcements - Monitor security sites for pre-announcements of patches and discussion of vulnerabilities and possible zero day exploits that they may address from sources such as the Lumension® Endpoint Intelligence Center, SANS, National Vulnerability Database, etc.
10. Confirm Reporting Up-to-Date - Review and update system records of last patch deployments, make sure that all computers are being regularly scanned. Deploy any missing Service Packs, Hotfixes or rollups from prior months if these are still outstanding. Remember that some patches won’t install if you have missing pre requisites.
11. Deploy Missing Updates and Prerequisites - Determine if your software is fully updated or if there are any missing Service Packs, hotfixes or rollups from prior months that are still outstanding.
On Patch Tuesday
12. Study Vendor Information - Microsoft and other vendors provide Webinars, email alerts and comprehensive online information on all new Patch Tuesday updates.
13. Prioritize Potential Patches - Use patch impact (Critical, Important, etc), asset risk and value to prioritize systems for patch testing and deployment.
14. Change Control - Follow any internal planning and approval processes for agreeing on patch deployment.
15. Staged Testing - Testing each patch is vital: automated deployment is very risky and not advised. Be certain to test the patch in each environment of your previously defined groups.
16. Installation of the Patches - Stage deployments by system groups and prioritization. Start with smaller, low risk groups, validate that no problems occur, and then work your way to larger and higher risk areas of the network.
After Patch Tuesday
17. Deployment History - Maintain accurate records of all patches deployed, for both internal and external reporting purposes.
18. Calculate Time-to-Deploy - Measure how long it takes to get all servers, desktops and laptops fully patched in your organization, this is a great metric to measure against. Remain vigilant for laptops and VPN connected systems that may connect days (or weeks) after the initial deployment.
19. Monitor for Compliance - Make certain that new or rebuilt systems are “baselined” for their appropriate systems group. Monitor for removal of patches.
20. Checks and Balances - If available use a network scanner, attack scanner or secondary system to validate your system security from a different perspective. This can help identify any anomalous situations due to malware activity within your network.
21. Metrics Improvement - Modify system settings, distribution parameters and so forth to optimize the system better for next month’s updates. WAN optimization, polling frequency and minimizing the patches being detected can all help further optimize performance. Look for computers that did not receive updates at all, or that took unusually long to receive updates.