Introduction
One India-based bank lost $13.5M due to not having the latest version of firmware on the network switch handling ATM services. Hackers exploited the switch vulnerability to withdraw money using fraudulent ATM cards from different countries.
Common Vulnerabilities and Exposures (CVEs), are the weaknesses in a piece of computer software or firmware in non-computer devices, which can be used to gain access by nefarious party who exploits these vulnerabilities to misuse organizational data or disturb business applications and causing damage to brand reputation.
Hardware and software vendors, from time to time publish patches or next releases to address vulnerabilities reported thru different sources, support services, and own testing processes. It is expected that all those consumers of the hardware/software products should be applying those patches or installing new releases and updates into their IT environments as and when those are available from OEM.
Large organizations and those process sensitive data e.g. banking, finance, travel, health care, etc. are always on the radar of hackers / nefarious parties for exploiting any type of vulnerability. With Industry 4.0, many manufacturing organizations using digital technology in their plant operations. These organizations have to continuously track new CVEs found and appropriately apply patches and upgrade software available from OEMs to safeguard the data and assets.
The following figures show the vulnerabilities reported in the past 6 years and vulnerabilities exploited by hackers, that were reported by impacted organizations.
It is necessary for every organization to keep watch on CVEs, remediations available and apply them from time to time to keep hardware and software assets safe. However, the ground realities are narrating different stories. Analysts found that only 5% of remediation actions are taken by organizations due to various constraints and challenges. This becomes more difficult when threats are from outside of our own ecosystems i.e. from cloud-based software services e.g. Log4j, SolarWinds, etc.
Reasons and challenges sighted by the IT Support team managers for delays in remediations are mainly: work volume, process, end-user availability, monitoring & follow-ups, longe process, repeat activities and human errors etc.
To overcome above listed challenges, a Robotic Process Automation (RPA) solution was implemented at medical devices manufacturing company to address critical CVEs within an SLA timeframe i.e. less than three days. Read the case study below:
Case Study
Manual Process
At the customer global locations, multiple teams coordinate the discovery and remediation process of CVEs, by providing data to vendors, downloading CVEs, classification, map exceptions and approvals, monitor downtime windows for performing the remediation processes from beginning to end. The volume of work is listed in the table below:
| Region / Geographies | North America, Europe, China, APAC, EMEA |
| Assets Types | All on-premises assets with IP Addresses (Servers, Laptops, Desktops, Printers, Network devices, Print Servers, etc) |
| Asset Count | 27K+ Assets |
| CVEs Reported every month | Avg: 125K CVEs |
| Critical CVEs every week | 5% – 7% [ Includes carried forward CVEs from the previous week and awaiting remediation. ] |
| Cycle time to resolve critical CVEs | 8 – 12 Days |
| IT Support team for remediation | Servers, Laptops, Desktops, Printers, Network switches and Routers, Print Servers, All on-premises assets having IP Addresses |
IT Team: Communicate the Asset Movement (Issue, surrender, extend, lost, ready for use etc.) to third-party vendor.
Vendor: Execute a software program to Poll and Scan these IT assets for software used in every IT asset in the customer network (LAN, WAN). Build a database for every asset, covering IP Addresses, CVEs observed, classification, remediation, OEM remediation links, etc.
Info Security Team: Download the report from the vendor website and organize data using a spreadsheet and identify critical CVEs to address. Apply exceptions built for having different approval or different processing steps. Create an Incident for the IT support team to work on the remediation process.
IT Team: Work on tickets created by the Info Security team for remediation of CVEs reported, apply patches, and upgrades on respective IT Assets.
A manual remediation process and a longer cycle of remediation, do not cover all critical CVEs, and hence the count continues to remain high. As well assets remained exposed to vulnerabilities with several other risks of thereof.
Challenges
The major challenge in the manual process of repetitive tasks was:
* Incomplete and incorrect Exception/Approval list resulting in targeting wrong assets and losing time.
* Priority conflicts to select assets for remediation.
* Duplicate CVEs assigned as tasks were not created for every CVE.
* Managing remediation in odd working hours as per maintenance window.
* Not documented SOPs, executing remediation/upgrades using safer ways.
* Human Errors, non-availability of SOPs, training, and work volume.
* Missing tracking mechanism of open CVEs.
These and other challenges, which caused delays in the remediation of CVEs reported exposing IT assets to vulnerabilities. The organization encountered malware entering the network and creating unwanted files on hundreds of MS Servers causing downtime and mammoth efforts for remediation activities and monitoring for long duration for any other incidence.
As part of Information Security, the organization has several tools adopted by individual teams to perform their activities effectively. Cross-function collaboration is required to ensure info security by following the organization’s procedures and policies, which was missing or not happening due to volume and readiness to adopt the change in own process.
Post this malware incident, the organization decided to explore options to accelerate this overall process and concluded to peruse automation using Robotic Process Automation (RPA), the best option of automation with minor process changes by respective teams.
The RPA Solution
Post analysis of overall monitoring CVEs and remediation process performed multiple functions viz. Info Security, Infra, Network, IT Assets, etc. divided and built three BOTs covering:
- Bot – 1: BOT to help Keep information up to date
Provides a flexible way to consolidate and share information using Email Tool, i.e. keep the Infosec database updated for data points used as input to the remediation process. The following diagram shows the use of the Email tool to build data required for input to vendors, Exceptions and approvals, and the remediation process x CVEs.
- Bot-2: Daily CVEs review, classification, and prioritization
Bot-2 works daily at a pre-defined schedule and performs multiple activities as part of the overall remediation process, viz. Download CVEs Report from the vendor website, classification, map exceptions/approvals, and perform the next steps i.e. (a) Create RPA Queue items for SOP Based remediation as input to the next RPA Bot (b) Create a ticket to IT Support Team for manual remediation.
Bot-3: SOP Based Remediation
Bot-3 read the queue item (created by Bot-2) and process the RPA queue item for all CVEs for One Asset by creating a calendar invite to the asset owner to follow the remediation steps to perform as per the enclosed SOP. In case remediation is delayed beyond threshold days, invite the manager of the asset owner to let her/him know the risks associated due to delayed remediation.
Benefits
Three RPA Bots (one license) help to keep the input information updated whenever available with the respective team by using simple email messages (in template) and performing 63%+ remediation activities. With the major work done by Bots, the IT support team is focusing on manual remediation, making more SOPs for automation and meeting SLAs, and working on noncritical CVEs. The following table shows the post-automation status of key activities
| Critical CVEs | 2% – 3% [ Mainly associated with servers to remediate as per downtime schedule ] |
| Cycle time to resolve critical CVEs | 2 – 2.5 Days |
| IT Support team for remediation | 4 FTEs i.e. 50% Savings ( from 8 FTEs) plus 1 FTE for the Info Security team ( earlier 2 FTEs) |
| ROI on Automation investments | ROI in < 8 Months, plus ongoing savings of efforts |
Automation has also benefited the customer with:
- A safer environment with timely remediation of critical CVEs
- Year on Year saving of human efforts on remediation
- Improved ESAT for employees released from routine, repetitive, and robotic nature activities
- Saving in manpower budget by deploying released bandwidth for other initiatives
Next Phase: Add Cloud-based assets CVEs discovery and remediation for organizational Info Sec policies compliance and monitor utilization.
