Scanning OT networks

Operational technology (OT) is defined by the National Institute of Standards and Technology (NIST) as programmable systems or devices that interact with the physical environment or manage devices that interact with the physical environment. Examples include industrial control systems (ICS), computer numerical control systems (CNC), building control systems, transportation systems, and many others. While OT systems and devices were once isolated independent systems running on specialized hardware and software, they have become increasingly interconnected, adopting industry standard operating systems and network protocols. As a result, it has become increasingly important for organizations to maintain an accurate inventory of network-connected OT assets.

Who is this playbook for and why?

OT environments typically have unique performance and reliability requirements due to the role they play in the delivery of critical services and the effect they can have on health and safety. As a result, it is important that you take a prudent approach to active scanning of OT assets. This playbook provides guidance to IT and OT personnel that want to leverage active scanning as part of their strategy for maintaining a comprehensive OT asset inventory.

How will runZero help?

runZero leverages active unauthenticated scanning to discover assets on your network, and the runZero Explorer is built with sensitive OT assets in mind. Unlike some technologies, the Explorer only sends normal traffic; it does not send malformed packets nor does it attempt to exploit vulnerabilities. Additionally, only TCP/UDP ports that provide actionable information for fingerprinting a device are probed. runZero also provides flexible scan configuration options, allowing you to customize the speed and depth of scanning to fit your environment.

What will I need to do?

In order to scan your OT environment, runZero recommends taking the following steps:

  1. Create a site for OT assets.
  2. Deploy a dedicated Explorer.
  3. Configure scan templates for OT assets.
  4. Run a limited scan & evaluate results.
  5. Run a full scan & evaluate results.
  6. Configure recurring scans.

Prerequisites

In order to configure scans of your OT environment, you will need the following:

  • A list of subnets allocated to your OT network.
  • A registered Explorer with appropriate access to your OT network.
  • If appropriate, firewall rules allowing Explorer communication through OT firewalls.

You should also know about any IP addressing schemes on your OT network, such as common patterns used across locations or networks. For example, you may wish to exclude PLCs from your initial scans to further reduce risk. If PLCs leverage a consistent IP addressing scheme or sit on specific VLANs, it will be easier to exclude these assets.

Implementation steps

The following instructions will show you how to safely scan your OT network.

Step 1: Create a site for OT assets

While not required in order to successfully scan your OT network, configuring a separate site (or multiple sites) for your OT environment will allow you to more clearly define your OT subnets and manage exclusions.

  1. In the appropriate organization, navigate to Sites and select New Site.
  2. Assign a Name to the Site (e.g. OT network).
  3. Add Registered subnets, including a description and relevant tags for each. Registered subnets will be automatically included in the Default scan scope so you do not need to define subnets in both locations.
  4. If appropriate, add Default scan exclusions.
  5. Click Save.
Tip: If there are critical assets within your OT network that you never want to scan, they can be added to the Default scan exclusions within the OT site configuration. This allows you to centrally manage your exclusions. The OT scanning templates defined in Step 3 will automatically use these exclusions, ensuring that your exclusions are applied consistently across all scans.

Step 2: Deploy dedicated Explorers

runZero recommends deploying dedicated Explorers for OT discovery. Whether you deploy one Explorer or multiple Explorers will depend on your OT network architecture, resource availability, and scanning strategy. Discovery works best when an Explorer is deployed on the network that it is scanning, with no intervening gateway devices such as firewalls or routers. This allows runZero to obtain the MAC addresses for each asset and avoid issues that can occur when scanning through stateful gateways.

Explorers will need to be able to communicate back to the console over 443/tcp in order to send scan data back to the console for processing and to obtain software updates. If your OT network is completely isolated from the internet with no inbound or outbound communication permitted, consider deploying a self-hosted runZero console or an offline scanner within your OT network. This will allow Explorers to communicate with the console without violating OT policies.

Review self-hosting runZero and offline mode configuration for more information on deploying and updating a self-hosted console. If you choose to implement a self-hosted console, runZero recommends placement in the industrial security zone. A proxy or broker service can be leveraged within your industrial DMZ to facilitate any required communication with your enterprise network.

Tip: While OT network architecture is too large of a topic to cover in detail here, it is important to consider security zones within your OT network and associated rules regarding communication between security zones. Whether you're employing the Purdue Model, ISA/IEC 62443 or some other best practices to help define your network architecture, deployment of seperate Explorers within each security zone is an encouraged practice. If you would like to learn more about recommended OT network architectures, review NIST Special Publication 800-82 Guide to Operational Technology Security.

The following are a few examples of Explorer deployment strategies.

Example 1: Centralized deployment

In a highly distributed environment where OT exists in 10s or 100s of field locations, a centralized deployment strategy may be more efficient. In this case, deploying Explorers in your primary and secondary datacenter locations is a strong starting point for scanning OT. In a centralized deployment model, it is especially important that the presence of firewalls and routers be considered. Rules will need to be implemented to ensure that each Explorer has appropriate visibility into field locations that it will be scanning. Deploying separate Explorers in the industrial DMZ and the industrial zone can help limit scanning through firewalls. You will also need to clearly understand what type of connectivity you have between your datacenter and field locations so that scans can be tuned appropriately (this will be covered in more detail in Step 3). One trade-off to a centralized deployment model is that layer 2 discovery will be less effective. While our Explorer leverages multiple techniques for enumerating MAC addresses, layer 2 probes are the most effective.

Example 2: Distributed deployment

In a less distributed environment where there are larger concentrations of OT at key locations, deployment of Explorers at each location may be a more effective strategy. In a distributed model, layer 2 discovery will be more effective since the Explorer will traverse fewer firewalls or routers (if any at all). This will also help distribute load on your OT network and help alleviate concerns related to bandwidth consumption in scenarios where higher latency connections are employed. However, bandwidth should still be considered since the Explorer will need to send its scan data back to the console. The amount of data sent back to the console will depend on the number of assets at the location and the frequency of scans.

Example 3: Hybrid deployment

In a hybrid deployment model, Explorers are deployed in both the datacenter and at field locations. If there are too many field locations to effecitvely deploy Explorers to each, then you should devise a set of criteria to determine which field locations warrant an Explorer. While each organization’s criteria may differ, this could be based on number of assets, criticality to operations, or even revenue generation. This model will help ensure that you are getting the most complete and accurate data at your highest priority locations and allows you to leverage the centralized Explorers at lower priority locations.

Step 3: Configure scan templates for OT assets

Since you can run multiple scans over time to cover various OT networks and/or sites, creating scan templates will simplify the scheduling of scans and help ensure a consistent configuration across each scan. Consistency is especially important when scanning OT. We recommend creating two templates: an OT Limited Scan template, and an OT Full Scan template. The purpose of the limited scan will be to perform an initial discovery of assets with only limited probing. This will give you an opportunity to assess the impact of scanning on your OT network and assets before transitioning to full discovery scans.

The following instructions do not go into detail regarding the function of each configuration item. Review discovering assets for more information on scan configurations.

Step 3a: Create an OT limited scan template

The purpose of this template is to determine what is alive on your OT network while minimizing the volume of traffic generated and avoiding proprietary or ICS ports during initial scanning. Performing a limited scan will also give you an opportunity to evaluate the impact of scanning on your OT network before moving on to more comprehensive scanning. This template is provided for guidance purposes and should be adjusted to meet the unique needs and risk tolerance of your organization.

  1. Add a template by selecting Tasks > Task library from the side navigation and then click Add template.
  2. Provide a Name for the template (e.g., OT Limited Scan Template).
  3. Set the Scan rate to a maximum of 500 packets per second.
  4. Navigate to the Advanced configuration tab.
  5. Set the Included TCP ports to 21,22,23,69,80,123,135,137,161,179,443,445,3389,5040,5900,7547,8080,8443,62078,65535.
  6. Set the Maximum host rate to 20.
  7. Set the Max group size to 2048.
  8. Set the Max TTL to 64.
  9. Toggle Limit scans to pingable hosts to Yes.
  10. Navigate to the Probes and SNMP configuration tab.
  11. Click the Disable all button under Disable all probes.
  12. Toggle the following probes to Yes individually:
    • LAYER2
    • NETBIOS
    • NTP
    • SNMP
    • SSH
    • SYN TCP port scan
    • TFTP
  13. Navigate to the Credentials configuration tab.
  14. Ensure that appropriate SNMPv2 or SNMPv3 credentials are selected.
  15. Save the template.
Tip: If there are additional ports that are commonly employed across your OT network, and you're comfortable making them a part of your initial scans, add them to the Included TCP ports configuration to improve results. Since we've limited scans to pingable hosts in Step 9, you can also add additional ports to the LAYER2 probe configuration in Step 13. The layer2-tcp-ports value is used as part of the multi-protocol ping scanner to identify pingable hosts.

Step 3b: Create an OT full scan template

The purpose of this template is to perform comprehensive discovery and fingerprinting on the OT network while still taking a conservative approach to the volume of traffic generated by scanning. This template is provided as guidance and should be adjusted to meet the unique needs and risk tolerance of your organization. For example, you may wish to exclude certain ports that are known to be used by legacy technology.

  1. Add a template by selecting Tasks > Task library from the side navigation and then click Add template.
  2. Provide a Name for the template (e.g., OT Full Scan Template).
  3. Set the Scan rate to a maximum of 500 packets per second.
  4. Navigate to the Advanced configuration tab.
  5. Set the Maximum host rate to 20.
  6. Set the Max group size to 2048.
  7. Set the Max TTL to 64.
  8. Add relevant DNS nameservers for your OT network.
  9. Navigate to the Probes and SNMP configuration tab.
  10. Click the Disable all button under Disable all probes.
  11. Toggle the following probes to Yes individually:
    Probes (click to expand)
    • BACNET
    • DAHUA-DHIP
    • DNS
    • DTLS
    • IKE
    • IPMI
    • KERBEROS
    • KNXNET
    • L2T
    • L2TP
    • LANTRONIX
    • LAYER2
    • LDAP
    • MSSQL
    • NETBIOS
    • NTP
    • OPENVPN
    • PCA
    • SIP
    • SSDP
    • SNMP
    • SSH
    • SYN TCP port scan
    • TFTP
    • UBNT
    • VMWARE
    • WEBMIN
  12. If you are comfortable doing so and it is safe in your environment, you may also wish to enable the following OT protocol probes by toggling their settings to Yes:
    Probes (click to expand)
    • ETHERNETIP
    • FINS
    • MODBUS
    • S7COMM
    • DNP3 - Note that for the `DNP3` protocol, some devices will only communicate with a single other device at a time. If this is true for devices on your network, scans could interrupt communication over DNP3. Therefore it is especially important to test before deploying the DNP3 probe in your network.
  13. Ensure that the tuning parameters for the various probes are set as appropriate for your environment. This includes, but is not limited to:
    Probes (click to expand)
    • MODBUS - the value of the modbus-identification-level option may need to be tuned depending on the capabilities of the MODBUS devices in your environment
    • S7COMM - the s7comm-request-extended-information option may be set to true to gather more information, if supported by the devices in your environment
    • DNP3 - the banner-address-discovery option may be set to require or prefer to require or prefer DNP3 address discovery via unsolicited messages sent by the probed outstation. The explorer-address option can be set to a decimal number indicating the DNP3 address to be used by the explorer when communicating with DNP3 devices.
  14. Navigate to the Credentials configuration tab.
  15. Ensure that appropriate SNMPv2 or SNMPv3 credentials are selected.
  16. Save the template.

Step 4: Conduct limited scans

Once templates have been created, it’s time to start scanning your OT network. Your initial scans should be of a small subset of your OT network. Start with one or two /24 networks to get started or consider scanning a low priority field location. Avoid scanning critical assets or high priority locations in the beginning. Ensure that appropriate personnel are informed prior to initializing a scan so that the health of the network and relevant OT devices can be monitored. Schedule the scan to initialize at a pre-determined date and time.

  1. Navigate to Tasks > Scan > Template scan.
  2. Type OT Limited Scan Template into the search box and select the radio button for the template.
  3. Click Continue to scan configuration.
  4. Select the Site configured in Step 1.
  5. Select an Explorer deployed in your OT environment.
  6. Add one or more subnets to the Deployment scope.
  7. Set a Start time for the scan.
  8. Click Initialize Scan.

Once your initial scan is completed, evaluate the scan data as well as the state of the OT networks that you just scanned. Review the list checks on the Running initial scans page. In addition to those checks, evaluate the following:

  • Consult with appropriate network personnel to evaluate the health of all networks that were scanned, including any routers, switches, or firewalls that were in the initial discovery scope or that were traversed during scanning.
  • Consult with appropriate OT personnel to evaluate the health of any OT devices that were in the initial discovery scope.
  • Evaluate bandwidth utilization during scanning at any remote locations that were in the initial discovery scope.

If issues occur as a result of intial scans, adjust the scan configuration of the OT Limited Scan Template accordingly and re-run the scans until you are comfortable with the results. Once you are comfortable with the results of your initial scans, extend limited scanning to the broader OT network. Keep in mind that you should continue to take a phased approach as you extend scanning. OT networks can be very diverse so your initial scans may not have the same results as scans on a different part of the network. Continue to keep appropriate personnel informed of scanning activity and maintain a log of all your activity so that you have a record in the event that issues occur.

Step 5: Conduct full scans

Once limited scanning is complete, it is time to perform more comprehensive scanning of your OT networks.

  1. Navigate to Tasks > Scan > Template scan.
  2. Type OT Full Scan Template into the search box and select the radio button for the template.
  3. Click Continue to scan configuration.
  4. Select the Site configured in Step 1.
  5. Select an Explorer deployed in your OT environment.
  6. Add one or more subnets to the Deployment scope.
  7. Set a Start time for the scan.
  8. Click Initialize Scan.

The process for full scanning should be similar to that of limited scanning. Follow the steps outlined in Step 4, continuing to keep appropriate personnel informed of scanning activity and checking network and OT device health following scans. If issues arise during scanning, adjust the configuration of the OT Full Scan Template and repeat scans until successful results are achieved.

Step 6: Configure recurring scans

Once you have completed full scans without causing any adverse affect to the health of your OT environment, you should schedule recurring scans to keep your asset inventory up-to-date. Consult with appropriate OT personnel on the timing and frequency of recurring scans and continue to monitor scans over time to ensure continued success. To do this, simply repeat Step 4 or Step 5 and set the Scan frequency as part of the scan configuration.

Additional considerations

  • If your OT network is completely isolated from the internet with no outbound communication permitted, consider deploying a self-hosted runZero console or an offline scanner within your OT network. This will allow Explorers to communicate freely with the console. Review Self-hosting runZero and Offline mode configuration for more information on deploying and updating a self-hosted console.

  • runZero discovers common OT/ICS ports and may actively probe such protocols. Examples include modbus (502/tcp) and EtherNet/IP (44818/tcp). These probes can be disabled idividually if you do not wish to actively scan these protocols.

  • IPv6 scanning in a native feature of the runZero Explorer. If an Explorer has an IPv6-enabled interface, it will perform IPv6 neighbor discovery and conduct a full scan on any newly-discovered assets. IPv6 addresses and DNS AAAA records can also be specified within the scan discovery scope.

  • Both the OT Limited Scan Template and the OT Full Scan Template example templates leverage conservative scan rate configurations. A slower scan rate will mean longer runtimes for scans. If you choose to increase the scan rate, it is recommended that you do so in small increments. Increasing the scan rate from 500 packets per secton to 1000 packets per second would be a reasonable increment. It is also recommended that you test new configurations on a small subset of your OT network prior to extending a new configuration to the entire OT network.

Relevant runZero resources

Updated
Previous: Finding gaps in vulnerability scanning Next: MSSP guidance