Beyond the PLC: Taking a Holistic View of Redundant Architecture

By: Allan Evora

PLC and controls only take up a small portion of an overall redundant system. Don’t get blinders on by putting a redundant PLC in your facility, only to leave instruments or networks or your SCADA system vulnerable.

Remember that true redundancy strives to eliminate single points of failure. Even if you design a redundant PLC but have a network with a single point of failure, your investment in redundancy may be somewhat diluted.

Let’s take a look at each of the five components that make up a redundant control system within a facility.

Controls (PLCs)

In a typical control system, you have two PLCs that do not contain any I/O modules. The controllers are strictly in place to execute control logic for the overall system. Those controllers are linked to remote I/O that reside outside the primary racks.

The PLCs are linked together via a cable which allows each controller to determine the status of the other. Only one is in control, but the other is sitting and waiting to take over control should the other have a fault. The primary difference between a non-redundant and redundant architecture is the presence of a second CPU.

 

Instruments

Instruments are what provide feedback to the controller. For a fully redundant system, you must consider your specific application and what would happen should one of the instruments that’s considered critical toward the control of the process or equipment fail.

Temperature sensors, such as thermostats, thermistors, or RTDs are a great example of instruments. If temperature control in your facility is critical, you may want to consider redundant temperature sensors.

This mentality also applies to equipment outside of the primary control system itself, such as motors, electrical switchgear, UPS, etc.

 

Network infrastructure

In the event of a failure in any segment of the network, the network should continue to operate. Typically, this is done via a self-healing ring. That technology has been around for many years and many manufacturers have created their own versions.

The point is, if you’re going through the expense of designing a control system, redundancy within your network infrastructure is one of the components you need to pay attention to.

 

Use deterministic networks

A response from a non-deterministic network (e.g., ethernet TCPIP) is not guaranteed, which is why it is highly discouraged in redundant systems. Most redundant architectures use deterministic networks (e.g., ethernet IP and Profinet).

Here’s the reason non-deterministic networks don’t work. When performing an action on the internet at home or work, a response occurs within a timeframe that totally depends on what is happening in the network. If two people issue a command at the same time, there could be a network collision, reset, and random timeout period.

Facilities can’t develop control systems based on randomness. Consistency is paramount, which is why a deterministic network with the correct design, configuration, and protocol is important.

 

SCADA

SCADA has many functions. It’s the operator’s HMI, historian for data logging, and also the mechanism that alerts you to the status of your high availability system. If it’s important to you that operators have visibility into and ability to interact with your control system at all times, consider specifying a redundant SCADA system.

Redundancy within SCADA has matured substantially. We like VTScada’s solution, but all major SCADA manufacturers have redundant solutions.

 

Commissioning

While commissioning isn’t part of your control system’s software or hardware, it’s a crucial aspect of redundancy. Commissioning is the formal process of point-to-point checkouts to test component functionality and operability of sequences designed into the control system.

Nowadays, most mission critical facilities employ a third-party commissioning agent or separate party who wasn’t involved in the design, development, or installation of the control system. They provide the owner piece of mind that if the system is called upon to function under one of the failure modes, it will do its job properly.

For example, we recently upgraded an EPSS SCADA system for a financial data center. The associated control system had a redundant PLC. As we went through the recommissioning process for the SCADA system, the owner took the opportunity to do a complete redundancy test. In addition to a series of tests to identify the operating and failure modes of generators and transfer switches, they conducted a full pull-the-plug test. They simulated a loss of utility power to see if the emergency generators came online, and even failed the primary controller to witness themselves that emergency power was provided to the load within the required timeframe.

They walked away confident their system would provide them with the ride-through from the generators and UPSs to allow the computing resources to continue to operate.

Moral of the story: when specifying a control system, take the time to commission that system properly.