Loading...
HomeMy WebLinkAboutItem 12o - Software Programing for the City SCADA SystemPurchase Order with SoCal SCADA Solutions August 20, 2024 Page 1 of 5 DATE: August 20, 2024 TO: Honorable Mayor and City Council FROM: Paul Cranmer, Public Works Services Director By: John Corona Utilities Superintendent Janessa Kalanjian, Management Analyst SUBJECT: PURCHASE ORDER WITH SOCAL SCADA SOLUTIONS FOR THE INSTALLATION, PROGRAMMING, AND CUSTOMIZATION OF THE IGNITION SOFTWARE PROGRAM FOR THE CITY SCADA SYSTEM AND REPLACEMENT OF SCADA REMOTE TERMINAL UNITS AT ORANGE GROVE WATER FACILITY IN THE AMOUNT OF $247,731 CEQA: Not a Project Recommendation: Approve SUMMARY The City of Arcadia Public Works Services Department (“PWSD”) uses Supervisory Control and Data Acquisition software, commonly referred to as “SCADA”, to monitor the City’s water system equipment. In Fiscal Year 2023-24, the current SCADA system underwent a comprehensive audit, which identified the software program as operating with outdated technology and in need of replacement. Staff determined that Ignition SCADA Platform (“Ignition”) was the software platform most compatible with Arcadia’s water system. In May 2024, the City Council approved the purchase of the Ignition software. Now that the new software has been purchased, the next step is to install Ignition at all the City’s water facilities, which requires complex programming and customization connect all monitoring equipment to the Ignition program. In addition to the software upgrade, there are two Remote Terminal Units (“RTU’s”) at the Orange Grove Water Facility that will be replaced with upgraded RTU hardware and be programmed with Ignition. To complete the installation and programming of Ignition software, it is necessary that the consultant have complete access to the water system, and that the consultant have a comprehensive understanding of both Ignition software and all components of the water operating system. Due to the sensitive nature of allowing access to the water system and in the interest of preserving security measures, City staff is recommending the installation be performed by our current SCADA consultant, SoCal SCADA Solutions (“SoCal SCADA”). SoCal SCADA has extensive knowledge of Arcadia’s water operating system, existing hardware and is an experienced programmer for Ignition software. They will be Purchase Order with SoCal SCADA Solutions August 20, 2024 Page 2 of 5 able to complete the necessary programming and customization in a timely manner without disruption to day-to-day operations, while providing the City added security during the transition. It is recommended that the City Council approve a sole source Purchase Order with SoCal SCADA Solutions to provide a complete software system installation, programming, and customization of the new Ignition platform for the City SCADA system, and the replacement of two Remote Terminal Units for the Orange Grove Water Facility in the amount of $247,731. BACKGROUND The City of Arcadia water system contains 11 active groundwater wells, 33 booster pumps, 15 reservoirs, and 2 forebays. Each water site contains various computer systems that operate and monitor the wells, boosters, and overall activity for each site. Because the water sites are operating 24 hours a day, it is necessary to constantly monitor the systems and be able to adjust and address emergencies remotely and in real time. All modern water systems use a SCADA program to monitor and remotely control each aspect of their water system. SCADA systems provide real time monitoring data, alarm reporting, historical data mining, and remote-control access to make necessary adjustments to each component of the water system. SCADA systems must be compatible with various types of equipment, be user friendly for water system operators, and offer robust and available technical support. Arcadia has used the current SCADA software system, Genesis32 SCADA (“Genesis32”), for over 25 years. Genesis32 is proprietary software that has not been maintained for over 10 years and operates on outdated 32-bit Microsoft technology. Because Genesis32 has not kept up with current technology, it is not guaranteed that it will continue to be compatible with future upgrades to our system. Should there be any significant failures to the software, it would be difficult to find a technician familiar with Genesis32 and there are no guarantees that the issue could be resolved. Based on this information, a comprehensive audit of the exiting SCADA system was conducted in Fiscal Year 2023-24, and recommended the City upgrade to a newer software. To determine which SCADA software would be most suitable for the City’s water system, PWSD worked with a technical consultant to perform an analysis of SCADA software currently on the market. The analysis reviewed all equipment currently used at each water site, the technical specifications needed for compatibility, identified the deficiencies in the current Genesis32 system, and compared these items to specifications and capabilities of newer software programs. They determined that while there are other SCADA systems on the market that would be compatible with our water system, Ignition Platform, provided by Inductive Automation, best addresses the insufficiencies of the current Genesis32 system while providing user-friendly software that has readily available technical support. On May 7, 2024, the City Council approved a sole source Purchase Order with Inductive Purchase Order with SoCal SCADA Solutions August 20, 2024 Page 3 of 5 Automation, LLC, for the purchase of the Ignition SCADA Platform software to replace the antiquated Genesis32 software. To upgrade outdated hardware, PWSD has been upgrading the Remote Terminal Units (“RTUs”) within the water system on an ongoing basis. RTUs are microprocessor- controlled devices that interface with all the monitoring equipment components at a water site and transmit their data to the SCADA system. RTUs act as a hub to transmit information collectively rather than from each component individually. Each water site may have multiple RTUs depending on the amount and size of the equipment at the site. The City has used Bristol Babcock 3330 (“Bristol”) RTU systems for nearly 20 years. While these units are compatible with the new Ignition software, they are no longer supported and will not be reliable in the long term. PWSD has been actively replacing the Bristol systems with Emerson Control Wave RTU (“Emerson”) hardware, a newer RTU system that is compatible with existing equipment and the new Ignition software. To keep the process cost effective, PWSD has been purchasing Emerson hardware components on an ongoing basis. Older, less reliable Bristol units were prioritized for replacement first, followed by other water sites, based on importance. The next facility to be upgraded to the Emerson RTUs is the Orange Grove well site. The Orange Grove Water Facility is the most crucial component of Arcadia’s water system. The facility includes four water wells that produce approximately 2,000 acre feet of water per year, four water reservoirs capable of storing nearly 15 million gallons of potable water, and six booster stations that distribute potable water to various areas of the water system. Since PWSD has the Emerson hardware on-hand, replacing the two units at Orange Grove simultaneously with the transition to the Ignition software would be the most efficient and cost-effective route. The RTU replacement involves prebuilding the hardware panel and will require numerous hours of testing and control confirmation during the replacement process. Once the hardware panels are built, they will be connected to the Ignition software system after its installation is completed. DISCUSSION The Ignition software must now be installed, programmed, and customized to operate as the City’s primary SCADA software. Programming requires detailed knowledge of the Ignition software as well as experience with the various monitoring equipment and components that make up the Arcadia water system. Programming also involves providing full access to the independent server that houses the SCADA software data, which poses a potential security risk due to the sensitive information and ability to augment the current system’s day-to-day activities. For these reasons, any potential consultant would need to undergo a background check and be cleared by the City’s IT consultant before being provided with the access needed to perform the upgrade. Due to the complex understanding of both the Ignition software and City’s existing water system components, and in the interest of minimizing security concerns, it is Purchase Order with SoCal SCADA Solutions August 20, 2024 Page 4 of 5 recommended that SoCal SCADA perform the water system upgrade to Ignition Platform SCADA software; and replace the two existing Bristol Babcock 3330 (“Bristol”) Remote Terminal Units. SoCal SCADA is the City’s current SCADA software maintenance provider and has comprehensive knowledge of Arcadia’s water system, as well as its various components and monitoring equipment. SoCal SCADA is the only local vendor proficient in the current Iconics Genesis32 software and has, therefore, been the City’s sole source provider for the existing SCADA system. They have provided satisfactory work in maintaining the system and troubleshooting errors as they arise. SoCal SCADA will be able to simultaneously keep the Genesis32 system operational, allowing City staff to maintain normal operations while they perform the upgrades, customization, and testing necessary to switch over to Ignition software. There is built in redundancy in the water system so that programming can be done without disrupting day-to-day activities. While there are other consultants that could perform the programming of Ignition, it would not be guaranteed that they are familiar with all the various components that make up the water monitoring system, and SoCal SCADA would likely still need to be involved to ensure the Genesis32 software remains fully operational during the transition. Having SoCal SCADA perform the work also reduces security concerns since they already have the access necessary to perform the work. Once the water system is fully transitioned to Ignition, the City will be less reliant on a single provider as Ignition is widely used and many companies provide technical assistance for the software. Full transition to Ignition software is expected to take two to three months, which includes programming, testing, and training City staff on the software. During this time, the two replacement RTU units for the Orange Grove water site will also be assembled, tested, installed, and programmed to the new software. Assembling the units should take about one week and involves building a “board” out of the individual hardware components, which the City has previously purchased, and connecting them into one cohesive system. SoCal SCADA will complete the RTU replacement simultaneously with the software upgrade. For these reasons, it is recommended that the City enter into a sole source contract for these services. ENVIRONMENTAL ANALYSIS The proposed action of upgrading software does not constitute a project under the California Environmental Quality Act (“CEQA”), and it can be seen with certainty that it will have no impact on the environment. Purchase Order with SoCal SCADA Solutions August 20, 2024 Page 5 of 5 FISCAL IMPACT Sufficient funds are available in the Department’s Fiscal Year 2024-25 Operating Budget for the installation, programming, and customization of the Ignition software for the City SCADA system, and the replacement of two RTU’s for the Orange Grove water facility. The maximum amount expected to be spent for this effort is $247,731. RECOMMENDATION It is recommended that the City Council determine that this action does not constitute a project under the California Environmental Quality Act (“CEQA”); and approve a Purchase Order with SoCal SCADA Solutions for the installation, programming, and customization of the Ignition Software Program for the City SCADA system and the replacement of SCADA Remote Terminal Units at Orange Grove Water Facility in the amount of $247,731.