John M. Daane, Director of Information Technology, Emerald Coast Utilities Authority
I was hired in January 2015, with the charge to bring the Utility into this century as far as Technology was concerned. The Utility’s technology was frozen in time and it was time to thaw it out. AS/400 applications, 15-year-old desktops, stand-alone servers, T1 network connections, tape backups, and many manual processes, were the spindle of the organization. Outages were a daily occurrence.
The Utility, the customers, and the governing Board of five elected officials wanted more. Apps to pay their bills online, data-driven decision making, real-time consumption data, expedient customer service, and a flawless sanitation service were amongst the demands that had to be met.
"The sleek design, compact into a 2U form factor, including nodes and storage, seemed like the right choice. Many of the servers and applications were not compatible with virtualization and needed to be upgraded first"
As any leader knows, to get anywhere successfully, one must know where they are, to begin with. Following this, the first year was spent creating a five year IT master plan. A comprehensive approach to figuring out how they get there so to speak. The plan was adopted by The Board, supported by the Executive Management Team, and funded. And this is where the fun began.
If the goal was to implement new applications and services, the underlying infrastructure had to be able to support it. To begin, the focus was on building a Virtualization platform running on a reliable network. Hyper-converged systems had my attention from the moment I laid eyes on them. The sleek design, compact into a 2U form factor, including nodes and storage, seemed like the right choice. Many of the servers and applications were not compatible with virtualization and needed to be upgraded first. This didn’t come without risk, and bare-metal restores were the only option if the upgrades failed. Some applications we were able to replace or upgrade, and some applications would just not run on a newer platform and had to be replaced or eliminated. Eventually, we were able to enter into a virtualized world and we now have 80 virtual servers, 3VMWare clusters, and room to grow. Virtualization in all aspects meets the “Better, Faster, and Cheaper” test.
The network needed an overhaul as well, especially the WAN connections. These came from multiple providers and varied in speed from 56K, T1, to 10Mb. A new fiber infrastructure seemed like the right choice and partnering with one of the major Fiber Vendors in the area we were able to build a new WAN topology that increased the overall bandwidth by 500 percent at 60 percent of the cost of the previous solution. Having all the connections in one invoice saved additional bill processing time, and all locations on one platform eliminated the many WAN related issues we were experiencing. “Better, Faster, and Cheaper” – check.
Running a utility in Florida requires phones, lots of them. A large number of customers use their phone to do business with us. The phone lines and phone system were not agile and easily expanded and upgraded. A decision to switch to a SIP provider and eventually implement a new phone system seemed imminent. Just switching to SIP increased capacity and lowered the cost significantly. Additional features that SIP offered over the traditional PRIs made it a better choice as well. The new phone system will go into place towards the end of 2018, putting the icing on the cake. Enhanced reporting for the call center, better call management, and Disaster Recovery features that were only dreamt of with the previous system.
A PC replacement program was put into motion, setting the life-cycle of computers at 5 years. 15-year-old machines running XP were the first target which took two years to clear out. We are now deploying new Windows 10 based machines to desks that have five to seven-year-old computers, and within the next year, no computer will be older than 5 years, running Windows 10. Solid state drives, all-in-one 23” screen form factor computers, with 5 years of warranty are going to be the norm. There is no comparison to the machines that we are replacing.
This finally leaves the ERP system. We have started the process, which will take about 2.5 years, to replace the AS/400 applications with a new system. A new system that will eliminate a lot of manual processes, provide better reporting, enable the end-users, and provide all the enhancements and features that the Board, our customers, and staff, are demanding. Features that seem so commonplace when you digitally interact with other utility and service providers, such as managing your own account, usage reports, consumption information, apps to pay your bill, and outage information. Features that require a 30-second conversation to start, and generate 2.5 years of work before they are accomplished.
We are about half-way through our five year IT master plan and by the time we go live with our new ERP system we will be at the 5-year mark. The support contracts for all the new systems will still have two years of life by the time we roll out the new ERP, which will give us time to settle into the new normal, and then start planning for the next new future, which will have to be “Better, Faster, and Cheaper”.