Big Data Aggregation and Reporting
Calyx is the powerful new data processing, reporting and visualization platform from ITM Monitoring, a market leader in the design, installation and management of geotechnical and structural monitoring solutions for major construction projects across the globe. Digital Tactics, having worked intimately with ITM to deliver big data reporting and visualization solutions for Crossrail (Europe’s largest construction project to date), were the natural choice for the development of the new system.
ITM Monitoring have an existing and respected online product for the basic reporting and analysis of sensor data, Argus, which has been subject to ongoing iterative development for many years. While it remains suitable for certain kinds of monitoring projects, in today’s interconnected big data world, Argus had begun to show its age and limitations vs. its competitors. In line with recent operational changes within ITM and brand refresh, a decision was made to commission Calyx: a fresh, cutting-edge re-imagining of the Argus platform, which better services modern user requirements and expectations as well delivering more efficient and advanced visualization tools than were previously available in Argus. Calyx had to intuitively service everything Argus had always done conceptually i.e. deliver systems to handle the import and export of sensor data from a wide variety of geotechnical instruments; systematically process and analyze this raw data, running it through complex engineering calculations; alert engineers through various channels (web, email, SMS…) to project risks and dangers post analysis; represent both raw and calculated data on a variety of technical charts; generate and send regular pdf format reports from the data; and a variety of more mundane but essential data management tasks. New functionality also had to be introduced. Calyx had to be appropriately responsive for use on tablet devices and smartphones in the field; it had to visualize geographically disperse sensors in a familiar and intuitive way; it had to provide visual tools for the management of sensor information across engineering schematics; it had to work natively with new, big data sets as easily as it did with traditional well understood formats; It had to integrate live webcam feeds of key engineering activities; it required improved charting, reports, speed and security features plus well-defined API functions for future integration with other building information modeling (BIM) tools.
Digital Tactics had worked alongside ITM on the Argus (PHP/MySQL) codebase, helping modify and support the software for use on significant big-data infrastructure monitoring projects such as Crossrail (Europe’s largest engineering project) and the upgrade of London Victoria Station. This familiarity with the Argus product coupled with a passion to drive the platform forwards on well proven, modern technologies lead to Digital Tactics being selected as ITM Monitoring’s natural partner for the planning, design and delivery of the Calyx platform. To this end Digital Tactics worked with ITM to produce a phased program of works that would deliver relevant portions of the Calyx platform as key industry dates occur throughout 2015, with the ultimate aim of achieving a fully featured cutting-edge solution by the end of the year. As well as pure delivery of a new product, support and maintenance of the existing Argus fleet also had to be considered. Therefore as well as a plan for execution of the build, a migration path for existing services was defined such that long term projects that warrant a migration to Calyx would be as painless as possible to transition.
Digital Tactics performed a detailed examination of ITM’s suite of existing operational Argus installations and defined commonalities in their core functionality. This core represented essential technical items which had to be delivered within the phase one build of Calyx, rationalized alongside a number of more conceptual usability improvements identified from the gathering of user stories. A number of core and loosely couple add-on software modules were identified for development atop a known and well-structured, modern enterprise level PHP framework ZF2 (Zend Framework 2). ZF2 was selected to provide a well tested and uniform baseline to create the software from, facilitating future automated testing and quality control processes as well as supporting reliable future code extensibility, management and maintenance (a stark but beneficial contrast to the historic proprietary nature of the existing Argus codebase). MySQL, running an optimized version of the Argus schema was retained as the database technology of choice due to familiarity and requirements around relational analysis, but with provision for use of NoSQL technologies in future big data projects as appropriate. As well as a full and frank dissection of the Argus backend code, the web frontend required severe attention to both bring the software into line with modern usability expectations, and to deliver the level of functionality we wanted to realize within the new interface. Given the dated look and feel of Argus, a wholesale redesign of the interface was delivered in line with ITM’s new brand guidelines and modern frontend technologies were introduced to leverage the power of the browser. Bootstrap, AngularJS, leaflet, highcharts, D3, jQuery, datatables and more have each been applied with www.skill to deliver an efficient and vastly improved user experience, as well as some innovative tools for engineers working with the platform day-to-day.
As of today phase one of Calyx is nearing release and huge strides have been made in platform development. The new Online Monitoring System (OMS) is a world away from its Argus roots, while retaining all the key functionality Argus facilitated. The interface is modern and quick to use with a variety of new features that will assist engineers in the interpretation and management of their monitoring data, without requiring any additional browser plugins or components. The software itself is now more extensible and maintainable than ever before, which enables ITM Monitoring to react rapidly to technical changes and innovations in the monitoring and civil engineering space facilitated by the boom in the Internet of Things (IoT). As a technical innovators, the team at Digital Tactics are immensely proud of what has been achieved to date with the platform and are confident that Calyx OMS will become a major asset for ITM Monitoring in this ever competitive technology driven industry.