Introduction – Second Network Hub Project
The day-to-day operation of CERN is heavily reliant on IT services, the long-term interruption of which would have a major impact on the running of the laboratory. Currently all the core networking equipment as well as fibre connectivity is located in the CERN Computer Centre (Bldg. 513). Hence, a major incident occurring in the Computer Centre, and particularly in the core networking area or one of the fibre rooms, could lead to a major outage of IT services for a prolonged period of time. In order to mitigate against this, CERN planned to implement a Second Network Hub with a new optical fibre infrastructure which would provide backup in case of incident and continuity of IT services.
Low loss redundant links
The fibre paths between the Hub and the IT terminal buildings were requested to be as feasible as possible redundant to the existing paths between the Computer Centre and those buildings. The number of intermediate interconnections along the fibre path was requested to be minimized in order to reduce the signal attenuation over the optical links.
The exceptionally large fibre activities (1500 fibres installed in 48 links, spanning over the CERN Franco-Swiss region) required an intensive coordination effort, including the management of external contractors and the coordination with other CERN groups. Forecasting of the needs and fast adaptability to condition changes were the main challenges faced during the project execution.
Organization of the field activities
Fibre activities often were performed in a challenging environment requiring special procedures, training and tooling. Most of the CERN cable trays are overloaded preventing easy duct installation. In addition, documentation was scarce for those areas where cabling has been operating for many decades.
A brand new optical fibre infrastructure
A new fibre infrastructure was designed in order to connect the Hub to the existing IT buildings across the CERN sites.
Direct links. The main strategic “end points” were connected to the Hub via “direct links” in order to avoid any interconnections along the path and to minimize the signal attenuation: “Entrance D” and “Entrance C” links as they are supposed to connect CERN to the outside world and to carry large volumes of data; Bldg. 513, 376, 212, 874 (CCC) and 887 as these buildings correspond to main geographical fibre distribution star points. This will contribute to provide flexibility to the infrastructure in case of upgrade in the following years; All the other end points were served via existing optical fibre star points
Redundant links. A completely new optical fibre route was deployed between Meyrin and Prevessin, delivering a fully redundant path with respect to the existing links between the two sites. The new optical links in Meyrin were deployed as far as possible from the Computer Centre, guaranteeing an alternative optical path in case of major failures occurring near the Computer Centre.
A tailor-made fibre star point in the HUB
Considering the high number of optical fibre links to terminate in the Hub and due to the need of having a new fibre star point in Prevessin to backup and complement the existing (overloaded) CCC star point, a dedicated space was allocated in the Hub for EN-EL to install 14 optical fibre distribution racks. These multi-user racks will serve all future CERN user needs including the “Second Network Hub” project needs, guaranteeing redundancy to the CCC star point located in building 874.
Work optimization by Geographical Campaigns
Cabling activities sharing the same working areas were grouped by “geographical campaigns” and fitted into the same planning slot. This allowed optimizing the resource distribution, the project cost and the installation time.
Optical fibre installations
Roughly 30 km of new optical fibre ducts and 33 km of optical fibre cables were installed during 2016 and 2017. About 1500 fibres were terminated in 25 different star points spread all over the CERN site. Long blowing operations (e.g. 5 km) are being executed to deploy direct links from the Hub to the end points in Meyrin. CERN staff daily supervised the worksites and the installation progress. A final validation was scheduled at the end of each installation to guarantee the respect of the technical specifications.
Optical fibre star point
14 optical fibre racks were installed in the new optical fibre star point. A false floor under the racks and a cable channel were built to route the optical fibre ducts from the technical gallery to the star point . Patch cords and patch cables with high-performance connectors were used to connect the terminated fibre links in the optical fibre star point to the IT equipment located in the IT room. A fibre guide system was installed for managing those patch cords and patch cables, going from the optical fibre star point to the far end of the rack alley in the IT room.
New optical fibre trenches
Three new trenches were excavated and built on the CERN site for an overall length of 750 m to accommodate new buried optical fibre ducts. A long consolidation activity was also undertaken to repair the existing ducts going from Meyrin to Prevessin in order to establish a completely new and redundant route between the two sites.
The construction of the new Hub was successfully completed on July 2017 in line with the original schedule. A new optical fibre infrastructure was built to connect the Hub to the IT terminal buildings across the CERN site. This will provide low loss links, fully redundant with respect to the links serving the Computer Centre. A new optical fibre star point with 14 multi-user racks was built inside the Hub in order to provide IT with the requested fibre connectivity and, at the same time, to guarantee the redundancy of the CCC star point located in building 874.