• Root cause of January outage identified
• Capacity and resilience improvements underway
• Analysys Mason appointed to undertake independent review
Root cause of January 27 outage identified
The cause of the recent issues with the XT Network South of Taupo is established and is due to traffic surges in the network overloading the Radio Network Controller in Christchurch.
During the outage on 27 January, the traffic surge was caused by thousands of users suddenly re-registering after a separate network routing fault took down some cell sites.
Capacity, resilience and coverage improvements underway
Improving the quality and reliability of service on the XT Network is our absolute number one priority and the entire team is focussed on this.
Since January 27, Telecom and its technology partner Alcatel Lucent have been working 24/7 to improve capacity and resilience. The programme of work involves two key elements:
• The addition of processing capacity and surge protection measures in the Radio Network Controller to prevent surge issues from one part of the network affecting customers in other parts, along with adding further Radio Network Controllers.
• Ongoing improvements to coverage and capacity, including new cell sites, the addition of signal amplification technology, and more fibre optic backhaul
The additional processing power and surge protection is helping to protect overall service, but we have seen localised problems in recent days caused by local traffic peaks.
We successfully migrated the first four cell sites to a new Radio Network Controller on Tuesday and last night we migrated a further 16 sites between 1.00am and 3.00am.
Some of those sites showed problems and service degradation once the traffic load began to increase in the morning.
We have subsequently rolled back the migrations whilst we do urgent engineering work to understand the cause of this degradation. All sites were fully-restored by 2.30pm.
Our plan is to install two new Radio Network Controllers by mid March.
Analysys Mason appointed to undertake independent review
Telecom has today appointed Analysys Mason to conduct its independent review of the XT mobile network.
The Analysys Mason Group (AMG) is a highly respected UK-based international consultancy specialising in telecommunications, technology and media and bring world’s best practice around design, build and operations of mobile networks.
AMG are bringing a multi-disciplined team of up to seven specialists who have in depth knowledge of the design, planning and operation of 3G and IP networks from a range of technology suppliers. Their consultants were hand-picked for their knowledge of radio, core, and transport networks and their understanding of operational processes.
AMG are often called on to undertake similar in-life network reviews for a range of clients.
The review will commence next week and cover the design, build and operation of the XT mobile network. It is expected to be complete within eight weeks.
During the review Analysys Mason will be engaging with Telecom staff, customers, Alcatel-Lucent and other key stakeholders.
Alcatel Lucent comment
Rajeev Singh-Molares, President Asia Pacific of Alcatel-Lucent, said today:
“Resolving the XT Network problems in New Zealand is the single most important operational matter in the world for Alcatel-Lucent right now, our entire Executive and Senior Engineering team, including Global Chief Executive Ben Verwaayen are involved.
“We apologise to the customers of Telecom New Zealand. The full resources of our global business are available to address these problems – we will do whatever it takes.”