What is a Network Operations Center (NOC) and how does it support everyday business? Will the development of the cloud change the approach to the NOC services offered? Can the implementation of automation improve the quality of work? I talk to Krzysztof Korski, Global Operations Manager at Comarch SA, about the functioning of the Network Operations Center.
Malgorzata Zabieglinska-Lupa: Let’s start with the basics, please explain what the Network Operations Center is and what it does?
Krzysztof Korski: The Network Operations Center – NOC (learn more about Network Operations Center here) in short, is where the proper operation of network devices such as routers, switches, servers and firewalls, along with data connections (terrestrial, radio, satellite, cellular, etc.) is supervised. These tasks are performed in 24/7 mode, which means continuity of service at night and during holidays and weekends. NOC is also a single point of contact for clients. This is a convenient solution for them, because they do not have to worry about where or whom to call in the event of irregularities. They need only one emergency number. If a failure or pre-failure condition is detected, NOC engineers take immediate diagnostic and corrective action, and support communication with the customer and subsequent support lines, performing more advanced repairs or changes in device configuration.
The term “NOC” itself contains three key words:
- Network – refers to the hardware infrastructure and telecommunications links that are the responsibility of the NOC maintenance team
- Operations – all activities related to supervision, reacting to incidents, handling requests, ensuring communication with clients, escalations and reporting,
- Center – this word reflects one or several places constituting a “command center” ensuring constant supervision over the progress of repair works, in order to comply with SLAs (agreements with clients describing the agreed level of service quality)
A good example from everyday life is the emergency number 911. In the event of a traffic collision, health problems or a fire at neighbors’, we can call for help and do not have to spend valuable time searching for the correct number. NOC has a similar function, except it is NOC that most often contacts designated parties on the client’s side to report anomalies, detected network security breaches, data transmission problems, unavailability of devices, deterioration of performance indicators (e.g. processor overload) or capacity problems with their components (such as no disk space). A common feature of both these solutions (911 and NOC) is that they are the first line of contact behind which there is a whole team of specialists ready to assist.
Apart from the human factor, what dedicated tools are vital to an NOC, without which you would not be able to operate?
Good, reliable and proven tools are actually only half the requirements for success. The other half consists of people, their experience, competence and ability to act under time pressure. There are various software packages on the market, both free and paid, offering many infrastructure supervision options. A dedicated tool must first be customized to the needs and specificity of a given business. Therefore, the choice of tool should always be preceded by thorough analysis, a verification of processes supported within the organization and of whether a given tool will enable the implementation of tasks in accordance with the applicable processes. The available budget is also an important factor, especially since many software packages require the purchase of support, which allows, for example, updating this software, adding new functionalities, removing errors or adapting the product to the specific requirements of the organization. Many wealthy companies invest lots of money at once, buying powerful tools yet using only a fraction of their functionalities. Companies that have a lot of IT potential have many programmers and experts, which means that they can afford a different approach such as adapting OpenSource tools and expanding them with necessary functionalities and/or tailoring them to meet the individual requirements or limitations resulting from contracts with clients.
What processes should be implemented to maximize the potential of NOC?
The processes supported by NOC should be strictly adjusted to the client’s specifics and business requirements. Experience shows, however, that sometimes you need a specific layer of “common sense”, i.e. good practices as well. That is why many companies take pride in implementing the ITIL methodology, sometimes without realizing that this alone is no guarantee of success. Rather, success is guaranteed through dialog with the client, the identification of their real needs, and then the adaptation of specific processes to these guidelines. The first process that should be implemented in NOC is event management. Thanks to this, NOC engineers are aware of changes that take place in client locations (e.g. server restart or terminal unavailability). Some of these events will be informative (e.g. backup performed correctly), but others will require a service response. To support them, it is necessary to implement the incident management and change management processes. When servicing end-users who submit service requests (for example, requests to reset a password or access a resource, or to deal with a lack of paper in the printer), there is a need to implement the process of managing these, which is called request fulfillment. The level of efficiency we achieve is possible thanks to easy access to the knowledge base, operational procedures and automation.
With growing pressure to do more for less, how can NOC teams raise their game and avoid being overwhelmed by an endless number of reported incidents?
It is possible, but it requires a well-thought-out approach. Automation is a good means of support, but it requires careful supervision. Response templates or checklists streamline our initial diagnosis processes. Up to date service manuals, operating procedures and access to the knowledge base and database of known errors are key to ensuring efficient failure handling and reporting. Training is equally important, both initially and to develop soft skills (such as customer service standards) and technical matters (for example, systems administration). Unfortunately, there will still the occasional flood of incidents and alerts, but good organization and carefully designed mechanisms that automate work can deal with this to a great extent. What remains for people working in the NOC is to make the right decision, and contact the client and subsequent support lines (if the case requires their involvement).
Is there a place for NOC in modern IT organizations? Is it not better to delegate this type of service to an external supplier?
I will answer with a comparison: is it better to build an airplane or to buy a plane ticket to fly to your holiday destination? It all depends on the needs, potential and budget of a given company. Building a thriving NOC is a significant investment in equipment (monitoring servers, information boards/graphic walls, tool configuration, maintenance, etc.), in the premises (requiring protected room, UPS and air conditioning) and in people (certification, a culture of wanting to deepen knowledge and ability to work under pressure). A company whose main business profile is far from IT (such as a chain store, bank or office) could theoretically build its own NOC, but it would probably be unprofitable. Hiring people with the highest level of certification is a considerable expense for the employer. If these employees only had to solve a technical problem occasionally, their cost effectiveness would be negligible. In a partner (outsourcing) model, a company using NOC services provided by an external company does not incur such high costs and at the same time has access to highly qualified engineers. In this way, the outsourcing company provides value by enabling the company to get the results they want. At the same time, the company does not take on the specific costs and risks associated with the provision of this service.
You’ve been working as an NOC manager for several years, during which time you have gained extensive knowledge and experience. What challenges will NOC face in the next five years? What changes are waiting for us?
For many companies, business is becoming more and more global. Large enterprises have headquarters in many countries, often on different continents. This generates many challenges, especially in terms of adapting to the ever-growing requirements from customers, such as providing support in the local language, addressing cultural and legal requirements, and implementing current standards (including tools) and best practices (such as ITIL) in order to operate efficiently in every situation, especially in the event of crisis.
Given that most enterprises already use cloud services or are in the process of migrating to them, why is NOC an indispensable element of any well-functioning business?
Cloud is just a service and technology skillfully hidden behind a nice interface. There are great monitoring solutions available on the market that work in the cloud, but they will not do anything constructive by themselves. People, with their knowledge and experience, are needed. Cloud solutions also have certain consequences related to the security of the data stored there. Are they well protected? Who has access to the data kept there? With cloud solutions becoming ever more common, the importance of NOC is still crucial in terms of security and support for other strategic areas with a similar degree of supervision. This is how Comarch’s Global Operations Center was created. It brings together the teams of the Network Operations Center, Security Operations Center and support teams for the loyalty systems provided by Comarch. Thanks to this solution, we improve internal communication and work more closely to solve complex issues. The benefit visible directly to the customer is the lower cost of obtaining the service, as well as the greater functionality of this solution, which directly translates into faster restoration of IT systems to their normal state. Customers simply expect this. Due to the increasingly sophisticated and advanced technologies used in business, there is no reason to assume that the need to supervise them will disappear.
Finally, I would like to ask you about Comarch GOC and your plans for the next 12 months. What changes can your clients expect?
Comarch Global Operations Center constantly adjusts to the needs of customers, and we do not plan to change this approach. I think that, in the 12-month perspective, the biggest challenge will be to reorganize processes, tools and other resources to be able to offer the highest quality services at a more competitive price. It’s a big challenge. This goal can be achieved by introducing process automation and optimization, and by sharing resources. It should be emphasized that lowering quality to be able to charge a lower price is certainly unacceptable. In Comarch, we maintain the highest quality of services. This philosophy has been working for many years, brings good results and is appreciated by customers who are increasingly aware of the need to implement service improvements constantly. It is natural that the scope of services must reflect current business needs and customer expectations. Based on the experience gained so far, I believe that this will be the key to success in the upcoming years. Thanks to this approach, the term “outsourcer” is slowly being forgotten, which is useful because it doesn’t have the most positive connotations. It is being replaced with the term “technical partner”, which we are delighted to say is how we are currently perceived through the prism of co-creating our clients’ business successes.