- Part one -
Change in data flow
A typical day for Paul and his remote colleagues consists of connecting hardware for new customers and tasks like unloading hard drives and solid state drives (SSDs). This does not imply very deep troubleshooting. For example, if a customer loses communication with one of their devices, their support team will check if the communication is working at the physical layer and, if necessary, change the network card and so on to ensure that access to device or platform restored.
In recent years, he has noticed some changes. Server racks of 1U or 2U are being replaced by 8U or 9U units, which support many different boards, including ultra-compact servers. As a result, there are far fewer requests to install individual server networks. There have been other changes in the past 4 or 5 years.
“At Tata, most of the equipment is represented by HP or Dell, and we now use their devices for dedicated servers and cloud protocols. They used to use Sun, but now it is very rare. We used NetApp as a standard for storage and backups, but now I see EMC has also appeared, and lately I've noticed a lot of Hitachi storage devices. In addition, many customers are opting for dedicated backup storage over managed or shared storage.”
Network Operations Center Control Centers
The layout in the NCC (Network Operations Center) section of the premises is much like a regular office, although the large screen and camera through which the UK office communicates with the NCC staff in Chennai, India may come as a surprise. However, they serve as a way to test the network: if the screen goes blank, both offices understand that there is some problem. Here, in fact, there is a first-level support service. The network is monitored from New York and hosting is monitored in Chennai. Therefore, if something serious really happens, in these places located far from each other, they will be the first to know about it.
Promotional video:
George describes the organizational structure of the center: “Since we are a network control center, we receive calls from people who have problems. We support 50 priority customers (all of them are the ones who pay the most for services) and every time they encounter a problem, it really is a priority. Our network provides a shared infrastructure, and a major problem can affect many consumers. In this case, it is necessary that we have the opportunity to inform them in a timely manner. We have an agreement with some customers that we provide them with the latest information every hour, and for some - every 30 minutes. In case of emergencies on the line, we constantly keep them informed while we solve the problem. Round the clock.
How an infrastructure provider works
As an international cable system, service providers around the world face the same challenges: damage to terrestrial cables, which most often occurs on construction sites in less closely monitored areas. These are, of course, the anchors that have lost their trajectory at the bottom of the sea. Plus, don't forget about DDoS attacks, in which systems are attacked and all available bandwidth is filled with traffic. Of course, the team is well equipped to deal with these threats.
“The equipment is set up to track the typical traffic patterns that are expected during a particular period of the day. They can consistently check traffic between 4pm last Thursday and now. If the inspection reveals anything unusual, the equipment can preemptively eliminate the intrusion and reroute traffic with another firewall, which can weed out any intrusion. This is called productive DDoS mitigation. Its other type is reciprocal. In this case, the consumer can tell us: “Oh, I have a threat in the system on this day. You'd better be on the lookout.”Even so, we can filter out as a proactive measure. There is also legitimate activity that we will be notified of, such as Glastonbury (UK Music Festival)so when tickets go on sale, the increased activity is not blocked."
System latency also has to be proactively monitored by clients like Citrix who run virtualization services and cloud applications that are sensitive to significant network latency. The need for speed is appreciated by such a client as Formula 1. Tata Communications manages the networked racing infrastructure for all teams and various broadcasters.
“We are responsible for the entire Formula 1 ecosystem, including the race engineers who are on site and who are also part of the team. We create an entry point at every race venue - set it up, run all the cables and provide all users. We set up various Wi-Fi access points for the guest area and other places. An engineer on site does all the work and can demonstrate that all communications are operational on race day. We monitor it using PRTG (Paessler Router Traffic Grapher - a program designed to monitor network usage - approx. New why) so that we can check the status of the KPIs. We provide support from here, around the clock and seven days a week.
This active customer, who runs regular events throughout the year, means that the asset management team must schedule dates for testing backup systems. When it comes to F1 week, from Tuesday through Monday of next week, these guys will have to keep their hands to themselves and not start testing lines in the datacenter. Even during my excursion, which Paul was leading, he was careful and, pointing to the block of equipment for the F1, did not open the flap so that I could get a closer look at it.
And by the way, if you're curious about how backup systems work, they have 360 batteries per UPS and 8 uninterruptible power supplies. This adds up to over 2,800 batteries, and since they each weigh 32 kg, their total weight is about 96 tons. The service life of the batteries is 10 years, and each of them is individually monitored for temperature, humidity, resistance and other indicators, checked around the clock. When fully loaded, they will be able to keep the data center running for about 8 minutes, which will give a lot of time for the generators to turn on. On the day of my visit, the workload was such that the batteries, if they were turned on, could provide the operation of all systems of the center for a couple of hours.
The center has 6 generators - three for each hall of the data center. Each generator can handle the full load of the center - 1.6 MVA. Each of them produces 1280 kilowatts of energy. In general, it receives 6 MVA - this amount of energy, perhaps, would be enough to provide power to half of the city. There is also a seventh generator in the center, which covers the energy requirement for the building. The room contains about 8000 liters of fuel - enough to survive a day in full conditions. With full combustion of fuel per hour, 220 liters of diesel are consumed, which if this were a car traveling at 96 km / h could take the modest 235 liters per 100 km to a new level - the numbers that make the Humvee look like like a Prius.
Last mile
The final stage - the last few kilometers from the network gateway or NOC to your home - isn't all that impressive, even if you take a quick glance at the end branches of your network infrastructure.
However, there were also changes. Installing new telecom cabinets side-by-side with old green cabinets, Virgin Media and Openreach are establishing DOCSIS and VDSL2 lines, increasing the number of homes and businesses connected to the network.
VDSL2
Inside the new Openreach cabinets for VDSL2 lines is a DSLAM multiplexer (Digital Subscriber Line Access Multiplexer in BT terminology). In the days of ADSL and ADSL2 technologies, DSLAM multiplexers were installed close to local switches, but the use of outdoor cabinets can amplify the signal of the optical cable going to the switch to increase the speed of broadband access for the end user.
DSLAM cabinets are powered separately and connected by connecting pairs to existing outdoor cabinets, such a bundle is a nodal telecommunication cabinet. The copper pair remains intact to the end user, while VDSL2 enables broadband access through the use of conventional outdoor cabinets.
This is an upgrade that cannot be done without the presence of technicians, and the NTE5 panel (network terminal equipment) inside the house must also be modified. Still, it is a step forward that allows ISPs to increase the speed from 38 Mbps to 78 Mbps in millions of homes, bypassing the amount of work required to install FTTH.
DOCSIS
This is a completely different technology of Virgin Media's hybrid optical-coaxial network, which allows to provide the home consumer with speeds up to 200 Mbps and up to 300 Mbps for enterprises. Although the technologies to achieve this speed are based on DOCSIS 3 (a coaxial data transmission standard) rather than VDSL2, there are some parallels here. Virgin Media runs fiber-optic lines to street cabinets, then uses copper coax for broadband and TV (still twisted pair for telephony).
It's worth noting that DOCSIS 3.0 is the most common last mile in the US, with 55 million of the 90 million fixed broadband lines using coaxial cable. In second place is ADSL - 20 million, followed by FTTP - 10 million. VDSL2 technology is hardly used in the United States, but it is occasionally found in some urban areas.
DOCSIS 3 still has a speed reserve that will allow cable providers to increase the speed to 400, 500 or 600 Mbps if necessary - and after that DOCSIS 3.1 will appear, which is already waiting in the wings.
When using the DOCSIS 3.1 standard, the incoming speed exceeds 10 Gbps, and the outgoing speed reaches 1 Gbps. These capacities can be achieved using the method of quadrature amplitude modulation, which is also used over short distances in submarine cables. However, on land, QAMs of a higher order - 4096QAMs - were obtained using the digital modulation multiplexing with orthogonal frequency division multiplexing (OFDM) scheme, where, as in DWDM, the signal is divided into several subcarriers transmitted at different frequencies in a limited spectrum. ODFM is also used in ADSL / VDSL and G.fast.
Last 100 meters
While FTTC and DOCSIS have dominated the UK wired Internet access market for the past few years, it would be a big neglect not to mention the other side of the last mile (or last 100m) problem: mobile devices and wireless.
More capabilities for the management and deployment of mobile networks are expected soon, but for now let's just take a look at Wi-Fi, which is basically an extension to FTTC and DOCSIS. Case in point: Recently deployed and nearly complete coverage of urban areas with Wi-Fi hotspots.
At first it was just a few daring cafes and bars, but then BT turned subscriber routers into open access points, calling it "BT Fon". Now it has turned into a game of large infrastructure companies - Wi-Fi network in the London Underground or Virgin's interesting "smart sidewalk" project in Chesham, Buckinghamshire
For this project, Virgin Media simply placed the access points under the manhole covers, which are made of a special radio-transparent composite. Virgin has many lines and nodes across Britain, so why not add multiple Wi-Fi hotspots to share with people?
In a conversation with Simon Clement, senior technologist at Virgin Media, it appears that implementing a smart pavement at first seemed more difficult than it actually was.
“Previously, we faced difficulties in interacting with local authorities, but this time it did not happen,” says Clement. services for the population and understand what work needs to be done to implement these services"
Most of the difficulties arise on their own or are related to regulations.
“The main challenge is to think outside the box. For example, standard wireless access projects involve installing radio points as high as administrative regulations allow, and these points operate at a maximum power level that is limited by the same regulation. We tried to install access points underground so that they work at the power of simple home Wi-Fi"
“We had to take a lot of risks during the project. As with all innovative projects, a preliminary risk assessment is relevant as long as the scope of work remains the same. In practice, this happens very rarely, and we are forced to regularly perform dynamic risk assessments. There are key principles that we try to adhere to, especially when working with wireless access. We always adhere to the limits of the EIRP (Effective Isotropic Radiated Power) standard and always use safe working practices when applied to radio. When dealing with radio emission, it is better to be a conservative."
Back to the future of cable Internet
Next on the horizon for Openreach's POTS network is G.fast, which can best be described as an FTTdp (Fiber to Distribution Point) configuration. Again, this is an adapter from fiber to copper cable, but the DSLAM will be placed even closer to the end user, above the telegraph poles and underground, and on the last tens of meters of the cable there will be the usual copper twisted pair.
The idea is to place the fiber as close to the customer as possible while minimizing the length of the copper cable, which theoretically allows connection speeds of 500 to 800 Mbps. G.fast operates over a much wider frequency range than VDSL2, so cable length has a greater impact on network performance. However, some doubt that in this situation BT Openreach will optimize the speed, since, due to the high cost, they will have to go back to the telecommunication node cabinet and sacrifice the speed to provide such services: it will drop to 300 Mbps.
There is also FTTH. Openreach initially deferred FTTH - they developed a better (read: cheaper) transmission method, but recently announced their "ambition" to begin large-scale FTTH deployment. FTTC or FTTdp technologies are most likely to be a short-term and interim solution for many users who use cable providers, which in turn are Openreach's wholesale customers.
On the other hand, there is no reason to believe that Virgin Media is going to rest on coaxial laurels: while its rival telecom giant is considering its moves, Virgin is delivering consistent FTTH services to 250,000 users and aims to reach 500,000 this year. The Lightning project, which will connect four million more homes and offices to Virgin's network over the next few years, includes one million new FTTH connections.
Virgin is currently using RFOG (Radio Frequency Over Fiberglass) technology and thus the ability to use standard coaxial routers and TiVo, but the significant FTTH clout in the UK gives the company several additional options in the future as demand for broadband user access increases.
The past few years have also been favorable for small and independent players like Hyperoptic and Gigaclear, which are launching their own fiber networks. Their coverage is still extremely limited to a couple of thousand residential buildings in the city center (Hyperoptic) and rural settlements (Gigaclear), but the growth of competition and investment in infrastructure never goes bad.
That's the story
That's it: the next time you watch a YouTube video, you will know in detail how it moves from the cloud server to your computer. It may seem very easy - especially on your part - but now you know the truth: everything runs on deadly 4,000 volt cables, 96 tons of batteries, thousands of liters of diesel, millions of miles of last mile cables, and excess in excess.
The system itself will also get bigger and crazier. Smart homes, wearable electronics and TV with movies on demand will need more range, more reliability, and more brains in flasks. It's good to live in our time.
Bob Dormon began his technological odyssey as a teenager, working at GSHQ, however, due to his passion for music, he went to master recording in London. For over twelve years he has been a regular contributor to music and Mac magazines. Fascinated by the relationship between man and technology, he became a full-fledged journalist, and for over six years was a member of The Register's editorial team. Bob lives in London and has an obscene amount of gadgets, guitars and vintage MIDI synthesizers.
Bob Dormon
The translation was carried out by the NewWhat project.
- Part one -