Normally LANs are used by a singl

e organization at best, but Wellington's 13-square-mile LAN comprises many of the city's businesses. The city council garnered a UNESCO Digital Access Award in recognition of its achievement in installing the 1,000 Mbps network.

Citylink's genesis dates back to the late 1980s, when Richard Naylor, IT manager for the city council, realized certain areas of Wellington were susceptible to power outages even at times when there was plenty of power in other parts of the city. A plant on one side of the city might suffer a shortage, while another remained at full power.

"We needed to balance the loads by connecting them," Naylor says. So he ran a fiber optic cable between the plants, allowing them to compensate by sharing power when one was hit by a shortage.

Naylor saw his distributed energy and plan work, and realized he could translate the concept to the Internet. By 1996, Naylor and his small team could be seen at night hanging fiber optic cables alongside the trolley cables that are strung over much of downtown. He soon took the idea of wiring up Wellington to the private sector and became the founder and CTO of Citylink. Today, Citylink is downtown Wellington's principal Internet backbone.

Inside Citylink: Fast Ethernet
Citylink runs at Fast Ethernet speeds of up to 1,000Mbps, about 650 times faster than a T1 line. Considering that many U.S. organizations use T1 lines to connect to faster Internet backbone providers, Citylink is offering speeds generally unmatched here. Of course, traffic slows as it leaves Citylink's network and moves onto another. Citylink has to connect to the New Zealand telcos, such as Xtra, to go to other parts of New Zealand, and through international telcos, like AT&T, to go abroad. New Zealand has a fiber optic pipe that crosses the Pacific Ocean, which maintains generally high bandwidth along the way.

A Wellington business within the network can plug in its own LAN--a LAN inside a LAN. By using Ethernet, the network distributes the data from node to node and doesn't need to route back to a service provider every time. In essence, Citylink now looks a lot more like a utility grid than a telecommunications backbone. "Telcos look like a big mainframe; we look like a big distributed network," Naylor says.

A mesh network looks much like a fishing net, with each knot in the net representing a building connected to the network; each building has many connections to other knots or buildings. If one or more connections are cut, most or all of the knots are still connected to the network.

"A traditional telco network is a looped string (a ring) with several knots on it. Cut the string in one place and all the knots are still connected," says Neil Dewit, the managing director of Citylink "Cut the string in another place and knots (buildings) fall on the floor."

That design has also allowed Citylink to offer high bandwidth for a relatively modest cash outlay. Because of the local area, unrouted nature of the network, Citylink can use inexpensive Cisco 3524 switches instead of more costly T1 routers. "We never had the luxury of spending lots of money," says Naylor. "We needed to be able to make do with less."

To support this, the company doesn't use bridges. It's a normal LAN with client-owned routers at the edge. Clients implement their own firewall protection. It connects to the Internet through Internet exchanges from ISPs or client leased telco WAN connections.

Build your own router to cut costs
As another cost-cutting measure, Citylink uses a generic computer running Debian Linux and SMTP management software, as well as a number of other open-source tools: NetSaint Network Monitor, NocMonitor, MRTG, and Cricket. And the company builds its own routers, rather than dropping the money on hardware. Naylor says a comparable Cisco router would cost him $25,000 NZ or roughly $11,500 U.S.; Citylink builds its own routers for $2,500.

The money Citylink saves on devices it spends on telco-grade SMF28 single-mode fiber from Corning, which it continues to hang alongside the trolley cables.

Wellington has become a model for other Pacific Rim municipalities. "We've had cities like Singapore and Melbourne and others come and ask how we did it, especially so cheap," says Neil De Wit, the managing director of Citylink.

QoS: No worries
Many IT departments say that prioritizing packets is vital if you want to run applications and send important files over the Internet. Because of Citylink's sheer speed and capacity, De Wit says adding quality of service (QoS) features isn't necessary. "QoS is a problem for others because they only have so much space in the pipe," he says. "We can fit all the traffic we want onto our Ethernet, so why do we need to worry about prioritizing?" Also, because of the generous capacity, DeWit says data collisions, which are often a concern on LANs, aren't such an issue with Citylink.

Citylink itself uses distributed Cisco 3524, 3508, and 2912 switches. The company builds and customizes its own routers and provides D-Link DI-701 Residential Gateways for customer premises.

With Wellington's population of only 166,000 (excluding the suburbs), severe bottlenecks are less of a concern than in larger urban areas.

For Wellington's businesses, Citylink provides some unique opportunities. "You could say, to some extent, it has become a test bed for multimedia applications," says Chris O'Connell, CEO of The Bridge, a multimedia developer and advertising agency in Wellington.

With 100 Mbps of capacity, businesses can easily implement video conferencing and voice over IP (VoIP). Those applications get limited use in the U.S. because of network latency; for video you need less than 50 milliseconds of latency, and for VoIP, less than 200. A typical national network, without last mile issues and an excessive number of hops, might see latency of about 50 milliseconds, though that's rarely guaranteed. Citylink's latency is generally around 4 milliseconds.

The high-speed environment also allows Wellington's corporate IT departments to stay current with technology developments despite being half a day away from the U.S. "A recent 90-plus MB file from Apple took two minutes to download from servers in Cupertino. That lets us keep our machines up-to-date very easily without having to wait for CD-ROM updates to reach New Zealand," says O'Connell.

Naylor, meanwhile, is looking forward to the next generation of Ethernet, which will deliver 10 Gbps of bandwidth. That means his network will be able to offer service up to five times the speed of the Abilene network that runs Internet 2 applications in the U.S. "We're going to be able to do almost anything we can imagine through TCP/IP," he says.

David Lipschultz is a freelance journalist living in Aspen, Colo.