Nobody knows what lies beneath New York City

Subterranean cartographers are bringing to light the dark, tangled truths buried under the streets.

Before a single raindrop fell, Alan Leidner knew the waters could rise and throw the city into darkness. On this point, the maps were as clear as a crystal ball. All you had to do was look.

It was 2010, and Leidner was consulting for the government services company Booz Allen Hamilton Inc., contracted by the U.S. Department of Homeland Security to identify potential threats and vulnerabilities in the nation’s critical infrastructure.

Leidner was examining a region that included New York and New Jersey. One day he was thinking about the area’s electrical power grid. He consulted some flood projection maps the Federal Emergency Management Agency had prepared. Then he stared at a map of the grid maintained by Consolidated Edison Inc., the region’s power supplier. And it just jumped out at him: The substation at East 13th Street, on the banks of the East River, was smack in the middle of a flood zone.

Leidner voiced his concerns with utilities, hospitals, and other major facilities. “The reaction was mostly, ‘Eh,’ ” he recalls, as we sit in the Tribeca offices of the Fund for the City of New York, where he directs the nonprofit organization’s Center for Geospatial Innovation.

When Hurricane Sandy arrived in 2012, barreling up the Eastern Seaboard and heading straight for New York, the National Oceanic and Atmospheric Administration projected a massive surge in New York Harbor. “I realized it would hit the flood maps that FEMA produced,” Leidner says, “which meant East 13th Street was about to be flooded.”

He churned out memos, with maps attached, urging the response community to prepare. Con Ed (as the utility is known) workers hastily constructed barriers around the transformers that connected to buried wire ferrying current for blocks. But when the water breached the river wall and spilled across FDR Drive toward the substation, the barriers weren’t enough.

Leidner called up some images on his laptop: a white-hot nova as the transformers exploded; and in the aftermath, an overhead shot of Manhattan, dark below 34th Street (save for a sliver of light in Battery Park City), a blackout that lasted three days. The damage included the shutdown of NYU Langone Medical Center and Bellevue Hospital—their backup generators failing, Leidner notes, because critical components were located in basements, subject to the same East River flooding that swamped the substation.

“We were churning out maps like crazy—something like 3,000 in six weeks. It was a real watershed moment for GIS”.

Leidner believes, fervently, in the power of geospatial data, “interfacing multiple map layers from different sources to come up with valuable intelligence,” as he explains it. In the ’90s, he led the creation of a map of New York City that stands as a pre-Google Earth model of urban cartographic complexity, troves of data integrated to reveal the location of everything from billboards to curbs. What it doesn’t encompass is the subterranean city, the sprawling network of infrastructure and the natural features that surround it.
Leidner is convinced that if such a map had been available before Sandy, as a resource shared and referenced by the multiple players who keep the city running, the precariousness of East 13th Street would’ve been obvious. But, he hopes, by the next major hurricane, planning ahead will be easier. Under his direction, New York is on the verge of completing the world’s most complex underground map—and therefore the most detailed realistic picture of the interlocking systems that make a city work. That, Leidner says, will improve public safety, help officials better manage rapid growth, and usher in the era of “smart” cities, in which sensors and other automated technologies manage the flow of daily urban life.

Because of data from satellites, we can now map the world down to about 6 inches. We’ve almost reached the point Jorge Luis Borges describes in his short story “On Exactitude in Science,” in which cartographers built “a Map of the Empire whose size was that of the Empire, and which coincided point for point with it.”
But the world beneath our feet remains shrouded in darkness. “Light and radio waves don’t go through dirt like they do air,” says George Percivall, chief technical officer for the Open Geospatial Consortium, which is helping to develop global standards for underground mapping. “The next frontier, in both a literal and figurative sense, is underground.”

New York City’s daunting infrastructural labyrinth is like the “Here be dragons” decorating ancient maps. Underneath the 6,000 miles of asphalt and concrete road lie thousands of miles of water, sewer, gas, telecommunications, and electrical infrastructure. And let’s not forget the 500 miles of underground subway tracks or Con Edison’s 100-mile steam delivery system. In its entirety, it’s known to no one. The individual details of the vast underground are hoarded and guarded by the various stakeholders. Con Edison has its electrical map; the Department of Environmental Protection (DEP) keeps track of water and sewer pipes; the Metropolitan Transportation Authority (MTA) could tell you where the transit tunnels are; and so on.

Imagine the city as a living organism, a body consisting of various systems—respiratory, nervous, skeletal—that share the same space and even intertwine. Now imagine surgery performed on that body by a surgeon who knows the location of only one system, who looks at the body and sees only blood vessels or bones. This is the odd condition of New York—a body subject to what, viewed through a wide lens, looks like perpetual triage. Each year, for repairs or to facilitate construction, the streets are sliced open 200,000 times—an average of almost 550 cuts per day, or 30 per street mile every year.

For every job, contractors are required to call in the keepers of this knowledge. Representatives from the relevant utility companies and city agencies are dispatched to sites, where they survey and mark out the location of underground infrastructure with spray paint. Walk just about any block in the city and you’ll see these urban hieroglyphics, the scar tissue that lingers long after the cuts are sealed. “GAS” is one of the more obvious ones, the unambiguity a sign of how dangerous it is to miscalculate and rupture a gas line. Still, mistakes are common and inevitable. Strikes on underground infrastructure cost the city an estimated $300 million every year.

Leidner’s map would let a user zoom through the city’s layers of pipes and wire, asphalt and tunnels, streams and granite to pinpoint a leaking sewer line or corroding gas line as easily as someone in Boise, Idaho, can now swoop down to check out a street view of a real estate listing in the Virgin Islands. To understand his obsession, you have to understand the pull for him of the technology that could make it possible.

Leidner’s career has unfolded concurrently with the rise of geographic information systems. GIS refers to any automated system that allows a user to generate, manipulate, and display geographic or geospatial data. Working in tandem with ubiquitous and freely accessible GPS, GIS is transforming our ability to visualize spatial relations in the physical world.
That map on your phone that pinpoints your location as a blue dot and displays nearby restaurants? That’s GIS. A complex map that allows the National Geospatial-Intelligence Agency (NGA) to visualize, to the centimeter, how a guided missile will behave given current weather conditions? Or one that lets relief workers plot the optimal distribution of medical supplies following a disaster or lets mobile phone providers allocate towers based on usage patterns? Also GIS.

It was in 1985, while working as the information technology director for the DEP, that Leidner first encountered GIS. Although he’d had some experience with paper mapping while doing zoning analysis in the ’70s, he knew next to nothing about computerized maps. What drew him was working with DEP’s hazmat unit, using GIS software to track incidents around the city. Here was a way to visualize emergencies, to get a sense of their impact. “My knowledge grew as I dealt with CAD [computer-aided design] systems, which were revolutionizing the way engineering projects in the city were designed,” he says. “Then I met Wendy, who was working on the water main drafting project.”

Wendy Dorf was then supervising the agency’s project of using GIS software to convert paper maps of the city’s water delivery system: the water mains, every pipe that connected buildings to the system, “and even the 100,000 hydrants,” she says, still marveling at the accomplishment. “Computer mapping was not sophisticated,” she recalls. “But lo and behold, after 10 years we had networked the entire system.” Leidner was awed by the scale of the project. The two geeked out over the possibilities of GIS and the city, solidifying a friendship that continues today.

After mapping the system that brought water to the people, DEP set forth on an equally ambitious project to use GIS to map the system that took it away, the sewers. The most obvious course of action would be to build one on top of the other, to create one map. But that would mean working collaboratively—ensuring that the sewer map was constructed with the same specifications, standards, and formats, anchored to what’s termed “control points” (spots whose location is already known with certainty), so that everything lined up correctly. Unfortunately, “the sewer guys hated the water guys,” Leidner remembers.

Leidner and Dorf were deep enough into GIS to understand that this was a missed opportunity. Why have separate documents? It was like seeing the water and sewer systems through unfocused binoculars. In fact, properly deployed, the technology could create a single accurate map encompassing not merely these two layers but also countless others. It would be called the base map, New York City’s infrastructural ecosystem depicted on one master document containing all the topographical and built features, as well as the water and sewer lines underneath. Mayor Rudolph Giuliani agreed to support the project via the city’s Department of Information Technology and Telecommunications (DOITT). The sewer guys had to get on board.

“It’s good hygiene if you’re a city manager to know where all the critical nodes are in your underground infrastructure”.

After the water and sewers were mapped, the base map expanded into an exercise in photogrammetry. Planes flew across the city, taking aerial photos that were linked to GPS and control points on the ground. Every pixel represented 1 foot of accuracy. Working with city agencies and overseen by Leidner and Dorf, experts at an outside mapping company scanned and digitized the photos and created outlines for the various components, including every curb line and building footprint. The base map was complete by 1999. It was distributed to all city agencies, many of which built their data upon it. “One map to bind them all,” Leidner says, putting it in Tolkien-esque terms.

Suddenly, the city’s many agencies had a way to coordinate their activities. This proved especially important in emergencies. When a water main ruptured, for example, representatives from the Office of Emergency Management could work with the Department of Buildings off the same map to assess the damage and take steps to repair it. Using the base map, the Department of Health identified places where water might pool in its successful fight against the West Nile virus.

“The underground map was always in our sights when we were building the base map itself,” Leidner says. “Because, in essence, the origin of the base map was discovering that water and sewer people were going to build their map layers in separate silos. We knew how valuable getting information about the underground was, how useful it was for operations.”

Their efforts to enlist the missing utilities and go deeper were put on hold when New York was attacked on Sept. 11, 2001. On that day, Leidner, like thousands of others, walked home from work. He had just reached his apartment on the Upper West Side when the call came in. “I never imagined myself playing any role in the disaster,” he says. “But city marshals came and drove me at 100 miles per hour, back downtown to Gramercy Park.” He was tasked with generating maps that could aid the rescue effort. He brought in Sean Ahearn, a GIS expert at Hunter College, along with representatives from Esri, a prominent GIS software company, NGA, and volunteers from GISMO—the winningly nerdy acronym for the GIS Mapping Organization, a professional group founded by Jack Eichenbaum, a longtime agitator for GIS who worked for the Department of Finance.

Using a few images captured with a smartphone, the x, y, and z coordinates of Chicago’s underground are created in 3D.SOURCE: RECONSTRUCT AND UI LABS
Dorf took the job of gathering all the data for utilities around the site. From a war room on a Hudson River pier in Midtown, the deep-infrastructure group pored over plans and renderings of the World Trade Center’s basement layers and overlaid them with utility maps. They mapped outage areas for water, telecommunications, and electricity. “We were churning out maps like crazy—something like 3,000 in six weeks,” Leidner says. “It was a real watershed moment for GIS.”

At one point, someone noticed that the World Trade Center plans showed a 200,000-gallon underground tank of Freon, used for the building’s cooling system. “I remember the hair on the back of my neck standing up,” Leidner says. “Part of our analysis was to locate this sucker in relation to fires we were mapping—using thermal imagery—that continued to burn underground.” If the fires reached the tank, the result could be catastrophic, because Freon, when heated, can transform into phosgene, a toxic gas used in chemical weapons. After careful analysis of hot spots on the pile, they were able to determine that the fires weren’t close enough to the Freon tank to cause an explosion.

Leidner and Dorf assumed that after this, the underground map project would continue to develop. The Freon incident alone showed how useful it was to have a clear idea of what lay underground. But the events of Sept. 11 also put their project in a different light. Con Edison, which by 2001 had been on the cusp of agreeing to initiate a fully digitized, shared map of the electric infrastructure, pulled its support, citing security concerns.

“9/11 giveth, and 9/11 taketh away,” Leidner says with a rueful laugh. “It gave because it created a lot of attention for the uses of GIS and showed how integrating that data for every kind of use was key. But it also reinforced the need for security. And now nobody wanted to part with the data.”

One of the bitter side effects of Sept. 11 is that it demonstrated not only the vulnerability of the country’s critical infrastructure—the part Homeland Security calls the “transportation sector”—but also how its energy could be redirected to become a destructive force. Leidner and Dorf are sympathetic to security concerns. But at the same time, Leidner likes to point out, the spray-painted symbols Con Edison leaves all over the streets of New York form a de facto map of the infrastructure for anyone paying attention. And wouldn’t a complete map that included Con Edison’s infrastructure even possibly help the utility pinpoint security problems, such as revealing critical nodes where the electrical system converges with major parts of other infrastructure, creating ripe targets?

“The whole world would like to do the same thing, and they’d really like to see it happen in New York”
Leidner retired from the city in 2004 and began doing private-sector consulting. Without his enthusiasm or Con Ed’s data, the map languished. Dorf also moved on to private-sector work. Jim McConnell, a commissioner with the city’s Office of Emergency Management, succeeded Leidner and Dorf as the biggest proponent of charting the underground. At the same time, skeletal parts of a future underground map began to emerge. To prepare for the 2004 Republican National Convention at Madison Square Garden, McConnell’s office worked with the MTA and federal agencies to create almost-3D maps of underground transit stations. Although not fully integrated, they could one day be layers of data in a unified underground map.

Six years ago, McConnell, Dorf, and Leidner got together and discussed giving it another try. Leidner thought he had a way to begin the project without the direct involvement of the city, which he hoped would come around as plans developed. He approached the Fund for the City of New York, a Ford Foundation organization tasked with financing innovative projects involving government and nonprofits. The fund’s president, Mary McCormick, had admired the work Leidner’s team had done after Sept. 11 and agreed to provide institutional support for the project. “When Alan came to me with his idea,” she says, “I said yes within two seconds.”

By then, the idea of creating underground maps was catching on around the world. In July 2004 construction workers in Belgium ruptured a massive high-pressure gas pipeline running between France and the Belgian port town of Zeebrugge, on the North Sea coast, killing 15 people and injuring hundreds. Virtually everything within a 1,300-foot radius was destroyed. Many of the bodies were burned beyond recognition and were thrown so far into nearby fields that rescue workers initially had trouble locating them.

Officials determined that the accident resulted from construction workers having incorrect data regarding the pipeline’s location. The Belgian government responded by ordering the creation, over the next three years, of an underground infrastructure map that depicts every asset owned or controlled by more than 300 utilities in Flanders, a region several times larger than New York City, with about three-quarters of the population. The almost 400,000 miles of subterranean cables, pipelines, wires, and conduits could circle Earth 16 times.

Not just anyone can gain access to the Belgian map. A contractor that wishes to dig underground must submit the coordinates of the work area via a computer portal called KLIP. The request goes out to all stakeholders with infrastructure running beneath the area, which are required to turn over their data.

The information is then synthesized and sent to the requester as a single document. “We have a decentralized architecture concept,” says Jef Daems, a KLIP official. “All network utility companies have their master database, and it is only a very small part of the network that is provided to the contractor.” Chicago has embarked on a project similar to the Belgian model, and mapping authorities in Singapore and London are also researching pilot programs.
New York’s existing three layers—the DEP’s water and sewer documents, plus the more recent mapping of subway stations—mean it’s starting its project many steps ahead. “Other than Flanders, I do not believe any other city in the world has gone as far as us,” Leidner says, “even though we haven’t yet prevailed upon private utilities to create maps of their networks.” Much of this future cooperation will likely hinge on Con Edison coming on board and sharing its data. (A Con Edison spokesperson declined to comment on the proposed underground map, beyond saying that “for multiple security reasons, this information is extremely sensitive and confidential.”) Meanwhile, Mayor Bill de Blasio has allowed Leidner’s team to approach city agencies to discuss the project, which is “under active consideration” to receive the full support of the mayor’s office, McConnell says.

As Leidner and his team navigate the political thicket of building the map, they continue to work closely with the Open Geospatial Consortium to develop technical standards for worldwide underground mapping. The effort has attracted pro bono consulting from mapping agencies around the world, including the Ordnance Survey of Great Britain and the Singapore Land Authority.

Whatever form the New York map takes, it will likely have two attributes that will make it more sophisticated than Belgium’s KLIP. First, while Belgium’s is only 2D, New York’s will provide visuals in three dimensions.

As with any paper map, latitude and longitude are the easiest attributes to depict, and the most crucial—you want to know there’s a gas line here as opposed to there before digging—but knowledge of depth provides a far more useful tool. But it also presents a knottier version of the problem all mapping systems face, especially when they combine different data sets into one map: finding control points. Ahearn, the Hunter College GIS mastermind who aided Leidner’s team after Sept. 11, argues that New York should use the city’s existing sewer map, which is already part of the base map, as an anchoring point. Knowledge of depth (the z-axis) is important for sewer officials, because sewage—as the old axiom says—must roll downhill.
New York’s sewer map is one of the most advanced in the world, with precise knowledge of the surface location of every manhole. “If you have two different manhole covers, each has a different x, y, and z location, so you have the invert elevation, the distance down to the pipe for one and the distance down for the top of the other,” Ahearn says. “So if that pipe is straight, you can calculate the x-y-z for any location along the pipe.” On the other hand, he adds, “if it’s curved, you need to know the radius of the curvature.”

Clear as urban mud, right? Now consider the other special attribute of the New York map. The inclusion of depth information will allow it to depict not only the infrastructure but also information about the soil levels that surround it, using data from hundreds of boreholes around the city. “Knowing the type of soil is very important for the behavior of the infrastructure,” says George Deodatis, a civil engineering professor at Columbia. “If the soil is very soft, you might have settlement, and some of the pipelines might start deforming excessively. If you have underground water, you have to start worrying about corrosion. If you have organic matter, there are issues with long-term behavior of pipelines. If you have a gas explosion, a certain type of soil will absorb the blast.”
The three major architects behind the underground map remain hopeful. “We’re confident we’re finally there,” Dorf says. “The whole world would like to do the same thing, and they’d really like to see it happen in New York.”

“It’s good hygiene if you’re a city manager to know where all the critical nodes are in your underground infrastructure,” Leidner adds. “Why they might be vulnerable, the interdependencies and single points of failure—this is all Homeland Security lingo that I learned—and what might cause a cascading effect.” His voice rises a bit, the excitement of a GIS crusader showing through. “You need to know that!”

This post was inspired by an article that got my attention on digg.com and because this is my first post it plays also the role of a test post. Thank you