Why Larry Smarr Is Pioneering Collaborative Innovation

In one laboratory, researchers are developing programs and computer visualizations that can map the complex urban boundaries in India. Nearby, the cofounder of the University of Illinois at Chicago’s Electronic Visualization Laboratory is working on high-speed networks and machine learning computing. On the first floor, 78-year-old Don Norman, author of The Design of Everyday Things, presides over a design lab. In one studio, an archaeologist is creating ultra-high-resolution 3D virtual reproductions of world cultural heritage sites that are under threat of destruction. Scientists huddle with first responders to combine readings from hundreds of meteorological stations to predict the movement of wildfires. In a corner office, two graduate students hack out code that enables more effective visualization of people’s internal organs to help pre-surgery planning.

This beehive of activity isn’t taking place on Google’s vast campus in Northern California, or in a futuristic corporate installation in Bangalore, or in a coworking space in Brooklyn. Rather, it’s all happening at the California Institute for Telecommunications and Information Technology, which is housed at two University of California campuses in the southern part of the state. And instead of being the brainchild of a 20-something Stanford computer science dropout, Calit2, as it is commonly known, is the creation of 68-year-old Midwestern physicist Larry Smarr. Smarr may not be a household name. For four decades, he has been among the world’s most important synthesizers of innovation — as a researcher on his own and as a collaborator in the discovery and commercialization of new technologies as varied as Web browsers and personalized medicine.

Growing up in Missouri in the 1950s, Larry Smarr would lie on his back in the clover fields, observing cloud dynamics. He captured fireflies in jars, taught himself astronomy from a college textbook, and used old car parts to build a Tesla coil, which knocked out the neighbors’ radio reception. When adults asked what he wanted to do when he grew up, Smarr’s response was highly sophisticated: He wanted to build an institute where the brightest people from all disciplines could be as creative and successful as possible.

What sounded like a lonely farm kid’s fantasy is a pretty apt description of Smarr’s career. Smarr is indeed director of a research hub where leaders in diverse fields of study collaborate across disciplinary boundaries. At Calit2, artists and designers mingle with mathematicians, biologists, and physicists to tackle big issues such as climate change, sustainable energy, and the mysteries of the microbiome.

Larry Smarr has never graced the front of Fortune or Forbes, though he did once appear on the cover of R&D magazine. His 1975 astrophysics doctoral dissertation included seminal research on black holes, bringing numerical rigor to Einstein’s theory of general relativity. In the 1980s, he collaborated with the nuclear weapons laboratories to bring the power of supercomputers to the nation’s academic researchers and corporations. He pushed for the transition of the Arpanet, an early data-sharing network funded by the Department of Defense, to the global Internet of today. Mossaic, the browser that first popularized the World Wide Web was developed on his watch at the National Center for Supercomputing Applications (NCSA), which Smarr founded in 1985 at the University of Illinois at Urbana-Champaign.

At Calit2, which Smarr founded in 2000, he is the principal investigator on the Pacific Research Platform, a secure high-capacity “data-centric freeway system” that will allow collaborators to share data up to 1,000 times faster than is possible on the current Internet. He collaborates on a program to create reference genome and microbiome catalogs for understanding the myriad microorganisms that share our bodies. As director, he is also involved in a host of other projects, such as building truly secure communications networks and sharing cryo-electron microscope images across the country.

Smarr’s ambition now is to remake the university so it is based on interdisciplinary collaboration. “If you look at any university, the reward structure is entirely based on single-warrior combat,” Smarr says. “But if you want to attack the real-world problems that we face, it’s a team sport, with specialists in many different fields. We’ve had a century of reductionism, but at some point you need to put all of that knowledge back together in a synthesis.”

Smarr traces his interdisciplinary approach to his background in astrophysics, which draws upon highly specialized disparate scientific fields such as electromagnetism, gravity, nuclear physics, atomic physics, hydrodynamics, particle dynamics, and relativity. “As an astrophysicist, you have to put together a model, from the extremely limited amount of information you get from observation, to create a system that can explain the observed data,” he says.

Farm to Tables

The late Jay Forrester, inventor of system dynamics, often said that growing up on an agricultural operation — in his case, a cattle ranch in Nebraska — was great preparation for a career modeling complex dynamic systems, because it involved observing stocks and flows, feedback loops, and the often unintended consequences of one’s actions.

Smarr’s family owned a commercial floral nursery in Columbia, Mo. They grew all their own vegetables, kept honeybees, and raised chickens. His grandfather had an eighth-grade education and rode to school on a horse with a rifle and a sidearm. But he was also a self-taught inventor and settled in Columbia so his children could walk to the university there.

“I attribute both my affinity to Larry and the fact that he’s always creative to his coming from the Midwest,” says Anne Petersen, a professor at the Center for Human Growth and Development at the University of Michigan and a former second-in-command at the National Science Foundation (NSF). “We all have this sense of needing to prove ourselves and contribute. He grew up with the belief that you can really do anything you want — you just have to work hard.”

As a child, Smarr helped build radios with his grandfather and telescopes with lenses purchased as surplus from the Edmund Scientific catalog. His father enlisted the entire family in building elaborate ponds with wooden piers and swimming platforms that are still in use 60 years later. Smarr created a big book of information, packed with conversion tables for measurements, a list of the kings of Israel, names of George Washington’s ancestors, directions for how to calculate the angle of a roof. He printed copies on an old press he found in the basement.

“I had this crazy, eclectic education because of stuff sitting around,” Smarr says. “We would never take things somewhere to get fixed; you would take them apart and fix them. You learn, when something needs doing, you just figure out how to do it. Today, when people say, ‘Larry, how do you know the future?’ I say, ‘You just look around, see what needs to be created, and you create it.’”

By his high school junior year at the University of Missouri Laboratory School, he was attending calculus courses at the university itself. After completing the undergraduate math and physics curriculum as a sophomore, he spent his last two years in college taking graduate-level classes. “Just for practice, they decided I could take the written Ph.D. exam, and I got the highest score ever while still a senior, so they decided they had to give me a master’s degree,” Smarr says.

He entered the Ph.D. program in astrophysics at Stanford University in 1970, just as Vietnam War protests were erupting. “There was tear gas on the campus every night, helicopters with bright lights, cops in full pursuit into the dorms, riots — and this was my first time away from home,” Smarr says. One night, Stalinist/Maoist history professor Bruce Franklin instigated a group to march to the computer center and burn it down. Smarr manned the barricades. “I organized my guys and said, ‘This is it.’ We literally formed a human chain, fingertip to fingertip, amid this endless group of people yelling and screaming at us, so we were all using our bodies to protect science. And this all happened under an eclipse of the moon.”

Smarr shared the protestors’ antiwar sentiment, but not their anti-science stance. He was quickly becoming one of the world’s top researchers in the then new concept of black holes. The term had been coined in 1968 to describe a region of space-time where gravitational effects were so powerful that nothing, not even light, could escape from inside it. Smarr was challenged by his Ph.D. adviser, Bryce Dewitt, an eminent scholar on the unification of general relativity and quantum theory, to use a supercomputer to solve Einstein’s equations. “I set out to create a computer program to do that, and that was my Ph.D. thesis,” Smarr recalls.

In his 1975 thesis, “The Structure of General Relativity with a Numerical Illustration: The Collision of Two Black Holes,” Smarr demonstrated that supercomputers could solve Einstein’s equations for what would happen when two black holes collided head-on: They would merge and produce gravitational radiation. In February 2016, when the Laser Interferometer Gravitational-Wave Observatory announced that a signal from gravitational waves had been discovered emanating from the collision and merger of two massive black holes more than a billion light-years away, Smarr’s numerical relativity was a critical factor in the discovery.

“Larry was just burning with passion about what he was doing with numerical relativity,” recalls Michael Norman, director of the San Diego Supercomputer Center and a longtime colleague. “He was absolutely the leader in taking Einstein’s gobbledygook equations and putting them on a computer so you could understand what was going on…. I think it’s accurate to say Larry is the father of numerical relativity.”

Computer Liberation

Back in the 1970s, the fastest computers available at top universities could take half a day to run Smarr’s math problems. In those days, and well into the 1990s, the supercomputers designed by the pioneering Seymour Cray were the fastest, most powerful data processors on Earth. But Cray computers could be found only at the two national laboratories in the United States that conducted classified work on the design of nuclear weapons: Lawrence Livermore and Los Alamos.

Although he had no interest in nuclear weapons, Smarr obtained a top-secret security clearance and an internship at Lawrence Livermore — just so he could use the Cray. Because he could get Cray time only when the nuclear weapons scientists finished their work, Smarr often pulled all-nighters and once collapsed from sleep deprivation and dehydration. In 1980, he became a summer scientific researcher at the Max Planck Institute in Munich. The Planck had the first Cray in Europe. “I was hit by the irony and the outrage of having to go to Western Europe to use an American-made computer in an open environment,” Smarr says.

In response, Smarr tapped out a manifesto on his typewriter, “The Supercomputer Famine in American Universities,” and distributed it like a piece of samizdat, as Xerox copies. Smarr wrote that supercomputers, which could be used to design nuclear weapons or model the collision of black holes, could also be used for nearly anything else — if only the National Science Foundation would support them. Next, he approached professors across the breadth of academic disciplines, asking each to write a paragraph about how supercomputers could enhance research in their fields.

At the time, Smarr was an untenured junior faculty member at the University of Illinois, but the task was tailor-made for his autodidactic polymath personality. Before approaching the senior professors whose support was critical, he gave himself a crash course in each of their disciplines, so he could speak the language. Despite strong opposition led by Peter Lax, a Hungarian-born mathematician and Manhattan Project alumnus, the NSF responded by establishing the Office of Scientific Computing in 1984, and chartered four supercomputer centers — at Cornell University, the University of Illinois, Princeton University, and the University of California at San Diego. (A fifth, at Carnegie Mellon in Pittsburgh, was added later.) Under Smarr’s leadership, the NCSA developed a vast range of software applications and protocols that were distributed free of charge.

“The NSF handed out money to buy a supercomputer and operate it, but Larry created the intellectual center that allowed him to recruit me and others to cover the waterfront of applications,” says Norman. Researchers were given a great amount of freedom without specific direction. “We just did our stuff and innovation came out,” Norman adds. For example, a group of researchers working on software development ultimately produced Mosaic. Said Norman: “There was no intent to create a Web browser, but there was an intent to create a place where the unexpected could happen.”

After freeing supercomputers from the national labs, Smarr turned his attention to the Arpanet, which had been used exclusively by scientists at those labs and a handful of academic researchers since its creation in 1972. Vice President Al Gore later took heat for claiming a crucial role, but as a senator he did in fact convene a September 1986 meeting in which he proposed building a fiber-optic network to link the NSF’s supercomputers. Leading scientists, Smarr prominent among them, urged the NSF to use the TCP/IP communications protocol of the Arpanet for NSF Net, which became the Internet.

Seemingly overnight in the late 1980s, the NCSA became one of the most frequently hit sites on the nascent Internet. The traffic was all dot-mil and dot-edu then; Smarr says he didn’t see a dot-com Web address until 1990. “This came at a moment where we as physicists had come out of an America that put all its resources into war,” he says. “I don’t think people understand how the NSF decided to take all this technology and make it available. The corporate world thinks they invented it? Well, forget it.”

But Smarr was happy to invite top corporations to the NCSA, and he charged them US$1 million a year to use its Cray. Owning a Cray became a mark of distinction for companies as diverse as Eli Lilly and the Ford Motor Company. Even Apple bought a Cray.

As the number of applications mushroomed, the NCSA took the lead in developing software that provided visual models of the immense data sets created. Here a Mac or PC was out of its depth, but a tiny startup, Silicon Graphics Inc. (SGI), stepped up with powerful desktop machines that could manipulate three-dimensional visual models in real time. In 1995, Smarr replaced NCSA’s Cray with SGI servers and became one of SGI’s largest customers until filmmaker George Lucas discovered the power of digital animation.

“Every once in a while, a new mode of discovery comes along,” says Edward Lazowska, a computer scientist and founding director of the University of Washington eScience Institute. First, he says, there was observation, then experimentation, abstraction, and modeling. “What the supercomputer program did was to firmly establish computational science across all disciplines. This was an incredible vision that Larry was largely responsible for.”

Second Act

In time, Smarr tired of the administrative duties associated with the NCSA directorship and longed to get back to research and collaboration. So when a call came from the University of California at San Diego, he was receptive. But the offer wasn’t quite what he had in mind.

In 1999, at the height of the Internet bubble, California governor Gray Davis devised a competition among the University of California campuses. The state would give $100 million to three centers, each of which had to provide a 2:1 match to the state grant with non-state fundraising. UC San Diego (UCSD) joined with UC Irvine to form Calit2, in a 70/30 partnership, and Irwin Jacobs, one of two former UCSD professors who had cofounded Qualcomm, convinced Qualcomm to put up $15 million in initial funding. (Both Qualcomm and the Jacobs family have played crucial roles in San Diego’s emergence as a major technology center.)

Robert Conn, dean of UCSD’s Jacobs School of Engineering at the time, asked Smarr to run the new institute. After Conn raised most of the funds, and UCSD offered a position to Smarr’s wife, Janet (as a distinguished professor in the department of theatre and dance, specializing in Renaissance culture and drama), Smarr agreed. In 2000, he became head of Calit2 and was given $250 million to spend.

Smarr brought with him certain lessons from the University of Illinois, not only from the NCSA, but also from the Beckman Institute for Advanced Science and Technology. Founded in 1989, the Beckman pursues research in nanoscale structures and processes, biological intelligence, imaging science, and human–computer interaction. Although he never held an administrative position at the institute, Smarr was heavily involved in picking its projects and talent. However, he felt that the Beckman’s practice of luring faculty stars from their departments led to unnecessary internecine conflict. In his new position, he didn’t want to attract UCSD stars to live at Calit2, just to bring their best work there. Everyone would compete for lab space, which was intentionally kept fluid and ad hoc. Real estate occupied by nanotechnology could quickly be turned over to communications, or to something unexpected. Smarr wanted to spend his time looking for opportunities and weaving together interdisciplinary teams.

“The heart of the matter is that no one was compelled to work with us, and we were not compelled to work with anyone,” says Ramesh Rao, director of the UCSD division of Calit2 since its inception. “A lot of it was staking out some ground, and people self-selected, people who had been successful, who had established their credentials and were looking for something new and different.”

The result is that in its 17 years, Calit2 has evolved into what Pradeep Khosla, UCSD’s chancellor, calls a “safe space where people won’t prejudge the outcome.” And that has made it a hotbed for innovation and collaboration.

When IBM scientists developed a new type of processor based on the human brain, they offered it early to Calit2 for testing and application development. “It’s really a sort of glue that allows projects that span academic disciplines, in a place where the project wants to be,” says Dharmendra Modha, a UCSD alumnus who is lead researcher of the Cognitive Computing group at IBM. “Larry’s touch, which magnifies and multiplies anything, makes it more valuable; it’s like good magic.”

The Big Pixel Initiative neatly illustrates the kinds of collaborations fostered by Calit2. It began with Albert Yu-Min Lin, an expert in the field of technology-enabled exploration. Calit2’s UCSD Division (known as the Qualcomm Institute) and UCSD’s School of Global Policy and Strategy partnered with the DigitalGlobe Foundation to grow a living laboratory related to everything spatial. Sponsored projects include detecting the boundaries of urban areas in India, tracking mining activity and the spatial dynamics of forest cover loss, and illuminating the human element in coral reef management.

Smarr dangles the lure of freedom to recruit leaders to Calit2. If you can raise the grant money, you can do almost anything at Calit2. Stars in their field take pay cuts to work there. “He stole me from Duke University, where I was obscenely paid,” says Stojan Radic, a professor of electrical and computer engineering and head of Calit2’s photonics lab. With funding from DARPA, the Defense Advanced Research Projects Agency, Radic is developing a next-generation wireless computation-free platform, which his group is currently reducing from a table full of custom electronics to a single chip. “We have the physics to prove it works, the engineering to put it together, and the test bedding and systems to build it out and, God willing, reach commercialization,” Radic says. “If I’d stayed at a private university like Duke, I could never have done this.”

Smarr doesn’t dictate, even benevolently; however, he expects people to produce and to share. Some projects sound fanciful, but many have direct relevance to industry, and 140 companies, from tiny startups to Qualcomm, pay $100 an hour to use the nanofabrication facility at Calit2’s Qualcomm Institute. The architecture of Calit2, down to the layout of power cords and communications cables, is intentionally fluid. A lab that doesn’t pull its weight can be replaced very rapidly.

“It’s magnificent,” says Vint Cerf, one of the fathers of the Internet and co-inventor of TCP/IP. “When he created the labs in Calit2, there were reconfigurable clean rooms of extremely high quality. He made a rule: ‘Dear faculty, if you want access, you have to demonstrate to me that you are collaborating with somebody in an interdisciplinary way. You don’t own this space; if you stop collaborating, you leave.’ He creates these incentives that make people want to collaborate and he supports them. This guy wields policy like a laser.”

The Quantified Self

Midwestern friends teased Smarr that he went native when he moved to San Diego. Off came the beard. He bought a home with a wine cellar and a sweeping ocean view. His wife got a Prius, and he drives a Tesla. He started eating healthfully and working out, and quickly shed more than 30 pounds.

In retrospect, Smarr says, maybe the weight came off too easily. He developed intestinal distress, which his physician said was caused by diverticulitis, a common inflammation or infection in small pouches in the digestive tract. But Smarr’s symptoms worsened. And because he didn’t believe the diagnosis, he switched doctors and began gathering data on himself. He collected stool samples and sent them off for analysis and submitted to multiple CT scans. He became famous — or infamous — as the most quantified human being on the planet. A 2013 BBC special, in which Smarr cheerfully showed a reporter the poop in plastic bags in his freezer, cemented his notoriety.

With a battery of supercomputers at his disposal, along with the world’s most sophisticated visualization technology, Smarr created 3D models of his own intestinal tract. When I visited Calit2, I toured his much-magnified colon on the Cinerama-sized, gazillion-pixel wall display, and donned a headset for an even more immersive view in the virtual reality cave. It brought to mind the 1966 movie Fantastic Voyage, in which scientists are shrunk to enter a human body — minus Raquel Welch.

But it wasn’t sci-fi; the tools and tests showed that Smarr had Crohn’s disease, an autoimmune condition that causes severe inflammation of the colon. His C-reactive protein (CRP), a blood test marker for inflammation in the body, was off-the-charts high, and the makeup of his microbiome, the sum of bacteria and viruses that live in the gut, was full of irregularities. Smarr turned himself into a laboratory animal, experimenting with diet and other treatments, collecting data, and publishing his results.

After some initial ridicule from the established gastrointestinal community, other scientists began using Smarr’s data, often in comparison with the American Gut Project, a huge crowdsourced project that collects fecal specimens from the population at large.

“Now there are all these people writing papers, where before it was just me, this lonely guy collecting stool samples,” says Smarr. “But I knew I could drive the science. I was so tired of people not understanding this is a scientific revolution that I decided that if I had to endure a few years of public humiliation, it was worth it.”

Smarr tried to get his inflammation under control, and succeeded to a limited degree. But in the fall of 2016, a blood test showed his CRP reading again at a high level, and a subsequent CT scan showed a significant restriction in his colon. Smarr reluctantly scheduled surgery for December, and he turned his own operation into an exercise in technological collaboration.

Smarr’s abdominal surgeons had an unprecedented advance look at his intestines, using the 3D models created at Calit2. During the surgery, they augmented the feed from cameras inside his gut with these images, viewing both on high-definition monitors. (The company that makes the robotic surgical tools used in the operation leaped at the chance to monitor Smarr’s procedure. The five-hour operation to remove the diseased portion of Smarr’s colon will be viewable in 3D at Calit2.) He experienced no complications and recovered rapidly.

“The plan is to work between the company and the surgeons and then do another set of surgeries, to see if we can make this routine,” Smarr says. “From a risk management point of view, the more you know about the details before you go in to cut, the less likely you are to make a mistake.”

Smarr is impatient for the next level in the intersection of high tech and health. “Medicine doesn’t treat the body as an interactive system,” Smarr says with typical zeal. In time, he is confident that big data will enable what his friend, gene sequencing pioneer Leroy Hood, calls “P4 medicine,” which is short for “predictive, preventive, personalized, and participatory medicine.” The idea is to detect and deter disease early, to foster health rather than treating illness.

A Collaboration Engine

On a fine fall day, Smarr is huddled in his sunny office with two postdoctoral fellows, who are modeling his microbiome data on a laptop, using a program they have just written. When Smarr asks to see a different kind of graphic, they quickly hack out the new code. “This is the new world,” Smarr says, delighted. It’s just the sort of cross-disciplinary collaboration for which Calit2 was built. “These kids are just as fluent in machine learning and algorithms as they are in the biology. I can’t believe how exciting this is; I never thought I’d be this excited again.”

As we crossed UCSD’s vast campus in Smarr’s blue Tesla X, the stereo was blaring Bob Seger. Smarr may live in the future, but he is also a man of a certain age. Still, no one is pressing him to retire, or to groom a successor; there’s too much to do. He believes the convergence of information technology, nanotechnology, and telecommunications is inevitable, and no place is better positioned for it than Calit2. And he is not content for Calit2 to drive cultural change at UCSD — he wants to set an example for organizations of all kinds.

“There are a lot of lessons here for any corporation, any government agency, any large-scale organization,” Smarr says. “The question is how can you get the best return, after the investment you make in capital and people, in that organization. There has to be a superstructure that will lift up and temporarily hold together a multidisciplinary team that attacks a problem.” The key to progress is to synthesize world views, scientific philosophies, and competencies to make sense of the immense data that can be collected and analyzed. Says Smarr: “You can’t predict the future, but if you’re going to create it, you have to be collaborative.”

Reprint No. 17408