Humanity’s penchant for more authentic and novel experiences appears to be deeply ingrained in our genes, defying rational explanation. The maturation of 3D imaging technology has ushered in the transition from two-dimensional to three-dimensional visualization, promising to spark a major revolution in optical innovation and drive a comprehensive upgrade in medical technology for humanity.

Recently, researchers at Utsunomiya University in Japan have developed a novel 3D imaging technology that utilizes multiphoton absorption of femtosecond laser pulses to generate microbubbles within a volumetric liquid screen, enabling these bubbles to be fixed at designated positions to render three-dimensional images.

Researchers stated that this technology will help doctors observe patients’ holographic anatomical images before surgery in the future, while also enabling architectural and topographic profiling studies in the field of architecture. The detailed findings have been published in the recently released high-impact optics journal OptiCA.

3D imaging technology generates three-dimensional virtual images through lenses, virtual headsets, and other media by performing 3D scanning, computer modeling, and rendering of objects. Compared with planar projection imaging, 3D imaging technology can present a more comprehensive representation of objects and enable interactive experiences with three-dimensional visuals, which is one of the reasons why many augmented reality companies are focusing on the development of 3D imaging technology.

As the technology matures, 3D imaging has been widely applied in fields such as entertainment, manufacturing, healthcare, and construction. Healthcare and consumer electronics are the two sectors experiencing rapid growth in demand for 3D imaging technology. In the medical field, 3D imaging enables doctors to visualize human organs in 3D holography, thereby addressing complex diagnostic challenges.

VCBeat (WeChat: vcbeat) will take you into physicist Larry Smarr’s 3D world for the first time, offering a firsthand experience of his entire colon surgery process.

Larry Smarr Showcased Detailed Insights into His Gut Health at the University of California, San Diego

After spending considerable time with Larry Smarr, you may have the privilege of being invited to La Jolla, California, to view his colon.

More than one million Americans suffer from inflammatory bowel disease, and Smarr is one of them. But unlike most people, Smarr also runs a research institute that masters cutting-edge technology. The research institute has many supercomputers, a large number of graphics cracking programmers, towering digital display screen walls, and mysterious cave-like automatic virtual systems (VR caves), all of which provide better support for Smarr to develop his personal version of digital 3D technology, which he calls "Transparent Larry".The remarkable feature of this version is its perfect replication of the colon, including every corner and crevice of the intestine, as well as inflamed tissue.

Smarr, a 69-year-old physicist, is widely known for his efforts to create a nationwide campus supercomputer network, which eventually evolved into today’s Internet. He now runs a future research institute called Calit2.

VCBeat has learned that the institute, located at the University of California, San Diego and the University of California, Irvine, is dedicated to advancing research across numerous fields, including medical research. Over the past decade, he has been applying technology to himself to identify data pertaining to personal privacy; the potential outcomes of this trial remain unknown to him.

But as his health began to deteriorate and surgery became necessary, Smarr saw a prime opportunity to integrate his stored big data and advanced visualizations into the surgical suite’s urgently needed software upgrades.

He hopes that his research institute will dedicate itself to the development of automated 3D imaging software for use in surgical procedures over the coming years.

“I think I know more about the condition of my internal organs than my surgeon does, and it’s not good,” said Larry Smarr.

In November last year, a few days before his surgery, he conducted an experiment on himself. At that time, he invited his surgeon, Sonia Ramamoorthy, to visit the cave-like automatic virtual system (VR cave) at his research institute.

“I think I know more about the condition of my internal organs than my surgeon does,” he said. “It’s indeed in bad shape.”

Ramamoorthy was not surprised by Smarr’s somewhat unusual invitation, given their uniquely close relationship. “Unlike any patient I’ve seen before,” she noted, “he came to his appointment with a PowerPoint presentation.”

In fact, Smarr found that even modern operating rooms have their flaws:Even the computer-generated 3D imaging technology used in most simple video games is not usable by physicians, despite its ability to detect structural changes that affect surgery.。

Jürgen Schulze of the University of California, San Diego, provided a 3D animated model of Larry Smarr’s diseased colon (green), as well as 3D animated models of the spine (white), intervertebral discs (yellow), major blood vessels (blue and red), bladder (light blue), and spleen (magenta).

“When 3D imaging technology is available, surgeons love it,” said Eric Wickstrom, a Professor of Biochemistry and Molecular Biology at Thomas Jefferson University Medical College in Philadelphia. For more than a decade, he has been dedicated to developing 3D imaging technologies for cancer surgery, and in 2013 he published research findings demonstrating that surgeons consider 3D imaging highly beneficial.

“Surgeons have repeatedly stated, ‘The biggest problem we encounter is operating on unexpected blood vessels,’” said Wickstrom.

However, Wickstorm stated that 3D technology is rarely used in operating rooms because it is too time-consuming, requiring the use of stacks of patients’ conventional 2D images to create realistic visuals.He stated that this process requires more research efforts and programmers to support it, thereby enhancing its level of automation and speed.

Smarr’s 3D model has evolved over many years, thanks to the extensive efforts of virtual reality expert Jürgen Schulze. First, he modeled each organ using a stack of 2D magnetic resonance imaging (MRI) scans. He then integrated these models to create a 3D representation suitable for viewing on standard computer monitors. The key to this step was achieving automation, thereby making the routine use of such images easier and more cost-effective.

When Ramamoorthy stepped into the darkened cave and put on a pair of 3D glasses, she was stunned. Imaging in the abdominal region lagged behind other areas because this region contains many organs with varying densities, and the abdomen moves with each breath and contraction of the intestinal muscles.

Ramamoorthy quickly identified a “no-fly zone”: Smarr’s inflamed colon was draped over his bladder. Through the opening created by the surgical instruments, urine could flow into the abdominal cavity (fortunately, the two organs were not in direct contact). She also noted that Smarr’s large intestine had elongated more than anticipated and was adhered to his spleen. As an experienced surgeon, she immediately recognized the need to modify the surgical plan and requested the 3D images be made available in the operating room.

“The utility of 2D imaging is not that significant; it doesn’t allow me to think from a 3D perspective,” said Ramamoorthy. With 3D imaging, she can preview the intraluminal conditions and even navigate through them in virtual reality. “To me, it’s like being in a brand-new candy store,” she remarked. However, when Ramamoorthy asked Schulze whether he would introduce 3D imaging into the operating room, he felt uneasy. “I have hemophobia,” he said.

Ultimately, Schulze agreed to go to the operating room.

“He truly deserves credit for his courage,” Smarr said.

The day before the surgery, Schulze was fiddling with cables in the operating room, attempting to connect his laptop to the surgical robot and project images onto the monitors at the Jacob Medical Center. The $943 million medical center had opened just days before Smarr’s surgery. The new operating rooms, equipped for laser therapy, robotic surgery, and magnetic resonance imaging (MRI), seemed an ideal setting for the application of pioneering imaging technologies.

Everything in the operating room was proceeding normally. As Ramamoorthy secured the four-arm da Vinci surgical robot to Smarr’s body, others in the operating room fixed their gaze on the 85-inch display mounted on the wall.The image on the display screen is divided into four quadrants, showing Smarr’s 3D virtual colonoscopy, MRI scans, footage from the robotic arm entering his abdominal cavity, and a close-up view of the interior of the colon, which is also captured by the surgical robot.。

“Looking at my teenage sons and their Xbox consoles, all the technology is in there, so why can’t it be used for patients?” said Dr. Sonia Ramamoorthy.

“The VR imagery looks like a starship,” said Smarr. Ramamoorthy likened the visualization technology to driving with Google Maps navigation. “You know your current location, how far you are from your destination, and whether there is a traffic accident ahead,” she noted, rather than merely having a vague idea of where you might be.

Cutting-edge 3D imaging technology has always been a frequent topic of discussion among the hospital’s physicians. Now, Ramamoorthy has the opportunity to use it on a patient; she has long hoped to extend the application of this technology to other patients, particularly in more complex surgical procedures.

“Looking at my teenage sons and their Xbox consoles, all that technology is right there, so why can’t it be applied to patients?” she said.

Smarr tracked 72 data collection points on his body, gathered health data, and displayed it on a video wall in his laboratory. The wall consisted of 32 monitors.

About twenty years ago, Smarr became obsessed with studying everything related to his body, starting with diet.

Smarr spent twenty years in Illinois, where he was responsible for establishing the National Center for Supercomputing Applications. In 2000, he joined UCSD (University of California, San Diego). When he arrived at the seaside campus, he was obese and overweight, and he was surprised by the healthy lifestyles of the people around him.

“I had never eaten kale before,” he said. “But I was afraid the school would send me back.” So Smarr quickly hired a personal trainer, adopted the Zone Diet, and began taking nutritional supplements.

Smarr lost 30 pounds, transforming one-sixth of his body weight into lean muscle. Yet he still keeps his old driver’s license in his wallet as a reminder of what he used to look like. No one chooses to undertake such efforts lightly. Wondering whether the number of fish oil capsules he was taking was optimal, Smarr began undergoing blood tests. Subsequently, he became addicted to all the data he could collect, constantly craving more.

“I looked at the hospital and said, ‘What’s wrong with these doctors?’” said Larry Smarr.

“It’s like a loose thread on a sweater; I just kept pulling at it,” he said. He also encountered numerous challenges: he had to contend with physicians who advised against self-ordering blood tests and refused to order them for him. “They said the blood tests were not covered by insurance because they were considered preventive,” he explained. He paid for the tests out of pocket, estimating that this alone cost him tens of thousands of dollars. “We always say that healthcare should be patient-centered, but we still have a long way to go,” he stated.

In 2005, although he felt well, his blood test results indicated health issues. His C-reactive protein (CRP) level was five times the upper limit of normal, and later rose to 15 times that level. “I didn’t know what CRP was, so I looked it up on PubMed and searched for it on Google,” said Smarr. Although he had no formal training in biomedicine, he had served as an advisor to several directors at the National Institutes of Health. He knew that CRP is a general marker of inflammation. The test results showed that something was wrong with his body, but he did not know what the underlying problem was.

Smarr did not suspect that his colon was problematic, as his 2005 colonoscopy results were normal. It turned out that his disease was atypical in many respects. His condition had a late onset; therefore, his physicians did not anticipate subjecting a patient of his age to intestinal surgery, as such an intervention would cause extensive scarring, particularly given that the intestinal damage was confined to only a small segment of the colon.

After his CRP levels spiked again in 2008, Smarr went to see his doctor. Smarr, a former astrophysicist, recalled that he was promptly sent home. “They asked me, ‘What symptoms do you have?’ I said, ‘I have the test data right here!’ If I had observed a similar anomaly in the sky, I would have pointed all my telescopes at it! But they just said, ‘Come back when you develop symptoms.’”

Severe abdominal pain soon set in. “It was entirely predictable,” Smarr said with a sigh. “I looked at the hospital and wondered, ‘What’s wrong with these doctors?’” After ten days of antibiotic treatment, the pain subsided, but his C-reactive protein (CRP) levels were not fully brought under control.

He felt uneasy.He decided to examine the genetic data he had obtained from the 23andMe website (he was also an early user of the platform). He identified a single nucleotide polymorphism (SNP)—a type of genetic variant—indicating that his risk of developing Crohn’s disease was increased by 80%, with potential for sporadic onset.。

In 2011, his CRP levels deteriorated significantly, prompting him to begin collecting stool samples for analysis. He has undergone 90 stool tests at a cost of $375 per test, and the testing is still ongoing (never open his refrigerator).

“If I hadn’t done this 10 to 15 years ago, I would have been wrong,” said Larry Smarr.

At his core, Smarr is still a computer scientist, obsessed with the study of feces because it contains a wealth of data, including indicators related to the immune system and gut health. “There are one billion bacteria in a single gram of feces, and each bacterium has five million DNA bases,” he said. “To me, it looks like a disk drive.”

By mid-2011, stool analysis revealed that his fecal lactoferrin level was 125 times the upper limit of normal; this protein is associated with inflammatory bowel disease. By the end of 2011, his C-reactive protein (CRP) level had reached 27 times the upper limit of the healthy reference range.

Colonoscopy revealed that his colonic wall was thicker than normal, which was caused by his own immune system.

At Calit2, Smarr can display a decade of his health data—cholesterol, glucose, insulin, vitamin D, CRP, and more—in graphical form on a digital visualization wall composed of 32 high-definition screens. These metrics are part of the 72 parameters he tracks. “This is me, my body,” Smarr stated.

When he realized that the same technology could not be adopted by the majority, he came to deeply understand that consumers should take their health into their own hands and seek data from within themselves.

A Controversial Viewpoint:Many physicians believe that routine blood testing in healthy individuals does more harm than good.

Testing can yield false-positive results. After all, repeated testing can induce anxiety and drive up medical costs. Although evidence has shown that Smarr was right to monitor his own C-reactive protein (CRP) levels, in many cases, an abnormal test result is merely an anomaly and does not necessarily signal a serious health threat. If patients rush to treat every condition based on test results, they will not only face increased medical expenses but also risk harming themselves, as treatments inevitably carry side effects.

Given these risks,The Society of General Internal Medicine does not even recommend that healthy adults undergo annual physical examinations or routine laboratory testing.

However, Smarr does not endorse this approach. He is aware that people consider him somewhat eccentric, but he does not care. “If I hadn’t done this 10 to 15 years ago,” he stated, “I would have been wrong.”

Obsessed with the data he had collected on his own condition, Smarr was reluctant to undergo treatment. “I didn’t want to mess up these beautiful time series,” he said.

For him, gritting his teeth and enduring pain to collect data was nothing new. Once, despite suffering from severe pneumonia, he still worked throughout the night under the Very Large Array radio telescope in New Mexico, unwilling to forfeit precious observation time. “I am a scientist,” he said. “To obtain data, you do what you must.”

But in 2012, as his abdominal bloating and rectal bleeding worsened, Smarr decided to abandon data collection. “Although I am committed to data acquisition, I am not stupid,” he said.

He completed a one-month course of antibiotics and corticosteroids. His CRP level decreased to 1.3, which is relatively normal, but he still hopes to see it drop further. He noted that his symptoms did not alleviate as his CRP levels declined, which is unusual.

In 2016, he underwent a full-body computed tomography (CT) scan, including virtual colonoscopy. “They inflate you with air from behind,” Smarr said with a smile. The results revealed that the intestinal wall was excessively thick in certain areas, while the lumen, or the interior of the colon, was only a few millimeters wide. It was indeed so narrow that even pediatric colonoscopy instruments could not pass through. A portion of the intestine had an abnormal kink, trapping fecal matter inside and leading to the production of methane gas.

“My wife said, ‘Oh my God, you look pregnant,’” Smarr stated. “I felt like a balloon about to burst.”

It is time for surgery.

In November 2016, guided by 3D imaging, Dr. Sonia Ramamoorthy performed colon surgery on Smarr, which lasted five hours.

Dr. Santiago Horgan, Director of the Future Surgery Center, points out areas of concern in the Smarr colon on the large screen in the operating room.

Thanks to virtual preview, Smarr’s surgery proceeded smoothly. Ramamoorthy used a robotic system to resect and cauterize the inflamed colon, then anastomosed the bowel and restored the lumen. Subsequently, Ramamoorthy removed two large pieces of unsightly red tissue, which Smarr referred to as “the devil’s children.”

VR expert Schulze was relieved to witness the entire procedure, which involved minimal blood loss. Smarr lost only 15 milliliters of blood, an amount comparable to that drawn in a routine blood test.

A significant benefit of 3D imaging is shorter surgical times, which typically leads to better patient outcomes.Ramamoorthy estimated that, by knowing the expected outcome in advance, at least 30 minutes were saved during the five-hour surgery. She positioned the robot attached to Smarr higher than usual to address his adherent spleen, a process that saved an additional 45 minutes.

Smarr recovered quickly after surgery. However, as a patient, he was quite unusual.

Within hours of regaining consciousness, Smarr resumed data collection. He was determined to monitor his postoperative C-reactive protein (CRP) levels to assess how the surgery affected his body. However, the nurse informed him that the physician had not yet authorized CRP testing. Smarr refused to allow the nurse to draw his blood unless he could secure approval from a physician permitting him to undergo CRP testing like other patients.

Smarr presented detailed, multidimensional images of his gut.

“They thought I was a fool, a problematic patient,” he stated. Beyond the neatly organized data points, Smarr believes that CRP testing should be a routine postoperative procedure to detect conditions such as sepsis. “It’s just an $11 blood test following a $150,000 surgery,” he said.

He got his wish, watching his CRP levels drop from 61 to 42, then to 30, and finally below 1 mg/L in the days that followed—his first time below this threshold in over a decade. VCBeat discovered that he had also infiltrated Fitbit, much to the astonishment of its employees. A team of his colleagues attached electrodes beneath his abdomen, hoping to detect the precise moment his colon resumed peristalsis.

Smarr has now made a full recovery, showing no signs of slowing down his work.

“Are you trying to grab my colon?” said Larry Smarr.

His goal is to develop software capable of automatically generating 3D images of organs. He believes that medical imaging companies could adopt such software within three to five years. He also envisions his research institute playing a key role in training the next generation of experts in virtual reality (VR) for medical imaging.

Now, he offers physicians interested in 3D imaging a digital tour of their own bodies, and they sometimes discuss the magic of 3D imaging technology with Ramamoorthy.

“Do you want to hold my colon?” he would ask as he took out the 3D-printed colon.

Whenever he has the chance, he always praises Ramamoorthy’s medical skills. “I love my new intestine; it’s fantastic,” he said at a recent case conference.

We may not remember the first 3D movie we ever watched, but the magical world of Avatar and the vivid scenes of mammoths and squirrels leaping in Ice Age likely left a deeper impression. I believe that as 3D technology has evolved to its current state, it should no longer be regarded merely as a toy offering fleeting visual novelty. In an era where technological advancements are accelerating at an unprecedented pace, no matter how unimaginable the innovations may seem, they must ultimately return to serving human-centric needs.

Although 3D imaging technology has not yet been widely adopted in healthcare, or even in surgical procedures, and projection technology has not fully entered our daily home lives, I believe that one day we will be able to receive remote medical treatment from the comfort of our homes, far from hospitals. Through 3D imaging technology, we will interact with doctors as naturally as if we were meeting face-to-face. Human emotional connection will inevitably grow infinitely closer, and technology will become inseparable from our lives, breathing with us like oxygen. Such a scenario may not be far off. The efforts Smarr is striving to make will ultimately become part of the transformation of human life.