Meet the Reinventor: Qian Lin Is Teaching Computers to “See”

By Sarah Murry

Qian Lin can predict the future. Or, at least she can teach computers to see it. 

Through her research at the intersection of artificial intelligence (AI) and computer vision, she is working on innovations that help printers correct flaws in images in real time and assist computers spot microscopic errors in a printhead rolling off the assembly line.

Qian joined HP as an intern in 1992, after completing her PhD at Stanford University in electrical engineering, where she developed specialized expertise in image processing while working on a project for NASA. During her nearly 30-year career she has been responsible for 45 filed patents and has 21 currently pending.

 “There are so many things I’d like to see happen, and I have the privilege of leading people who share the same vision.”

— Dr. Qian Lin, HP Fellow, VP of AI and Emerging Compute

Lin has helped develop intelligent assistants for imaging services, vision systems for print quality and 3D-part identification, and machine learning and edge computing solutions. With partners, she invented HP’s adaptive halftoning technology that’s been incorporated into more than 200 million printers and has played a central role in advancing HP’s digital camera imaging pipeline. If you’ve ever had red eyes automatically fixed in a family photo or shared a panoramic vista from your vacation, thank Dr. Lin.

One of Lin’s early patent disclosures was inspired by a visit to a museum, where she was trying to take a photo of a painting she liked. “Digital cameras were not that advanced then,” she explains. “My photo of the painting came out blurry. I solved the problem by taking multiple low-light shots and then used image processing to boost up the quality. That became the foundation for a technology that’s become common in digital cameras.”

Deep learning at the center  

The past decade has seen the democratization of photography through the smartphone revolution, with more pictures taken every two minutes than were taken throughout the entirety of the 1800s, according to some estimates. In fact, InfoTrends predicts more than a trillion images will be taken this year alone. 

“Cameras generate so much data, and a lot of times you need immediate action and analysis from this data,” she says. “This research area is one that I’m very passionate about and its increasingly becoming more important to HP.” 

Lin and her team were the creators of Pixel Intelligence, a powerful portfolio of computer vision algorithms that helps print service providers make sense of visual data. The algorithms can find faces within an image or find the same face in multiple images with great accuracy. They can also recognize specific objects (such as a specific logo), detect whether people are smiling or their eyes are shut, and sift through hundreds of images to make a collage of the best photos. Moreover, they work in real time, opening up new avenues for improvement in devices like smart home assistants that constantly monitor their surroundings.

The common language of Pixel Intelligence and Lin’s current research is an area known as computer vision, a branch of AI that helps computers to understand visual information. In recent years, computer vision tasks have become more accurate with machine learning models. “At HP, we’ve started using it to enhance our ability to understand images,” she says. 

This creates potential commercial applications and new capabilities such as in social sharing. When you upload a photo on a social media site, the application downsizes the photo so it's faster to transmit, which reduces the quality of photos. Lin and her team have looked at using deep learning to scale up these photos for large-format printing. They’re also researching how to “recompose” photos, to create images that are more professional or creatively composed, even when shot on a smartphone. 

The organization and retrieval of images, Lin says, also has huge opportunities. How do you make sure the photo and memories captured on smartphones are not lost when people replace them throughout their lives? Or how do you sort thousands of photos of your kids? Lin is curious about what kind of software could enable you to tell a photo story from babyhood to college. “We are thinking about how to use AI to tell stories with your photos,” she says. “It can mine through collections, events, and digital archives, too.”

Business opportunities for computer vision

Computer vision is a blue-sky area of research that many at HP and beyond see as ripe for innovation. At a basic level, it involves training computers with algorithms to replicate what human brains can do when they identify and interpret an image -- but to do it with the speed of a machine and on a massive scale.  

Lin and her team are looking at several projects that apply computer vision to new business opportunities. One looks at how to build printers that can improve image quality. If you think about the printing process right now, it’s an open loop, explains Lin. Closing the loop would mean the printer itself can optimize images. “The printer can identify deviations, understand and ‘see’ what’s being printed, and make it perfect,” she says.

In the broader ecosystem of HP, computer vision can potentially be applied manufacturing printheads. Right now, it takes a small army of people to inspect each one. “This is tedious work being done by human operators with microscopes. It’s extremely repetitive,” Lin says. “So, how can we use deep learning computer vision algorithms to inspect those products as they are being made, and make sure the quality passes our criteria?”

Persistence and predictions

Lin’s ability to develop new technologies over her career and create new opportunities in AI and deep learning is among the many reasons she’s been named an HP Fellow. “A Fellow needs to work across organizations, champion ideas and push technology development,” she says. 

Lin says in the next decade HP will bring new tools and technologies to the marketplace, and she is excited to be part of it. “I see a lot of opportunities to apply AI and machine learning in our products,” she says. “There are so many things I’d like to see happen, and I have the privilege of leading people who share the same vision.”

Learn how bias gets baked into AI, and what researchers are doing about it.