LinkedIn is launching a fresh AI-powered people search engine this week, after what feels like a very long wait for what should be a natural offering in generative AI.
It comes a full three years after the launch of ChatGPT and six months after LinkedIn launched its AI-powered job search offering. For tech leaders, this roadmap illustrates a key lesson for enterprises: Deploying generative AI in real-world enterprise environments is challenging, especially at the scale of 1.3 billion users. It’s a leisurely, brutal process of pragmatic optimization.
The following coverage is based on several exclusive interviews with LinkedIn’s product and engineering team responsible for the launch.
First, here’s how the product works: the user can now type a query in natural language, e.g. “Who knows about cancer treatment?” in the LinkedIn search bar.
LinkedIn’s venerable keyword-based search engine would have been stumped. Only references to “cancer” would be looked for. If a user wanted more advanced results, they would need to perform a separate, challenging keyword search for “cancer” and then “oncology” and manually try to combine the results.
However, the fresh artificial intelligence-based system understands intention search because LLM understands semantic meaning under the hood. It recognizes, for example, that “cancer” is conceptually related to “oncology,” much less directly to “genomic research.” As a result, it displays a much more correct list of people, including oncology leaders and researchers, even if the exact word “cancer” does not appear in their profiles.
The system also balances this importance with usefulness. Instead of simply showing you the best oncologist in the world (who may be an unattainable third-degree contact), it will also look at who in your immediate network – such as a first-degree contact – is “pretty relevant” and can serve as a key bridge to that expert.
See the sample video below.
But perhaps the more vital lesson for enterprise practitioners is LinkedIn’s “cookbook”: an iterative, multi-step process of distillation, co-design, and continuous optimization. LinkedIn needed to perfect this in one product before trying it in another.
“Don’t try to do too many things at once,” writes Wenjing Zhang, vice president of engineering at LinkedIn, in a post about the product launch, which also spoke to VentureBeat last week. He notes that previous “sweeping ambitions” to build a unified system for all LinkedIn products have “halted progress.”
Instead, LinkedIn focused on conquering one industry first. The success of the previously launched AI Job Search service – which resulted in job seekers not having a four-year degree 10% more likely to get hiredaccording to vice president of product engineering Erran Berger — delivered the design.
Now the company is applying that plan to a much bigger challenge. “It’s one thing to be able to do this across tens of millions of jobs,” Berger told VentureBeat. “It’s another thing to do it in the north of a billion members.”
For enterprise AI developers, the LinkedIn journey provides a technical playbook of what it is Actually going from a successful pilot to a product at scale of billions of users.
Up-to-date challenge: a graph with 1.3 billion members
The job search product created a solid recipe on which to base a fresh people search product, Berger explained.
The recipe started with a “golden dataset” of just a few hundred to a thousand real query and profile pairs, meticulously cross-referenced with a detailed 20- to 30-page “Product Policies” document. To scale this for training, LinkedIn used this little golden kit to get the huge base model to generate massive volume synthetic training data. This synthetic data was used to train a The 7 billionth parameter “Product Policy” model – a high-fidelity relevancy assessment that was too leisurely for live production, but perfect for teaching smaller models.
However, the team hit a wall early on. For six to nine months, they tried to train a single model that could balance strict compliance (relevance) with user engagement signals. The “aha moment” came when they realized they had to solve the problem. They transformed the 7B policy model into: Teacher Model 1.7B the focus was solely on meaning. They then combined it with separate tutor models trained to predict specific member activities, such as job applications related to a job product or networking and tracking to find people. This multi-teacher team created cushioned probability scores that the final student model learned to emulate through KL discrepancy loss.
The resulting architecture operates as a two-stage pipeline. Bigger first Parameter model 8B supports wide sampling, casting a wide net to pull candidates from the graph. The highly distilled student model then takes over for fine-grained ranking. Although the job search product has successfully implemented a 0.6 billion (600 million) Student of parameters, the fresh people search product required even more aggressive compression. As Zhang notes, the team trimmed their fresh student model from a 440M to just Parameters 220Machieving the speed needed for 1.3 billion users with less than 1% relevancy loss.
However, applying this to people search broke the venerable architecture. The fresh problem included more than just that classification but also search.
“A billion records,” Berger said, is “a different beast.”
The team’s previous fetch stack was built on CPUs. To cope with the fresh scale and latency requirements of “fast” searches, the team had to move indexing to GPU-based infrastructure. This was a fundamental architectural change that was not required by the job search product.
Organizationally, LinkedIn has used a number of approaches. For a while, LinkedIn had two separate teams — job search and people search — trying to solve the problem in parallel. But when the job search team achieved a breakthrough using a principles-based distillation method, Berger and his leadership team intervened. They brought in the architects of victory in search of work — product manager Rohan Rajiv and engineering manager Wenjing Zhang — transfer your “cookbook” directly to a fresh domain.
Distillation providing a 10-fold augment in efficiency
After solving the search problem, the team faced a ranking and performance challenge. This is where the cookbook was adapted, using aggressive fresh optimization techniques.
Zhang’s technical position (I will post the link when it is available) provides detailed information that our audience of AI engineers will appreciate. One of the most vital optimizations was the size of the input data.
To feed the model, the team trained other LLM with reinforcement learning (RL) for one purpose: summarizing the input context. This “summarization” model was able to reduce the size of the model input by 20 times with minimal information loss.
The combined result of a 220M model with a 20x reduction in the input signal? AND 10x augment in ranking throughputenabling the team to efficiently share the model with its massive user base.
Pragmatism over hype: building tools, not agents
During our conversations, Berger was adamant about something else that might catch people’s attention: The real value for today’s enterprises is improving recommendation systems, not chasing “agent noise.” He also declined to talk about the specific models the company uses in searches, suggesting it hardly matters. The company selects models based on which one it considers most effective in accomplishing the task.
The fresh artificial intelligence-based people search engine is a manifestation of Berger’s philosophy that it is best to first optimize the recommendation system. The architecture includes a fresh “intelligent query routing layer,” Berger explained, which itself is based on LLM. This router pragmatically decides whether a user’s query—e.g., “trust expert”—should go to the fresh natural language semantic stack or to the venerable, reliable lexical search.
This entire sophisticated system is designed as a “tool” that: future the agent will benefit, not the agent himself.
“Agents products are only as good as the tools they use to perform tasks for people,” Berger said. “You can have the best reasoning model in the world, and if you’re trying to use a people search agent, but the people search engine isn’t very good, you’re not going to be able to achieve that.”
Now that the people finder is available, Berger hinted that the company will one day offer it to agents who will utilize it. However, he did not provide details about the date. He also said that the recipe used to search for jobs and people will be used in the company’s other products.
For enterprises creating their own AI roadmaps, LinkedIn’s playbook is clear:
-
Be pragmatic: Don’t try to boil the ocean. Win one industry, even if it takes 18 months.
-
Codify the “cookbook”: Turn this victory into a repeatable process (strategy documents, distillation pipelines, co-design).
-
Optimize Continuously: True 10x gains are coming After seed model during pruning, distillation, and artistic optimizations like an RL-trained summarizer.
LinkedIn’s journey shows that when it comes to real-world enterprise AI, the emphasis on specific models or chilly agent systems should take a backseat. Sustainable, strategic advantage comes from mastery pipeline — “native AI” cookbook of collaborative design, distillation, and ruthless optimization.
(Editor’s note: We will be publishing a full-length podcast with LinkedIn’s Erran Berger soon on the VentureBeat podcast channel, which will discuss the technical details in more detail.)