Industry - Sogou Search Ads Relevance Measurement

Role: Product Manager | UX Designer

Overview

Relevance measurement project is an extremely data driven project. My KPI is based on the percentage of relevance level and the effect ads have on normal search results. This project's sole purpose is to form rules of the standards of relevant on different levels and use strategies to eliminate bad results from search results. To measure the DCG score, PM is required to understand each query's intent and analyze what user is searching and determine if the result is relevant. 

KPI Achievement

I created the first measurement guideline which helped Devs created a much better ranking algorithm and led to this project from a losing project into a winning project in just 2 months. I led this product from losing to Baidu (core competitor) 0.41% to winning 2.22% in only 5 months. I surpass my annual KPI of 2015 four months early. 

What I did as a product manager?

Created the 1st official relevance guideline

https://drive.google.com/open?id=0B3ZxCtKjo44tTU9pTWlnRlo4Tkk 

Created the 1st Spam and Junk guideline

https://drive.google.com/open?id=0B3ZxCtKjo44tVGxyUi1FVkNXVjA 

Strategy

Through massive amount of data analysis, came up with several strategies to help improve algorithm on solving deadlink and irrelevant results. 

Recruiting and Training interns

One of the jobs for a relevance measurement product manager at Sogou is to recruit interns to do massive amount of data labeling for product manager to review the data and ensure the accuracy of data for dev to analyze. My job as a product manager is to hire them, training them and manage their work load as well as work quality to ensure data quality.