后来那个猎头跟我说 面试官对我product回答非常满意，觉得我communication很好，所以给我一个SQL second screen。。。希望这次不要犯晕才好。
为了回馈大家。Proudct问了俩问题
Product就是问Acceptance rate降了咋办
我的思路是这样的
首先开是不是降了。看year over year是不是去年同一周也一个情况，可能就是fatigue after holiday season啊。说不定最近DAU也低了，大家没有log in 怎么去respond呢。还有是不是每个月都是月末比月初低呢。有这个pattern不用太在意。
还有就是rate降了多少，有没有significant change，比对前一周还有月初。值不值得worry
可能request比同期多了，比如launch了新的feature，people you may know 在newsfeed，或者group participation 增多，network以外加人增多。或者career group，sales group这种人数曾多，不熟悉的人发request，acceptance rate就降了。反之notification／navigation变了不那么容易注意到friend request就没回这种
还有就是确定了降了，看user segment谁少了。也是看engagement什么的有没有变化。
第二个product问题就是people you may know on news feed （这个地里也有）
我好像也是一个思路吧。measure friend request number per views per customer, acceptance rate.
overall on FB, participation how much % of users exposed had interattion with this feautre
how much % of friend request sent over through People you may know.这个数字有没有change over time grow and srink呢还是grow at slower rate呢
还有就是不同age group，还有segments这些metrics是不是都一样的result
最后就是别的feature有没有受影响。比如search friend，adjacent posts的comment like有没有受影响
还有eventually 看看对retention有没有影响。time spend，number of friend， etc。
SQL题我一上来看acceptance rate还挺高兴，以为就是谁有最多friend那种题，其实就是找一天发的request看这些request的acceptance rate。 我以为是当天的，后来改成left join以后我只用 a.target_id = b.send_id 了。今天不知道为啥脑子有点沉。
正确答案很简单的
select sum(if(b.send_id is not null,1,0))/sum(if(a.action=’‘send’’,1,0)) as acceptance_rate
from friending a
left join friending b on b.action=’‘accept’’ and a.target_id=b.send_id and a.send_id = a.target_id
where a.date = ‘‘date’’
Product就是问Acceptance rate降了咋办
Metrics drop investigation question
思路：Clarify assumptions => Math decomposition => Brainstorming framework => Statistical Analysis for Root cause => Recommendation for each issue
Clarify Assumption
NSM impact: We want better network quality - higher engagement on FB
Metrics recipe: Denominator is accept, Numerator is request. Each part can move both direction. Need to know which part change.
Product details: PPL can send request by search and recommendation.
Brainstorming Framework
Normal Seasonality: YoY, MoM trend
Internal Investigation
User segment: Which users? New? old? segmentation?
Which product: recommendation or search?
Vertical Funnel: View drop? More request? More refuse?
Horizontal Product: How about IG?
External Investigation
DAU Accounting: New users increase a lot causing that? DAU drop causing low response?
Holiday, big event: any news, holiday need to be aware of?
Statistical Analysis for Root Cause
Seasonality: Check YoY, MoM => If no issue
Holiday: Same as above
Internal: Identify the segment, product funnel, action => analyze data more
External: % of non-accepted request by new user. It’s common
Recommendation
Seasonality => Wait / Improve model on that
Holiday => Same. Opportunity sizing and new strategy.
Internal => Further data analysis, survey, AB testing new features (painkiller), Debug outage.
External => Improve algorithm for new users.
People you may know on news feed.
What features to use to find people you may know?
Clarify: Increase NSM by increase the network quality & quantity. For new friends only.
Brainstorming:
Network overlap. Co-worker overlap. Close friend overlap. Follow similar public account.
Bio: Same city, school (close year), company, hobby
Together proxy: Tagged in same post; Location close in same day (overlap in year).
Analysis
We can check correlation by these metrics to Y (already friends)
Recommendation:
Y can be already friend.
Test acceptance rate to know if recommender is good proxy.
Often need ML model like NN two tower matching to optimize similarity score.
Feature can be compressed into embedding
What metrics to monitor for AB testing?
Brainstorming
Request # by new feature; Request # / Viewed #
Acceptance rate; Accepted #
SPAM rate
For MLE: Average rank of request. Distribution on rank
Grouping Metrics & Evaluate
Drive metrics: Request # / Viewed # for the new feature
Supporting metrics: Acceptance rate from new feature (within 1 week).
Guardrail: Spam reporting #
For MLE: Average rank of request. Distribution on rank