10/10 initial technical phone interview. There was a HR screening before this. Good luck to everyone who’'s recruiting!
SQL:-- Q1: how many posts were reported yesterday for each report Reason?-- Table: user_actions-- ds(date, String) | user_id | post_id | action (’‘view’’,’‘like’’,’‘reaction’’,’‘comment’’,’‘report’’,’‘reshare’’) | extra (extra reason for the action, e.g. ‘‘love’’,’‘spam’’,’‘nudity’’)
– Q2: introduce a new table: reviewer_removals, please calculate what percent of daily content that users view on FB is actually spam?–no need to consider if the removal happen at the same post date or not.-- ds(date, String) | reviewer_id |post_id
Product:
How would you test if this filter works?
List some metrics
If we experiment, how would you conduct it?
A/B testing
How to select a sample group
Random to avoid bias
How many people would you select for your sample group
Use formula for n (minimum sample size)
After getting results from A/B testing, what to do next?
T-test on metrics to see if there’‘s a difference
What’‘s a t-test? What’‘s t-score? What’‘s P-value? Explain p-value to someone who doesn’‘t know stats.
Let’'s say the filter worked but revenue went down, what would be your hypothesis?
If it’‘s rev/user dropped, perhaps the number of users increased because of the effective spam filter, but they don’t spend money.
But aside from rev per user, total revenue also dropped. Perhaps user distribution changed, there’‘re more users who don’'t spend money now.
Perhaps the filter downgrades the spam posts and now all the spams are clustered at the same spot (e.g. the 20th post and forward). So users stop looking scrolling after the 20th. Check activated time to validate.
Given revenue decrease, how would you make recommendations? (doesn’'t have to be yes or no answer)
Short term vs. long term: how much does revenue drops? User experience vs. revenue. Short term revenue drop vs. long term brand perception and long term revenue gain.
If user distribution changed: find cause and tackle that unimaged users
