(Part 2) 8 Common Roles in Trust & Safety
This is a three-part series that aims to raise awareness and educate readers on the topic of Trust & Safety (T&S) in Tech companies. If you are new here, please check out Part 1 (‘What is Trust & Safety’) before reading the article below.
In recent years, there has been a spike in job openings for Trust & Safety roles. Tech companies like TikTok, Airbnb, Twitter, Google, Netflix, Facebook etc. have been actively hiring for Trust & Safety teams. In Part 2 of this series (article below), we will learn about the 8 common roles in a typical T&S team and how the different roles work together to build a safe platform for Users. Note: Users refer to both customers or providers using the platform e.g. buyers, sellers, content creators etc.
Quick background: I was first exposed to Trust & Safety 3 years ago in Amazon. I’m currently a Program Manager in Google’s Trust & Safety team.
The content in this post is strictly personal and it does not represent views of any organization.
Synonym for Trust & Safety
While ‘Trust & Safety’ is a common term used across tech companies, there are some companies that name their T&S teams as ‘Business Integrity’ , ‘Customer Trust’, or ‘Fraud & Abuse’ etc. There are different ways to address the same team or role. This is something to take note of if you are on the lookout for job opportunities in the T&S space.
Common Roles within a T&S team
In this section, you will learn about the 8 common roles that make up a typical T&S team. As there will always be nuances in these roles across different companies, it is important to read the job description to understand specific requirements before applying.
I tried to think of analogies to make the section below more comprehensible. As a result of recency effect, I eventually drew analogies between roles in T&S and roles in the Netflix show, ‘Squid Game’. (read: spoiler alert, proceed to read at your own risk!)
1. Policy Specialist/ Manager
Policy Specialists act as the heart of a T&S team. They define what constitutes a violation. Policy Specialists develop guidelines and build standards for what Users can do or cannot do on the platform. The challenge of this role is in striking a balance between developing policies that keep the platform safe from harmful Users or content, and allowing good Users or content to be freely expressed on the platform. Policy Specialists also engage with external regulators, policymakers and industry partners to support the development of policies to maintain a safe platform.
- Suitable for: Candidates who are passionate about policy making, internet technologies, cybersecurity laws, privacy, content moderation, regulations, international/geopolitical events and how they may affect a tech company etc.
- Drawing parallel with Squid Game: The host defines the rules of the game. He decides how the game is played and what you can or cannot do in each round. He also dictates the punishment/enforcement action of the game i.e. death.
2. Investigation Specialist (aka Risk Specialist, Enforcement Specialist or Operations Specialist)
Based on the policies developed by Policy Specialists in (1), Investigation Specialists then investigate allegations of policy violations reported by internal stakeholders (includes detection model) or external stakeholders (includes Users, regulatory authorities etc.). Investigation Specialists will refer to policy guidelines and use a combination of manual and automated tools to gather information and conduct the investigation. Through their investigation, they will determine if the User’s behaviour or content is violating any of the mentioned policies. If yes, the Investigation Specialist will take enforcement action against the User (i.e. remove the User account or content). If no, the Investigation Specialist will pass these reviews (i.e. no action taken against the User account or content). Beyond performing investigation, the Investigation Specialist is also in a good position to identify potential policy gaps, and feedback to the Policy team for further refinement.
- Suitable for: Candidates who have an eye for detail, ability to understand and apply policies to diverse scenarios, ability to manage sensitive content calmly
- Drawing parallel with Squid Game: Red Guards — Red Guards monitor and review if any players lose or violate the rules of the game. For players who lost or violated the rules of the game, the Red Guards ensure that they are punished (i.e. killed). They also ensure that players who won or did not violate the rules of the game proceed to the next round without any action taken against them.
3. Incident Manager (aka Escalation Manager, Incident Response Specialist)
Incident Managers focus on responding, investigating and resolving high severity escalations. These escalations are usually time sensitive, and they can be reported by executive stakeholders, regulatory authorities or high profile Users etc. This role requires high-level judgement especially when dealing with edge cases or grey area scenarios that might not be properly defined in existing policy guidelines. For most companies, you may think of an Incident Manager as an advanced Investigation Specialist who focuses on more severe and time sensitive violations. He/she is also able to exercise discretion instead of strictly following the policy guidelines. At times, Incident Managers also triage reviews to Investigation Specialists for resolutions, lead root cause investigations and work with other teams to implement changes to avoid a similar issue from repeating again in future. This might include writing Correction of Error (COE) reports.
- Suitable for: Candidates with strong verbal and written communication, ability to remain calm and methodical under high-pressure situations, build good rapport with cross-functional teams to resolve escalations
- Drawing parallel with Squid Game: Front Man — Whenever there is a high severity escalation (e.g. Player 001 begging everyone to stop rioting, black market organ harvesting being exposed), the Front Man is able to quickly respond, investigate and resolve these time sensitive escalations, so that the game can proceed as planned. These scenarios might not have been clearly spelled out in the rules of the game, but the Front Man will need to exercise his judgement to determine on what to do next quickly.
4. Customer Support Specialist/ Manager
This is usually an external facing team who serves as a help desk or contact point for Users. Users may be facing a diverse range of T&S related issues such as appealing an enforcement action, being a victim of any risk type. Customer Support Specialists will assist Users in resolving their problems. They act as a good source of feedback to improve existing User experience from a Policy, Program, and Product standpoint.
- Suitable for: Candidates with strong verbal and written communication, service-oriented mindset, ability to maintain a professional demeanour while handling difficult situations
- Drawing parallel with Squid Game: In general, I would say Squid Game has a poor customer support team. They only come to you when they want to recruit you. When you need help, they “vanish”.
5. Data Analyst (aka Business Analyst)
A good Data Analyst is one who understands business needs, and is able to drive targeted analyses to recommend solutions that align with overall business goals. On a daily basis, Data Analysts write queries, develop, build and monitor performance metrics dashboards. They also generate insights to improve efficiency of operations, understand the user impact of policies in order to know how it affects metrics, conduct analyses and derive actionable insights to enhance policy, tool or user experience in general.
- Suitable for: Candidates usually possess SQL, data visualisation and/or Python skillset. Refer to job description for specific requirements.
6. Program Manager (PgM)
A T&S Program Manager acts as the glue that ensures that policies, programs, and tools work together to protect Users and prevent abuse. The role of a PgM is to plan and drive varied projects in order to achieve overall T&S goals e.g. reduce harmful content by XX%, reduce payment fraud by YY% etc, improve User trust by ZZ%. To effectively achieve these T&S-level goals, a PgM is often required to 1) scope problems, design solutions, identify risks, remove roadblocks, and implement solutions at scale from inception to completion stage. A PgM will also need to 2) collaborate closely with cross functional teams (Policy, Product, Operations, Engineers etc.) to drive projects that advances trust & safety e.g. work with Operations team to improve operational processes, work with Policy team to implement policy changes, work with Product Managers and Engineers to pilot new detection models etc. Refer to job description for specific requirements.
7. Product Managers (PM)
A T&S Product Manager identifies needs arising from Users in order to keep the platform a safe one. A PM owns the product strategy and roadmap of a product’s lifecycle. They write Product Requirement Documents and work closely with other internal stakeholders from PgM, Operations, Policy and Engineering teams to implement product features that balances near-term results with longer-term investment needs. PMs can work on both internal-facing or external-facing products. For example, internal-facing T&S products may include developing features for tools that will allow Investigation Specialists to have additional information when performing their investigation or allow more accurate detection of bad Users or content on the platform. External-facing T&S products may include developing features for tools that enable Users like yourself to report violations easily. Refer to job description for specific requirements.
8. Engineers (e.g. Machine Learning Engineer, Software Developer Engineer, Backend/Frontend Engineer)
Depending on the type of engineering roles, the job scope will differ. It can range from building scalable tools (e.g. for investigation review), to improving backend system design and architecture, to developing classifiers, machine learning models and algorithms to protect the platform and Users from harm or abuse etc. Refer to job description for specific requirements.
That’s a wrap for Part 2! Thank you for reading and I hope you gained a better understanding of the 8 common T&S roles that companies hire for. Are there any roles mentioned above that interest you?
If you find this article helpful, do give a thumbs up and share it with friends who might benefit. Stay tuned and follow me for Part 3 of this series, as we dive into the key concepts and challenges around T&S.
