(Part 1) What is Trust & Safety?
In recent years, there has been a spike in job openings for Trust & Safety (T&S) roles. Tech companies like TikTok, Airbnb, Twitter, Google, Netflix, Facebook etc. have been actively hiring for Trust & Safety teams. But what exactly is Trust & Safety? Why is it important? What do people in Trust & Safety do?
I seek to cover these questions in a three-part series: Part 1 (this article that you are reading) summarises what T&S is about; Part 2 will cover common roles in T&S that you can apply for; and Part 3 will touch on key concepts and challenges around T&S. Follow me on Medium if you would like to stay updated on Part 2 and Part 3 of this series:)
The content in this post is strictly personal and it does not represent views of any organization.
What is Trust & Safety?
Let’s begin by breaking up the terms. According to Cambridge Online, trust is defined as “having confidence in something, or believing in someone”, while safety is defined as “the state in which you are safe and not in danger or at risk”. For a company, the definition of Trust & Safety translates into minimizing any potential risks that Users are exposed to when using its products or services. Companies achieve this by building safe and secure products, so that Users can confidently use them without fear of harm.
1. The concept of T&S isn’t new.
You might have heard of payment fraud such as bank scams or fake text messages tricking you into giving your personal information. Payment fraud or financial risk is a common example or subset of T&S. However in recent years, the coverage of T&S has also expanded beyond financial risks, to content risk and product risk etc. The expansion of coverage is driven by a combination of the rise in digital and information technologies, and government regulatory pressures. These reasons resulted in an increasing need for companies to take responsibility for the overall safety of their platform (read: more focus on user experience). The different types of risk will be covered in point 3 below.
2. T&S is largely policy-driven.
Policy is the heart of T&S work. A T&S policy team develops guidelines and standards to dictate what Users can or cannot do on the platform. They identify potential threats to Users, platform abuse and other potentially risky or unwanted behaviours that will jeopardise the safety of Users, and translate them into policies which will serve as the “moral compass” for Users on the platform. Essentially, Users who violate these policies will be penalized or removed from using the product or services.
3. T&S coverage is contextual.
T&S coverage will differ across companies, across products. An easy way to think of T&S would be to segment based on the types of risk that a company’s product or service is exposed to. Here are examples of common risk types that a company might be exposed to:
- Content risk — This refers to the integrity of content that is posted by Users. Examples of content platforms include Facebook, TikTok and Twitter. These companies will have T&S policies in place to ensure that Users on the platform are protected against content risk such as fake content, spam, abusive content, extremist content, intellectual property infringement or other malicious content. Content risk may also be in the form of fake reviews, fake ratings or abuse reviews (i.e. intentionally giving bad reviews to reduce competition). Companies that face such content risk include marketplaces like Airbnb, Taobao and Carousell.
- Financial risk — More commonly known as payment fraud, this refers to fraudulent behaviour where there is an intent to deceive Users for personal financial gains. Financial risks are applicable to platforms that involve payment transactions (e.g. eCommerce sites, Internet Banking etc.) or platforms that allow communication between Users (e.g. WhatsApp, Telegram, Gmail, Outlook). The company will have T&S policies in place to reduce risks arising from fraud or malicious behaviours e.g. phishing scams, use of stolen credit cards (i.e. you did not make a purchase but it was billed to your credit card account). Financial risks can also exist within a company, where there are policies in place to prevent misuse of company funds or reimbursement.
- Product risk — This refers to the integrity of the physical or digital product itself. Think eCommerce sites such as Shopee, Lazada, Amazon, Zalora. The company will have T&S policies to ensure that 3rd party sellers are transparent in the products they sell. For example, there will be policies in place to regulate the sale of fake goods, damaged goods or expired goods etc. Imagine paying the full price of Nike Air Jordans or adidas YEEZY shoes, expecting to receive an authentic product, but the delivered product turns out to be counterfeit. Ouch! An example of digital product risk could be paying for an online gift voucher that has already been redeemed by someone else.
- Malware risk — This refers to intrusive software that can potentially damage or destruct device systems or accounts. A common example would be compromised accounts (i.e. bad actors hacking your account and taking control of it), or data privacy leakage where customers’ data like mobile number, credit card details, email address are being stolen. In 2020, you might recall incidents where prominent US figures’ (e.g. Jeff Bezos, Bill Gates) Twitter accounts were hacked. Other examples of malware include virus, phishing, trojans, ransomware, adware, spyware, Denial of Service (DoS), session hijacking, credential reuse etc.
As you can possibly imagine, any of the risks mentioned above will negatively impact user experience, and hence damage the brand reputation of the company. It is thus crucial for the company to put in place policies that regulate content or eradicate potential risks on the platform, so that Users can confidently trust and use the product without fear of harm. Additionally, given the changing behaviour of Users and increasing government regulatory pressures, it has become imperative for companies to continuously invest in this space in order to keep up with ever changing abuse patterns, trends and behaviours.
Now that we’ve covered the definition and examples of T&S, we’ll move on to Part 2 of this series, where we will learn about the common roles in a T&S team. Follow me on Medium if you would like to stay updated on this series.
If you enjoy reading this article, do give a thumbs up or share it with friends who might benefit from this sharing. Thanks for reading!
.
.
.
Reference:
