The Definitive Guide to Content Moderation

This is the complete guide to content moderation. In this guide, you’ll learn:

Part 1

The basics and benefits of user-generated content

Part 2

The risks of UGC and how content moderation can help

Part 3

Content moderation methods: Human-based, AI-based, and more

Part 4

How to choose a moderation partner

Ready to discover how content moderation can help your organization reap the benefits of user-
generated content while avoiding the risks? Let’s dive in.

Part 1: The Basics and Benefits of User-Generated Content

User-generated content (UGC): Any content that has been created by end users, often to promote a
brand’s products or services online.

User-generated content can include content that may be shared on online platforms such as websites,
social media accounts, and other marketing channels, including:

Images

Text

Video

Audio
UGC may also take the form of blog posts, social media comments, forum posts, podcasts, reviews, and
testimonials. UGC is not created by the brand that is being promoted, which can actually be to your
advantage as a business.

Today’s consumers are in the position to ask “Why should I trust your brand?” Authentic content
created by users allows you to answer “Because other users just like you trust our brand,” without ever
creating a banner ad or advertising on a billboard.

There are several benefits to implementing UGC in your marketing campaigns that are worth noting:

1. Brand exposure to new audiences.

Generally, millennials demonstrate trust in influencers, brand ambassadors, and followers that are
relatable and consistently create authentic pieces content. When these influencers create and share
content that features your products and/or services, this puts your brand in front of a traditionally hard-
to-reach audience, giving you free exposure. Additionally, this decreases your expenses, as your brand
will be able to reduce the amount of time and money spent on creating marketing content in house and
promoting it via targeted paid media.

2. Engagement among existing followers.

To better grow your audience and turn leads into sales, make retaining the customers and followers that
your brand already has as much of a priority as attracting new customers. There are several ways that
UGC increases engagement with your existing audience, and the following land at the top of the list:

UGC keeps followers engaged with your brand, creating brand enthusiasts in the process.

When you share UGC with your own audience, it facilitates trust in your brand’s offerings and boosts
your brand’s credibility.

When followers see their own thoughtfully created, unique content featured by your brand on a
platform visited by large audiences, that shared UGC makes them feel appreciated, special, and excited
to spread the word about your brand.

Considering that 70 percent of consumers trust online consumer opinions, sharing customer
testimonials about your service or product in the form of UGC acts as social proof that helps followers
with their buying decisions.

3. Improved search engine rankings.

Original, dynamic UGC has the power to be relatable to consumers while also improving search engine
rankings.

Here’s how the latter works:

Generally, the more content you have on a web page, the more power it can have in the eyes of search
engines. When some of that content is user generated, the variety of words and phrases increases, as
the UGC creators use their own wording. Often, this is wording that your company may not have
considered using, or legally be allowed to use. The result is a page with an array of rich keywords for
search engines to see, and a substantially more well-rounded website overall. Additionally, if that UGC
includes photos and/ or videos, then this enhanced level of mixed media can further improve your
webpage’s rank.

By deploying UGC, you can improve brand exposure and development, engage new and existing
audiences, provide word-of-mouth endorsement, improve search engine rankings, and give your brand
the edge it needs to compete in today’s market.

Part 2: The Risks of UGC and How Content Moderation Can Help

There are risks associated with publishing content created by your organization’s community members –
A fact that savvy brands do not overlook.

User-Generated Risks Include:

Unmoderated content published in real time exposes your brand to offensive content

Unmonitored two-way interactions get offensive

UGC posts get out of control, taking your brand down with it

Risk 1) Unmoderated Content Published in Real Time Exposes Your Brand to Offensive Content

Publishing user-generated posts and video in real time means that the information is going live
immediately instead of being curated by a moderation team before-hand. By allowing unmoderated
content to be published in real time, offensive content can be easily uploaded, damaging your brand.
Anytime you work with UGC in real time, you run the risk of broadcasting highly visible content that
could be offensive, even if a user intended it to be funny. Or, a user could share content that simply
doesn’t align well with your company’s values.

For instance, a celebrity sponsored by your high-end jewelry brand may have a live interview broadcast
from a nightclub on your brand’s website. The celeb, a bit intoxicated, may misinterpret a seemingly
acceptable chat question and respond with a barrage of profanity and threats. At the very least, such
incidents are embarrassing. In the worst case, you may have upset and lost some of your brand’s
followers.

UGC may very well give your brand an edge as a marketing tool, but that content could do more harm
than good. Fortunately, you can mitigate the risk of these occurrences with UGC moderation.

Work with a company that has a combination of trained professionals on staff and artificial intelligence
(AI) moderating UGC in real time. Both are required, since humans are able to distinguish the nuances of
language, photos, and videos that AI may fail to analyze correctly.

Risk 2) Unmonitored Two-Way Interactions Get Offensive

Brands that offer services or apps featuring two-way interactions must be especially vigilant about
preventing hate speech, nudity, drug use, violence, or other inappropriate material from being shared.
Dating websites and apps are most at risk, but even “gig economy” brands like grocery delivery services
or ride sharing platforms should have moderation processes in place.

Anytime a customer, client, contractor, or employee can send a photo that might be offensive, content
moderation is necessary. No matter who the user is, it’s important to recognize that two-way
interactions are beneficial, but also come with increased risk of offensive or harmful communications.
Here are some examples:

Dating Apps

Dating app moderation can be as simple as setting photo standards, such as permitting only a full
human face, and never a meme or a photo with two people in it. Or, it can be as complex as navigating
specific terms of nudity that you set, such as allowing partial nudity in profile pics or in two-way
interactions.

Delivery Services
Content moderation in a grocery delivery service, for example, may involve stopping a “hangry”
customer whose food order was canceled from sending the personal shopper an inappropriate photo or
offensive message. It’s just as important that your customers be protected, so all UGC sent by
subcontractors should be checked before it reaches them. For example, allowing a shopper to verify
with a customer that “this is the item you wanted” with a photo of that item can quickly become a viral
social event if the shopper decides to be “creative” with the photo he or she takes.

Back of House Customer Service Platforms

Moderating customer communications is intended to protect employees from antagonistic customers

and customers from disgruntled employees. This may involve preventing customer service reps from
generating bills with questionable content or reacting offensively, or protecting reps from being berated
by customers in live chat. Using a block list downloaded from the web, however, won’t be sufficient
prevention. To thoroughly protect your brand and prevent customer billing debacles (such as the
changing of a customer’s name to “A-hole Brown” on a bill) look for proficient, accurate profanity
filtering technologies that feature custom block & allow lists, the ability to scan text embedded in
images, support for multiple languages, and other safety measures so that your customers, staff, and
brand are thoroughly protected.

Transportation and Gig Helper Apps

In-app chat moderation must be a priority in the case of communications between a mobile
subcontracted workforce and users, such as a ride service hailing company or outsourced task app.
Volatile messages between a subcontracted staffer and a user may damage your credibility and your
bottom line. To mitigate these risks, look for an experienced content moderation partner that offers
efficient profanity filtering technologies featuring moderation of in-app chat among employees/
subcontractors and customers.

How can you protect your brand against two-way interaction risks?

Answer: Make sure that employees, customers, subcontractors, and users cannot send a photo, video or
message without moderation. Offensive messages cannot be stopped without pre-moderation, so seek
the professional services of a company that can provide the necessary checks and balances.

Risk 3) UGC Posts Prove Brand Damaging

Through UGC campaigns, companies are using photos, tweets, and videos submitted by consumers to
produce engagement and interest in a way that is cost effective. But your brand could take a hit if you
let unacceptable material onto your platforms. Before launching UGC campaigns, your business must
institute a well-constructed policy that is enforced by a team of professionals.

How can you prevent inappropriate user-generated content from landing on your company’s platform?
Answer: Implement profanity filters, image moderation, and video moderation, using both live teams
and AI, to filter UGC. Moderating content is critical to making UGC work seamlessly for your company or
clients. Stringent moderation can ensure UGC messaging remains relatively controlled, on brand, and
beneficial to your bottom line.

Part 3: Content Moderation Methods

You can and should capitalize on user-generated content, but in order to make it a worthwhile
investment, UGC must be accompanied by moderation. Sounds simple, until you realize that there are
various forms available to choose from.

The particular form of content moderation that is appropriate for you can depend on client
requirements, business needs, industry standards, and your online community. Before settling on the
form your business will implement, examine the various ways to conduct moderation, and take all
demands into consideration, as well as goals for your brand’s online presence.

The 4 most common ways to conduct content moderation rely on either human moderators, AI, or both,
and include:

Human-Based AI-Based AI + Human Hybrid Approach

In-House Content Moderation Crowdsourced Moderation AI-Based Content Moderation Expert

Partner Outsourcing

Human moderators ensure that your brand is always seen in the best possible light by spending hour
after hour scanning content. Crowdsourcing moderation to a network of people in the form of an
open call.Crowdsourced moderators are usually anonymous and typically not specialists. AI
algorithms tackle the immense task of locating and removing millions of posts containing nudity, hate
speech, weapons, drugs, or offensive gestures. AI can be used to remove any overtly objectionable
content such as pornography or hate symbols. And a human team can review the content for more
nuanced and brand-specific criteria.

Pros: With a content moderator in-house, you have more control over your content moderation
operation. Moderators work alongside you to update guidelines based on immediate needs. Pros:
Real-time, high speed moderation at low prices. Pros: AI can moderate content faster and at
lower cost than a human. AI can be taught to detect certain words and patterns of content, as well as
learn to recognize profanity and other harmful content. Pros: Accuracy and expertise. Successful
prevention of blatantly offensive, brand damaging content and assurance that the featured images, text,
and videos support your brand’s mission. As you grow, moderation can scale up and down with ease.

Cons: It’s costly to have full-time 24×7 moderators on staff. It’s also time consuming, as moderators will
require rigorous training and supervision. Cons: Moderators are not familiar with your distinct
brand criteria. There is no guarantee that crowdsourced moderators will be unbiased. False positives,
false negatives, privacy violations, and image theft have been known to occur. Cons: AI cannot
understand context, resulting in occasional flagging of harmless content or, even more concerning,
failing to catch inappropriate content. False positives and false negatives have been known to occur. AI
trained on millions of examples may still make mistakes that a human would not. Cons: Far more
accurate for brand and mission-critical needs, but can be more expensive than crowdsourced solutions
or AI-only moderation.

There are several moderation variations, depending on whether moderation is AI-based or human-
based, and how one or both are conducting moderation. Let’s examine the most common variations of
moderation and each one’s ability to maintain security and brand credibility:

HUMAN-BASED

Pre-Moderation

Pre-moderation is intended to ensure that your online community is not exposed to harmful content,
safeguarding your brand against legal ramifications in the process. Text, images, video, and all other
content are scrutinized by moderators trained to review UGC submitted by your audience before
allowing it to become viewable. Pre-moderation is well suited for any company seeking to maintain their
online reputation while growing their brand.

Pre-scanning UGC does not allow for real-time posting, which is why some businesses shy away from
this approach. Delaying content from going live can frustrate online community members who are
accustomed to seeing their posts instantly.

At WebPurify, we address this concern by offering AI-based pre-moderation. AI rejects anything with a
high probability of containing harmful content before it can go live, allowing anything with a low
probability to post immediately. Any “on the bubble” content can be held back for post-moderation by a
live team within a few minutes.

Post-Moderation

Post-moderation displays user-generated content on your app or website immediately, while replicating
it in a queue so a moderator may review the content after it goes live, so as not to slow down the user
experience. Dating apps, some social sites, and other platforms will often use post-moderation in
response to users’ demand for immediate posting.

Relying on a post-moderation approach comes with significant risk. For instance, dating platforms see
tens of thousands of images come through daily. By allowing these images to go live on a site before
screening them, companies are taking a significant risk, and their resources are often left playing catch-
up as they attempt to take down offensive posts before it upsets users. Even if content is taken down, it
is often too late, as users have already taken a screenshot of the experience and shared it.

Fortunately, an expert partner can help mitigate the risk for companies that elect to use post-
moderation. WebPurify works with platforms whose users live stream their videos in real time (which
cannot be delayed by pre-moderation). With a combination of our live teams and technology, we
address any issues within minutes of these broadcasts starting.

Reactive Moderation

Reactive moderation relies on the community to flag concerning content, which is then surfaced to your
well-trained internal support team or moderation partner. This form of moderation is often used as a
safety net in conjunction with pre- and post-moderation to catch any untoward content that slipped
through the cracks.

When used as the primary method of moderation, reactive moderation gives members the
responsibility of flagging content on the community platform or website that they deem offensive,
typically through the use of a reporting button. When a community member clicks on the button, an
alert is filed with the website’s moderation team or administrators, flagging the UGC that must be
reviewed. The content in question will then be removed if it is determined to be in violation of the site’s
regulations.

Since reactive moderation depends on community members, this method comes with the risk of making
offensive content visible on a brand’s website, albeit briefly, and many organizations are not willing to
take this risk and compromise their brand image.

Distributed Moderation

Distributed moderation is best described as a “jury of your peers” approach where a brand charges their
users with the task of content moderation. In distributed moderation, a rating system enables online
community members to vote based on an average score.

This score is determined by several community members, and the voting process decides whether or not
content submitted by fellow users adheres to the online community’s values and aligns with posting
regulations. Generally, voting is complemented by supervision from the community’s senior moderators.
Distributed moderation is often used by smaller businesses that are drawn to the member-forced
moderation method because of their limited budget. Unfortunately, some community members will be
turned off by the lack of company supervision. And when it comes to mission critical content, relying
solely on the community to enforce the rules is an approach that can too easily lead to the posting of
brand-damaging content. Distributed moderation is recommended only in combination with other
moderation methods, if at all.

AI-BASED

Automated Moderation

Run by specially designed technical tools, automated moderation is responsible for filtering offensive
language and other violations in multimedia content by implementing complex artificial intelligence
solutions. An automatic and faster way of identifying offensive posts, AI moderation can also help to
block the IP addresses of users that are classified as abusive.

Automated technology, however, is limited by its inability to distinguish the nuances of photos, videos,
and text in the absence of human review. For this reason, the most effective moderation solution is one
that pairs AI-based moderation with live moderation so acceptable content is not filtered out and
harmful content is detected.

THE HYBRID APPROACH

A Perfect Partnership of AI and Human Content Moderation

The risks of using AI alone were brought to light during the Covid pandemic when many large social
media giants were forced to send their live teams home, solely relying on AI. Their moderation programs
were much less effective, with a high volume of harmless content being blocked and vice versa. AI is a
computer, only working in probabilities and numbers. As such, AI functions as a content filter, but not a
comprehensive solution.

While AI provides data that your organization can use to make decisions, it has distinct limits. AI may
catch images that are harmless or fail to catch everything it’s programmed to. On the other hand, it
could take months for teams of professional moderators to go through millions of images that AI could
process and rank in minutes, or even seconds.

A hybrid approach to moderation, combining human and AI efforts, is necessary to effectively monitor
user-generated content. At WebPurify, we’ve been using a hybrid system of AI and live human
moderation to scrub UGC for hundreds of brands for over a decade.
AI can be used to detect and reject any overtly objectionable content (such as hate symbols or offensive
gestures) before it can go live, and allow any content with a low probability of being inappropriate to
post immediately. Any borderline content can be held back for a human team to review for more
nuanced and brand-specific criteria a few minutes later. This allows your company to prevent blatantly
offensive, brand-damaging content and ensure that the images, text, and videos that are featured
support your brand’s mission.

Hybrid Image Moderation: AI and humans working together

A Word on Mental Health

If you choose to hire an in-house moderation team, it’s imperative that you prioritize the moderation
team’s mental health, as well as their overall working conditions. If you choose to work with a
professional moderation agency, be sure they have a comprehensive mental health program in place for
those who will be moderating your platform’s content.

Part 4: Choosing a Moderation Partner

If you’re undecided as to whether or not you will handle moderation yourself or work with a partner,
consider the pros and cons of both approaches.

Self Moderation Partnership with Moderation Agency

Pros:

With your own internal teams

Your business has full end-to-end control over what’s being said.

You can address content that you know to be a risk, while promoting content that enforces a positive
brand image.

Pros:

By teaming up with a professional moderation company whose entire focus is on providing moderation
services, you can leverage their expertise.

Hiring and training a team is not necessary, as you have access to the agency’s highly-trained team.

24×7 support is provided by the agency’s moderation team.

Moderation is fast and easy to scale up as you grow.

Cons:

You have the responsibility of building your own moderation processes, systems, and software.

You’re responsible for hiring and training a 24×7 team.

If high volumes of content are submitted, your team will likely be unable to keep up with the task,
allowing inappropriate content to slip through, or lengthy delays for publishing user submissions.

Even in the case of less content, there is a chance that the team will be exposed to disturbing content
during the moderation process, making it imperative to put a robust mental health program in place.

May cost more than outsourcing once all costs are considered.

Cons:

Onsite supervision of moderation teams is not available, although a strong moderation partner should
have robust QC supervision systems in place.

While typically affordable, these external costs should be budgeted for.

If you don’t take the time to properly evaluate prospective options, the public consequences can be
irreparable. When choosing a moderation partner, confirm that the company you are working with does
not crowdsource under any circumstances and the team moderating your platforms and apps is working
in a controlled, professional atmosphere.

If a moderation company cannot guarantee that it will never store or share any of your data, then they
aren’t the moderation partner to trust with your brand. To ensure that the partner you choose to
moderate your UGC is qualified to fill this integral role, refer to this questionnaire as a framework:
Questionnaire for Selecting a UGC Moderation Partner

Conclusion: If it doesn’t already, your business should leverage user-generated content as a significant
way to promote your brand and be competitive. Along with this incredible marketing machine called
UGC, however, comes the need for effective content moderation to protect your brand’s reputation,
ensure customer security, prevent risk, and improve online engagement for the success of your
company and the benefit of your users.