Tag: Facebook content moderators

  • Belfast’s Broken Record Crackles On

    There is a strong impression of the same old story of the Troubles in Belfast, all over again. The new element is the pandemic affecting teenage lives. Words and images by Fellipe Lopes in collaboration with Daniele Idini.

    It is hard to see a purpose behind the recent violent protests in Belfast that have fleetingly come to global prominence. On the ground, community leaders identify continuities, as recreational rioting is all too familiar, ever since the Good Friday Agreement of 1998.

    There is, however, a perception that the intensity of this year’s disturbances are a product of uncertainties around the new Irish Sea border and Northern Ireland’s position in the post-Brexit context.

    Brexit has clearly increased insecurity in the Unionist or Loyalist community, many of whom voted against withdrawal from the EU. Now the establishment of a commercial border between Northern Ireland and the rest of the United Kingdom, through the Northern Ireland Protocol, designed to protect the Good Friday Agreement, has created a sense of betrayal. This has hardened the rhetoric of struggle and conflict.

    The Unionist side of the argument is that this creates further distance from the rest of the U.K. amidst a sense that they are the ones being sacrificed in the divorce from the E.U.. Moreover, talk of a possible United Ireland in the South and persistent grievances over the failure to prosecute senior Sinn Féin politicians for a perceived failure to abide by social distancing rules at the Bobby Storey funeral fans the flame.

    © Fellipe Lopes
    © Daniele Idini

    Added to all this, Northern Ireland’s lockdown has affected the lives of young people in particular: directly through extended school closures, and loss of job opportunities, but also by creating a despondency that leaves many feeling they have little to lose.

    Finally, the long-standing educational segregation between Catholic and Protestant communities perpetuates a sense of “us” and “them.”

    So, how to explain these violent outbursts? What is different now from the usual activities? And what role has the pandemic played, especially on the youth involved? We spoke to a politician John Kyle, an activist Eileen Weir and an academic Dr Katy Hayward to find out more.

    John Kyle (PUP). ©Fellipe Lopes

    According to John Kyle, a Councillor on Belfast City Council, and former interim leader of the centre-left ‘Loyalist’ Progressive Unionist Party in Northern Ireland:

    “There are different factors involved. The first is that young people have been restricted. They have been unable to go to school, unable to meet with their friends, unable to go to the movies, unable to do, you know, stuff like that. If then there’s an opportunity to go out and get a bit of excitement, they will take that opportunity. Secondly, here in Northern Ireland, there is a tradition of what we call recreational rioting. In other words, if there were kids at an interface between two communities in the summer nights, sometimes, they arrange on Facebook to meet up for a fight. And so you might get 100 kids, sometimes 30, sometimes 50 kids. They meet up, start shouting at each other and throwing stones and bottles. And that’s what we call recreational rioting.”

     

    Daniele Idini with John Kyle. © Fellipe Lopes

    But Dr Katy Hayward, an academic and writer based at Queen’s University in Belfast says:

    “Something different has happened in the past few weeks, and that is that you’ve had a change in the political situation whereby the rhetoric in particular, from Unionist communities, and leaders has been one in which people have begun to really, doubt the process of basically the core elements of the Peace Process, for a long time. And amongst Loyalist communities, there has been a sense of frustration and anger about the Good Friday Agreement and compromises there. This is now beginning to get into the wider mainstream political field. And that’s primarily as a result of the conditions of Brexit, also you’ve had a new dimension, which is the criticism of the police service”

     

    Katy Hayward at Queen’s University Belfast. ©Daniele Idini

    On the other hand, Eileen Weir, Peacemaker and Community relations worker at Shankill Women’s Centre speaks of persistent social problems:

    “yeah, I think it is partly down to boredom. The things that people are not picking up on is that we have interface violence all year round. Right? This is not the first. They are usually not to that extreme where things are being burned or anything, but the two communities come together for a fight all year round and every year. We have a big, big drug problem in our communities and a lot of our young people are addicted to some type of drug, whether it’s illegal or prescribed. So we have a big mental health problem in our communities. It’s like they put a plaster on a thing that actually needs an operation. Our rate of suicide, and not just with young people with adults as well, is through the roof.”

     

    Eileen Weir in front of the Shankill Woman’s Centre. © Fellipe Lopes

    Walking around the predominantly Protestant areas of Shankill and Lanark Way, as well as in the Catholic Springfield Road and Falls, reveals a tranquil environment with bustling local shops, active community centres and friendly characters, more than willing to share their views on the current situation.

    These are among the areas of Belfast most affected by the long period of the Troubles and the scars are apparent. Chatting to locals from both communities living alongside the so-called the Cupar Way Peace-Wall the general impression is of a desire to leave behind a troubled past.

    Shankill Road © Fellipe Lopes
    Falls Road © Fellipe Lopes

    According to Eileen:

    “we do have peaceful communities, OK? I mean, the woman that I work with, you know, we’re from North and West Belfast and we have a fantastic relationship. I don’t like to label anybody. But within that group I have Republicans, I have Nationalist, I have Loyalists, but building relationships is not enough. And we can all build relationships. But that relationship doesn’t happen overnight.”

     

    Clonard neighbourhood. © Fellipe Lopes
    Shankill neighbourhood. © Daniele Idini

    By Monday April 19th the moratorium on Loyalist protests in the wake of Prince Philip’s death had lapsed. That morning, in front of the Irish Secretariat, a two man protest against the Northern Irish Protocol, organised by the Loyalist Communities Council’s Chair David Campbell and former politician and co-founder of the group David McNarry, was in full swing.

    Before long, however, it was interrupted by David McCord, whose brother Raymond was allegedly killed by members of the UVF in 1997. With a picture of his late brother in hand, he confronted the two men who were holding a banner that had appeared in different locations across Northern Ireland the previous night, addressed to the large media presence before them.

    © Daniele Idini

    There is a big gap between the government and these communities that reflects uncertainties over the decisions being made, especially the government’s failure to recognise the serious impact of their policies. Our conversations in the neighborhoods reveal a lack of understanding of the practical implications of Brexit and the Northern Ireland Protocol. This incoherence is reflected in the seemingly pointless violence.

    Now there is particular concern around the escalating misbehaviour of teenagers, leading shop owners in the affected areas to coordinate closures.

    Meanwhile on social media, calls to resume Anti-Northern Ireland Protocol protests across the region started to circulate. This included tweets about one to be held that evening at Lanark Way, where in previous weeks the violent riots had taken place.

    In the hours leading up to the anticipated protests on Monday April 19th in the Springfield Road area, rumours circulated about community efforts to prevent local teenagers from taking part.

    Springfield Road Interface. © Fellipe Lopes

    But by that evening approximately thirty teenagers were blocking Lanark Way on the Shankill Road side, setting fire to furniture and rubbish in the middle of the street.

    © Daniele Idini
    © Daniele Idini
    © Fellipe Lopes
    © Fellipe Lopes

    The protest soon descended into stones and pieces of iron being thrown at police cars. Over the course of the disturbances we saw no visible expression of what was motivating the protesters; no posters criticizing the governments, or otherwise communicating reasons for the violence.

    After a couple of hours, the PSNI asked journalists to leave the area and move to Springfield Road, as they closed the gate. This raises an important question as to what happens in the fight between teenagers and the police when media outlets are absent. A lack of media coverage fuels growing resentment towards the PSNI.

    Lanark Way interface Gate © Fellipe Lopes
    Lanark Way interface Gate. © Daniele Idini

    Katy points to this new antipathy towards the PSNI:

    “you’ve had this new dimension, which is the criticism of the police service. That’s something very new. So for some time, Loyalists have been feeling that the police service no longer backs them … what changed in the past week was that the First Minister of Northern Ireland called on the chief constable to resign, and we had all the political leaders of Unionism calling for the same thing.”

     

    @ Fellipe Lopes

    Katy adds:

    “They don’t trust political representatives or institutions to manage Northern Ireland’s interests. So you can see here all these factors exacerbating the sense of insecurity and that it only takes a few individuals. We know those individuals who have been present for a long time, presenting the Peace Process, presenting the compromise, thinking that violence pays. There’s a new narrative in Loyalism now that looks at what happens with the Protocol … They say the threat of Republican violence led to the Irish Sea border. So that’s another element.”

    Another important dimension is the difficulty faced by women in particular during the pandemic, as Eileen puts it:

    “the reason why this impacts on women more so is that there is homeschooling – working from home and often caring for an elderly relative. A lot of women’s jobs are low paid … zero hours contracts. We get very little recognition for the work of women. In the present situation, if it wasn’t for community workers being on the ground, you’d be seeing a hell of a lot more problems. And really, that’s not what community workers are for. That’s what our political representatives are for.“

     

    Eileen Weir. © Fellipe Lopes

    We also raised the question of a United Ireland and whether it is healthy to talk about this now. Katy said:

    “the more you talk about it, the more unnerving it is for Unionism. And that’s perfectly understandable because, of course, it’s anathema to the Unionist identity and certainly that is a factor. Because Brexit has taken Northern Ireland outside of the European Union, that’s a bereavement for some people and particularly for Nationalists. Northern Ireland is on the wrong side of that European external border, but that’s kind of overlooked because that leads to, certainly amongst Nationalists, a growing desire for Irish unity. They wish to see that happen and happen soon. But that has increased the uncertainty and insecurity for Unionists and we have both communities feeling more insecure in their position, thinking that the other is secure because basically they got what they wanted.

    Unsurprisingly, John holds a different view:

    “I personally think that the best outcome is a Northern Ireland that is part of the United Kingdom, but that has a close relationship with the Republic of Ireland. People here who identify as Irish have got free movement between the North and the South. They can work anywhere in Ireland and celebrate their Irishness and vice versa. There is good things in the British system and good things in the Irish system.”

     

    The Cupar Way ‘Peace Wall’. © Daniele Idini

    Right now there is considerable focus on the decisions that the governments are making. The other part is about the lives and futures of these teenagers. Several communities in Belfast have paid a high price for the lack of access to education and job opportunities. Teenagers are now paying this bill. With their futures appearing bleak many have been drawn into unsuitable activities.

    Discussion of the protest should look beyond the intricacies of the Brexit deal and the Northern Ireland Protocol. Society and government should be forming a long-term plan for the social and cultural integration.

    On the streets you see violence involving teenagers and a decisive response from the police. What we did not discover is whether external forces are guiding and supporting these teenagers. Perhaps the real question is who stands to gain from the disorder?

    These communities are experiencing challenges very similar to what happened from the 1960s to the 1990s. The Good Friday Agreement created peace, but the record player is still playing the same scratched tune, with an extraordinary new phenomenon at play, which is the pandemic.

    The response of these adolescents represents another side to the impact of the pandemic. Mental health problems present as an issue in the lives of many families. The social projects around the Shankill and Falls Road do their best, but at this point the government has a responsibility to intervene, not only through policing, but also with social supports, inclusion programmes, enhanced access to education, job opportunities, improved living conditions and greater support for social workers, and psychologists.

    The pandemic is leaving a distinct legacy in Belfast: the ashes from the street bonfires; the stones hurled at police and journalists, and a new generation of teenagers accustomed to fighting the police. But the locals we spoke to all want peace and integration. They don’t want this generation to experience the same story as their parents.

    The fear is reborn of a new chapter to the Troubles. What happened over the last few weeks in Belfast has reawakened fears in these communities. No doubt this is a political issue, but the psychological scars are borne by individuals, families and communities.

    You can support Cassandra Voices reporting either on an ongoing basis through Patreon or a one-off PayPal donation to admin@cassandravoices.com.

    © Fellipe Lopes
  • Unforgettable Year: August 2020

    Many Europeans enjoyed a blissful August while storm clouds gathered overhead. That month photographer Daniele Idini travelled from North to South of Italy, finding a country in severe economic distress, and desperate to resume the good life.

    Image (c) Daniele Idini

    Dr Marcus de Brun, meanwhile, saw a perfect storm forming on the horizon. He predicted there would be a resurgence of the virus in Ireland this winter based on the following factors:

    An elevated number of potential viral hosts, which is a consequence of suppression of natural-immunity.

    Increased life of the virus in the external environment due to decreased daylight

    Raised levels of social anxiety and subsequent susceptibility to illness/infection

    Continued persistence of the virus at low levels within Irish society

    Another contributor, Alex Ugur, meanwhile, described government precautions as an unethical human experiment.

    It’s been difficult finding the words to express my worsening mood and deepening depression. I’m referring specifically to my subconscious responses to altered public behaviour and the marks left by social reaction to Covid-19. For the first time in my life, I’m noticing increasing anxiety and, with the stress, a direct link to declining health. I’ve been struggling with this worsening dynamic over the last month or two, trying to get to grips with it. Trying to better understand its cause. I’m sure I’m not alone in this.

    2/7/1986 President Reagan with William F Buckley in the White House Residence during Private birthday party in honor of President Reagan’s 75th Birthday

    Meanwhile, David Langwallner drew an important distinction between Neoliberalism Neoconservatism. He wrote that while the former is a form of libertarianism, combining unregulated, laissez faire economics, and the legitimation of a hedonistic lifestyle, Neoconservatism,

    on the other hand, is hardly even capitalist in outlook. It is really an offshoot of a more authoritarian leftism combined with a fundamentalist, morally self-righteous neocolonialism informed by ‘Christian’ values. It is associated in particular with the administrations of George W. Bush, with Paul Wolfowitz and Richard Perle its most prominent ideologues.

    Image (c) Daniele Idini

    Next, Boidurjo Rick Mukhopadhyay offered an insight into the careers of content moderators on platforms such as Facebook, YouTube and Google:

    There are thousands of content moderators, who are paid to view objectionable posts and decide which need to be removed from digital platforms. Many are severely traumatized by the images of hate, abuse and violence they see on a daily basis so that we, our families and children get to see ‘WARNING: The following post or content may be disturbing to some viewers.’

    From Spain, Connor Blennerhassett brought a report on the ordeal suffered by vegan activist Juan Manuel Bustamante, who spent sixteen months in jail on trumped-up terrorist crimes: ‘a Kafkaesque nightmare that saw him pass through five of Spain’s most notorious prisons, often locked up in solitary confinement and denied a vegan diet by his captors, who also beat him. It ruined his family’s finances and lead him to attempt to take his life after his release.’

    Icaria, Greece

    Over in Greece Frank Armstrong found a hardening of borders, and attitudes, in the wake of the pandemic, and drew wisdom from the writings of Albert Camus:

    Albert Camus in The Rebel (1951), identified an enduring tension between a Caesarian Marxist project that permits all manner of atrocity on the journey to earthly paradise, and an approach he identifies with Ancient Greece, characterised by moderation, incrementalism and respect for tradition. He suggests:

    The profound conflict of this century is, perhaps, not so much between the German ideologies of history and Christian political concepts, which in a certain way are accomplices, as between German dreams and Mediterranean traditions … in other words, between history and nature.

    Vietnam. Image (c) Hectic Fish

    Also, for the first time since his arrival, Hectic Fish was also able to travel around Vietnam, he proceeded to the territory of the Mnong accompanied by a copy of Rachel Carson’s The Marginal World ‘the otherworldly essay that opens The Edge of the Sea.’

    The shore is an ancient world, for as long as there has been an earth and sea there has been this place of the meeting of land and water. Yet it is a world that keeps alive the sense of continuing creation and of the relentless drive of life. Each time that I enter it, I gain some new awareness of its beauty and its deeper meanings, sensing that intricate fabric of life by which one creature is linked with another, and each with its surroundings.

    All photography by Konstantine Lunarine (c)

    August’s Musician of the Month was the alluring Undine, who expressed herself in timeless poetry.

    There was also fiction from Sarah Johnson with ‘The Candidate for the Roberts Prize’ where ‘The significance of discovery lies exactly in the degree to which it can be appreciated and put to use by the human community.’ And Glenda Miller’s ‘The Club’ in which an experience of cancer prepares her for the agonies of the birthing process.

    Seasonal poetry featuring came from Oliver Tickell, while Kevin Higgins satirically looked forward to a new politics.

    After Recent Unfortunate Results

    Next election onwards,
    there’ll be a second vote for those
    who turn up with, under their arm,
    a print copy of one of the larger newspapers
    and answer a few unobtrusive questions
    to prove they’ve consumed it correctly.

    A third for those who also present receipts
    that show they’ve dined sufficiently
    in restaurants with at least four stars,
    and a note from the maitre d
    that they know their way around the cutlery.

    A fourth for the lucky few in possession – to boot –
    of a ticket for one of those pampering spas
    at which one temporarily discards
    worldly things to have one’s darker parts
    irrigated of all subversive thoughts.

    So when all’s said and counted,
    people who shouldn’t matter
    can go back to not mattering.

    Unforgettable Year: January 2020

    Unforgettable Year: February 2020

    Unforgettable Year: March 2020

    Unforgettable Year: April 2020

    Unforgettable Year: May 2020

    Unforgettable Year: June 2020

    Unforgettable Year: July 2020

  • WARNING: The (Open) Secret lives of Content Moderators

    Tick Yes or No: ‘I understand the content I will be reviewing may be disturbing. It is possible that reviewing such content may impact my mental health, and it could even lead to post-traumatic stress disorder (PTSD).’[i]

    Last year, a sixteen-year-old Malay girl posted a poll on Instagram asking her followers whether she should live or die.[ii] 69% voted for death and she took her own life. The followers who voted that she should die neither took action to protect their ‘friend’ nor shared empathy or concern.

    People are awful.[iii] This is what my job has taught me”, says a former Facebook content-moderator who recently sued the social media giant after experiencing psychological trauma as a direct consequence of his work. The Wall Street Journal recently described content moderator as ‘the worst job in the US[iv] , and the same applies to other countries, which this article elaborates on.

    Very little is known about the role, mental health toll or other work experiences of content moderators. They may work for YouTube, Facebook, Google and other such platforms that we are all pretty much ‘addicted’ to.

    A few studies are now looking into the working conditions for people[v] who determine what ‘material’ or ‘content’ can be posted to Facebook or Twitter or YouTube. Their job is to decide on whether content adheres to the ‘community guidelines’ of online platforms. They work day and night so that we the users are saved from exposure to videos of graphic violence or child abuse as well as hate speech, among the constant stream of user generated material uploaded on to social media feeds.

    There are thousands of content moderators, who are paid to view objectionable posts and decide which need to be removed from digital platforms. Many are severely traumatized by the images of hate, abuse and violence they see on a daily basis so that we, our families and children get to see ‘WARNING: The following post or content may be disturbing to some viewers.’

    The heavy mental health toll on content moderators who are hired on a ‘freelance’ or ‘gig’ basis cannot be underestimated.

    Never-ending Uploads and Ever-Expanding Platforms

    A staggering three hundred hours of video content is uploaded on to YouTube every minute, while over ninety-five million photos[vi] are uploaded to Instagram each day, along with over five hundred million tweets sent out on Twitter (or 6,000 tweets per second). Therefore, it is virtually impossible for human moderators to vet every piece before a content is uploaded and goes live (with some potentially going ‘viral’). Popular platforms such as these serve user-generated content uploaded by a global community of contributors.

    The uploaded content is just as diverse as the user base, meaning inevitably that a significant amount is offensive to most users and, by extension, the platforms. Users routinely upload (or attempt to upload) content such as: child abuse, animal torture, and disturbing, hate-filled messages.

    Facebook outsources the hiring of content moderators and provides office space. Its sites are largely outside the United States – mainly in south, south-east and east Asia, but the operations have expanded to the US, more specifically in California, Arizona, Texas and Florida.[vii] Content moderators work at a computer workstation where they review content –  a steady stream of text posts, images and videos. These can range from random personal musings to information with ramifications for international politics. Some of it may seem rather benign – just words on a screen that someone didn’t like. While the worst may be incredibly disturbing. On a regular basis moderators have to witness beheadings, murders, animal abuse, and child exploitation. Therefore, one might wonder, what toll on mental health does this take?

    One previously unreported aspect of a moderator’w job is the numerical quotas that these subcontractors[viii] are forced to meet: each moderator is required to screen thousands of images or videos per day in order to maintain their employment.

    Facebook alone has an army of about 15,000 people in 20 locations[ix] around the world, who decide what content should be allowed to stay on Facebook, and what should be marked as ‘disturbing’, whether execution videos from terrorist groups, murders, beatings, child exploitation or the torture of animals. In addition to the stress of exposure to disturbing images and videos, there is also the pressure to make the right call about what how to mark the content. A wrong decision taken under stress will have penalties, financially for the worker, and also may have mental health effects on other human lives.

    Platforms, as we know them, reserve the right to police user-generated content through a clause in their Terms of Service (which none of us read, or do we? Should we?), usually by incorporating their Community Guidelines as a reference. For example, YouTube’s Community Guidelines prohibit  ‘nudity or sexual content’, ‘harmful or dangerous content’, ‘hateful content’, ‘violent or graphic content’, ‘harassment and cyberbullying’, ‘spam, misleading metadata’, ‘scams’, ‘threats’ videos that would violate someone else’s copyright, ‘impersonation’ and ‘child endangerment.’

    ‘Now you see me’

    The Cleaners, a recent documentary, features interviews with several former moderators who were previously outsourced by a subcontractor in the Philippines. The interviewees exposed their experiences of filtering the very worst images and video the internet has to offer. In the Philippines, workers operate out of jam-packed malls, where they spend over nine hours a day moderating content for as little as $480 a month.[x] With few workday breaks and no access to counselling, many of these individuals end up suffering from insomnia, depression and post-traumatic stress disorder.

    Records also show the average pay of a full-time online content moderator in the US is around $28,000, but globally and by a large measure a significant amount of hiring is done through outsourcing and on a temporary basis. In Ireland, research shows that typically a Facebook employee would be paid a basic rate of €12.98 per hour,[xi] with a 25% bonus after 8pm, plus a travel allowance of €12 per night – the equivalent of about €25,000 to €32,000 per year. Yet the average Facebook employee in Ireland earned €154,000 in 2017.

    On average, the workload involves moderating about 300 to 400 pieces of content[xii]  – called ‘tickets’ – on an average night. On a busy night, their queue might have 800 to 1,000 tickets. The average handling time is 20 to 30 seconds – longer if it’s a particularly difficult decision.

    ‘We are trash to them, just a body in a seat’ shares a content moderator. Every work minute is strictly bound.[xiii]  Harsh working conditions characterised by specified bathroom breaks and a meagre nine minutes of wellness time engenders a stress that is exacerbated by employers’ downplaying the importance of mental health care.

    The continuum of content in those quotas range from tone-deaf jokes; kids dressed up as history’s great dictators that may constitute hate speech; nude images; domestic violence images, and then the really graphic and inhumane ones that inevitably surface. The content moderators have about twenty-four hours[xiv] within which they have to classify the posts under bullying, hate speech, and other content as appropriate.

    Like other forms of gig workers, digital reputation or future work orders come from high ratings. Several former moderators felt pressurised to achieve a 98% quality rating. This would mean that the auditor would agree with 98% of their decisions taken on a random sample of tickets. Moderators are therefore scrutinised for the smallest mistakes. An unending stream of extremism, violence, child sexual abuse imagery and revenge porn, does not give moderators time to consider the more subtle implications of particular posts.

    Artificial Intelligence (AI) cannot nail this one… just yet!

    Moderators are human beings, so mistakes are inevitable. However, to shatter one misconception on this front: Artificial Intelligence (AI) cannot help much in this field. They currently act as triage systems; for example, by pushing suspect content to human moderators and weeding out some unwanted material on their own. But AI cannot solve the online content moderation problem without human help. For example, AI uses either a visual recognition to identify a broad category of objectionable content or match content to an index of banned items (for example, illicit materials, child abuse, terrorist content, etc.) – and then it allocates a ‘hash’ or an ID so that if these are detected again, the uploading process will be disabled. But then guess who will need to set the parameters before the automation can work!?

    Automated systems using AI and machine learning still have a long way to go before they can carry out content moderation independently (free of human help that is). We are surely not there yet.

    Content moderation is arguably one of the most important tasks that BPOs perform today, fulfilling outsourced contracts for social media giants ranging from Facebook and TikTok to Live, among many others. This has led to a process-driven BPO[xv] industry that has become the refuge for quick-fix content moderation based on subjective criteria. Add to that how many of the mods are often young people (their average age is less than thirty), who sometimes join even before finishing college degrees, and the problems begin to add up.

    The Need for (Content Upload) Speed and…Training!

    One might have assumed that US companies who hire moderators would have a good understanding of these issues, but it turns out that they really don’t. It has been reported for instance that Facebook doesn’t provide ongoing cultural education for these moderators to bring them up to speed. The one exception is when a particular issue goes viral on Facebook, and there’s a sudden need to bring everybody up to speed in real time. With this laissez faire approach it is unsurprising how many Court, Senate and Congressional hearings Mark Zuckerberg has had to attend over the past four years (and not just for the Cambridge Analytica scandal).

    One former moderator shared how he witnessed images of child sexual abuse[xvi] and bestiality with me while weeding out content that was unsuitable for the platform. He suffered from psychological trauma as a result of these working conditions and a lack of proper training.

    Accenture is one of the companies that hires contract workers to review content for big networks like Google, Facebook, and Twitter. There is a well-documented history of content moderators reviewing[xvii] including graphic and disturbing imagery – with jobs taking significant mental health tolls, and leading to psychological trauma.

    In order to share more of what goes on during content moderation, the freelancers have to break the nondisclosure agreements first, and this is an area where there is journalistic investigations and research work pending. One of the burning questions is whether the company has anything to say about the psychological and emotional impact of watching the brutality, pornography, and hate that the moderators have to look at on a daily basis?

    Some Debt Cannot be Repaid

    Facebook has already paid out a $52 million settlement to content moderators suffering from mental health problems such as Post Traumatic Stress Disorder (PTSD).[xviii] In light of repeated allegations and the seriousness of the situation, the company has agreed to compensate American content moderators and provide extra counselling during their tenure. The social media giant will pay a minimum of $1,000 to each moderator.[xix]  The settlement covers 11,250 moderators which is a glimpse at the colossal number (in hundreds of thousands) of moderators involved in this work globally.

    “I know it’s not normal, but now everything is normalized[xx],” said a moderator who declined to share his name and other details because of the confidentiality clause he signed when he took the job. Non-disclosure agreements are non-negotiable for moderators, and are forcibly imposed by the platforms. For example, YouTube content moderators are reportedly being told they could be fired if they don’t sign ‘voluntary’ statements acknowledging their jobs could give them PTSD (i.e. post-traumatic stress disorder).

    Reports also shows that Accenture managers repeatedly coerced site counsellors to break patient confidentiality.[xxi] Although these allegations were refuted by Accenture, such fault lines between workers and management are bound to affect organisational morale.

    Further studies are elusive on whether companies such as Accenture are shifting the responsibility of mental health care onto individual employees, and thus avoiding liability in the face of increasing lawsuits from dormer moderators. In response to growing allegations, certain social media giants have reinstated their commitment towards safeguarding their employees’ mental health and have clinical psychologists on call.

    The Valley of Uploads

    While some of the specifics remain intentionally obfuscated, content moderation is done by tens of thousands of online content moderators, mostly employed by subcontractors in India and the Philippines, who are paid wages well below what the average Silicon Valley tech employee earns. We need more studies and investigations on this as time progresses, as our hunger for newer ‘tailor-made’ media feeds continues to grow.

    The general assumption is that the large tech companies can easily hide the worst parts of humanity, otherwise freely available on the internet. There is no easy solution. With billions of users and unending uploads, there will never be enough moderators to check everything before it is shared with the world.[xxii]

    Legal challenges and new methods of reporting abuse help to narrow the risks, but the task is nonetheless Sisyphean. The complexities are ongoing, ever-growing and multi-faceted. The trade-off between a ‘quick fix’ of myriad issues would still create a dispersed range of unintended externalities to the stakeholders involve. This list includes the users, content moderators, companies, lawmakers and legal systems monitoring these behemoth digital platforms.

    [i] Madhumita Murgia, ‘Facebook content moderators required to sign PTSD forms’, Financial Times, January 26th, 2020, https://www.ft.com/content/98aad2f0-3ec9-11ea-a01a-bae547046735

    [ii] Jamie Fullerton, ‘Teenage girl kills herself ‘after Instagram poll’ in Malaysia’, May 15th, 2020 https://www.theguardian.com/world/2019/may/15/teenage-girl-kills-herself-after-instagram-poll-in-malaysia

    [iii] Marie Boren, ‘Life as a Facebook moderator: ‘People are awful. This is what my job has taught me’’ Irish Times, February 27th, 2020, https://www.irishtimes.com/business/technology/life-as-a-facebook-moderator-people-are-awful-this-is-what-my-job-has-taught-me-1.4184711.

    [iv] Jennifer O’Connell, ‘Facebook’s dirty work in Ireland: ‘I had to watch footage of a person being beaten to death’’, Irish Times, March 30th, 2019, https://www.irishtimes.com/culture/tv-radio-web/facebook-s-dirty-work-in-ireland-i-had-to-watch-footage-of-a-person-being-beaten-to-death-1.3841743

    [v] ‘Managing and Leveraging Workplace Use of Social Media’, SHRM, January 19th, 2019,  https://www.shrm.org/resourcesandtools/tools-and-samples/toolkits/pages/managingsocialmedia.aspx

    [vi] Daisy Soderberg-Rivkin, ‘Five myths about online content moderation, from a former content moderator’. October 30th, 2019, https://www.rstreet.org/2019/10/30/five-myths-about-online-content-moderation-from-a-former-content-moderator/

    [vii] ‘Inside Facebook, the second-class workers who do the hardest job are waging a quiet battle’, Washington Post, https://www.washingtonpost.com/technology/2019/05/08/inside-facebook-second-class-workers-who-do-hardest-job-are-waging-quiet-battle/

    [viii] Terry Gross,  ‘For Facebook Content Moderators, Traumatizing Material Is A Job Hazard’, NPR, July 1st, 2019,

    [ix] Ibid, O’Connell, March 20th, 2019.

    [x] Ibid, Soderberg-Rivkin, October 30th, 2019.

    [xi] Ibid, O’Connell, March 20th, 2019.

    [xii] Ibid O’Connell, March 20th, 2019.

    [xiii] Prithvi Iyer, Suyash Barve, ‘Humanising digital labour: The toll of content moderation on mental health,’ Digital Frontiers, April 2nd, 2020, https://www.orfonline.org/expert-speak/humanising-digital-labour-the-toll-of-content-moderation-on-mental-health-64005/

    [xiv] Ibid O’Connell, March 20th, 2019.

    [xv] Prasid Banerjee, ‘Inside the secretive world of India’s social media content moderators’, LiveMint, March 18th, 2020, https://www.livemint.com/news/india/inside-the-world-of-india-s-content-mods-11584543074609.html

    [xvi] Kelly Earley, ‘Irish content moderators prepare lawsuit against Facebook and CPL’ December 4th, 2019, https://www.siliconrepublic.com/companies/irish-content-moderators-facebook-cpl-recruitment

    [xvii] Paige Leskin, ‘Some YouTube content moderators are reportedly being told they could be fired if they don’t sign ‘voluntary’ statements acknowledging their jobs could give them PTSD’, January 24th, 2020, https://www.businessinsider.in/careers/news/some-youtube-content-moderators-are-reportedly-being-told-they-could-be-fired-if-they-dont-sign-voluntary-statements-acknowledging-their-jobs-could-give-them-ptsd/articleshow/73594478.cms

    [xviii] Untitled, ‘Facebook to pay $52m to content moderators over PTSD’, BBC, May 13th, 2020, https://www.bbc.com/news/technology-52642633

    [xix] Ibid

    [xx] Elizabeth Dowskin et al, ‘Content moderators at YouTube, Facebook and Twitter see the worst of the web — and suffer silently’, July 25th, 2019, https://www.washingtonpost.com/technology/2019/07/25/social-media-companies-are-outsourcing-their-dirty-work-philippines-generation-workers-is-paying-price/

    [xxi] Sam Biddle, ‘Trauma Counselors Were Pressured to Divulge Confidential Information About Facebook Moderators, Internal Letter Claims’, The Intercept, August 16th, 2019, https://theintercept.com/2019/08/16/facebook-moderators-mental-health-accenture/

    [xxii] Ibid, Soderberg-Rivkin, October 30th, 2019. https://www.rstreet.org/2019/10/30/five-myths-about-online-content-moderation-from-a-former-content-moderator/