This post was updated at 14:40 CET to note result from <a href="https://www.consilium.europa.eu/en/press/press-releases/2018/12/06/terrorist-content-online-council-adopts-negotiating-position-on-new-rules-to-prevent-dissemination/">6 December 2018</a> meeting of the Home Affairs Council.
There’s a reason why, when courts function properly, they offer more <a href="https://santaclaraprinciples.org/open-letter/">due process</a> than corporations when it comes to making decisions about free expression. Deciding what speech can take place in public forums in democratic societies is not an easy task. While standards range from permissive jurisprudence of the First Amendment to the broader prohibitions against hate speech found in German law, at least these standards can be litigated, discussed, and understood. So why is the European Commission <a href="https://cdt.org/press/cdt-and-human-rights-organisations-urge-ministers-to-reconsider-proposed-eu-terrorist-content-rules-citing-threats-to-free-expression/">trying to push through</a> a regulatory proposal that would not only force private corporations to regulate broad swathes of expression, but would actually require the use of opaque filters and artificial intelligence driven algorithms to do so? The only answer is that this proposal, <a href="https://www.consilium.europa.eu/en/press/press-releases/2018/11/30/main-topics-and-media-events/#">set to be considered today</a>, is political—and it must not pass.
The proposal
The European Commission has proposed a regulation on “<a href="https://www.parlament.gv.at/PAKT/EU/XXVI/EU/04/57/EU_45743/imfname_10862334.pdf">dissemination of terrorist content online</a>,” based on <a href="http://www.europarl.europa.eu/legislative-train/theme-area-of-justice-and-fundamental-rights/file-preventing-the-dissemination-of-terrorist-content-online">a process</a> which started in September 2017. The Commission adopted a recommendation on “illegal content” more broadly in <a href="https://ec.europa.eu/digital-single-market/en/news/commission-recommendation-measures-effectively-tackle-illegal-content-online">March of 2018</a>, and proposed specific regulations on “terrorist content” in September of this year. A recent draft introduced slight revisions from the <a href="https://ec.europa.eu/commission/sites/beta-political/files/soteu2018-preventing-terrorist-content-online-regulation-640_en.pdf">September version</a>, but not in a way that addresses the grave concerns raised by civil society and business alike. The most recent version can be found <a href="https://www.parlament.gv.at/PAKT/EU/XXVI/EU/04/57/EU_45743/imfname_10862334.pdf">here</a>. The regulation is being discussed today in the <a href="https://www.consilium.europa.eu/en/meetings/jha/2018/12/06-07/#">Justice and Home Affairs Council</a> and supporters are trying to push a “<a href="https://www.consilium.europa.eu/en/council-eu/decision-making/">general approach</a>,” which means less opportunity for discussion. <a href="https://www.politico.eu/pro/european-ministers-endorse-position-on-terrorist-content-proposal/">Update from Politico on the meeting</a>: the majority of Ministers in the Home Affairs Council approved a “general approach,” but “countries including Finland, the Czech Republic and Denmark said they could not support the general approach. Others like Slovakia said they needed more time and more work at the technical level.”
The ostensible goal of this regulation is to force “hosting service providers” to remove “terrorist content” within one hour upon receipt of a removal order from any EU member states “competent authority,” and to use proactive measures, “including automated means,” to detect such content and prevent reappearance. The quotation marks all signify places in the regulation where definitions are unclear or poorly worded. The proposal would create specific obligations for hosting service providers to remove, to report on removals, and to coordinate their required “proactive measures” against terrorist content with authorities on an ongoing basis. Member states designate their own competent authorities, and set their own penalties, which can be up to “up to 4% of the hosting service provider’s global turnover of the last business year” if the state concludes that the hosting service provider has systematically failed to comply with the regulation. Furthermore, terrorist content is defined by each member state, and can vary from country to country. For example, in Spain a single law prohibits the glorification of terrorism alongside “humiliating victims of terrorism”—conflating two very different forms of expression. 
The problems
Much has already been written about the dangers of this regulation. Unfortunately, however, WITNESS has first-hand experience with how this proposal could go very, very badly—both inside and outside of the European Union—through dangerous copycat legislation and overuse of automated content moderation.
We agree with our allies, including the signatories to <a href="https://cdt.org/insight/letter-to-ministers-of-justice-and-home-affairs-on-the-proposed-regulation-on-terrorist-content-online/">a letter sent to the Ministers</a> on December 4, that the regulation is poorly written, unnecessary, and overbroad–and that it appears to be getting pushed through as a ploy, coming before the 2019 parliamentary elections. For good general background information and legal analysis, check out these articles:
<ul>
<li>European Digital Rights Initiative: “<a href="https://edri.org/eus-flawed-arguments-on-terrorist-content-give-big-tech-more-power/">EU’s flawed arguments on terrorist content give big tech more power</a>”</li>
<li>Center for Democracy and Technology: “<a href="https://cdt.org/blog/the-european-commissions-draft-regulation-on-terrorist-content-requires-significant-revision/">The European Commission’s draft regulation on ‘terrorist content’ requires significant revision</a>”</li>
<li>Stanford Center for Internet and Society: “<a href="http://cyberlaw.stanford.edu/publications/new-eu-proposal-prevention-terrorist-content-online-important-mutation-e-commerce">New EU Proposal on the Prevention of Terrorist Content Online: An Important Mutation of the E-Commerce Intermediaries’ Regime</a>”</li>
</ul>
We have two concerns about how, based on our experience, this regulation will be harmful.
Automated content moderation 
First and foremost, this proposal encourages increased use of machine-learning algorithms for identifying terrorist content, as well as increased automatic takedown through shared databases of material that has been deemed terrorist content. This architecture already exists, and it is shrouded in secrecy and error.
Through the <a href="https://gifct.org/">Global Internet Forum to Counter Terrorism</a> (GIFCT), many major companies already have a <a href="https://www.blog.google/around-the-globe/google-europe/update-global-internet-forum-counter-terrorism/">shared database</a> of extremist content that violates their Terms of Service (which means it might not even violate the law.)This database helps companies take down content before it is ever seen. Unfortunately, the GIFCT provides almost no information about this database publicly, including information about quality checks or reassessment. Errors in this database will be propagated throughout all of GIFCT’s members if not corrected.
Facebook and YouTube, who are GIFCT founding members, also <a href="https://blog.google/around-the-globe/google-europe/four-steps-were-taking-today-fight-online-terror/">already use</a> machine-learning algorithms <a href="https://newsroom.fb.com/news/2018/11/staying-ahead-of-terrorists/">to detect and remove</a> so-called extremist content. We have been dealing with the fallout of YouTube’s extremist content algorithm since <a href="https://blog.witness.org/2017/08/vital-human-rights-evidence-syria-disappearing-youtube/">August of 2017</a>, and alongside our partners the Syrian Archive, we have observed removals of hundreds of thousands of channels and videos that are <a href="https://www.nytimes.com/2017/08/22/world/middleeast/syria-youtube-videos-isis.html">documenting human rights abuses in Syria</a>.
What we don’t know is almost anything at all about those algorithms. We don’t have the most basic assurances of <a href="http://www.fatml.org/resources/principles-for-accountable-algorithms">algorithmic accountability</a> or <a href="https://www.acm.org/binaries/content/assets/public-policy/2017_joint_statement_algorithms.pdf">transparency</a>, such as accuracy, explainability, fairness, and auditability. Platforms use machine-learning algorithms that are proprietary and shielded from any review. Groups like WITNESS have to look to patterns and hearsay to get any idea of how they’re working. Unless they’re specifically <a href="https://news.mit.edu/2018/mit-lincoln-laboratory-adaptable-interpretable-machine-learning-0905">designed to be “interpretable,”</a> these algorithms can’t be understood by humans, since they learn and grow over time—but we can’t even get access to the training data or basic assumptions driving the algorithms. There has never been any sort of <a href="https://na.theiia.org/periodicals/Public%20Documents/GPI-Artificial-Intelligence.pdf">third-party audit</a> of such proprietary technology, although we would strongly support this if companies were open to it.
When it comes to accuracy, we have seen how even existing systems are already having disastrous effects on freedom of expression and documentation and research of human rights abuses. As noted, Syrian Archive has seen hundreds of thousands of videos go missing, and whole channels shut down. We have reviewed many of these videos personally. They are coming from groups recognized by journalists and human rights bodies like the United Nations. When they do depict extremist activities, they often indicate that they are trying to show the world the human rights abuses taking place in Syria. But many of these videos simply have no link to extremism—they are showing demonstrations, or the aftermath of a bombing.
Syrian Archive isn’t the only group relying on such <a href="https://www.law.berkeley.edu/wp-content/uploads/2018/02/Bellagio_report_2018_9.pdf">open source knowledge</a> to investigate human rights abuses. That list includes the <a href="https://www.irishtimes.com/news/world/europe/social-media-evidence-a-game-changer-in-war-crimes-trial-1.3243098">International Criminal Court</a>, <a href="https://www.hrw.org/report/2017/05/01/death-chemicals/syrian-governments-widespread-and-systematic-use-chemical-weapons">Human Rights Watch</a>, the United Nations, and even <a href="http://www.themedialine.org/featured/germany-sweden-take-lead-prosecuting-syrian-war-criminals/">prosecutors in Sweden</a> and Germany trying to <a href="https://www.hrw.org/report/2017/10/03/these-are-crimes-we-are-fleeing/justice-syria-swedish-and-german-courts">prosecute perpetrators of war crimes</a> and terrorism in Syria. These investigators don’t have an alert system in place with platforms that would allow them to review deleted material and save evidence that could later be crucial from being lost forever.
We have no doubt that this regulation strongly incentivizes companies to open the floodgates on automated content moderation.
A bad precedent for the world
WITNESS works at a global scale, and we have seen how policies made in democratic societies can be used to repress human rights in the wrong hands, or even simply how policies in one setting don’t work elsewhere when they are copied. At its most basic level, this regulation takes decisions about what speech is legal or illegal away from courts and lawmakers, and places those decisions with corporations and poorly defined “competent authorities. It encourages platforms to create machinery that easily could be used in undemocratic societies to silence critics. And it encourages the idea that fighting “terrorist content” can justify any legislative excess—an idea already embraced by <a href="https://www.forbes.com/sites/annaborshchevskaya/2016/07/08/brave-new-world-russias-new-anti-terrorism-legislation/">Russia</a>, <a href="https://www.eff.org/de/deeplinks/2018/07/draconian-new-cybercrime-bills-vietnam-and-egypt-will-only-increase-censorship">Egypt</a> and many other countries with terrible human rights records. 
The role of the European Union in setting norms that can potentially affect the human rights of billions is only growing larger. What’s more, no legal system has caught up with the Internet, and every new piece of legislation regulating it matters. Legislative responses to problems raised by the Internet, can and do spread globally, even when they <a href="https://freedomhouse.org/report/freedom-net/freedom-net-2018/rise-digital-authoritarianism">pose a threat to free expression</a>, like the many <a href="https://www.wired.co.uk/article/malaysia-fake-news-law-uk-india-free-speech">legal responses to misinformation</a> popping up around the world . That’s why we’re deeply concerened about the bad precedent that would be set globally should this regulation pass.
This is not hypothetical. Germany’s NetzDG law, which creates a <a href="https://www.hrw.org/news/2018/02/14/germany-flawed-social-media-law">dangerous system for rapid removal</a> of “illegal content” on social media, has already been c<a href="https://www.hrw.org/news/2018/02/14/germany-flawed-social-media-law">ited as a positive example</a> by many countries, including Russia, a country with an obviously atrocious record when it comes to <a href="https://freedomhouse.org/report/freedom-world/2018/russia">free expression</a>. In fact, <a href="https://rsf.org/en/news/russian-bill-copy-and-paste-germanys-hate-speech-law">Reporters Without Borders says</a> that a dangerous Russian law passed in 2017 “”is copy-and-paste of Germany’s hate speech law.” Even within Germany’s democratic system, NetzDG has already been used to silence <a href="https://www.dw.com/en/german-satire-magazine-titanic-back-on-twitter-following-hate-speech-ban/a-42046485">Tweets that were parodying</a> hateful speech from a far-right Alternative für Deutschland politician.
Governments around the world already abuse platforms’ terms of service to silence critics. Imagine how expertly governments that already abuse Facebook’s terms of service to silence dissent, <a href="https://www.buzzfeed.com/meghara/facebook-cambodia-democracy">such as Cambodia</a>, will abuse legislation like the terrorist content proposal in their own countries?
What’s next?
Although this regulation is being pushed hard, it’s not likely to pass before the European Parliament goes on its winter break on December 13. WITNESS will monitor the regulation, and will be sharing the above analysis of it with Ministers and Members of Parliament. We will continue to support efforts of our allies, such as this <a href="https://www.laquadrature.net/2018/12/03/44-organisations-demandent-a-macron-de-renoncer-a-son-projet-de-censure-anti-terroriste/">open letter from La Quadrature du Net</a>. For the most up-to-date news, follow us on Twitter: @mnemonicorg
This work was originally published by <a href="https://witness.org">WITNESS</a> and is licensed under a <a href="http://creativecommons.org/licenses/by-nd/4.0/">Creative Commons Attribution-NoDerivatives 4.0 International License</a>.

European “terrorist content” proposal is dangerous for human rights globally

.css-68jq5d{padding-bottom:1.45rem;color:#ffffff;-webkit-transition:0.25s;transition:0.25s;}.css-68jq5d:hover{-webkit-text-decoration:underline;text-decoration:underline;}BLOG

European “terrorist content” proposal is dangerous for human rights globally

BLOG