The Technology 202: Facebook struggles to keep violent ‘boogaloo’ content off its platform

with Cat Zakrzewski

Content from the extremist anti-government “boogaloo” movement is thriving on Facebook, outside researchers and a review by The Washington Post found. The persistence of the content highlights Facebook’s ongoing struggle to keep domestic extremists from using its platform for recruitment and propaganda.

Researchers at the global nonprofit group Avaaz found nearly two dozen Facebook pages affiliated with the “boogaloo” movement, a generally anti-government and anti-law enforcement ideology. Despite the amorphous nature of the online movement, members have become a notable physical presence at both rallies against pandemic shutdowns and more recently Black Lives Matter protests against police brutality. 

Posts on pages flagged by Avaaz between May 28 and June 18 included explicit calls for armed violence as well as more borderline content such as anti-government memes employing euphemisms for violence. Some pages also shared misinformation about the protests, such as memes claiming police were placing bricks to cause riots and conspiracy theories about Bill Gates and George Soros. The majority of the pages were created within the past six months and had a collective following of tens of thousands of users. 

“It’s all targeted towards building distrust of police and basically saying that the U.S. needs armed rebellion.” Fadi Quran, campaign director at Avaaz, says.

The violent “boogaloo” content should not be on Facebook.

Facebook changed its community standards in May to ban pages and groups using “boogaloo” and affiliated terminology including “Big igloo and “Big Luau,” when accompanied by images of armed violence. And earlier this month, Facebook said it would stop recommending other boogaloo content to members of boogaloo groups, Reuters reported. 

The Washington Post sent Facebook 15 different posts from 12 of the groups flagged by Avaaz appearing to endorse violence in some form. Five of the pages had some version of “boog” in their names.

As of press time, Facebook had removed 11 of the 12 flagged pages. Seven of the pages were removed for multiple pieces of content violating Facebook’s policy against violence and two were removed because accounts administering the pages violated site policies (the accounts have also been removed). One page was still being reviewed. Facebook removes content in violation of its community standards after it is reviewed by a moderator.

But other content recommended to visitors of boogalo-related Facebook pages is also problematic.

The Post was easily able to find multiple instances on Facebook where boogaloo-themed pages and private groups featured a sidebar of suggested other boogaloo content. Facebook uses these suggestion tools to drive users to other content on the platform, but the feature has been criticized by researchers for directing users toward extremist information.

“We continue to remove content using boogaloo and related terms when accompanied by statements and images depicting armed violence,” Facebook spokeswoman Sarah Pollack said in a statement. “We are also preventing these Pages and groups from being recommended on Facebook.”

Facebook could not offer numbers on how many boogaloo pages and groups have been removed since it introduced new policies regarding the movement.

The boogaloo movement is hard to define — which makes it difficult to moderate.

Members are strong opponents of anything they perceive to infringe on the Second Amendment. Many of the memes on the Facebook pages express disdain for both President Trump and presumptive Democratic nominee Joe Biden. Some accounts and memes express support for Libertarian presidential candidate Jo Jorgensen. Some pages reference her in their name.

Some posts on both public Facebook pages and in private groups reflect the movement’s anti-government views and calls for armed rebellion. For instance, two pages reviewed by The Post featured posts that gave recipes for molotov cocktails, including one featuring Mickey Mouse calling them a “surprise tool.”

A post from one group flagged by the Avaaz researchers that Facebook has removed. (Facebook)

“It’s like Jell-O. It just keeps changing and moving,” Megan Squire, a professor of computer science at Elon University who studies online extremism, said of the movement. “That makes it concerning because really bad people are taking advantage of that and moving their ideas in.”

For instance, while some pages have recently publicly disavowed white supremacy, the movement has still attracted members who believe in the ideology, Squire and other researchers have found.

The language and imagery used by the movement is constantly evolving, making it difficult to track, as well, Squire says. Facebook account administrators often share “backup pages” to divert users if Facebook bans the original ones. After Facebook banned several high-follower pages earlier this month, they popped up under new names using phrases such as “Liberty” or “REDACTED.” 

“At least from what we can see there really isn’t much more action beyond the kind of whack-a-mole effect. The backups aren’t removed,” says Katie Paul,  director at the Tech Transparency Project. “Or only one of the dozen groups that a particular page moderates is removed.”

Paul says it’s similar to Facebook’s ongoing struggle to keep white supremacists off the platform.

Even after Facebook banned white-nationalist content in 2019, groups and pages representing the ideology have continued to surface on the platform under other names, BuzzFeed, the Guardian and numerous other media outlets have found. 

Just last week Facebook removed an additional 900 social media accounts linked to white-supremacist groups that discussed bringing weapons to police brutality protests. The removals show that even after banning certain kinds of content, keeping it off the site has proved difficult for the billion-dollar company.

There’s a connection between the movement and recent real-life incidents of violence.

Federal prosecutors in California, Texas, Nevada and Colorado have issued a series of criminal charges against self-described boogaloo supporters. The movement and its follower have also sparked concern from the intelligence community, which warned last week that adherents of the ideology could target Washington, D.C., “due to the significant presence of U.S. law enforcement entities, and the wide range of First Amendment-Protected events hosted here,” Natasha Bertrand at Politico reported.

Researchers say public Facebook pages like those flagged by Avaaz are often the first step to users finding private Facebook groups and links to other platforms where more explicit content, including militia guides and manifestos, and plans for real-life meetups, are found. 

Three suspects arrested earlier this month on terrorism-related charges were members of private “boogaloo” groups first flagged in April by the Tech Transparency Project.

Facebook only removed two of the groups after those arrests; the company removed a third after another self-identified “boogaloo boy” Steven Carrillo was charged with killing two Oakland police officers. 

One of the groups is still active on Facebook, Paul says. Two of the Facebook groups were managed by the same users who still run multiple accounts and have created an intelligence-sharing group to replace the one that no longer exists.

Manifestos and tactical manuals shared in the removed groups are still circulating in the new private groups, Paul says. And so is praise for Carrillo.

While researchers acknowledge that moderating boogaloo groups presents a significant challenge, they still say Facebook isn’t doing enough.

“This movement lives online,” Paul says. “Many of these guys are not based in the same location. If Facebook deplatformed these groups, it would make it much more difficult for them to connect with one another.”

Squire says that Facebook’s recent moderation push has caused “confusion” but that boogaloo followers are using other channels such as the messaging app Discord to regroup. Both researchers noted it’s unlikely the movement would  entirely give up Facebook because its tools and recommendation algorithms make it easier to find new members than other social media platforms.

Some researchers are worried the movement could cause chaos in the 2020 elections.

Quran, who has also studied how just a handful of high-follower accounts and pages are behind most coronavirus misinformation, expressed concerns the massive followings of Facebook boogaloo pages and groups could be weaponized to spread election disinformation or spark violent protests.

“The scale and danger of what Facebook is doing to the societies ahead of this election cannot be magnified enough,” Quran says. “Boogaloo is just one piece of a puzzle of a much more complex system, but they’re a piece of the puzzle that can be extremely disruptive.”

Our top tabs

Twitter hid another tweet from President Trump behind a warning label. 

It’s the fifth time in recent weeks that the company has cracked down on Trump’s tweets for violating its policies, my colleague Rachel Lerman reports. The company Trump’s tweet violated its policy on abuse, specifically “the presence of a threat of harm against an identifiable group.”

The tweet said: “There will never be an ‘Autonomous Zone’ in Washington, D.C., as long as I’m your President. If they try they will be met with serious force!”

The company hid the tweet behind a gray box, and it also prevented people from retweeting or liking it. The company took similar action against Trump’s post that said “when the looting starts, the shooting starts,” several weeks ago. It also has labeled tweets that violate other policies on misinformation.

The move threatens to escalate the battle between Trump and tech companies, who he thinks are biased against conservatives.

White House press secretary Kayleigh McEnany tweeted: 

The company also suspended @CarpeDonktum, who was credited with creating a crudely edited video of two toddlers that President Trump shared last week. Twitter ultimately removed the video per a copyright complaint, after labeling it as manipulated media. From CNN’s Donie O’Sullivan:

Senate Republicans want to end “warrant-proof” encryption.

Legislation introduced by Senate Judiciary Committee Chairman Lindsey O. Graham (R-S.C.) and other Republicans would require tech companies to help law enforcement officials with a warrant to access encrypted data on their systems. 

“The Lawful Access to Encrypted Data Act” would authorize the attorney general to issue directives requiring tech companies comply with the law, but not the authority to tell tech companies exactly how to do it. The lawmakers argue data-protecting technology shields terrorists, child predators and other criminals from law enforcement. Encryption advocates and tech companies have long contended creating a backdoor for law enforcement would also open the floodgates for bad actors to break into their systems. 

The new legislation is being introduced just two days before the panel is scheduled to discuss Graham’s “EARN IT Act,” a bill with bipartisan support to strip tech companies of liability protections if their users share material exploiting children. Encryption advocates have warned the bill seems positioned to pressure companies such as Facebook against encrypting their services. Both bills have a slim chance of passing ahead of the November election.

The new legislation follows a months-long push by Attorney General William P. Barr against encryption. 

Barr tried unsuccessfully to push Apple to help unlock encrypted iPhones used by a Saudi Air Force student who opened fire last year at a U.S. military base in Pensacola, Fla. He also has criticized Facebook’s push to encrypt all its messaging services, alleging it would be more difficult to track down online child predators.

Germany’s top court ruled that Facebook broke competition laws. 

The move could embolden European regulators who have been increasingly scrutinizing technology companies, Adam Satariano reports for the New York Times. The case upheld a novel application of antitrust law, which argued Facebook ran afoul of rules by combining data it collected about users across its different platforms, including WhatsApp and Instagram.  

“There are neither serious doubts about Facebook’s dominant position on the German social network market nor the fact that Facebook is abusing this dominant position,” the court said. “As the market-dominating network operator, Facebook bears a special responsibility for maintaining still-existing competition in the social networking market.”

Facebook will have to make changes to its data collection processes in Germany. The company will have to allow users to block Facebook from combining their data from the main social network with their activities on other apps and websites.  

It’s not the final ruling on the matter. A lower court must issue a ruling as well.  

Rant and rave

Facebook said it would stop selling Oculus Go, its cheapest virtual reality headset. 

Some virtual reality fans were sad to see the headset go:

Others were already trying to plan to time the eBay market for the recently retired device: 

Hill happenings

A pair of senators wants the Securities and Exchange Commission to look into allegations the company hid illegal activity from investors.

The complaint, filed last month by the National Whistleblowers Center, claims Facebook violated its fiduciary duties by not informing shareholders about drug sales and other illegal activity, as Nitasha Tiku first reported. 

The claims raised in the complaint “represent a troubling pattern on the part of Facebook, a prioritization of revenue over public safety,” Sens. Robert Menendez (D-N.J.) and Bill Cassidy (R-La.) wrote in a letter to SEC Chairman Jay Clayton shared first with The Technology 202.

The letter cites allegations from one whistleblower, an ex-employee at a cybersecurity firm hired by Purdue Pharma to police online counterfeiters of its drug OxyContin, who said Facebook refused to take down illegal offers to sell the drug. The senators say allegations Facebook lobbied other platforms, including Twitter, not to engage with OxyContin to remove counterfeits is “even more concerning.”

If true, the allegations could make Facebook liable under federal securities law. Facebook previously denied withholding risks from shareholders.

Facebook has “regularly disclosed potential risks related to content in our SEC filings, including at least four times in the last year,” Facebook spokesman Joe Osborne told Nitasha last month.

Advocacy groups are launching a new campaign to pressure Congress to subpoena Jeff Bezos.

The campaign asks members of the public to contact lawmakers to urge a subpoena of the Amazon chief executive. The groups, which include Athena, Fight for the Future, the Open Markets Institute, Demand Progress and the Institute for Local Self-Reliance have launched a call page called SubpoenaBezos.com with a contact tool to guide users.

“Jeff Bezos may be the richest man on Earth, but that does not make him above the law,” said David Segal, executive director of the Demand Progress education fund. “If Bezos continues to dismiss Congress, and by extension us, then it is the duty of our representatives to help him understand reality by way of subpoena. ”

Bezos recently signaled he was open to appearing in front of the House Judiciary Committee for a hearing on antitrust concerns. But the company says it still has questions regarding timing, format and outstanding document production. 

The campaign follows a letter to the committee last week signed by 66 groups from across labor, small-business and other sectors demanding the panel issue a subpoena to Bezos and other key tech executives.

Bezos owns The Washington Post.

Workforce report

There’s a growing push by Amazon employees for the company to address racial inequality.

A group of employees are working on a proposal to demand the company’s leadership make diversity a guiding factor in the hiring, review and promotion of workers, The New York Times reports. Dozens of employees involved in the effort cite experiences of discrimination at the company. 

In the weeks since the death of Floyd in police custody, the company has responded with “listening circles” between black employees and executives, the Times reports.

Warehouse workers have also reported discrimination, says the Times. A warehouse contractor last week filed a complaint with California’s fair employment agency after saying he was fired in June for allegedly taking a photo of racist graffiti at the warehouse then posted to Twitter.

Trending

Mentions

• Former Citi FinTech CEO Yolande Piazza has joined Google Cloud as vice president, financial services, according to a news release.
• The Global Internet Forum to Counter Terrorism (GIFCT) announced Nicholas J. Rasmussen as the first full-time executive director of the organization.

Daybook

Today:

• The Aspen Tech Policy Hub will host an event on “Protecting Your Digital Reality”  at 12 p.m.
• The Senate Commerce Committee will hold an oversight hearing to examine the Federal Communications Commission at 10 a.m.
• The Energy and Commerce Committee will host a hearing on online disinformation on  at 11:30 a.m. The hearing will cover disinformation related to covid-19 and the recent racial unrest.

Coming up:

• The Bridge and All Tech is Human will host a discussion on Section 230 Thursday at 12 p.m.
• The Senate Judiciary Committee will markup the EARN IT Act of 2020 on Thursday.
• Carnegie’s Partnership for Countering Influence Operations and Twitter will host an event on influence operations on Twitter on July 9 at 1 p.m.

All events are online unless otherwise indicated.

Before you log off

We all miss baseball: