At heated City Council hearing, Philly tech officials “failed” their first AI test

At a Philadelphia City Council hearing on AI in government last week, city technology officials affirmed the city’s commitment to responsible use of technology. 

But the city’s chief information officer, Melissa Scott, and Mayor Cherelle Parker’s chief legal counsel, Kristin Bray – who also oversees Philly Stat 360, a city best-practices agency – failed to offer details and, in many cases, to answer basic questions, leaving those in attendance unconvinced of Philadelphia’s readiness for the rapidly evolving cyberspace.

“This was an open-book test, and they failed,” Rue Landau, the council’s tech committee chair, told City & State afterward. “It was definitely concerning to not have them come prepared.”

The response from a packed room of community members at Philly CAM, a multimedia center, was just as damning. Against a backdrop of signs bearing messages like “AI is watching us – who is watching AI,” one speaker called the city’s presentation “embarrassing,” and another grumbled, “We about to get played.” Again and again, speakers raised serious and urgent concerns about privacy, surveillance and the accountability of both AI-wielding agencies and the big-tech behemoths managing the data.

Prior to the Wednesday hearing, Landau said, she had submitted a list of questions to Philadelphia’s technology officials. But Scott and Bray offered few specifics about Philadelphia’s current uses of AI or its safeguards to protect residents’ privacy. 

In response to Landau’s direct questions, the officials repeatedly told Landau they would need to get back to her with answers.

“I thought I heard you say that you’re not using AI in the realm of public safety here in Philadelphia,” Landau asked Scott at one point.

“Now, to our knowledge, we’re not,” Scott answered.

Landau responded that the City of Philadelphia does, in fact, currently use several AI tools in public safety, including facial recognition and gunshot-detection technologies as well as drones.

Bray said that the city plans to issue formal AI guidelines for employees in spring 2026, and Scott repeatedly gave the example of training employees to take precautions while uploading data into software programs. But as many panelists pointed out, the scope of AI runs the gamut.

Landau said that she was especially concerned about the impact of artificial intelligence – and its potential for abuses – on the city’s most vulnerable populations. That concern was echoed by numerous panelists and community members, who worried about the unintended consequences of poorly regulated AI on everything from employment to criminal justice.

“This technology is moving very fast,” Landau said. With a U.S. Supreme Court that “doesn’t seem very set on enforcing the current laws that we have,” she called on the city to step up and define safeguards to avoid “perpetuating discrimination, perpetuating inequities.”

That concern was echoed by panelist Alli Finn of the AI Now Institute. “With the push for deregulation on AI at the federal level, it is local governments, such as this one, that are on the front lines of protecting constituents,” she said.

Landau challenged Bray and Scott about how the city is protecting residents’ sensitive information from potential misuses by AI vendors – for instance, endangering immigrants by making data available to immigration authorities. Other speakers raised worries about technologies that could track residents’ whereabouts and activities or misidentify people.

“What oversight do you have with a company … to make sure that they are not changing the terms of what they’re using with your data?” Landau asked. Scott’s answer – about training and security assessments – failed to satisfy Landau, as did Bray’s repeated but vague assertions about the city’s commitment to responsible use.

Councilmember Jim Harrity was among the speakers raising concerns about the city’s strategy around potentially invasive technology, which he called “a slippery slope.” 

He acknowledged he was “very leery” of AI, referencing the 2002 dystopian film “Minority Report,” in which police use technology to arrest people before they commit crimes. 

“I don’t want to give up my freedom for somebody else’s idea of safety,” he said, drawing applause. “We need to be very careful … There are consequences to things we do now, down the line.” 

In addition to the human consequences of surveillance gone amok, Harrity also cautioned that misuse of AI public safety tools could lead to expensive lawsuits, as has happened in other municipalities.

At one point, Landau asked Scott to identify the city’s AI experts. Scott responded: “If anyone in the world would say that they were an AI expert, they will be lying, because it’s continuously transforming.”

But several speakers testified that they were, in fact, AI experts – including Finn; fellow panelist Sorelle Friedler, a Haverford College computer science professor who previously worked at Google; and Kendra Albert, a Philadelphia-based technology lawyer and nationally recognized scholar in algorithmic justice who has worked at Harvard Law School’s Cyberlaw Clinic.

Landau told City & State later that she is confident that Philadelphia’s robust academic and technology communities include AI experts, and hoped that the city’s proposed AI governance committee would engage them. She also vowed to convene many more hearings to address the issue.

“I know firsthand that we are not the most advanced, technologically” speaking, she said of Philadelphia city government, adding that she was concerned about a poorly conceived “scramble to catch up.” “We should be far more along in creating policies and systems and oversight and monitoring and educating our city employees than we are.”