In 2007, Facebook released the "Facebook Platform". Following the scandal where Cambridge Analytica used the platform to harvest data of some 87 million users and influence the US election and Brexit referendum, Deputy General Counsel Paul Gewal at Facebook confirmed that this was not a data breach - the system was operating as intended.
That the system was designed to reduce user privacy and abuse user trust for the profit of corporate payers went almost unnoticed in the uproar over how the data was used, but it marked a fundamental shift in the relationship between software developers and their users. Until then, users could trust that software they used was designed and built for them.
*
Having begun with a promise "it is free and always will be", Facebook's model to fund via advertising dollars meant that their software would be built for advertisers - and their users would be the product. During that golden age of computing, millions of hours of the best engineering skills in the world turned to the purpose of extracting user time, attention and engagement.
A wave of predatory, extractive and harmful design patterns emerged and slowly warped about half the population of the earth from humans trying to connect with their communities into a harvestable commodity. It simultaneously transformed the moral landscape of our digital world from one of trust, to one of exploitation. Like planned obsolescence before it, the moral valence flipped from an honest trade of producing the best possible products which users would pay for, to the most maximally extractive product they users won't quite abandon. This is a very delicate and a very deliberate process.
*
Now, two families in Texas are marking the new low-water mark in the moral sewer of big tech. Their claim against Google-backed Character.ai is that "through its design" AI-powered chatbots use "addictive and deceptive designs" which harm kids. Harm which extends in at least one case, the filing claims, to suicide.
Social Media Victims Law Center founder Matt Bergman said in an interview "It's akin to pollution". It is a cry echoing around the world, with Stephen Fry using the same phrasing in his recent speech at King's College. "They are the worst polluters in human history, worse than any chemical plant ever. You and your children cannot breathe the air or swim in the waters of our culture without breathing in the toxic particulates and stinking effluvia that belch and pour unchecked from their companies into the currents of our world."
If you are a software engineer, and you work for one of these companies, your conscience should be piqued. These organisations have demonstrated that shareholders will not demand accountability, that the leadership will not prioritise it and that regulation is a joke not even worth laughing at. The return of ethical systems and products to our industry must be led by those who actually build them and that means you.
Famously, engineers value objective truth and social good over subjective loyalty and personal greed. Google's famous "Don't be evil" motto was the perfect epitome of this value system, before it was retired by a bunch of MBAs. It seems only a grass roots movement of ethical engineers refusing to build evil systems can save our digital world.
Comments
Post a Comment