At tech companies, let’s share the reins Kathy Pham profile picture by Kathy Pham 
Sunday, March 3, 2024, 06:18 AM
Kathy Pham is a Computer Scientist, Senior Advisor at Mozilla; Vice President of AI and Machine Learning at Workday; Adjunct Lecturer at Harvard. Opinions here are her own and not on behalf of any affiliated organization.

In 2023, we learned a lot about AI models: What’s possible, like discovering a new class of antibiotics. What's uncertain, like how exactly businesses will integrate chatbots into their operations. And how the technology might be regulated and implemented across governments, per the Blueprint for an AI Bill of Rights, the NIST Risk Management Framework, the EU AI Act, and the United States’ executive order on AI.

But for those within the industry — or those paying close attention — there was another big learning. That the people building the foundation of the most influential technology of our era are overwhelmingly from a very narrow set of disciplines: computer science, software engineering, and even more specifically, machine learning.

It makes sense for the people writing the code to be computer scientists and programmers — that’s our trade. But building these models is about a lot more than code. It’s about identifying use cases, making design decisions, and anticipating and mitigating potential harms. In short: It’s about making sure the technology isn’t just functional, but also trustworthy. And this requires skill sets beyond machine learning and traditional computer science.

I know this well from my own experiences across academia, non-profits, government, and industry. The hallmark of truly impressive technology is productive applications — and companies only build the best and right things when they pull together experts across disciplines.

Five years ago, I published an essay about the consequences of this bias in the industry and how we might fix it. I argued that if we want to change the tech industry — to make software more fair and less harmful — then we need to change what classes are required for a computer science degree.

AI has grown exponentially since then. Its tremendous computing capacity means even more unpredictable, unintended harms. My original thesis remains true: We need to expand computer science curricula to include the humanities. But those changes take time. Right now, we also need to break down the persistent engineering / non-engineering divide directly at the industry level.

There’s no shortage of stories about AI systems’ harms: hallucinations that generate incorrect information, disinformation, sexist and racist outputs, toxic training sets. In many cases, these are problems that engineers overlooked, but that others — like cognitive scientists and media theorists — confronted after the fact. The order of operations is clearly backward: These harms should be addressed before the system is deployed. Proactive, not reactive, should be the status quo, and that’s only possible with more varied expertise. It’s true that some companies have made progress on this front with their trust and safety teams. But those teams are still siloed and splintered.

There are some bright spots these companies can emulate: In recent years, Airbnb has centered the work of Laura Murphy, a legal and civil rights expert, to fight discrimination in product features. In 2018, Salesforce established an Office of Ethical and Humane Use — a place for internal frontline employees, executives, and external experts across a broad range of functions to guide decision-making. (I was an inaugural advisory board member.) When AI really entered the zeitgeist, the office was ready for action.

It isn’t a big ask for the engineers to talk with the lawyers or social scientists. A simple conversation — “Hey, if I create Booleans with two choices for gender in my model, is that OK?” — can head off bias and the need for an audit later on. I’m often reminded of my late father, a truck driver. There are countless engineers working to streamline the trucking industry. But how many of them spend time speaking with actual truckers?

In addition to overt harms like bias, there are also harms of omission. A dearth of humanities experts means a missed opportunity for AI systems with clear use cases and problem-solving capabilities. Right now, many AI systems are developed and deployed first, with a goal of finding a purpose later. Consider OpenAI’s 128K context window or Anthropic’s 200K context window: technically impressive, and lacking clear objectives until they are used for meaningful applications for society. Research and development without a clear goal isn’t necessarily a bad thing, but these systems use tremendous amounts of money and energy to train. And we also know real goals exist, like better cancer detection algorithms for people with darker skin. There are powerful examples of what’s possible with this approach, like the Kiazi Bora app in Tanzania. Its creators identified a problem first — a lack of agricultural literacy among women in East Africa — and then built an AI chatbot to help solve it.

For AI systems to be more trustworthy, tech companies must prioritize a broad range of expertise. Yes, there has been encouraging progress on this front: Initiatives like All Tech is Human are connecting engineering to the social sciences and humanities, programs like Mozilla’s Responsible Computing Challenge are reimagining computer science curricula as encompassing humanistic studies, and teams like Workday’s Responsible AI are convening boards that spans disciplines.


Comments

Add Comment
Fill out the form below to add your own comments.









Insert Special:
:o) :0l







Moderation is turned on for this blog. Your comment will require the administrators approval before it will be visible.