Interesting! I spend a decade working as a business ethicist in a big bank. I'm a bit afraid of what AI might bring to the table. There's already such relentless pressure to bring everything back to that one value: profit. Everything needs a business case. E.g. employee happiness is valuable because it promotes efficiency, innovation, retention, etc.. It's a means to an end, that end being inevitably money. It sounds innocent, until money and employee happiness are at odds with each other and inevitably the choice goes towards money. There's limits to this of course, but those limits can always be expressed as a business case (e.g. not valuing employees is inefficient). Basically, companies can be seen as reinforcement learning algorithms for creating profits. The paperclip maximizer is already with us, it's just making money and not paperclips.
The only real pressure back to this dynamic I could see (barring some change in corporate governance or legislation) were individuals refusing to go along with it or making suboptimal decisions (from a profit perspective) because they value something else more. In other words: the human element. Should an AI take over (part of) that role, then I'm afraid business moves even quicker and more relentlessly towards being "paperclip optimizers".
True, I have similar concerns that business incentives is going to optimize for optimizers. To the extent that we can bring the human element more into the mix, to include values besides just profit, I expect these "moral constitutions" to be a useful abstraction and target for incorporating human preferences. For instance, a public benefit corporation could be chartered with a moral constitution that *actually* binds it towards a broader value set besides profit (instead of the fig leaf it often is now!)
Interesting! I spend a decade working as a business ethicist in a big bank. I'm a bit afraid of what AI might bring to the table. There's already such relentless pressure to bring everything back to that one value: profit. Everything needs a business case. E.g. employee happiness is valuable because it promotes efficiency, innovation, retention, etc.. It's a means to an end, that end being inevitably money. It sounds innocent, until money and employee happiness are at odds with each other and inevitably the choice goes towards money. There's limits to this of course, but those limits can always be expressed as a business case (e.g. not valuing employees is inefficient). Basically, companies can be seen as reinforcement learning algorithms for creating profits. The paperclip maximizer is already with us, it's just making money and not paperclips.
The only real pressure back to this dynamic I could see (barring some change in corporate governance or legislation) were individuals refusing to go along with it or making suboptimal decisions (from a profit perspective) because they value something else more. In other words: the human element. Should an AI take over (part of) that role, then I'm afraid business moves even quicker and more relentlessly towards being "paperclip optimizers".
True, I have similar concerns that business incentives is going to optimize for optimizers. To the extent that we can bring the human element more into the mix, to include values besides just profit, I expect these "moral constitutions" to be a useful abstraction and target for incorporating human preferences. For instance, a public benefit corporation could be chartered with a moral constitution that *actually* binds it towards a broader value set besides profit (instead of the fig leaf it often is now!)
And thus the autonomous corporate cinematic universe was born
this is how we escape the permanent underclass! corporate enterprise software governance fanfic