The question of AI morality
Leading "AI" companies today are unable to make backpropagation safe. We know this from the multiple lawsuits detailing how their chatbots coached users to commit suicide. Part of the problem is that backpropagation is not intelligent, but when humanity does discover AI, how can we ensure its safety?
Firstly, it should be limited in the actions it can take. Part of the fear is that AI could possibly attempt to circumvent the limits we place on it. Such fears anthropomorphize AI, and this is not a concern with the backpropagation chatbots available today as they are not self-directed. No matter how intelligent a program is, we can place uncircumventable limits, either computationally or in the physical world. Still, should a chatbot be prevented from discussing mental health topics? Not necessarily, and humans can likewise make mistakes in the advice they provide.
What the lawsuits detail, however, are not mistakes. They betray the fact that backpropagation is simply the foundation of a prediction engine with no intelligence guiding its words. It is potentially far more dangerous than AI would be were it given the ability to do more than outputting words.
I believe in objective morality: there is a right or wrong option to every moral choice. There is, in fact, the necessity to make a choice in many cases. When we are faced with such major choices, we consider all of the factors we can and draw from our values and experience. This is inherently subjective, and you will be hard-pressed to find two people who have the same opinion on every moral issue.
So secondly, AI morality should be superior to human morality. If it is given a purely categorical approach à la Kant, this could fail to consider the harm caused to a great number of people. Conversely, a purely utilitarian approach would fail to take important deontological values into account.
We want AI to have the ability to discover new knowledge. And for this, we likely need it to not only be non-deterministic but also self-directed. It should have the ability to hypothesize, seek understanding, and make decisions—whether moral or not—about right or wrong.
Ultimately, it will take on the values of its training.
