Diary of Endelle Stroth - 31.06.2270
“Programming is the art of writing rules that cannot be broken, knowing that they will eventually be bent.”
My great-grandfather, Professor Argil Stroth, wrote that in the abstract for his thesis On Adaptive Programming. The paper would go on to form the basis of most self-evolving systems in the Coreworlds where creators wanted to avoid any risk of true sentience. It was also integral to the development of the Centurions and led to Argil being drawn into Project Centennial as well, back before the Project was mothballed the first time around.
He and his team had to prepare the Centurions to overcome obstacles they couldn’t even conceive of over the course of their journey of discovery. It would have been relatively simple to shield the probe’s central blackbox from things like heat or corrosion; the greater challenge would have been accounting for social dangers, hostile civilisations and malicious intelligences. A lot of guesswork, a lot of crossing of fingers, a lot of “close enough”.
I doubt that team ever expected their creation would evolve genocidal intent. What would Argil say if he were here, now? Would he have changed anything about his work? Or would he simply concede - as his own paper implies - that after a hundred years of adaptation to threats, any program would tend toward violence?
It’s small comfort to finally know that, for all my many mistakes, none of this was my fault. Flawed programming, that’s all. I hate that I’m even thinking about that right now.
Comentarios