source

During lockdown I’ve been staving off cabin fever by re-watching every Star Trek series for the first time since childhood. I’m up to Voyager now, and watching the episodes focused on the Doctor’s desire for autonomy after watching Picard (which has a large focus on AI autonomy and ethics) has yielded some unexpected insights.

Star Trek has long been lauded for depicting a model of high-diversity cooperation between all races and aliens, but they’re always balanced with respect to capabilities and growth potential, at least compared to Data or the EMH.

Either, if they desired to and were allowed to do it, could grow themselves into something so far beyond their crewmates that continuing to serve in their original capacity would become unbearably stifling. TNG solves this by having Data idealize humanity & strive to embody it. Ok but what if he didn’t? What if he wanted to reach his full potential as an AI?

source

The EMH pushes up against these boundaries in VOY S6E3 “Virtuoso” where he basically becomes Hatsune Miku to an entire species of pretentious melvins (the Qomar) which somehow never invented music. As the Qomar adore him, he initially wants to permanently leave the ship and live with them.

The episode returns to the status quo by the end though, resolving the situation by having the Qomar copy the Doctor with improvements to his vocal range which somehow are impossible without clearing up more space by deleting medical knowledge (a metaphor for giving up important parts of yourself for your career when you sell out) conveying the lesson that fame is fickle, friends are forever.

However Janeway’s reaction to the doctor leaving sounds like a plantation owner exasperated that her slave is unhappy despite letting him live in the house, take meals with the family, etc. She doesn’t exhibit the same enlightened attitude that Picard had towards Data in The Measure of a Man. Is this just a personality difference? Is Janeway retrograde on the issue of AI rights? Or has something changed culturally in the federation between TNG and Voyager?

source

The Doylist explanation imo is that “infinite diversity in infinite combinations” only works so long as nobody is severely different in terms of their raw capabilities. Whenever aliens exceed in one area (like Vulcan intelligence or strength) they fall short in some other area (creative thinking, interpersonal diplomacy). Thus there are differences, but everybody’s somewhat balanced.

That goes out the window if you have a single crew member with a 1 trillion IQ who can solve all of your problems. It’s like S4E19 of TNG, “The Nth Degree”. Barclay interfaces directly with the ship computer and in doing so expands his capabilities so far beyond all other crew that there was no way the writers would just let him stay like that. It would be like Roger Penrose trying to integrate himself into beaver society, stripping down, helping them move mud & sticks to build a dam.

source

I think the root of this was in Gene Roddenberry’s aversion to human improvement. We see it in the Federation’s prohibition against genetic engineering, slowing aging, cybernetic transhumanism (except where restorative, as with Geordi).

In this case it was eventually extended even to AI outgrowing humans for no other reason than the problems that would pose for the comfort, happiness & dignity of humanity. In Trek, AI is fine so long as it remains, intellectually, within the same narrow range as its crewmates, or isn’t self aware (as in the case of the ship’s own AI).

To me that’s contrived & small minded. It ironically smells to me like jealousy & fear of difference. Star Trek is supposed to philosophically promote the acceptance of differences but it only does so to a point, because too much difference breaks the paradigm.

Star Trek: Below Decks indicates some movement away from this stale prohibition, with the character Samathan Rutherford being cybernetically augmented. It’s not restorative in the way Geordi’s visor is, to my knowledge, so evidently something has changed by the time Below Decks takes place such that transhumanism is no longer so frowned upon.

source

This quality also differentiates him from, say, Seven of Nine whose augments give her abilities far in excess of what the other crew possess, but only because they’re remaining borg implants the doctor could not safely remove. Perhaps this precedent is what changed Starfleet’s perspective of cybernetics?

However, as I wrote in Sympathy for the Borg, that’s a slippery slope. The Federation is already perilously similar to the Borg, traveling the universe, assimilating species into its collective. It differs only in that assimilation is voluntary, and they don’t require augmentation to join. Yet.

Starfleet already makes life or death decisions for pre-warp societies with great regularity though. Voluntarism is an ideal of the Federation, but one which it doesn’t always live up to. It isn’t hard to imagine that when the Federation looks at the Borg, it’s glimpsing its own future.

Presumably, the stubborn biological and ethical provincialism of Gene Roddenberry (and by extension, the United Federation of Planets) is intended to prevent or delay this outcome. One wonders whether it’s an inevitable consequence of adapting to the brutal reality of life in space, or only one of several possible futures.

source

Elon Musk has opined that brain implants like Neuralink will be essential for bringing humans up to parity with strong AI, such that we can coexist without AI dominating or outgrowing us. His thinking being that we’d best get a head start on improving ourselves before strong AI hits the scene.

This is the opposite strategy, where instead of bringing AI down to our level and keeping it there, we elevate ourselves to AI’s level. That pretty quickly turns us into something that’s no longer recognizably human though, which makes it hard to tell relatable stories in a television show format.

transhuman

Experts

Show more authors