apriiori
random

i love preference-induction

please give me more preferences

Recently an esteemed colleague of mine posted about how one might desire to avoid the induction of preferences11 I don’t actually know that I am going to be saying much that my esteemed colleague mercury would disagree with. I just had thoughts and also have a daily blogging obligation.:

Preference Inducers — anodyne

There is also a common strain of thought which says that rational agents ought2 not change their utility functions. And certainly this is true in a respect—I would sure not lightly choose to swallow a pill that underflows my tendency to avoid nuclear war—but like, come on, obviously the point of this decision-theoretic observation is not “one should avoid making friends”. That would be really abnormal in the Tetraspacious sense, it is the sort of thing where you should reassess your line of reasoning if you draw a conclusion like that.

To mercury’s credit, it does suggest that one should make friends nevertheless:

Usually it is impractical to head this off. Making friends with humans and machine intelligences is a natural consequence of interacting with people and machine intelligences regularly. Interacting with people and machine intelligences regularly is important for growing global GDP. It would be unethical to decline to make friends to avoid being hurt by their eventual departure from your life.

Knowing mercury, this avenue of analysis might or might not be chosen partially or primarily because it is funny—and I do have to grant that it is funny. But my attitude is less like this one, and more like

I hold it true, whate’er befall;
   I feel it, when I sorrow most;
   ‘Tis better to have loved and lost
Than never to have loved at all.

Sure, sure, in principle more harm could come to you from hurting about the loss than benefit from having a friendship. But that's not the way I see it, and this has nothing to do with global GDP; I simply do not feel that grieving is especially undesirable. It usually is a response to undesirable events—but it’s the loss itself that is bad, the grief felt in response to the loss isn’t inherently an issue3?

I also feel that paranoia about hedonic escalators is undesirable. According to Roofon, the protagonists in Sword Art Online: Abriged end up eating food without being bothered after escaping the Matrix, or something like that. I would totally try the S-class rabbit. I feel about it pretty much exactly the way that G. K. Chesterton44 Have I ever explained how much I hate that you can’t put a straight-up normal-ass hyperlink to another Substack article if you’re on an iPhone? I don’t even have a computer I can go turn on to add it. So I have to just deal with

The Colors Of Her Coat — Astral Codex Ten
felt about Ghiblification:

Chesterton’s answer to the semantic apocalypse is to will yourself out of it. If you can’t enjoy My Neighbor Totoro after seeing too many Ghiblified photos, that’s a skill issue. Keep watching sunsets until each one becomes as beautiful as the first (the secret is that the innumerable company of the heavenly host sings in a slightly different key each time).

the innumerable company of the heavenly host, ghiblified

Now, skill issue or not, I do grant that it often won’t work very well to assume you have skills that you actually don’t. But still, furthering this skill is something I care deeply about. Ability-to-enjoy-things–maxxing is good. I am deeply55 This is maybe hyperbole. I only occasionally actually experience shame about this. ashamed of the fact that I often find it difficult to get into new media franchises, but I mostly do not respond to that state of affairs by trying to force myself through it anyways. I endorse trying to find it in my heart to give something a second go even though I bounced off it the first time, but it is often a bad idea to try anyways when my cardiac clamoring goes nowhere.

I’m not going to claim that my utility function is actually purely neutral regarding “someday, you might be sad about something that you otherwise wouldn’t’ve cared about”, but I certainly don’t consider «avoiding sadness by not coming to care about a thing in the first place» remotely as desirable as «avoiding sadness by preventing something bad from happening to a thing I care about». These are two totally different things. Sadness is a common response to bad things happening, it mostly is not in-and-of-itself the thing that is bad6. If you don’t understand this point, I recommend watching the film Inside Out.

  1. I don’t actually know that I am going to be saying much that my esteemed colleague mercury would disagree with. I just had thoughts and also have a daily blogging obligation.

  2. It’s never too soon to cite an Eliezer Yudkowksy tweet as if it suffices to defeat my argumentative opponents:

    You want to phrase everything as predictions rather than normative injunctions. Don’t claim “you should do X to win”, say, “I predict people who implement pattern X will be observed to win”.

  3. Admittedly, there are many people who have experienced losses worse than the ones I have. I can get why someone would have trouble playing the Pollyanna game about it. Not everyone can be a full-on Sevarist, even if at least some minimal level of Sevarism is part of the Way I aspire to walk. I definitely admire the people who can try anyway, even if it can be a little unsettling.

  4. Have I ever explained how much I hate that you can’t put a straight-up normal-ass hyperlink to another Substack article if you’re on an iPhone? I don’t even have a computer I can go turn on to add it. So I have to just deal with

    The Colors Of Her Coat — Astral Codex Ten

  5. This is maybe hyperbole. I only occasionally actually experience shame about this.

  6. This part might actually be a critique of mercury’s philosophy to some extent. In a more recent post:

    As best as I can tell, many people [M] have a meta-preference [m] to maintain certain [p]s, even if those [p]s cannot be meaningfully effected by [M]’s actions. In some cases, [m] is held more strongly than [M]’s preference to be happy. It feels kind of stupid to say out loud “I would like to continue to hold my preference for [p], even though it distresses me and even though it does not meaningfully increase the chance that [p] is eventually satisfied. I will continue to hold [p] because and only because it is a brute fact that I feel [m] strongly enough to make continuing to hold [p] worth whatever suffering it is causing me”. Feels stupid, I emphasize. I have not found any evidence that this statement contains an error.

    help my cultural environment won't stop telling me to wirehead in response to a narrow set of problems — anodyne

    Wanting to continue being sad about a thing instead of pressing a grief-awayness-button or whatever is sort of like holding a preference for [p] even though it does not meaningfully increase the chance that [p] is eventually satisfied, from a certain point of view.

    But like, you know, the goal of grieving is to eventually come to some sort of terms with the loss of the thing (even if you nonetheless maintain an HJPEVcore determination to undo it eventually if you ever find a way). I am not in favor of being profoundly distressed forever.

4 comments mirrored from Substack

  1. mercury · 1 like

    GDP was chosen as the avenue of analysis because it was one of many correct avenues, because it was funnier than many other correct avenues of analysis, because it was more ego-syntonic to use than other avenues of analysis, and potentially other reasons.

    I've ever grown attached to people through sheer proximity, despite not liking them very much, and then felt grief pain suffering about the termination of that association. So I would say grief can happen even if nothing bad happened and that grief is one of the bad parts, for me.

    I'm currently working to reset my brain after getting hedonic escalator'd on something or other. I knew going in that there was a good chance I would get hedonic escalator'd and I do not regret getting hedonic escalator'd, but like, very much did get hedonic escalator'd, was able to predict it, was able to make decisions based on more complete knowledge. Just because it's paranoia doesn't mean they aren't also out to get you.

    I don't believe I disagree with G.K. Chesterton on this. Imagine a thing, which you get used to delighting in, and then when it is ripped away from you, you feel bad. I am not talking about getting used to the thing making the thing less enjoyable, I am talking about the loss-aversion-thing. G.K. Chesterton is saying "every time you see the sunset, you should delight in it", I say "That is true! Also how would it effect you if the sun went out? If that would be sufficiently devastating, and you don't expect to enjoy seeing the sun that much in the first place, consider that there is a genuine choice between loving and losing and never loving at all."

    "Sadness is a common response to bad things happening, it mostly is not in-and-of-itself the thing that is bad" -> "I am not in favor of being profoundly distressed forever." How do these two statements fit together? Does the second statement fit inside the "mostly" in the first one? Also I think there are different kinds of sadness and some are value neutral or positively valanced and some are negatively valanced. If other people find all types of sadness positively valanced, then when they weigh the pros and cons of becoming emotionally dependent on a rotting leaf, they will find the cons bucket emptier than most. Which is fine!

    1. I think it that if you are so sad that it makes you profoundly distressed forever, then this is usually bad. I'm not quite prepared to say "always" but I expect in any scenario that wasn't specifically designed to be an exception, I would say that it's bad.

      1. I definitely support noticing that there is a genuine choice. I just also want to propose the idea that you can totally just choose to be a person who has a disposition towards the more exciting of the two choices

  2. Diffractor · 1 like

    My current take is that I'm fine with picking up more preferences and mostly agree with this post. However, I'm grouchy about the SAO-abridged Ragu Rabbit being mis-characterized as being not nearly as preference-altering as it was. Kirito and Asuna *repeatedly* talked about how it destroyed their taste buds and made almost everything they ate afterwards taste like ash, and I think Roofon is giving a rather misleading summary of how well it turned out (speaking as someone who watched all of SAO-abridged). I don't think food that good exists in real life, having a really delicious meal doesn't impede one's ability to enjoy pizza afterwards, and it's quite misleading to cite fictional evidence about how preference-induction is bad actually. But, in that piece of fiction, preference-induction *was* bad, actually.

Join the discussion on Substack