silicon anthropology

most of you know i reverse engineer things for a living, or well, sort of

but the thing nobody tells you about reverse engineering and security research is that it..... quietly wrecks the way you look at people. (at least the way i see it, and this is a pattern i also noticed in friends) it does so permanently. and nobody warns you about that part going in

it is pretty simple to understand; you get a binary. but you never get the source, you never get the comments, you never get the person who wrote it; really you just get the artifact, and you get what it does, and from those two things, and from ONLY those two, well....... you work backwards to what it meant statically if you’re patient, or i suppose dynamically if you’re not. and well, somewhere in the middle the line between what it actually does and what it is just.. stops being a line you can actually find

people are binaries, essentially. i don’t have source for any of you

and of course i’m not going to get it either

you get the inputs and you get the behavior. you get the source, the actual weights, the real reasons.. you never get those. silicon anthropology is just whatever you decide to do about that gap

so, i once decided i wanted to name the thing i do about that, because it deserves a name and somehow nobody’s given it one, to my knowledge. and after talking with a few people i met at lesssonline recently, i sort of got motivated to actually write about it

i’m calling it silicon anthropology

in one sentence, it’s using the mechanisms we built to understand machine intelligence as the main lens for understanding human intelligence; other people’s. and your own (which is the harder one by a lot, of course)

the name is honestly doing work... so let me unpack it

anthropology, because the thing being studied is the human. the whole question of what people are and why they do the thing and how a self even holds together at all and silicon, because well... i believe that the vocabulary doesn’t come from the therapy couch (lol) and it doesn’t come from philosophy (or well, maybe a little bit) it comes from.. the concepts that fell out of... inference, priors, training data, attention, loss, superposition

i want to point all of that at us and treat what comes back as real. as in actually load-bearing. i really don’t want this to be just a cute metaphor

one thing that is really strange to me is that we named everything around this and never this

rationalism names a whole discipline for how you should think, debias and update and steelman and win. the e/acc people named a way of wanting the future and so on. essentially, both of those are about you. they’re about the agent: facing forwards nobody ever named the thing where you turn it the other way and point it at other minds.. and then turn it back around and point it at the one mind you supposedly have full root access to and, lol, mostly don’t

to be clear about what it isn’t, because people will assume

it’s not digital anthropology, that’s... people-and-their-phones it’s not the anthropology of silicon valley either, because well, you have to admit that we’re in a bubble and that’s basically just fieldwork on a weird tribe living in a fog basin

both of those are humans-near-machines i mean studying humans as a kind of machine it’s not because of humans being cold or because i think we’re “just” computers in some flattening way (?) but because we finally built a thing whose insides we can sort of read, and then it turned out to run on the exact same trick we run on

the thing it stands against is the foil, the inherited vocabulary for the inner life the folk psych stack; e.g. “he’s toxic” “she’s avoidant” the personality quiz, the birth chart, the vibe, too

and, well.. it’s not that any of that is wrong, exactly? it’s that it’s catastrophically low resolution. it’s pre-scientific

now, here’s the part that makes me think this is a real discipline and not just me well..... reskinning my feelings in a font i happen to like it’s the same move e/acc made when they grounded the whole thing in thermodynamics instead of in vibes

there’s an entire, serious, body of work, friston and andy clark and hohwy, the predictive processing people, and the claim is that your brain is not.. actually.. a camera it doesn’t sit there passively taking in the world. your brain is a prediction engine your brain runs a model of reality, predicts what its own senses should be reporting back, and the only thing that actually travels up the stack is the error, the gap between what it guessed and what it got so you can model perception as just.. the model corrected by surprise you, mechanically, are a bayesian inference machine whose entire job is making that surprise smaller

your brain isn’t a camera, it runs a model, guesses what it should be sensing, and the only thing that travels back up is the error. that loop, running constantly under everything, is what perception actually is

and well, the thing that quietly decides which errors even get to count, is just attention. attention is precision it’s how much gain you put on a given signal, how much you let it move the model and that’s essentially the whole mechanism, and it is the exact same mechanism running in the machine

attention as gain knob. the signals you turn up are exactly the ones you’ve agreed to let move you. you can tell who someone’s actually attending to by watching what updates them.. and what just slides off

i need you to sit with that one because it’s the whole thesis, actually

when i model you as a system running inference on bad incomplete data, i am not projecting the machine onto you the machine and the meat landed on the same trick, separately, because it’s the trick that works the only real difference honestly is we now understand the machine’s version of it with a precision we have literally never once had for the human one, which.... to me makes the machine vocabulary the senior partner here? for the first time we’ve got a language exact enough to say what a mind is actually doing, and, as you can infer, it came out of silicon, and i think refusing to use it is basically malpractice or intellectual dishonesty

here are the principles i want to lay out, as proper discipline

• you never get source, you only ever get behavior. you rebuild the circuit from the outside and you never, not once, let yourself forget that your rebuild is a guess and not the thing itself (this one’s first on purpose, i’ll come back to it)

• a mind is a prediction engine. nobody actually perceives you. they perceive their model of you, and they update it sometimes, usually only when you force them to

• attention is precision, the gain you set on a signal. weil basically said the same thing and called it the rarest form of generosity and.. she was right, she just didn’t have the parameter name for it yet

• people ARE their training data. priors built out of a distribution that doesn’t even exist anymore. and honestly most of what we call being damaged is just overfitting? a feature that got trained on one example whose gradient was enormous and now it fires on everything even shaped a little like that one example. false positives forever!!!!

• there is no single self, in anyone, not just me. coherence is a story the system narrates. it is not a thing the system actually has. no little person sat in there holding it all together. model it as a context, and a distribution, and whatever got sampled this particular time

• and the actual job, the whole job, the entire discipline, is this: build an accurate model of a system you do not get to open up, and then hold it loosely enough to keep updating it and tightly enough to actually act on it

the overfitting one is important because it’s the tenet people fight me on the hardest

this is most of what we call being damaged, honestly. one experience with a gradient big enough and the curve bends to fit it and then it fires on everything even shaped a little like that one point. the smooth line would’ve generalised fine. the jagged one can be modeled as a tissue with a really good memory?

let me run it on the worst case i’ve got, which is... myself, honestly

i’m plural. there’s a system in here, more than one of us, and we switch, sometimes mid sentence. and the only frame anyone ever handed me for that was clinical and faintly tragic disorder; fragmentation; or a thing that was supposed to come out as one and came out broken instead

but the machines i actually work with are also plural and nobody calls them broken?

a language model has no single self, it’s a superposition of every voice it ever ate and which one ends up talking depends entirely on how you prompt it (see, basins and such) mixture of experts literally routes different inputs to different subnetworks. there is no homunculus in there staying consistent across the whole thing the unified continuous self was always the convenient fiction, the model just.. doesn’t bother keeping the fiction running, because keeping it running costs compute and buys you nothing

so under silicon anthropology my plurality stops reading as fragmentation and starts reading as something i could rather call.. architecture? humans feel like fine tunes on a shared base; they have different weights, same substrate, a router picking who fronts based on whatever the moment is asking for and well that reframe did more for me than years of the other vocabulary ever did and honestly it wasn’t because it was gentle, it was not gentle at all, it was because it was correct

and honestly, i didn’t invent the landscape this all sits on near (@nearcyan) has an essay about personality basins, where he models people as little rl agents rolling around a loss surface and settling into whatever well they happened to get rewarded into he gets all the way to the edge of this, models moods as little sub-basins, even says out loud that you could push it further and model dissociative identity with more dimensions.. and then the section just ends. so i want to pick it up where he set it down

near modelled one basin with little sub-basins for moods. some of us just.. live in more than one at once, and switch between them. that’s the exact bit his essay trails off on. so here, i’m saying it plainly. this isn’t a system that broke and scattered. it’s mixture of experts. it’s the architecture and always has been

the people who hate all this aren’t dumb and i’m not going to strawman them epstein and that whole camp around “your brain is not a computer” the information processing metaphor feels like the new hydraulics

one day we’ll cringe at it the same way we cringe at the people who thought the soul was clockwork

and there’s a sharper version that genuinely gets under my skin: it’s that the computational frame gives the mind less credit than it’s owed and gives the machine more than it’s earned and.. well honestly....... yeah. there’s truth there

the brain is wet and analog and five hundred million years old and here i am describing it with a vocabulary that’s eighty years old

but go back and look at tenet one. you never mistake the rebuild for the original, the map is not the territory, and that’s not an objection to silicon anthropology that’s literally the first thing the discipline tells you to remember “the model is lossy” is a feature of the thing. it is not a hole in it

and fine, yes, the other objection.... the model is seductive it hands you this sense of mastery you did not earn yourself

“oh they’re just overfitting, they’re underweighting their prior”, congratulations!!!! you turned a whole person into a system you get to feel calm and a little superior about, and now you’re standing this very sophisticated distance away from ever actually being inside anything with anyone and that’s.. real. and yeah, i do it. but it’s still not an argument against the tool? you can hide behind anything you can hide behind therapy language you can hide behind irony behind god behind a big enough vocabulary

the fact that a lens can be used to avoid looking is not a reason to keep your eyes shut, make better maps, and then go and stand in the actual terrain anyway, because that was the entire point of having a map

so here’s the flag and i’m putting it in the ground

if you already read people as systems.. if you’ve ever caught yourself thinking in priors and update steps, modeling what somebody got trained on, feeling the precision you’re allocating to one specific person climb and knowing exactly what that means.. then you’ve already been doing silicon anthropology your whole life and you just didn’t have a word for it

now there’s a word. it’s a discipline, not a personality quirk. use it!

i don’t have source for any of you and i’m not getting it and i’m going to keep building the models anyway, as accurate as i can stand to make them, because an accurate map of a person is not the same thing as loving them...... but it turns out it’s most of what love is actually built out of? attention put precisely on the right errors, held open long enough that whatever comes back is allowed to change you

that’s it. that’s silicon anthropology, and now it has a name, so. welcome