an off-the-cuff set of responses to "The Mythology of Concious AI"
directxman12.dev
response-to-the-mythology-of-concious-ai.md
edited
1(this is an off the cuff set of responses to
2https://www.noemamag.com/the-mythology-of-conscious-ai/ that grew too long for
3a reasonable bsky thread)
4
5broadly, the bulk of the first 2-3 points seem to rest on a comparison between
6_platonic_ digital computation and _material_ biology. it asserts that modern
7digital computation is "pure" and divorced from its substrate, while biology,
8and more specifically neuro-biology, is messy and inherently tied to its
9substrate. it accuses those who believe in the simulatability of consciousness
10to be operating off of an over-simplified model of cognition, and yet it
11commits this very same fallacy when it discusses computation (and to some
12degree, the physics of continuity).
13
14it makes repeated reference to the idea that computation is platonic, but
15platonic computation is the realm of computer scientists; to anyone who
16actually has to interface with a computer it is patently obvious that
17computation is _not_ substrate-indepentent. consider both the story of the
18fpga optimizer that created a model that worked due to interference from nodes
19placed physically close to each other (which seems to mirror the "messiness"
20that the author attributes solely to physical neurons), or, for an example
21rooted more closely in standard computer equipment, timing or heat-observation
22attacks in cryptography. one might assert that this represents
23substrate-dependency for a "class" of computing substrate and not an individual
24instance. in that case, i would direct one's attention an bit-flips in non-ecc
25ram (or ram-refresh in general), or the placement of blocks on ssds as they age;
26it takes many intermediate layers and much abstractive work to mantain the illusion
27of substrate independence that is fundamental to the way we approach _higher-level_
28modern computing (if you'll forgive me for being slighly petty, i'd say that in just
29the same way that ai programmers reason about neurons in an abstracted model, so
30too do neuroscientists write numpy-affected python without worrying about ssd
31maintenance, ram refreshes, or even simd).
32
33the article's discussion of the physics of continuity also seems questionable
34to me (although i will admit that my physics knowledge is significantly shakier
35than it used to be, so take this part with an _extreme_ grain of salt); the author
36makes a big deal out of modern computation being discrete but physics being
37continuous, however, it's not particularly clear to me that physics/reality _is_
38continuous, or rather what continuity means in a physical sense (e.g. given that
39many models of reality break down as u reach the planck length. my impression was
40that this was an open area of discussion amongst physicists). again, we find
41ourselves in the space of "simplified models for use in a particular domain do
42not necessarily match the complexity of domain-specific models or reality".
43
44the author also, at points, dips into a deeply neurotypical description of
45consciousness; they assert that "consciousness does not stutter from one state
46to another, it flows" which seems to ignore the lived experiences of both some
47forms of plurality and memory repression. to make this concrete, one of the
48more jarring experiences in my life was suddenly remembering a whole aspect of
49my childhood and feeling like an entire timeline had been grafted into my
50reality in a single moment; my perception did not flow, it jumped.
51
52while the author hand-waves away several other issues with their arguments
53(largely down to "we could simulate the physics that make neurons messy"), they
54attempt to address this in section 4 by asserting that "simulation is not
55instantiation". this to me, is a religious argument; the author presents this
56as _obvious_, not axiomatic to their argument. but this is not obvious. what
57separates a fundamentally rich simulation from reality? what does "wet" mean
58besides the observable phenomena that encompass wetness, and the ability to observe and
59inspect those phenomena? it seems to me that the author commits a fallacy of assuming
60an absolute frame of reference rather than acknowleging that any discussion is rooted
61in a particular frame of reference, and may change if shifted.
62an attempt is made, sideways, to at least address some source material here,
63but it rests on circular logic: the author presupposes that simulations cannot be
64conscious in order to prove that simulations are not reality, and thereby prove
65that u cannot simulate consciousness.
66
67it doesn't seem worth it to me to address the end of the article; it devolves into
68religious arguments about souls. if you want to have a religious argument, at
69least have the courtesy to participate in the minyan first and bring some
70cookies for the nosh.
71
72i do think, perhaps, there are some interesting discussion points (as well as some
73extremely well-tread ones) to be found burried in this article, but they are presented
74in a deeply intellectually sloppy manner with an unhelpful framing.
75
76## follow-up notes (added/edited after i posted this)
77
78- the author presents conciousness by an absolute definition:
79 conciousness is the way that a being experiences its existence.
80 the author hand-waves away what "experiences" means here (although
81 they briefly touch on this in the section about life, which i _do_
82 find interesting and somewhat compelling). i think this presents
83 a much more interesting discussion point as in constrasts with the
84 idea of conciousness as a set of observed phenomena coallesced into
85 a single concept by an observer -- i.e. nothing is absolutely concious,
86 but rather appears concious under a given measurement by a given observer.