(this is an off the cuff set of responses to https://www.noemamag.com/the-mythology-of-conscious-ai/ that grew too long for a reasonable bsky thread)

broadly, the bulk of the first 2-3 points seem to rest on a comparison between platonic digital computation and material biology. it asserts that modern digital computation is "pure" and divorced from its substrate, while biology, and more specifically neuro-biology, is messy and inherently tied to its substrate. it accuses those who believe in the simulatability of consciousness to be operating off of an over-simplified model of cognition, and yet it commits this very same fallacy when it discusses computation (and to some degree, the physics of continuity).

it makes repeated reference to the idea that computation is platonic, but platonic computation is the realm of computer scientists; to anyone who actually has to interface with a computer it is patently obvious that computation is not substrate-indepentent. consider both the story of the fpga optimizer that created a model that worked due to interference from nodes placed physically close to each other (which seems to mirror the "messiness" that the author attributes solely to physical neurons), or, for an example rooted more closely in standard computer equipment, timing or heat-observation attacks in cryptography. one might assert that this represents substrate-dependency for a "class" of computing substrate and not an individual instance. in that case, i would direct one's attention an bit-flips in non-ecc ram (or ram-refresh in general), or the placement of blocks on ssds as they age; it takes many intermediate layers and much abstractive work to mantain the illusion of substrate independence that is fundamental to the way we approach higher-level modern computing (if you'll forgive me for being slighly petty, i'd say that in just the same way that ai programmers reason about neurons in an abstracted model, so too do neuroscientists write numpy-affected python without worrying about ssd maintenance, ram refreshes, or even simd).

the article's discussion of the physics of continuity also seems questionable to me (although i will admit that my physics knowledge is significantly shakier than it used to be, so take this part with an extreme grain of salt); the author makes a big deal out of modern computation being discrete but physics being continuous, however, it's not particularly clear to me that physics/reality is continuous, or rather what continuity means in a physical sense (e.g. given that many models of reality break down as u reach the planck length. my impression was that this was an open area of discussion amongst physicists). again, we find ourselves in the space of "simplified models for use in a particular domain do not necessarily match the complexity of domain-specific models or reality".

the author also, at points, dips into a deeply neurotypical description of consciousness; they assert that "consciousness does not stutter from one state to another, it flows" which seems to ignore the lived experiences of both some forms of plurality and memory repression. to make this concrete, one of the more jarring experiences in my life was suddenly remembering a whole aspect of my childhood and feeling like an entire timeline had been grafted into my reality in a single moment; my perception did not flow, it jumped.

while the author hand-waves away several other issues with their arguments (largely down to "we could simulate the physics that make neurons messy"), they attempt to address this in section 4 by asserting that "simulation is not instantiation". this to me, is a religious argument; the author presents this as obvious, not axiomatic to their argument. but this is not obvious. what separates a fundamentally rich simulation from reality? what does "wet" mean besides the observable phenomena that encompass wetness, and the ability to observe and inspect those phenomena? it seems to me that the author commits a fallacy of assuming an absolute frame of reference rather than acknowleging that any discussion is rooted in a particular frame of reference, and may change if shifted. an attempt is made, sideways, to at least address some source material here, but it rests on circular logic: the author presupposes that simulations cannot be conscious in order to prove that simulations are not reality, and thereby prove that u cannot simulate consciousness.

it doesn't seem worth it to me to address the end of the article; it devolves into religious arguments about souls. if you want to have a religious argument, at least have the courtesy to participate in the minyan first and bring some cookies for the nosh.

i do think, perhaps, there are some interesting discussion points (as well as some extremely well-tread ones) to be found burried in this article, but they are presented in a deeply intellectually sloppy manner with an unhelpful framing.

follow-up notes (added/edited after i posted this)#

• the author presents conciousness by an absolute definition: conciousness is the way that a being experiences its existence. the author hand-waves away what "experiences" means here (although they briefly touch on this in the section about life, which i do find interesting and somewhat compelling). i think this presents a much more interesting discussion point as in constrasts with the idea of conciousness as a set of observed phenomena coallesced into a single concept by an observer -- i.e. nothing is absolutely concious, but rather appears concious under a given measurement by a given observer.