Architecture

In discussions about AI alignment (especially AI takeoff), the term architecture is used to mean ... something like the basic design of the AI system (like what kind of machine learning is being used in what way, what the high-level organization of the components is). Architecture is used in contrast with content, which means something like all the low-level details, the specific knowledge that's encoded in, ...? I'm honestly not sure I can distinguish these two very well, and I wish people would define these terms more formally. This might be pointing to the same distinction Richard Sutton makes in his Bitter Lesson post.

synonyms? "mental architecture", "cognitive architecture", the "architecture of the AI"

"Hanson defines the “content” of a system to be its small modular features, while its “architecture” is its most important, least modular features." [1] [2]

"For similar reasons, I’m skeptical of a blank-slate AI mind-design intelligence explosion. Sure if there were a super mind theory that allowed vast mental efficiency gains all at once, but there isn’t. Minds are vast complex structures full of parts that depend intricately on each other, much like the citizens of a city. Minds, like cities, best improve gradually, because you just never know enough to manage a vast redesign of something with such complex inter-dependent adaptations." [3] -- how does Hanson reconcile this with how small the human genome is?

"Hanson mentions Peter Norvig’s recent paper, where Norvig was arguing with Noam Chomsky and saying that it’s wrong to expect there to be a simple elegant theory of linguistics. Instead there are just many messy details that one has to get right, with no key architecture." [4] -- I think this is conflating two levels. There's the source code level or top level [5] and then there's the learned model or subsystem level. The top level can be very simple even if the learned model is not simple (and has lots of messy details).

Robin Hanson: "I think our dispute in part comes down to an inclination toward architecture or content. That is, one view is that there's just a clever structure and if you have that basic structure, you have the right sort of architecture, and you set it up that way, then you don't need very much else, you just give it some sense organs, some access to the Internet or something, and then it can grow and build itself up because it has the right architecture for growth. Here we mean architecture for growth in particular, what architecture will let this thing grow well? [...] My opinion [...] is that it's largely about content. There are architectural insights. There are high-level things that you can do right or wrong, but they don't, in the end, add up to enough to make vast growth. What you need for vast growth is simply to have a big base. [...] I think that for minds, what matters is that it just has lots of good, powerful stuff in it, lots of things it knows, routines, strategies, and there isn't that much at the large architectural level." [6]

is the difference between humans and chimpanzees mostly about architecture or content? there's also the question of humans now vs humans thousands of years ago, where it seems clear that the difference is culture/"content".

if progress happens via compute, is that architecture or content? if progress happens via larger/better datasets, is that architecture or content?

if you come up with a simple learning algorithm that then has to spend a lot of time interacting with the world and using compute, in order to become smart, is that architecture or content? I feel like we need to distinguish the "top level" algorithm from the "secondary level/discovered-by-top-level" algorithm. The top level architecture could be very important, even if the secondary level is basically lots and lots of content. "It seems to me for this particular argument to carry, it's not enough to say you need content. There has to be no master trick to learning or producing content." [7]