Knowledge base vs LLM
What if we could create a digital inventory of all human knowledge, an encyclopedia but ordered in such a way that complex queries can be answered? I can hear you shout: ‘Isn’t that a bit like a LLM trained on Wikipedia?'
Kind of … Except that such an LLM would have created the inventory itself, leaving the connections between facts opaque. This is so because all text input data fed into an LLM is unstructured and treated on equal footing. It’s the internal attention mechanism that starts organising the text. So, although we can ask complex queries in the form of natural language and retrieve an answer in the same medium, there is no knowledge inventory accessible to us. It’s as if the LLM is giving a press conference but leaves us dry with ‘no further questions’ after the first round.
Is that better than having no query capability at all? Well, that’s really a mis-apperception of the playing field. The choice isn’t between LLM-like query magic and nothing at all. The choice is between LLM-like query magic and incomplete yet carefully crafted knowledge bases (search engines can answer queries but do so by pointing to documents on the web rather than providing the answer themselves). Using the earlier press conference analogy, the choice is between listening to a spokesman with a vast vocabulary and asking the actual protagonists at the risk of coming home empty-handed.
Medical Ontology exposing LLM limits.
The foundational model of anatomy (FMA) is such an attempt an create an inventory of human knowledge, in this particular case, all knowledge related to the structural organisation of the human body at micro- and macroscopic levels. A prima facie uncontroversial choice of fundamental relationship to be used in this knowledge base is the partonomic inclusion relationship. For example, cytoplasm part_of cell. Little spoiler… this slicing up of the conceptual space doesn’t cut it (pun intended). I’ll explain to you why. However, let’s first focus on what this means within the here-discussed wider context.
We pride ourselves on having LLMs which can assist us in digging into vast amounts of knowledge, conveniently ignoring the fact that we have no real insight into how facts are connected within the model, while our own attempts to connect facts in the most rigorous fashion have turned out to be treacherous endeavours. The press conference gives us the impression of having received some reliable answers while even the protagonists themselves can’t really recall what has happened. If we’re a member of the public relationships committee, this might not worry us too much, but as a citizen with a vested interest in the upholding of liberal democracy, I might not want to rely too heavily on such reports.
Partonomic inclusion has gone wrong
Now, how can a relationship so seemingly innocent and straightforward as partonomic inclusion be so misleading?
Here are some seemingly uncontroversial examples of such a relationship:
human testis part_of human being
Yes, but there are male and female human beings. So, every human testis belongs (or belonged if the human was subject to an unlikely accident) to a human, but not every human has a human testis.
Solution?
The part_of relationship is split into two sub-relationships: has_part and part_for. So, now we can state that the human testis part_for human being without committing to human being has_part human testis. The part_of relationship is salvaged as the conjunction of these two sub-relationships: A part_of B =def A part_for B & B has_part A.
replication fork part_of nucleoplasm
Yes, but only so during certain times during the cell replication cycle (i.e., the S phase).
Biology read: https://www.nature.com/scitable/topicpage/replication-fork-stalling-and-the-fork-protection-14435782/
Solution?
At first glance, there seem to be at least two possible ways out. Firstly, we could introduce time-dependent or phase-dependent mereological relationships. So, part_of could get a time-index to distinguish between different times at which a relationship is obtained. However, since the cell replication cycle consists of clearly defined phases, it’s logically more apt to introduce phase-dependent partonomic inclusion relationships. Secondly, instead of fiddling with the relationship itself, we can enrich the representational capacity of the relata (i.e., the things related). So, a replication fork is not just a replication fork tout court, nor is the nucleoplasm always the same entity. The partonomic inclusion of the replication fork in the nucleoplasm during the S phase can be represented as follows: replication fork part_of nucleoplasm_S.
synaptonemal complex part_of chromosome
Yes, but not in all cases. For example, in protozoan ciliates, the formation of a synaptonemal complex is not required for pairing homologous chromosomes.
Biology read: https://pubmed.ncbi.nlm.nih.gov/24336924/
Solution?
In this case, recourse is found in the ancillary axioms, which define the logical relationships between the fundamental mereological relationships at hand. If we want to represent ‘part of’ as ‘can be part of,‘ then we can introduce a less fundamental part_of_can relationship by defining it as an existential import relationship: A part_of_can B =def ∃C(C is_a B & A part_of C). So, for this particular example, we’d have synaptonemal complex part_of_can chromosome, which is to say that there exists some chromosome for which a synaptonemal complex is part of it. The attentive reader will, at this point, see an alternative solution akin to one earlier exposed. Indeed, instead of introducing a secondary relationship, one can stick to the fundamental relationships and represent the facts by demultiplexing the relata. So, the fact ‘synaptonemal complex part_of chromosome_Animalia’ would be part of the knowledge base but not the fact ‘synaptonemal complex part_of chromosome_Ciliophora.’
Conclusion
In short, prima facia well-rounded relationships need to be broken up and defined in more primitive terms or require ancillary axioms in order to capture the complexity of the reality one desires to represent. Moreover, these solutions are in tension with solutions that prefer relational and definitional parsimony at the cost of an increase in entity cardinality. Since these issues occur in medical ontologies designed by professionals, it should be clear that this task is under no circumstance an easy one. It highlights a perplexing double bind between, on the one hand, the subtlety of reality’s relational composition and, on the other hand, the flexibility of the natural language to squeeze and squash that marvelous detail into communicable trunks.
Now, what are the implications of this insight on the state of AI’s contemporary playing field? If we aim for accuracy of representation, there might still be a role for classical AI or resurrections of its core ideas within deep learning paradigms. Conversely, if we aim for smoothness of communication, LLM might actually be capturing exactly what we want… were it not for the fact that we, humans ourselves, have already mastered the skill of human language so that LLM become what sex dolls are in comparison to real intimacy.
Sources
Paper on which this article is based: Bary Smith and Cornelius Rosse - The role of foundational relations in the alignment of biomedical ontologies: https://pubmed.ncbi.nlm.nih.gov/15360852/