Abstract
The making of knowledge engines in natural language processing has been shaped by two seemingly distinct paradigms: one grounded in structure, the other driven by massively available unstructured data. The structured paradigm leverages predefined symbolic interactions, such as knowledge graphs, as priors, and designs models to capture such priors. In contrast, the unstructured paradigm centres on scaling transformer architectures with increasingly vast data and model sizes, as seen in modern large language models. Despite their divergence, this thesis seeks to establish conceptual connections that bridge these two paradigms.
These connections form a new recipe for developing general knowledge engines, where the guidelines include modelling both the seen and the unseen, the latter being relatively underexplored. Efficiently modelling the seen necessitates structure formation, regardless of whether the data is inherently structured or not. Conversely, modelling the unseen benefits from active destructuring of the learned cache, which promotes robustness and adaptability.
By bridging the two paradigms, this thesis establishes structure and destructure as complementary forces in the design of knowledge engines that support transparent, controllable, and adaptable intelligent systems.
Table of Content
| Part | Description |
|---|---|
| Part Opening | Building Knowledge Engines. 📖 Chapter 1 Introduction |
| Part I Structure 🧱 | Structure – The Foundation of Knowledge Engines. 📖 Introduction to Structure 📖 Chapter 2 Language Modelling Completes Knowledge Graph Structures 📖 Chapter 3 Uncovering Interpretable Structures in Pretrained Language Models 📖 Summary on Structure |
| Part II DeStructure 🌀 | DeStructure – Addressing the Limits of Rigid Knowledge. 📖 Introduction to Destructure 📖 Chapter 4 Inductive Knowledge Graph Learning with Active Forgetting 📖 Chapter 5 Improving Language Plasticity via Pretraining with Active Forgetting 📖 Summary of Destructure |
| Part Closing | Toward General Knowledge Engines. 📖 Chapter 6 Conclusions 📖 Chapter 7 Future Work |
A full copy of the thesis can be downloaded with the following options
Bibtex
@phdthesis{chen2025structure,
title={Structure and Destructure: Dual Forces in the Making of Knowledge Engines},
author={Chen, Yihong},
url={https://discovery.ucl.ac.uk/id/eprint/10211291/},
year={2025},
school={University College London}
}