I’ve recently begun using SemanticScholar heavily. This system has an option to save articles to a “library” and also create folders within the library. As of Feb 13 I’ve created 7 folders. Many of the articles overlap within this categorization system. I’ve summarized my current understanding of each of these folders below. Although category theory and artificial intelligence provide common threads, several of these folders stand outside these two areas.

AlphaGAT

This deserves a whole article, so I’ll briefly outline the idea here. The goal of AlphaGAT is to develop a general-purpose software system connecting GATlab.jl with a deep neural network. The goal of this software would be to do planning/reasoning within a GAT by expressing the GAT as a rewriting system. The rewriting system would be computed as an e-graph. The deep neural network would be trained on synthetic data developed from the GAT of interest with the particular goal of “unrolling” the e-graph into a computational system.

The goal of this system would be to take advanted of the parallels between planning and reasoning. LLMs are still poor at planning, and explicit KRR systems still seem to be required. Both planning and reasoning can be expressed as rewriting systems. Rather than committing to a particular knowledge representation formalism, focusing on a sophisticated computer algebra system as a system of knowledge representation systems has the potential for a “multi-modal” KRR system as described by Sowa.

Both AlphaGeometry and FunSearch seem relevant here. AlphaGeometry uses a DNN trained on synthetic data prepared by an algebraic solver to incorporate “auxilliary constructions” into the knowledge base of the algebraic system, which adds this construction to the knowledge base and re-applies the algebraic rewriting system. FunSearch uses a genetic algorithm based search technique to develop functions that solve combinatorial systems. FunSearch seems applicable to the “unrolling” step where the e-graph is transformed into a sequential application stated in code.

The “alpha” series of papers from Google were the cutting edge of AI for last 10 years of so. These are “neuro-symbolic” systems, with a neural network connected to a computational system that provides a “reinforcement” signal. A review of the FunSearch paper from Google also called them “deep reinforcement learning” systems.

Hypergraph Transformers

Can the transformer architecture be applied to hypergraphs? In particular, the “hypergraphs with interfaces” that are developed by Wilson and Zanasi [@wilson:2023:data-parallel]. They are developing as sytem to do these construction within Python. This is perhaps due to limitations in the previous CATlab code, which hopefully have been addressed within GATlab.

Hypergraphs with interfaces are used to represent symmetric monoidal categories with a special Frobenius struture on the wires. Zansi and various collaborators have considered other rewriting systems.

Note that AlphaGeometry uses a “sequential” transformer, so that a more “native” transformer architecture may not be necessary.

Liu et. al. develop a hypergraph learning framework that uses a positional encoding of nodes and edges and “hypergraph structure regularization in the loss function, capturing connectivities between nodes and hyperedges.”