A deep dive into how Memory layers work and how they supercharge LLMs, so much so that next-generation AI architectures will miss out if they do not use these.
Image generated with DALL-E 3
LLMs are massive knowledge stores of information within their parameters (primarily as the weights of linear matrix transforms in the dense layers).
However, as the parameter size grows, so does the computational and energy cost.
Can these be replaced with simple and cheap key-value lookup mechanisms?
Much previous research has been done to address this, but never at the scale of the AI architecture at present.
But Meta researchers have finally figured things out and developed Memory layers that can supercharge our current LLMs.
These layers replace the Feed-forward network (FFN) of one or more Transformer layers.
And the results are surprisingly good!
Transformer visualised (Image from author’s book “ AI In 100 Images ”)
Memory layers improve the factual accuracy of LLMs by over 100%, along with improvements in…