Yogthos on Nostr: The researchers have developed a new type of LLM that doesn't use matrix ...

The researchers have developed a new type of LLM that doesn't use matrix multiplication making it more efficient and less energy-intensive.

The model uses ternary values and simpler arithmetic operations instead of matrix multiplication.

Expensive self-attention mechanism is replaced with a simpler, more efficient unit called MLGRU.

The new model has shown performance comparable to state-of-the-art models on several benchmark tasks.

#machinelearning

https://arstechnica.com/information-technology/2024/06/researchers-upend-ai-status-quo-by-eliminating-matrix-multiplication-in-llms/2/