November 26, 2022 · 1 min read

Variational Tensor Neural Networks for Deep Learning

Deep neural networks (NN) suffer from scaling issues when considering a large number of neurons, in turn limiting also the accessible number of layers. To overcome this, here we propose the integration of tensor networks (TN) into NNs, in combination with variational DMRG-like optimization. This results in a scalable tensor neural network (TNN) architecture that can be efficiently trained for a large number of neurons and layers.

Multiverse Computing

The variational algorithm relies on a local gradient-descent technique, with tensor gradients being computable either manually or by automatic differentiation, in turn allowing for hybrid TNN models combining dense and tensor layers. Our training algorithm provides insight into the entanglement structure of the tensorized trainable weights, as well as clarify the expressive power as a quantum neural state. We benchmark the accuracy and efficiency of our algorithm by designing TNN models for regression and classification on different datasets. In addition, we also discuss the expressive power of our algorithm based on the entanglement structure of the neural network.

Full paper here.

About Multiverse Computing

Multiverse Computing is the leader in quantum-inspired AI model compression. The company’s deep expertise in quantum software led to the development of CompactifAI, a revolutionary compressor that reduces computing requirements and unleashes new use cases for AI across industries. Headquartered in Donostia, Spain, with offices in the United States, Canada, and across Europe, Multiverse serves more than 100 global customers, including Iberdrola, Bosch, and the Bank of Canada.