Publications
All the latest publications, straight from us.
May 13, 2024

CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks

Large Language Models (LLMs) such as ChatGPT and LlaMA are advancing rapidly in generative Artificial Intelligence (AI), but their immense size poses significant challenges, such as huge training and inference costs, substantial energy demands, and limitations for on-site deployment. Traditional compression methods such as pruning, distillation, and low-rank approximation focus on reducing the effective number of neurons in the network, while quantization focuses on reducing the numerical precision of individual weights to reduce the model size while keeping the number of neurons fixed. While these compression methods have been relatively successful in practice, there is no compelling reason to believe that truncating the number of neurons is an optimal strategy. In this context, this paper introduces CompactifAI, an innovative LLM compression approach using quantum-inspired Tensor Networks that focuses on the model’s correlation space instead, allowing for a more controlled, refined and interpretable model compression. Our method is versatile and can be implemented with — or on top of — other compression techniques. As a benchmark, we demonstrate that a combination of CompactifAI with quantization allows to reduce a 93% the memory size of LlaMA-2 7B, reducing also 70% the number of parameters, accelerating 50% the training and 25% the inference times of the model, and just with a small accuracy drop of 2% - 3%, going much beyond of what is achievable today by other compression techniques. Our methods also allow to perform a refined layer sensitivity profiling, showing that deeper layers tend to be more suitable for tensor network compression, which is compatible with recent observations on the ineffectiveness of those layers for LLM performance. Our results imply that standard LLMs are, in fact, heavily overparametrized, and do not need to be large at all.

January 25, 2024

CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks

Large Language Models (LLMs) such as ChatGPT and LlaMA are advancing rapidly in generative Artificial Intelligence (AI), but their immense size poses significant challenges, such as huge training and inference costs, substantial energy demands, and limitations for on-site deployment. Traditional compression methods such as pruning, distillation, and low-rank approximation focus on reducing the effective number of neurons in the network, while quantization focuses on reducing the numerical precision of individual weights to reduce the model size while keeping the number of neurons fixed. While these compression methods have been relatively successful in practice, there’s no compelling reason to believe that truncating the number of neurons is an optimal strategy. In this context, this paper introduces CompactifAI, an innovative LLM compression approach using quantum-inspired Tensor Networks that focuses on the model’s correlation space instead, allowing for a more controlled, refined and interpretable model compression. Our method is versatile and can be implemented with — or on top of — other compression techniques. As a benchmark, we demonstrate that CompactifAI alone enables compression of the LlaMA-2 7B model to only 30% of its original size while recovering over 90% of the original accuracy after a brief distributed retraining.

January 15, 2024

How Quantum Computing Can Help Meet the Goals of Sustainable Finance

Large financial institutions play an important role in driving society’s sustainability goals. Banks and hedge funds can support the decarbonization of industrial and institutional clients while also influencing capital flows through advice to individual clients. For example, financial institutions can increase the credit availability for corporate energy transition actions. At the individual level, banks and investor advisors can help clients who want to enhance the environmental, social and governmental (ESG) profile of their investments.

December 12, 2023

Quantum Computing and the Business Transformation Journey

Quantum Computing impacts business transformation in firms and requires a new mindset for operational excellence.

November 06, 2023

Hacking Cryptographic Protocols with Advanced Variational Quantum Attacks

Here we introduce an improved approach to Variational Quantum Attack Algorithms (VQAA) on crytographic protocols. Our methods provide robust quantum attacks to well-known cryptographic algorithms, more efficiently and with remarkably fewer qubits than previous approaches. We implement simulations of our attacks for symmetric-key protocols such as S-DES, S-AES and Blowfish. For instance, we show how our attack allows a classical simulation of a small 8-qubit quantum computer to find the secret key of one 32-bit Blowfish instance with 24 times fewer number of iterations than a brute-force attack. Our work also shows improvements in attack success rates for lightweight ciphers such as S-DES and S-AES. Further applications beyond symmetric-key cryptography are also discussed, including asymmetric-key protocols and hash functions. In addition, we also comment on potential future improvements of our methods. Our results bring one step closer assessing the vulnerability of large-size classical cryptographic protocols with Noisy Intermediate-Scale Quantum (NISQ) devices, and set the stage for future research in quantum cybersecurity.

October 30, 2023

Efficient tensor network simulation of IBM's largest quantum processors

We efficiently simulate IBM's largest quantum processors, Eagle, Osprey, and Condor, using graph-based Projected Entangled Pair States, achieving unprecedented accuracy with simple tensor updates.

October 30, 2023

Decoding algorithms for surface codes

Quantum error correction through surface codes, critical for reliable quantum computing, demands efficient decoding algorithms balancing speed, complexity, and accuracy.

June 05, 2023

Quantum Computing for Smart Grids

Paper by Gianni del Bimbo, Daniel García Guijo and Esperanza Cuenca Gómez.

April 27, 2023

Quantum Computing and Sustainability

Case Study by Gianni del Bimbo, Rodrigo Hernández Cifuentes, Esperanza Cuenca Gómez, Daniel García Guijo and Angus Dunnett

April 18, 2023

Application of Tensor Neural Networks to Pricing Bermudan Swaptions

The Cheyette model is a quasi-Gaussian volatility interest rate model widely used to price interest rate derivatives such as European and Bermudan Swaptions for which Monte Carlo simulation has become the industry standard.

April 14, 2023

Study by One of the World’s Largest Banks Concludes Quantum Is Useful Today

How quantum-inspired algorithms solve the most complex PDE and machine learning problems to achieve real business advantage now.

April 13, 2023

Improving Gradient Methods via Coordinate Transformations: Applications to Quantum Machine Learning

Machine learning algorithms, both in their classical and quantum versions, heavily rely on optimization algorithms based on gradients, such as gradient descent and alike.

March 28, 2023

Case Study: Ethical Considerations of Quantum Radars

A Practical Approach by Esperanza Cuenca Gómez and Pablo Martín Ramiro

December 28, 2022

Quantum-Inspired Tensor Neural Networks for Option Pricing

Recent advances in deep learning have enabled us to address the curse of dimensionality (COD) by solving problems in higher dimensions. A subset of such approaches of addressing the COD has led us to solving high-dimensional PDEs. This has resulted in opening doors to solving a variety of real-world problems ranging from mathematical finance to stochastic control for industrial applications. Although feasible, these deep learning methods are still constrained by training time and memory.

December 06, 2022

Financial Risk Management on a Neutral Atom Quantum Processor

Machine Learning models capable of handling the large datasets collected in the financial world can often become black boxes expensive to run. The quantum computing paradigm suggests new optimization techniques, that combined with classical algorithms, may deliver competitive, faster and more interpretable models.

November 26, 2022

Variational Tensor Neural Networks for Deep Learning

Deep neural networks (NN) suffer from scaling issues when considering a large number of neurons, in turn limiting also the accessible number of layers. To overcome this, here we propose the integration of tensor networks (TN) into NNs, in combination with variational DMRG-like optimization. This results in a scalable tensor neural network (TNN) architecture that can be efficiently trained for a large number of neurons and layers.

October 09, 2022

A Differentiable Distance Approximation for Fairer Image Classification

Naively trained AI models can be heavily biased. This can be particularly problematic when the biases involve legally or morally protected attributes such as ethnic background, age or gender. Existing solutions to this problem come at the cost of extra computation, unstable adversarial optimisation or have losses on the feature space structure that are disconnected from fairness measures and only loosely generalise to fairness.

October 06, 2022

Variational Quantum Non-Orthogonal Optimization

Current universal quantum computers have a limited number of noisy qubits. Because of this, it is difficult to use them to solve large-scale complex optimization problems. In this paper we tackle this issue by proposing a quantum optimization scheme where discrete classical variables are encoded in non-orthogonal states of the quantum system.

October 06, 2022

Variational Quantum Continuous Optimization: a Cornerstone of Quantum Mathematical Analysis

Here we show how universal quantum computers based on the quantum circuit model can handle mathematical analysis calculations for functions with continuous domains, without any digitalization, and with remarkably few qubits. The basic building block of our approach is a variational quantum circuit where each qubit encodes up to three continuous variables (two angles and one radious in the Bloch sphere).

August 24, 2022

Financial Index Tracking via Quantum Computing with Cardinality Constraints

In this work, we demonstrate how to apply non-linear cardinality constraints, important for real-world asset management, to quantum portfolio optimization.