Copyright Notice: 2025 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Abstract

This paper investigates efficient compression techniques for embedding representations in semantic communication systems. As communication networks evolve to support AI agent-based interactions, there is a growing need for methods that preserve semantic content while reducing transmission overhead. We explore Product Quantization (PQ) as a specialized compression approach for embeddings constructed using Matryoshka Representation Learning (MRL), which organizes semantic features hierarchically by relevance. This structure allows a reduction in model complexity by using truncated embeddings instead of full-size representations. Within a framework based on network-application cooperation and following to the Separation of Concerns principle, we analyze how PQ serves as an effective source coding strategy for semantically structured representations. Our comparative analysis demonstrates that PQ delivers better task accuracy performances and efficiency compared to simple truncation methods, especially when compressing MRL-based embeddings.

1 Introduction

Semantic communication represents a paradigm shift towards communication systems that prioritize service-oriented metrics tailored to represent application needs, rather than focusing on traditional network-centric metrics intended to optimize bit-accurate transmission. Guo et al. in [1] highlight AI agent-based communications as a key use case motivating the transition from a traditional to a semantic data communication framework, facilitating data sharing and task execution in collaborative agent-to-agent communication scenarios, also envisioned in the future AI-native 6G network [2]. In ways to directly integrate this semantic awareness into the design of communication systems, [3] points out that many works have proposed semantic representation techniques based on Knowledge Graphs, Deep Neural Networks (DNN), Natural Language Processing, Toposes, and Causal Representation Learning. These semantic representations extract the underlying characteristics of the source information and expose them as algebraic structures defined in a semantic space.

Particularly, some of the DNN-enabled approaches focus on constructing semantic representations as high-dimensional, real-valued vectors known as embeddings. In essence, they can capture the underlying semantic features of diverse data types and structure them within a vector space, where the semantic relationships inherent in the data are represented as mathematical relationships between vectors. Leveraging this semantic structuring capability, embeddings currently enable many multimodal AI-based tasks [4], [5], [6]. Also according to [3], the field of semantic communication has primarily focused on end-to-end optimization of the communication process [7]. By developing such methods, the semantic properties of embeddings are leveraged to properly represent the application-specific requirements while compacting the amount of information transmitted.

However, general-purpose operated networks are subject to the Separation of Concerns principle, which fundamentally distinguishes between two domains: application concerns that involve understanding data content and purpose, and network concerns that focus exclusively on providing reliable data transmission without regard to the meaning of the information being conveyed. As a consequence, this paradigm delimitates the level of dialogue between network and application entities, currently impeding end-to-end optimization capabilities for common general-purpose operated networks. To address this limitation, [8] proposes several approaches for progressively increasing the network-application cooperation, and thus gradually incorporating semantic processing into the network. In this work, building upon this cooperative framework, we are interested in exploring the utilization of embeddings as general-purpose data representations. To ensure communication compatibility between several applications, general-purpose embeddings models acting as standard data formats are considered. Moreover, as the sustainability of information networks remains a growing concern, optimizing existing network bandwidth utilization is essential. As traditional communications have developed specialized compression techniques for established data formats, the field of semantic communications shall also explore efficient source coding methods for these novel general-purpose embedding representations.

Proposed as a promising technique for compacting embeddings, the Matryoshka Representation Learning (MRL) method [9] addresses the need for data representations that simplify downstream task executions. By ordering semantic features by relevance within nested structured embeddings, this approach enables flexible compression through truncation at different levels while maintaining a suitable semantic representativeness for the task at hand. Exploiting the MRL property in embeddings construction, this paper examines Product Quantization (PQ) [10] as a specialized embeddings compression approach for semantic-aware networks. Assuming cooperation between network and application, our analysis explores how PQ can function as a dedicated source coding strategy that utilizes the structured semantic content inherent in Matryoshka representations. The analysis also compares PQ compression results to those obtained by performing truncation compression.

2 Semantic Communication Architecture

The architecture studied is depicted in Fig. 1. Initially, a text dataset related to a given task of interest is accessed, and a general-purpose embedding model is used to encode the data samples into their corresponding semantic representations. Then, a compression technique is performed to compact the embedding representations, with the resulting compressed vectors being encoded into binary format for transmission across the communication channel. At reception, after performing the corresponding binary decoding step depending on the evaluated compression scheme, the embeddings are reconstructed and provided to a task-adapted model, which performs its target task and expresses its performances according to a metric of interest. The compression techniques are finally evaluated in terms of both task execution quality and network performance metrics.

2.1 Matryoshka Representation Learning

Matryoshka Representation Embeddings (MREs) are nested vector representations where smaller dimension embeddings are contained within larger ones, similar to Russian nesting dolls. Those nested representations are trained using a multi-objective optimization approach where the model learns to encode information hierarchically across different dimensional subspaces simultaneously. The training process ensures that lower-dimensional projections retain the most important information while higher dimensions add more nuanced semantic details.

As shown in Fig. 2, the MRL construction is expressed as a composition of multi-granular nested embeddings. Parting from an embedding \(z \in\) \(\mathbb{R}^d\), smaller vectors are composed by the first elements of \(z\) with decreasing size \(l\). In the example, \(l \in L = [d, d/2, d/4, d/8, d/16]\), where \(L\) is the set of possible granularities.

During the training phase, each of these fragments is associated with a training loss function \(\mathfrak{L}(l_i)\), with a global training loss function \(\mathfrak{L}(z)\) expressed as,

\[\mathfrak{L}(z) = \sum^{|L|}_{i=1} \alpha_i \mathfrak{L}(l_i),\]

where \(l_i\) is the \(i^{th}\) component of the set of possible granularities \(L\) and \(\alpha_i\) represents the weight or the importance attributed to each multi-granular loss \(\mathfrak{L}(l_i)\). Based on this construction, a Matryoshka embedding model optimizes its global training loss function over a given set of training data. Each individual loss \(\mathfrak{L}(l_i)\) is defined by a base loss function. Because the smaller fragments are involved in more \(\mathfrak{L}(l_i)\) loss terms than the larger ones, they have more impact on the global loss, leading the model to place the most relevant semantic features within its first dimensions. It should be noted that this training technique can be used as a fine-tuning of an already existing embedding model.

During the testing phase, smaller representations are built by truncating the full-sized vectors, hence suppressing the last vector dimensions, which are supposed to be the least informative ones. This construction can then output more compact informational structures at different granularities, which can be flexibly adapted to specific complexity requirements of various task modalities; e.g., adaptive retrieval or adaptive classification inference tasks. This compression approach is simply referenced as Truncation hereafter.

2.2 Product Quantization

As a classical vector compression technique from source coding theory [10], the PQ method splits a high-dimensional vector \(z \in\) \(\mathbb{R}^d\) into \(M\) smaller sub-vectors of dimension \(d/M\). Using a piece-wise quantization method such as K-means clustering, each sub-vector space or subspace is quantized independently into a set of \(K\) sub-codewords or centroids of dimension \(d/M\). Then, the nearest neighbor centroids are computed for each sub-vector in terms of the squared Euclidean norm and indexed as their quantized representations. Finally, the nearest neighbor indices are used to compose a lower-sized structure, the PQ code, which retains the necessary information to reconstruct an approximation of the original vector \(z\).

During a training phase, training vectors of dimension \(d\) are split into their associated sub-vectors, and then \(M\) sub-codebooks containing \(K\) centroids are determined from quantizing each of the \(M\) composed subspaces. A consecutive testing phase consists of computing the PQ codes for each test vector based on the trained sub-codebooks.

3 Experimental Methodology

The adopted methodology particularly aims to investigate two research hypotheses: the first one focused on attesting performance gains in favor of processing Matryoshka embeddings against their non-MRL counterparts when performing the same Truncation compression scheme; and the second one aiming to identify the potential benefits of applying PQ as a specialized embedding compression scheme against simple Truncation results, especially when leveraging the MRL property.

3.1 Massive Text Embedding Benchmark

The Massive Text Embedding Benchmark (MTEB) [11] is an evaluation framework designed to assess the performance of embedding models dedicated to the representation of text data across a large variety of tasks and datasets. We rely on such a framework to set up the experimental scenario and evaluate how the studied compression techniques impact the quality of task executions.

In this paper, we consider generalist embedding models addressing a variety of ML tasks (e.g., classification, information retrieval, clustering) rather than task-specialized embedding models. This allows us to verify the potential of such generalist models in view of the aforementioned general-purpose operated networks. Precisely, we used tomaarsen/mpnet-base-nli¹, referred to as non-MRL model, and tomaarsen/mpnet-base-nli-matryoshka², referred to as MRL model. According to [12], both models share the same pre-trained model version, the corpus over which they are trained, and the base loss function used to optimize their training steps. This implies that the sole distinction between the models is the MRL structure integrated into the MRL model, enabling the evaluation of how incorporating the MRL property within the embeddings affects task performance outcomes. Setting up the MRL construction, the base loss function is a multiple negatives ranking loss, introduced in [13]. Also, both models output embeddings composed of 768 dimensions, and the Matryoshka model optimizes its loss function for the granularities [768, 512, 256, 128, 64], with \(\alpha_i = 1\) for each of these values.

We consider then two different datasets, each associated with a respective MTEB task: the Stanford Sentiment Treebank binary (SST-2) datasetfor binary classification and the BANKING77 corpusfor multilabel classification. Both datasets are composed of text data, with their samples describing natural language examples for two different lexical fields (i.e., movie reviews and customer banking queries, respectively). Additionally, the related binary and multilabel classification tasks allow us to objectively assess task performances at an application level by estimating the true label classes associated with each test sample.

• SST-2: Used for sentiment analysis, SST-2 consists of text movie reviews labeled according to two well-distinguished classes in terms of sentiment polarity: positive (1) and negative (0) sentences. The dataset has 67,349 training samples, 872 validation samples, and 1,821 test samples. Due to a restriction of accessing its test corpus from MTEB functionalities, this work only uses the validation set to perform testing inference for the binary classification task, even though the entire training set was accessible and taken into account to perform the training processes.

• BANKING77: Designed to contain banking service data for intent classification, BANKING77 consists of a collection of customer inquiries that are labeled according to their associated banking-related topic; e.g., account management, loans, transactions. As its name suggests, the corpus is composed of 77 unique labels describing customer intents, as well as 10003 training samples and 3080 test samples.

3.2 Evaluation Metrics

3.2.1 Number of Bits Transmitted per Embedding

As a network-centric metric, the number of bits transmitted per embedding reflects the amount of resources required by the network during the transmission of a single semantic representation.

For Truncation schemes, each component of the embedding vectors is encoded as a float32 using the IEEE 754 [14] standard prior to communication. The number of bits transmitted per embedding is defined then as \(n_{bits} = 32 \times n_{dims}\), where the integer value \(32\) refers to single precision IEEE 754 and \(n_{dims}\) represents the chosen number of dimensions for truncation.

For PQ schemes, we quantize each of the split sub-vectors with two centroids. Consequently, each index composing a PQ code is transmitted as a single-bit representation. The number of bits transmitted per embedding is expressed then as \(n_{bits} = n_{dims}\), where \(n_{dims}\) represents the number of PQ sub-vectors created and transmitted individually as a single bit.

For Truncation compression, \(n_{dims} \in S_{g} = [1,2,4,8,16,32,64,128,256,512,768]\), while for PQ compression, \(n_{dims} \in S_{g} = [1,2,4,8,16,32,64,128,256,768]\) as the division of \(768\) by \(512\) yields a non-integer value for the sub-vectors’ dimension.

3.2.2 Classification Accuracy

As an application-centric metric, the classification accuracy is a concrete measure setting the quality of the task performances. Following the MTEB standards, it is computed as,

\[Acc = \cfrac{CP}{TP},\]

where \(CP\) stands for the number of correctly predicted sentences matching their associated true label, and \(TP\) stands for the total number of predictions, which represents exactly the total number of samples composing the test set.

3.2.3 Average Accuracy Difference

To illustrate quantitatively the effect of multi-granular versions of the original full-sized embeddings, we analyze an average accuracy difference metric stated as follows.

\[D = \cfrac{1}{|S_{g}^{*}|} \sum_{n_{dims} \in S_{g}^{*}} \left(Acc_{n_{dims}} - Acc_{\text{ref}}\right),\]

where \(Acc_{n_{dims}}\) is the classification accuracy obtained by using a granularity \(n_{dims}\), and \(Acc_{\text{ref}}\) is the maximum classification accuracy achieved when the original full-sized embeddings are directly used to perform the task. For a fair comparison, only a common number of bits for both Truncation and PQ schemes is considered. In this study, the common values of the number of bits transmitted per embedding for both compressions are \([32,64,128,256]\), resulting in \(S_{g}^{*} = [32,64,128,256]\) for PQ and \(S_{g}^{*} = [1,2,4,8]\) for Truncation. \(|S_{g}^{*}|\) denotes the cardinality of the set \(S_{g}^{*}\). Since different embedding models yield distinct reference values, we assess performance degradation as a relative deviation from each model’s maximum reference performance, establishing the benchmark performance for each pair \([\text{dataset}, \text{embedding model}]\). Average accuracy difference allows us to examine the cumulative task degradation observed from adopting different granularity configurations.

3.3 Main Hyperparameters

3.3.1 Channel Condition

We assume an error-free transmission, so we can assess the task performance purely from an application perspective. This assumption is realistic for general-purpose operated networks, in which error correction mechanisms are implemented and packets are usually provided error-free to the application.

3.3.2 Classifier Setup

MTEB sets a logistic regression as the standard task-adapted model to perform the classification process. The classifier training and testing phases are detailed below.

• Training phase: During its training step, the classifier is fed with labeled training embeddings generated by the embedding model. Following the standardized MTEB configuration, 8 equilibrated and shuffled training samples per label are taken into account, leading to 16 and 616 training sentences for SST-2 and BANKING77, respectively. Because the Truncation compression modifies the number of embedding dimensions, a dedicated classifier is trained for each value of \(n_{dims}\), resulting in 11 classifiers after the training phase. In case of PQ compression, the standard classifier at 768 dimensions is used, without any compression applied.

• Testing phase: Consecutive to each training stage, the testing stage is performed over the entire set of test embeddings. These test vectors are compressed and transmitted according to the aforementioned experimental scheme presented in Fig. 1. Finally, the classifier uses the reconstructed test embeddings to perform its task, evaluated with the aforementioned metrics.

By denoting one training and testing phase as one MTEB experiment, it was observed that the default MTEB configuration maintains identical training batches across separated MTEB experiments. This configuration limits our ability to assess the effects of random training sample selection on the classifier performance. To properly ensure independence between MTEB experiments, we have adapted this standard configuration to always perform a random selection of training batches. The code implementing this modification has been made publicly available by the authors³.

The reference and Truncation approach results are obtained using 10k MTEB experiments each. For PQ, we conducted simulations with 100 distinct clustering initializations, each evaluated across 100 MTEB experiments, yielding a comparable set of 10k simulations. All clustering models were trained on the complete training dataset.

4 Numerical Results

Fig. 3 and Fig. 4 present the classification accuracy for each run with granularity \(n_{dims}\) along the sets \(S_g\) defined for PQ and Truncation compressions. The reference value \(Acc_{ref}\) for each pair \([\text{dataset}, \text{embedding model}]\) is illustrated as a horizontal line, representing the maximum classification accuracy obtained without any alteration of the embeddings. For SST-2 results, \(Acc_{ref} = [0.8819, 0.8911]\) for non-MRL and MRL models, respectively. For BANKING77 measures, \(Acc_{ref} = [0.8237, 0.8308]\) in the same respective terms. The task performances are presented over the number of bits transmitted per embedding during transmission through an identity channel. As vertical dashed lines delimited by colored bounds, we provide the accuracy intervals over the 10k independent MTEB experiments for each compression configuration. Additionally, Table 1 shows the measured average accuracy difference over the common compression range for both compression techniques.

Comparing Truncation compression results in both Fig. 3 and Fig. 4, we observe that the compression of representations constructed using the MRL construction consistently yields better performance than their non-MRL counterparts for both datasets. This finding validates our first research hypothesis, attesting to the contribution of the MRL construction in better executing different tasks at flexible embedding granularities. Also, the MRL model requires fewer bits per embedding to reach its reference performance values in both cases, although its minimum accuracy values (i.e., when the classifier is incorrectly classifying test labels in a worst-case scenario) surpass those of the non-MRL model for small granularities over SST-2. This misclassification variability is caused by the fact that MTEB only considers 8 training samples per class as a standard pipeline parameter, consequently producing very small batches of training samples (i.e., with only 16 training embeddings per batch) to train the classifiers. Thus, when assessing SST-2 results, the classification models are more sensitive to the representativeness of a very low number of training samples to estimate the label distribution, and especially when dealing with hardly truncated vectors as observed in Fig. 3. Besides, such variability is not observed for BANKING77 results, where larger training batches of 616 samples produce more stable and resilient classifiers.

Regarding the potential of the Truncation compression itself in preserving task quality, we can see that the accuracy gaps from each accuracy value with respect to its reference are kept on similar levels over low-bit configurations, only modestly increasing from granularity 16 for both models. This remark is also perceived when comparing average accuracy difference values for Truncation compression, with close performances over both datasets. This suggests that Truncation schemes modestly exploit the structured organization within Matryoshka embeddings to perform compression, still exhibiting substantial task degradation for low-bit representations.

Regarding PQ, results have shown an interesting potential in leveraging embedding properties to perform compression, especially when dealing with Matryoshka representations. Analyzing SST-2 accuracy results, we observe that even very small PQ granularities present close performances to their respective references for both non-MRL and MRL models, but more consistently to the Matryoshka case. The 2-bit case in the PQ + Non-MRL curve presents a notably high accuracy relative to its reference line. This suggests that a 2-bit PQ compression already provides a clear binary label discrimination and simplifies the task execution for the classifier, potentially dealing with a lower correlation between dimensions while preserving the label discrimination property defined in the original embedding space. SST-2 performances may also suggest that, in the MRL cases, Matryoshka embeddings could potentially better organize intrinsically correlated relevant dimensions to group samples into 2 representative clusters by construction over multiple granularities. This capability would allow to represent both distributions of binary labels with more accurate centroids, and then facilitate the task execution to low-complexity classifiers. For BANKING77, however, the PQ + MRL curve is slightly under its non-MRL counterpart, suggesting that the classic PQ construction does not leverage in the same manner the Matryoshka property to efficiently describe the multilabel distribution at the classifier level. The PQ compression results for SST-2 are similarly affected by suboptimal classifier training conditions inherent to the evaluation pipeline of the MTEB benchmark, inducing high variability of misclassifications. However, similar to Truncation schemes, PQ maximum accuracy values equally reach their references, and indicate a clear advantage for Matryoshka embeddings in consistently achieving reference performance even for very low-bit configurations.

Directly comparing Truncation and PQ compression results over their common compression range for both embedding models, it is noticed that PQ accuracy results are largely higher than their Truncation counterparts, which is emphasized by a maximal average accuracy difference measure in Table 1. The best average accuracy difference for SST-2 is obtained by the PQ + MRL model, indicated with a negative sign that represents task accuracy degradation in relation to the reference value (as observed in Fig. 3). For BANKING77, the best average accuracy difference is still observed in favor of PQ (i.e., PQ + Non-MRL model), with a slightly lower value observed for its MRL counterpart. These results corroborate the previous analysis on the PQ performances when exploiting embedding properties to perform compression. Also, even assessing accuracy performances outside the common compression region, both PQ curves at \(768\) bits present quite similar classification accuracy when compared to their corresponding Truncation curves at \(8192\) bits, attesting then a gain of efficiency in a factor of around \(10\) to the PQ method.

Finally, these several observations allow us to partially validate our second research hypothesis (i.e., the potential of PQ as a specialized embedding compression, especially with Matryoshka embeddings). Also, they suggest that, as important as it is to individually recognize the most informative embedding dimensions before compressing, it is also important to consider the informational relationship between grouped dimensions/features to compose the PQ semantic subspaces, as well as the quantization strategy applied, allowing the label distributions in each of these subspaces to be adequately represented. As a potential consequence, it is expected that there exists a PQ feature combination coupled with an adaptive quantization scheme that maximizes the information carried by the quantized subspaces and that best characterizes the data distributions to the task-adapted model.

5 Conclusions and Perspectives

This work has proposed an investigation of Product Quantization as a specialized embedding compression strategy that leverages network-application semantic integration and the Matryoshka property within structured semantic representations. The study also validated the benefits of integrating Matryoshka construction into embeddings at various levels of granularity, demonstrating that this approach maintains task execution quality when subjected to truncation operations as a first compression strategy. Aligning both network and application metric perspectives to design the communication process, the evaluated techniques yielded promising findings in preserving the quality of the studied tasks while compacting semantic representations.

Further work will be carried out to investigate the generalization of the PQ technique in compressing representations from a general-purpose perspective rather than a data-specialized construction. Such studies will also be interested in exploring the potential of more sophisticated quantization techniques, particularly hierarchical clustering approaches, for optimizing sub-codebook composition and structure. Relying on the property of embeddings as multimodal semantic representations, the generalization of the proposed compression analysis across diverse tasks, datasets, and data modalities will also be investigated.