DPTDR: Deep Prompt Tuning for Dense Passage Retrieval

DPT originates from prompting and prompt tuning (Liu et al., 2021a). Given some discrete or continuous prompts, PLMs like GPT-3(Brown et al., 2020) can achieve impressive zero-shot and few-shot performances for knowledge-intensive tasks. However, studies find that prompt tuning fails to perform well for moderate-size models (Liu et al., 2021b). Thus, DPT(Li and Liang, 2021; Liu et al., 2021b) is proposed by inserting prompts at deep layers to steer PLMs towards desired directions more capably. It obtains comparable performance to FT across a range of NLP tasks. DPTDR is mainly related to DPT, focusing on dense passage retrieval instead of NLP. There also exist works of pretraining prompts for prompt tuning(Gu et al., 2021), which shows effectiveness in few-shot learning using billion-size models, as we will explore as well in the context of DPT.

Pretraining We have witnessed a series of unsupervised pretraining works proposed for dense retrieval, such as ICT, BFS, WLP, and independent cropping (Lee et al., 2019; Chang et al., 2020; Izacard et al., 2021). Following works also try to pre-train retriever and reader jointly for question answering (Guu et al., 2020). coCondenser (Gao and Callan, 2021b) follows a contrastive learning framework using Condenser structure (Gao and Callan, 2021a) by adding an explicit decoder to learn representations better. There are also semi-supervised and weakly-supervised works. DPR-PAQ (Oğuz et al., 2021) pre-trains a PLM using 65-million-size synthetic QA pairs on the target corpus. GTR (Ni et al., 2021) pre-trains T5 (Raffel et al., 2019) on 2-billion-size community QA pairs from T5-base to T5-xxlarge. We follow unsupervised contrastive learning as our pretraining strategy for DPTDR. However, we aim to ensure compatibility with any PLM, thus resulting in different sample building processes and model structure choices.

Negative mining DPR (Karpukhin et al., 2020) proposes to train retrievers using BM25 negatives. ANCE (Xiong et al., 2020) extends that by mining negatives periodically from previously-trained dense retrievers. RocketQA and RocketQAv2 (Qu et al., 2021; Ren et al., 2021) introduce the idea of denoised negative sampling by selecting negatives with high confidence scored by a re-ranker. DPTDR unifies the above into a general negative mining strategy.

Let $C$ be a corpus consisting $N$ passages, denoted by $p_1,p_2,...,p_N$. Given a question $q$, the task of dense retrieval is to find a passage $p_i$ that is considered relevant to the question.

The dual-encoder Normally, a dual-encoder is applied. First its passage encoder $E_p(\cdot )$ embeds a passage $p$ to a $d$-dimensional dense vector. Then a vector search index (Johnson et al., 2019) of passages is built for retrieval. At inference time, the question encoder $E_q(\cdot )$ embeds the question $q$ to a $d$-dimensional dense vector, and $k$ passages closet to the question based on the vector similarity will be retrieved. In practice, the similarity score is computed as the inner product:

For PLM-based dual-encoder, we usually take the representation at the first token (e.g., [CLS] symbol in BERT (Devlin et al., 2018)) as the output dense vector.

Deep prompt tuning We then apply DPT in the PLM-based dual-encoder, as illustrated in the left part of Figure 1. To prepend multi-layer prompts for the dual-encoder, we initialize a trainable prefix matrix $M$ of dimension $l\times d$ for each layer of the PLM, where $l$ is the length of the prompt and $d$ is the hidden size of the PLM. Since the prompt resides at the deep layers of PLM, it has a full capacity to steer the PLM towards the desired direction and output meaningful dense vector for questions and passages. Note that a verbalizer (Schick and Schütze, 2020) plays a vital role in mapping words to labels in canonical prompt tuning. However, we remove it in dense retrieval since the output dense vector is what we need. Let $E_p^{\prime }$ as the prompted passage encoder and $E_q^{\prime }$ as the prompted question encoder, and the similarity score is computed:

Training The objective of the training is to learn dense vectors so that the similarity between relevant pairs of questions and passages ranks higher than irrelevant ones. Given a pair of question $q$ and positive passage $p_i$, along with $n$ negative passages, we optimize the loss function as the negative log-likelihood of the positive passage:

Generating negative passages is critical for the performance, and we will explain it in Sec. 3.3. During training, we freeze parameters of the backbone PLM and only update the deep prompts, where approximately 0.1%-0.4% parameters of a PLM get trained.

Inference As illustrated in the right part of Figure 1, since the backbone PLM is frozen, it is possible to deploy it ahead as online PLM services and then pass the trained prompts as pre-computed key values together with tokenized inputs to get dense vectors. It is at the core of how we save efforts and costs of deployment and increase the utility of computing resources. In practice, the cloud service providers or infrastructure teams of search companies are able to focus on the PLM as a central service, while users can quickly train deep prompts for different retrieval tasks and obtain efficient and compelling retrieval performances without any deployment.

Although DPT brings in many advantages, it is worth noting that it does not accelerate the inference speed because the forward computation is not reduced but increased slightly.

The goal of RIP is to either pre-train deep prompts or PLMs using contrastive learning. We first describe the task as follows. Let $C$ denote a corpus consisting $N$ passages. For a passage $p_i$, we split it into $l$ sentences, denoted by $s_i^1,...,s_i^l$. Given a sentence $s_i^j$, the task of pretraining is to distinguish its context sentence $s_i^{j^{\prime }}$ from sentences of other passages $s_k^l$, where $k\ne i$. Formally, we randomly select a pair of sentences from each passage as context sentences to form a batch of training data $B=\{s_i^1,s_i^2{\}}_{i=1}^m$, where $m$ is the batch size. Then we define the contrastive loss for $s_i^j$ over the batch as:

In contrast to prior works(Gao and Callan, 2021b; Izacard et al., 2021), we directly sample sentences as opposed to text spans. Since sampling text spans is a non-trivial technique where factors such as the probability of short sentences and how to keep the spans linguistically meaningful can have a complicated effect on the pretraining, we remove this complexity in our approach. We also conduct an experiment observing sentences work even better than text spans in MS-MARCO corpus (Sec. 4.4).

Under the contrastive learning task, there exist two pretraining ways tailored for DPT, depending on the pre-trained objects (i.e., the deep prompts or the PLM backbone).

Pre-train deep prompts One is to pre-train deep prompts with a vanilla PLM. Later we initialize a DPT-based retriever using the pre-trained deep prompts and the vanilla PLM. However, experiments in Sec. 4.4 show that it suffers from catastrophic forgetting and exhibits no superior performance to randomly initialized prompts.

Pre-train the PLM The other is to pre-train a PLM, and then we initialize a DPT-based retriever using randomly-initialized deep prompts and the pre-trained PLM. Notice that we intend to allow any PLM to be easily pre-trained for DPT so that users may employ their own PLMs. Thus we contrast prior works such as coCondenser(Gao and Callan, 2021b), a state-of-the-art model structure in contrastive pretraining, by removing the workload to modify any model structures. Surprisingly, it yields better performance than coCondenser in Table 8. Therefore, we refer RIP strategy as pretraining of PLMs for the rest.

For any PLM, We also intend to remain its original self-supervised tasks, such as masked language modeling(MLM; Devlin et al., 2018; Sun et al., 2019), denoted as $L_s$. Therefore, the final loss of pretraining over the batch is:

After pretraining, the resulting model can be deployed once as online services and taken as the backbone model for DPT training.

We also develop unified negative mining for DPT, as interpreted as "Multiple Retrievers & Hybrid Sampling." "Multiple Retrievers" is to incorporate negatives from as many retrievers as we can. We use a BM25 retriever as the initial retriever and train a DPT-based retriever using BM25 negatives. Later we treated retrieved negatives from the BM25 retriever and the first DPT-based retriever as un-denoised hard negatives. Users are allowed to introduce any other retrievers if possible. "Hybrid Sampling" is to select denoised hard negatives from un-denoised hard negatives retrieved by the above multiple retrievers. We borrow an existing re-ranker released by RocketQA (Qu et al., 2021) and select those negatives with high confidence. For training the final DPT-based retriever, we mix the denoised hard negatives, un-denoised hard negatives, and easy negatives from in-batch or cross-batch training.

We believe unified negative mining is critical for the performance of DPT-based retrievers, as it provides negatives of high quality and diversity.