Guiding Text Generation with Constrained Beam Search in 🤗 Transformers
Back to Articles Guiding Text Generation with Constrained Beam Search in 🤗 Transformers Published March 11, 2022 Update on GitHub Upvote 16 +10 Chan Woo Kim cwkeam Follow guest Introduction Why It's Difficult Example 1: Forcing a Word Traditional Beam Search With Constrained Beam Search Example...
Back to Articles Guiding Text Generation with Constrained Beam Search in 🤗 Transformers Published March 11, 2022 Update on GitHub Upvote 16 +10 Chan Woo Kim cwkeam Follow guest Introduction Why It's Difficult Example 1: Forcing a Word Traditional Beam Search With Constrained Beam Search Example 2: Disjunctive Constraints Traditional Beam search Constrained Beam Search Banks More About Constraint Classes and Custom Constraints Conclusion Introduction This blog post assumes that the reader is familiar with text generation methods using the different variants of beam search, as explained in the blog post: "How to generate text: using different decoding methods for language generation with Transformers" Unlike ordinary beam search, constrained beam search allows us to exert control over the output of text generation. This is useful because we sometimes know exactly what we want inside the output. For example, in a Neural Machine Translation task, we might know which words must be included in the final translation with a dictionary lookup. Sometimes, generation outputs that are almost equally possible to a language model might not be equally desirable for the end-user due to the particular context. Both of these situations could be solved by allowing the users to tell the model which words must be included in the end output. Why It's Difficult However, this is actually a very non-trivial problem. This is because the task requires us to force the generation of certain subsequences somewhere in the final output, at some point during the generation. Let's say that we're want to generate a sentence S that has to include the phrase p1={t1,t2} p_1=\{ t_1, t_2 \} p1={t1,t2} with tokens t1,t2 t_1, t_2 t1,t2 in order. Let's define the expected sentence S S S as: Sexpected={s1,s2,...,sk,t1,t2,sk+1,...,sn} S_{expected} = \{ s_1, s_2, ..., s_k, t_1, t_2, s_{k+1}, ..., s_n \} Sexpected={s1,s2,...,sk,t1,t2,sk+1,...,sn} The problem is that beam search generates the sequence token-by-token. Though not entirely accurate, one can think of beam search as the function B(s0:i)=si+1 B(\mathbf{s}_{0:i}) = s_{i+1} B(s0:i)=si+1, where it looks at the currently generated sequence of tokens from 0 0 0 to i i i then predicts the next token at i+1 i+1 i+1 . But how can this function know, at an arbitrary step iSource: Hugging Face — Published — Category: Models