This is a new project/product idea. Premise: storytelling = Persuasion-as-a-Service.

ArXiV 1902.01109 (Storytelling Strategies)

Intro

- goal: investigate decompositions of story generators. decompositions: > allow more abstract models to be generated first > allow specialized modeling > applicable to any text dataset, no manual labeling required > previous work: Plan-and-Write (2019)
> The models are trained on 300k stories from WRITINGPROMPTS (placeholder).

Model

- overview > story generation challenge: maintaining coherence across many sentences
> previous work: self-attentive architectures (placeholders)
> goal: build decomposable, coarse-to-fine story stop generator
- decompositions: > uses negative log likelihhod of problem = L = -log sum p(x|z)p(z)
> problem: marginalization (?) over z usually intractable.
> workaround: use variational upper loss limit:
> q(z|x) = 1
> optimizer: z* = arg max p(z|x)
> L <= -log (p(x|z) - log p(z)) -- allows models to be trained separately.
- architectures: > uses convolutional seq-to-seq Gehring 2017 placeholder
> uses attention module Bahdanau, .. 2015 placeholder
> does weighted sum of encoder output
> decoder uses gated multi-head SAM Vaswani,.. 2017

Sequence Models

- uses semantic role labeling (SRL) = IDs predicates & arguments in sentences => assigns roles
- concats predictes & args ID'd by model, parses sentences with delimiters (tokens)
- verb goes first; arguments in canonical order
- focus on main narrative = only retain core args.
- one of decoder's multihead SAMs = a "verb attention head"

> if text does not contain verbs (yet) = zero vector
>

Modeling Entities

- Generating Anonymous Stories
> NER entities
* uses named-entity recognition (NER) model to ID people, orgs, locations.
* identical strings get same placeholder token
> Co-reference based entities
* can't detect different entity mentions from different strings
* uses coreference resolution model Lee et al (2018) (placeholder)
* all spans in same cluster = same entity placeholder string
* can't detect singletons - replace non-coreferent entities with unique placeholders
- Generating Entity References
> pointers
> subwords
* word-based vocabs = inadequate due to sparsity?
* instead use BYTE-PAIR-ENCODING (BPE) Sennrich 2015 placeholder
> NER entity references
* Each string maps to one SURFACE FORM (possibly multiword)
* model maps combo placeholder(token,story) to placeholder surface form
- co-reference generation
* more challenging (b/c different mentions of same entity can use different surface forms)
* each mention reference built with:
> BoW context window around entity -- allows local context to ID name,pronoun,reference.
> previously generated references for same entity (ent0 == "Bilbo" and ent0 == "him")
FairSeq (github)

Setup (data, baselines, training, generation)

Experiments

- decomposition comparison
> automation
> human-powered
- SRL decomposition effects
- model comparison
> subwords
> additional contexts
> qual. example
- entity anonymization
> entity name diversity
> entity cluster coherence
> example
* Related Work
- StoryGen with Planning
- Entity Language Models
- Non-Autoregressive Generation
* Conclusion
ArXiV 2004.14967 (Plot Machines)
ArXiV 1809.10736 (Plot Shaping)
ArXiV 1812.02784 (StoryGANs)