[D] how to do NLP Q/A on the content of a set of books?
I’m trying to make a nlp that can answer questions about e.g a specific/static body of 100 books, each containing 200 pages. I don’t think I can simply supply that body of text a as ‘context’ because it’s too big. On the other hand, fine tuning an existing language models would more be about learning domain specific terminology, but I doubt if would actually capture the knowledge contained in those books, right? How should I go about this? Are there any good references I could learn from?
Posted Jun 13, 2022, 12:37 PM UTC
37
Here is a good walkthrough on how to make such a system for long-form question answering, or LFQA. Once you get the basic idea, you can swap out a lot of the component algorithms to optimize for your use case.
Essentially, it comes down to first encoding the set of books (as paragraphs or sentences, depending on how fine-grained you want to search), then putting them in a searchable database, finding the top N best candidates for the query in the database, and passing those top N candidates in as the context to a second, question-answering model. There are a few huggingface models that work well for that last step, including yjernite's from the link or tinyroberta-squad, for example. Naturally it depends on the hardware you have available as well. Sentence-Transformers with Approximate Nearest Neighbour or something like ElasticSearch tends to work well for embeddings and retrieval, in my experience.
This is a good advice.
I would also recommend checking out a Bert model trained on the MSMARCO dataset. Recently we had a implement a search engine where I work and this model is the one that gave us the best results, beating the squad ones.
Thanks for the reference! I found Bert-MSMARCO https://huggingface.co/Capreolus/bert-base-msmarco and "PARADE" https://arxiv.org/abs/2008.09093
Thanks. That's a nice hybrid & scalable solution. Its basically creating a search engine, and it allows one to easily add new books. The "best candidates search" acts like an attention mechanism, although with attention the set of selection of "top N candidates" gets collapsed into a single information item, which then gets decoded into a single answer. It depends a bit on the application if you want that or not. I think quoting source fragments as answers is the easiest and most transparent Q/A algorithm for me.