So perplexity can kind of weakly analyze the first few pages of small file size pdfs one at a time, but I’d love to have something that would allow me to upload several hundred research papers and textbooks that could then be analyzed for consensus and contradictions and give me more meaningful search results and summaries than keyword searching alone. Does anything like this exist in a fairly user friendly accessible format?

  • rufus
    31·
    7 months ago

    I don’t think you can use Retrieval Augmented Genaration or vector databases for a task like that. At least not if you want to compare the whole papers and not just a single statement or fact. And that’d be what most tools are focused on. As far a I know the tools that are concerned with big PDF libraries are meant to retrieve specific information out of the library. Relevant to a specific question from the user. If your task is to go through the complete texts, it’s not the right tool because it’s made to only pick out chunks of text.

    I’d say you need an LLM with a long context length, like 128k or way more, fit all the texts in and add your question. Or you come up with a clever agent. Make it summarize each paper individually or extract facts, then feed that result back and let it search for contradictions, or do a summary of the summaries.

    (And I’m not sure if AI is up to the task anyways. Doing meta-studies is a really complex task, done by highly skilled professionals of a field. And it takes them months… I don’t think current AI’s performance is anywhere near that level. It’s probably going to make something up instead of outputting anything that’s related to reality.)