Skip to content

sucv/EasyPaper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

8 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation


Logo

Easy Paper

Table of Contents
  1. About The Project
  2. Getting Started
  3. Usage
  4. Venues
  5. Limitation
  6. Contributing
  7. Acknowledgments

🌟 About The Project

EasyPaper is a Human-in-the-Loop AI tool designed for large-scale literature review across fields or topics. It offers:

  • A CS database with 80K top-tier papers from recent years and the whole arXiv.
  • Accurate RAG using vector-less tree-based indexing and AI.
  • Deep Research and Chat agents built with Langchain and DeepAgents.

Built With

πŸš€ Getting Started

The system works as a self-hosted React/FastAPI web application.

πŸš€ Prerequisites

Make sure you have the followings:

  • node.js for Frontend
  • Python 3.12+ for Backend
  • Marker API for PDF OCR
  • One Text Embedding API key (Paid API or Ollama) for RAG
  • One AI API key (Paid API or Ollama) for Chat and Deep Research

πŸš€ Installation

  1. Clone the repo
    git clone https://github.com/sucv/EasyPaper.git
  2. Install NPM packages
    # under EasyPaper/frontend/
    npm install
  3. Install Python packages
    # under EasyPaper/backend/
    pip install -r requirements.txt
  4. (Optional) Download the database (containing 80K top-tier CS papers) from Release, and put it into EasyPaper/chroma_db/
    # under EasyPaper/
    mkdir chroma_db
    # Then put the downloaded chroma.sqlite3 into EasyPaper/chroma_db
    # Note that the database is indexed using the qwen3-embedding:8b from Ollama.
    # If you use the database, you have to specify qwen3-embedding:8b for embedding_config in your config.yaml
  5. Provide your API keys and/or Ollama base url
    # under EasyPaper/
    cp .env.example .env
    # Then put your API keys or Ollama base url there
    # Add LangSmith for free and accurate token usage and pricing tracking
  6. Edit the EasyPaper/config.yaml to specify your providers, models and configurations
  7. Launch the app
    # under EasyPaper/
    python start.py
    # Then visit http://localhost:5173/

πŸ“– Usage

EasyPaper starts with creating a Project. You may also continue from an existing Project. A Project can have multiple Idea, each Idea can have multiple reports from different aspects.

πŸ“– Query the database and Select the paper

EasyPaper supports three data sources:

  • A sqlite3 database containing 80K top-tier CS papers crawled from a crawler.
    • About 80% are downloadable (Top CS conferences).
    • The rest 20% contain only title, author, year, and publisher (Mostly top CS journals and a few conferences)
  • The arXiv API
  • Your own PDFs
    • Could be the top journal papers you manually downloaded
    • If you are non-CS people and have some PDFs to analyze just like the PaperQA.

Choose your query method from Vector, Boolean Expression, or arXiv, select the Year and Venue, then click Search. Select the interested paper in the results or check the citation count.

  • Both the Vector and Boolean Expression will query with the Title only. Currently there is no way to add other field. See Example for the Boolean Expression.
  • For arXiv, it's the official API, which allows "Title, Author`, and many more. See the [official example].

If you are interested in adding more venues or years to the database, please refer to paperCrawler and crawl by yourself. Then generate the Chroma vectorstore by running:

# under EasyPaper/
populate_chroma.py your_crawl.csv

πŸ“– Add to the cart

Add the selected paper to the Cart (also load your local PDFs to the cart), select or create an Idea and send the cart papers to your Idea Panel.

πŸ“– Define your Idea

The Idea serve as the prompt, guiding the AI to find relevant information from the papers in an Idea Panel. The Idea can be named as one single word or sentences. It is your best interest to think about your Idea.

Content in the Idea Panel is preserved on your file system.

# Example 1 for getting the methodology details:
methodology details

# Example 2 for getting methodology and experiment  results
method and experiment

# Example 3 for getting the related works and reference
relevant works of the proposed method and the reference section

πŸ“– Download and Retrieve

Click the Download , followed by the Retrieve. EasyPaper will download the paper, run the OCR, run the vector-less indexing, and retrieve the relevant nodes using AI.

Before Retrieval, EasyPaper will show the page number of each PDF and the estimated cost. You may choose the page range for length ones. If a pdf has already been downloaded or retrieved, it will be skipped, respectively.

Once the relevant segments are retrieved, you can view or export them from the Column Action.

πŸ“– Deep Research

Choose the task and LLM, then click the Run the Task. Once completed, you may view or export the report. The research starts by delegating paper-wise task to the worker LLM, aggregating the results, finally adding the summary.

You may add your custom task by defining the yaml file in EasyPaper/tasks. Once any new yaml is added, EasyPaper will display it in the dropdown manual before Run the Task.

More than one Research can be conducted for an Idea, sequentially.

πŸ“– Chat

In the Paper QA panel, you may create or continue a chat, and also choose the Scope of the conversation to one or all Idea(s).

The Chat agent has the tools of list all papers and see the paper structure (i.e., the tree index). The agent will delegate paper-wise task to subagents, following by running the RAG.

The chat history is preserved on your file system.

πŸ’² Cost

In the top panel, EasyPaper shows an approximation to the cost so far. For accurate cost tracking, provide your LangSmith API key LANGCHAIN_API_KEY in your .env and set langsmith_enabled to true. You will then be able to see the detailed usage, content, and cost via LangSmith's console.

πŸ›οΈ Venues

Type Venue Year
Conference CVPR, ICCV, ECCV, ICLR, ICML, Neurips, AAAI, IJCAI, MM, KDD, WWW, ACL, EMNLP, NAACL, Interspeech, ICASSP Recent 5 years
Journal Nature, PNAS, Nat. Mach. Intell, TPAMI, IJCV, Proc. IEEE, ACM Comput. Surv., J. ACM, Info. Fusion., SIGGRAPH, TNNLS, etc... Recent 10 years

All journals and some conferences (MM, KDD, WWW) papers are not downloadable (up to 20% of the whole database) These entries have only title, venue, year, and author, without abstract and pdf url. The user need to download manually should they match the query and interest.

πŸ’€ Limitation

EasyPaper degrades to a GUI-version PaperQA for non-CS papers, as it can only rely on user provided papers or arXiv papers in that case.

🀝 Contributing

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

πŸ‘ Acknowledgments

  • The RAG chunking and indexing are inspired by PageIndex.
  • The surprisingly accurate OCR is powered by Marker.
  • The 80K papers are crawled using paperCrawler.
  • The tool is made using Claude AI.

(back to top)

About

Query, retrieve, research, and chat with 80K top CS papers and more.

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors