Use pymupdf4llm in pdf fast process mode

[Exino329]

pymupdf has an extension named pymupdf4llm that can convert documents into structured Markdown, JSON, and plain text in a suitable way for RAG pipelines : https://github.com/pymupdf/pymupdf4llm.

There is now a new pdf processor src/mmore/process/processors/pdf_pymupdf4llm_processor.py that extract pdf in a markdown structure in fast mode.

It can be configured by setting :

dispatcher_config:
  use_fast_processors: true
  ...
  ...
  ...
  processor_selection:
    ".pdf": PDFPyMuPDF4LLMProcessor

in a process config file.

Note that pymupdf4llm allows to save extracted images on disk :

md = pymupdf4llm.to_markdown(
    "document.pdf",
    write_images=True,        # save extracted images to disk
    image_path="./images",    # directory for saved images 
    image_format="png",       # output format
    dpi=150,                  # image resolution
)

However, after testing with the sample pdf data, it seems better to use pymupdf for the image extraction process.