Popular repositories Loading

1. mmore mmore Public

   Massive Multimodal Open RAG & Extraction A scalable multimodal pipeline for processing, indexing, and querying multimodal documents Ever needed to take 8000 PDFs, 2000 videos, and 500 spreadsheets …

   Python 36 10

2. MoE MoE Public

   some mixture of experts architecture implementations

   Python 13 2

3. nanotron nanotron Public

   Forked from huggingface/nanotron

   Minimalistic large language model 3D-parallelism training

   Python 6 9

4. Megatron-LM Megatron-LM Public

   Forked from NVIDIA/Megatron-LM

   Ongoing research training transformer models at scale

   Python 4 6

5. data-PDF-pipeline data-PDF-pipeline Public

   PDF pipeline for creating training corpora (mainly for llm, multimodal and alignment horizontals)

   Python 3

6. video2dataset video2dataset Public

   Forked from iejMac/video2dataset

   Easily create large video dataset from video urls

   Python 1