Welcome to libai’s documentation!

• Tutorials
  • Get Started
    • Installation
    • Quick Run
    • LiBai Model Zoo
    • Benchmarks
    • Run distributed inference
  • Basics
    • Config System
    • Features in LiBai
    • Training
    • Evaluation
    • Training & Evaluation in Command Line
    • Load and Save a Checkpoint in LiBai
    • Write Models
    • Write Dataloaders
    • Build New Project on LiBai
    • Distributed Configuration
    • Auto Parallel Training
    • Preprocessing Dataset
  • Advanced Tutorials
    • How to Customize Dataloader
    • How to Customize Parallelism
• Notes
  • How to use Huggingface’s pretrained weights in LiBai
    • LiBai Transformer vs Huggingface Transformer
    • QKV slicing logic
    • How to correctly load QKV weights
  • Detailed instruction on building Vision Transformer models in LiBai
    • Model Architecture
    • A simple Torch implementation of Vision Transformer
    • Implement 3D parallel Vision Transformer in LiBai
    • Details about LiBai’s implementation of the Vision Transformer model
  • How to load pretrained model in LiBai
    • Steps
    • Start Loading
  • Use Custom ModelLoader
    • Model Loader for HuggerFace
    • Model Loader for LiBai
  • Detailed instruction on using model parallel in LiBai
    • Transfer pytroch model to oneflow.
    • 2、Change the deep learning framework to oneflow.
    • 3、Using Libai’s api.
    • 4、Model parallel in Libai.
  • Detailed instruction for using distributed inference in LiBai
    • Check model.py
    • Build config.py
    • Refine pipeline_inference.py
    • Distributed run pipeline_inference.py
  • Frequently Asked Questions
    • Training
    • Model
• API Documentation
  • libai.config
  • libai.data
    • libai.data.data_utils module
    • libai.data.datasets module
    • libai.data.samplers module
    • libai.data.build module
  • libai.engine
    • libai.engine.default module
    • libai.engine.hooks module
    • libai.engine.trainer module
  • libai.evaluation
  • libai.layers
  • libai.models
    • VisionTransformer
    • SwinTransformer
    • SwinTransformerV2
    • ResMLP
    • BERT
    • RoBERTa
    • T5
    • GPT-2
  • libai.optim
  • libai.scheduler
  • libai.tokenizer
  • libai.utils
    • libai.utils.distributed module
    • libai.utils.events module
    • libai.utils.logger module
    • libai.utils.checkpoint module
• Changelog
  • v0.2.0 (07/07/2022)
  • v0.1.0 (22/03/2022)