Welcome to libai’s documentation!

• Tutorials
  • Get Started
    • Installation
    • Quick Run
    • LiBai Model Zoo
    • Benchmarks
  • Basics
    • Config System
    • Features in LiBai
    • Training
    • Evaluation
    • Training & Evaluation in Command Line
    • Load and Save a Checkpoint in LiBai
    • Write Models
    • Write Dataloaders
    • Build New Project on LiBai
    • Distributed Configuration
    • Auto Parallel Training
  • Advanced Tutorials
    • How to Customize Dataloader
    • How to Customize Parallelism
• Notes
  • How to use Huggingface’s pretrained weights in LiBai
    • LiBai Transformer vs Huggingface Transformer
    • QKV slicing logic
    • How to correctly load QKV weights
  • Detailed instruction on building Vision Transformer models in LiBai
    • Model Architecture
    • A simple Torch implementation of Vision Transformer
    • Implement 3D parallel Vision Transformer in LiBai
    • Details about LiBai’s implementation of the Vision Transformer model
  • Frequently Asked Questions
    • Training
    • Model
• API Documentation
  • libai.config
  • libai.data
    • libai.data.data_utils module
    • libai.data.datasets module
    • libai.data.samplers module
    • libai.data.build module
  • libai.engine
    • libai.engine.default module
    • libai.engine.hooks module
    • libai.engine.trainer module
  • libai.evaluation
  • libai.layers
  • libai.models
    • VisionTransformer
    • SwinTransformer
    • ResMLP
    • BERT
    • RoBERTa
    • T5
    • GPT-2
  • libai.optim
  • libai.scheduler
  • libai.tokenizer
  • libai.utils
    • libai.utils.distributed module
    • libai.utils.events module
    • libai.utils.logger module
    • libai.utils.checkpoint module
• Changelog
  • v0.2.0 (07/07/2022)
  • v0.1.0 (22/03/2022)