<Melvin-King>
  • Bio
  • Papers
  • Talks
  • Blog
  • Experience
  • Projects
  • Personal
  • Projects
    • Data vs Pipeline Parallelism in LLM Training
    • DesignBench: A Multi-framework Multi-task Benchmark for Front-end Code Generation
    • Email Application with PGP Encryption
    • Search Engine Project with Hybrid Information Retrieval Models
    • Robotic-guided Self-help Telerehabilitation in Metaverse for Stroke Patients
    • Software Engineering Course Project - Jungle Game
  • Publications
    • DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generation
    • How Large Language Model Empowers the Analysis of Online Public Engagement for Mega Infrastructure Projects: cases in Hong Kong
    • Large Language Model-based data-driven framework for digital transformation in construction industry
  • Blog
    • 🎓 Excited to attend the 30th congregation ceremony as a graduate!
    • 🎒 Touring around City of Pittsburgh
    • 🌆 Thrilled to present our research on ASCE i3CE2024, held by Carneige Mellon University!
    • 🏆 Glad to receive the certificate of achievement from Amazon Web Services (AWS)
    • 🌌 Honored to attend a lecture delivered the Nobel Laureate of Physics in 2020
  • Personal
    • Book List
    • Social Responsibilities & Commitments
    • Illustrations
    • Outdoors
  • Recent & Upcoming Talks
    • Conference Paper Presentation
  • Projects
  • Experience

Data vs Pipeline Parallelism in LLM Training

May 31, 2025 · 1 min read
Go to Project Site

Implements distributed training methods, including data parallelism and pipeline parallelism across multiple GPUs.

Last updated on May 31, 2025
Distributed Training LLM
Wang Ming (Melvin)
Authors
Wang Ming (Melvin)
MSc in Information Engineering
DesignBench: A Multi-framework Multi-task Benchmark for Front-end Code Generation May 31, 2025 →

© 2025 Wang Ming. Work licensed under CC BY NC ND 4.0

Published with Hugo Blox Builder — the free, open source website builder that empowers creators.