### Optimized Parallelism Strategies: DualPipe and EPLB

Source: https://github.com/deepseek-ai/open-infra-index/blob/main/README.md

This section introduces DualPipe, a bidirectional pipeline parallelism algorithm for computation-communication overlap in V3/R1 training, and EPLB, an expert-parallel load balancer for V3/R1. It also references tools for analyzing computation-communication overlap.

```APIDOC
Optimized Parallelism Strategies:
- DualPipe: Bidirectional pipeline parallelism algorithm for computation-communication overlap in V3/R1 training.
- EPLB: Expert-parallel load balancer for V3/R1.
- Profile Data: Tools for analyzing computation-communication overlap in V3/R1.
```

--------------------------------

### DeepSeek-V3/R1 Inference System Overview and Performance

Source: https://github.com/deepseek-ai/open-infra-index/blob/main/README.md

This overview details the DeepSeek-V3/R1 inference system, highlighting optimizations for throughput and latency through cross-node EP-powered batch scaling, computation-communication overlap, and load balancing. It also presents impressive production performance metrics, including high token throughput and significant cost profit margin.

```APIDOC
DeepSeek-V3/R1 Inference System Optimizations:
- Cross-node EP-powered batch scaling
- Computation-communication overlap
- Load balancing

Production Data (V3/R1 online services):
- 73.7k/14.8k input/output tokens per second per H800 node
- Cost profit margin 545%
```

--------------------------------

### DeepEP: Open-Source EP Communication Library for MoE Models

Source: https://github.com/deepseek-ai/open-infra-index/blob/main/README.md

DeepEP is the first open-source EP communication library for MoE model training and inference. It provides efficient all-to-all communication, supporting both intranode and internode operations with NVLink and RDMA, and offers high-throughput/low-latency kernels with flexible GPU resource control.

```APIDOC
DeepEP Features:
- Efficient and optimized all-to-all communication
- Both intranode and internode support with NVLink and RDMA
- High-throughput kernels for training and inference prefilling
- Low-latency kernels for inference decoding
- Native FP8 dispatch support
- Flexible GPU resource control for computation-communication overlapping
```

--------------------------------

### DeepGEMM: FP8 GEMM Library for Dense and MoE GEMMs

Source: https://github.com/deepseek-ai/open-infra-index/blob/main/README.md

DeepGEMM is an FP8 GEMM library that supports both dense and MoE GEMMs, powering DeepSeek's V3/R1 training and inference. It's a lightweight, fully Just-In-Time compiled library with core logic around 300 lines, yet it outperforms expert-tuned kernels across most matrix sizes.

```APIDOC
DeepGEMM Features:
- Up to 1350+ FP8 TFLOPS on Hopper GPUs
- No heavy dependency, clean implementation
- Fully Just-In-Time compiled
- Core logic at ~300 lines
- Outperforms expert-tuned kernels across most matrix sizes
- Supports dense layout and two MoE layouts
```

--------------------------------

### FlashMLA: Efficient MLA Decoding Kernel for Hopper GPUs

Source: https://github.com/deepseek-ai/open-infra-index/blob/main/README.md

FlashMLA is an optimized MLA decoding kernel designed for Hopper GPUs, specifically for variable-length sequences. It supports BF16 and paged KV cache, offering high memory-bound and compute-bound performance, battle-tested in production.

```APIDOC
FlashMLA Features:
- Optimized for variable-length sequences
- BF16 support
- Paged KV cache (block size 64)
- Performance: 3000 GB/s memory-bound | BF16 580 TFLOPS compute-bound on H800
```

--------------------------------

### 3FS & Smallpond: Parallel File System and Data Processing Framework

Source: https://github.com/deepseek-ai/open-infra-index/blob/main/README.md

3FS (Fire-Flyer File System) is a parallel file system leveraging the full bandwidth of modern SSDs and RDMA networks for high throughput, achieving massive aggregate read and GraySort benchmark throughputs. Smallpond is a data processing framework built on 3FS, supporting various data operations for V3/R1.

```APIDOC
3FS (Fire-Flyer File System) Features:
- Parallel file system utilizing full bandwidth of modern SSDs and RDMA networks
- 6.6 TiB/s aggregate read throughput in a 180-node cluster
- 3.66 TiB/min throughput on GraySort benchmark in a 25-node cluster
- 40+ GiB/s peak throughput per client node for KVCache lookup
- Disaggregated architecture with strong consistency semantics
- Supports training data preprocessing, dataset loading, checkpoint saving/reloading, embedding vector search & KVCache lookups for inference in V3/R1

Smallpond Features:
- Data processing framework built on 3FS
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.