### Initial MindOCR Setup on Windows Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/en/tutorials/frequently_asked_questions.md This bash snippet outlines the standard procedure for cloning the MindOCR repository and attempting to install it locally using pip. It serves as the starting point before encountering dependency errors. ```bash git clone git@gitee.com:mindspore-lab/mindocr.git cd mindocr pip install -e . ``` -------------------------------- ### MindOCR Benchmark Command Example Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/en/tutorials/frequently_asked_questions.md This snippet shows an example of a benchmark command used with MindOCR on an Ascend device. The output logs from such a command might contain the 'acl open device 0 failed' error, indicating an issue with the Ascend environment setup. ```bash benchmark --modelFile=dbnet_mobilenetv3_lite.mindir --device=Ascend --inputShapes='1,3,736,1280' --loopCount=100 - -wammUpLoopCount=10 ModelPath = dbnet_mobilenetv3_lite.mindir ModelType = MindIR InDatapath = GroupInfoFile = ConfigFilepath = InDataType = bin LoopCount = 100 DeviceType = Ascend AccuracyThreshold = 0.5 CosineDistanceThreshold = -1.1 WarmUpLoopCount = 10 NumThreads = 2 InterOpParallelNum = 1 Fpl16Priority = 0 EnableparalÍel = 0 calibDataPath = EnableGLTexture = 0 cpuBindMode = HIGHER CPU CalibDataType = FLOAT Resize Dims: 1 3 736 1280 start unified benchmark run IERROR] ME (26748,7f6c73867fc0, benchmark) :2023-10-26-09:51 : 54.833.515 Imindspore/lite/src/extend rt/kernel/ascend/model/model_infer.cc:59] Init] Acl open device 0 failed. [ERROR] ME (26748,7f6c73867fc0,benchmark):2023-10-26-09:51:54.833.573 [mindspore/lite/src/extend rt/kernel/ascend/src/custom_ascend_kernel.cc:141] Init] Model i nfer init failed. [ERROR] ME (26748, 7f6c73867fc0, benchmark) :2023-10-26-09:51:54.833.604 [mindspore/lite/src/extendrt/session/single_op_session.cc:198] BuildCustomAscendKernelImpl] kernel init failed CustomAscend [ERROR] ME (26748,7f6c73867fc0, benchmark) :2023-10-26-09:51 :54.833.669 [mindspore/li te/src/extendrt/session/single_op_sess ion.cc:220] BuildCustomAscendKernel] Build ascend kernel failed for node: custom_0 [ERROR] ME (26748,7f6c73867fc0,benchmark) :2023-10-26-09:51 : 54.833.699 [mindspore/lite/src/extend rt/session/single_op_session.cc:302] CompileGraph] Failed to Build custom ascend kernel [ERROR] ME (26748,7f6c73867fc0,benchmark) :2023-10-26-09:51:54.833.727 [mindspore/lite/s rc/extendrt/cxx_api/model/model_impl.cc:413] BuildByBufferImpl] compile graph failed. [ERROR] ME (26748, 7f6c73867fc0, benchmark):2023-10-26-09:51:54.835.590 [mindspore/lite/tools/benchma rk/benchmark_unified_api.cc:1256] CompileGraph] ms_model_.Build failed while running IERROR] ME (26748,7f6c73867fc0,benchmark) :2023-10-26-09:51:54.835.627 [mindspore/lite/tools/benchma rk/benchmark_unified_api.cc:1325] RunBenchmark] Compile graph failed. [ERROR] ME(26748,7f6c73867fc0, benchmark):2023-10-26-09: 51:54.835.662 [mindspore/lite/tools/benchmark/ run_benchmark.cc :78] RunBenchmark] Run Benchmark dbnet_mobilenetv3_lite.mindi r Failed : -1 ms_model_.Build failed while running Run Benchmark dbnet mobilenetv3 lite.mindir Failed : -1 ``` -------------------------------- ### Troubleshoot seqeval pip install error Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/zh/tutorials/frequently_asked_questions.md This snippet captures the error output from a pip installation command failing due to issues with the 'seqeval' package. The failure is attributed to a missing dependency, 'setuptools_scm', during the setup process, leading to a 'metadata-generation-failed' error. ```bash Collecting seqeval>=1.2.2 (from -r requirements.txt (line 19)) Downloading http://mirrors.aliyun.com/pypi/packages/9d/2d/233c79d5b4e5ab1dbf111242299153f3caddddbb691219f363ad55ce783d/seqeval-1.2.2.tar.gz (43 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.6/43.6 kB 181.0 kB/s eta 0:00:00 Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [48 lines of output] /home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) WARNING: The repository located at mirrors.aliyun.com is not a trusted or secure host and is being ignored. If this repository is available via HTTPS we recommend you use HTTPS instead, otherwise you may silence this warning and allow it anyway with '--trusted-host mirrors.aliyun.com'. ERROR: Could not find a version that satisfies the requirement setuptools_scm (from versions: none) ERROR: No matching distribution found for setuptools_scm Traceback (most recent call last): File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/installer.py", line 101, in _fetch_build_egg_no_warn subprocess.check_call(cmd) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/home/ma-user/anaconda3/envs/MindSpore/bin/python3.9', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpusgt0k69', '--quiet', 'setuptools_scm']' returned non-zero exit status 1. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 2, in File "", line 34, in File "/tmp/pip-install-m2kqztlz/seqeval_da00f708dc0e483b92cd18083513d5e7/setup.py", line 27, in setup( File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/__init__.py", line 102, in setup _install_setup_requires(attrs) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires _fetch_build_eggs(dist) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/__init__.py", line 80, in _fetch_build_eggs dist.fetch_build_eggs(dist.setup_requires) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/dist.py", line 636, in fetch_build_eggs return _fetch_build_eggs(self, requires) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/installer.py", line 38, in _fetch_build_eggs resolved_dists = pkg_resources.working_set.resolve( File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/pkg_resources/__init__.py", line 829, in resolve dist = self._resolve_dist( File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/pkg_resources/__init__.py", line 865, in _resolve_dist dist = best[req.key] = env.best_match( File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1135, in best_match return self.obtain(req, installer) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1147, in obtain return installer(requirement) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/installer.py", line 103, in _fetch_build_egg_no_warn raise DistutilsError(str(e)) from e distutils.errors.DistutilsError: Command '['/home/ma-user/anaconda3/envs/MindSpore/bin/python3.9', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpusgt0k69', '--quiet', 'setuptools_scm']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. ``` -------------------------------- ### Install and Update Packages Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/zh/tutorials/frequently_asked_questions.md Commands to update setuptools and setuptools_scm, and install the seqeval package with a specific mirror. ```bash pip3 install --upgrade setuptools pip3 install --upgrade setuptools_scm pip3 install seqeval -i https://pypi.tuna.tsinghua.edu.cn/simple ``` -------------------------------- ### Setup Local Development Environment Source: https://github.com/mindspore-lab/mindocr/blob/main/CONTRIBUTING_CN.md Steps to create a dedicated conda environment for MindOCR development, activate it, and install the project in editable mode. This ensures dependencies are managed correctly. ```shell conda create -n mindocr python=3.8 conda activate mindocr cd mindocr pip install -e . ``` -------------------------------- ### MindOCR Installation Instruction Source: https://github.com/mindspore-lab/mindocr/blob/main/configs/rec/crnn/README_CN.md Provides guidance on how to install the MindOCR environment, directing users to the main installation instructions. ```APIDOC Installation: Refer to MindOCR's [installation instruction](https://github.com/mindspore-lab/mindocr#installation) for environment setup. ``` -------------------------------- ### Install MindOCR on Windows (Initial Attempt) Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/zh/tutorials/frequently_asked_questions.md Shows the standard commands to clone the MindOCR repository and install it using pip. This process often fails on Windows due to the 'lanms' dependency not being compatible. ```bash git clone git@gitee.com:mindspore-lab/mindocr.git cd mindocr pip install -e . ``` -------------------------------- ### ABINet Configuration Example Source: https://github.com/mindspore-lab/mindocr/blob/main/configs/rec/abinet/README_CN.md Example YAML configuration for ABINet model training and evaluation. It details parameters for distributed training, batch size, dataset paths, and checkpoint saving/loading. ```yaml system: distribute: True # 分布式训练为True,单卡训练为False amp_level: 'O3' seed: 42 val_while_train: True # 边训练边验证 drop_overflow_update: False common: ... batch_size: &batch_size 96 # 训练批大小 ... train: ckpt_save_dir: './tmp_rec' # 训练结果(包括checkpoint、每个epoch的性能和曲线图)保存目录 dataset_sink_mode: False dataset: type: LMDBDataset dataset_root: dir/to/data_lmdb_release/ # 训练数据集根目录 data_dir: train/ # 训练数据集目录,将与`dataset_root`拼接形成完整训练数据集目录 # label_files: # 训练数据集的标签文件路径,将与`dataset_root`拼接形成完整的训练数据的标签文件路径。当数据集为LMDB格式时无需配置 ... eval: ckpt_load_path: './tmp_rec/best.ckpt' # checkpoint文件路径 dataset_sink_mode: False dataset: type: LMDBDataset dataset_root: dir/to/data_lmdb_release/ # 验证或评估数据集根目录 data_dir: evaluation/ # 验证或评估数据集目录,将与`dataset_root`拼接形成完整验证或评估数据集目录 # label_file: # 验证或评估数据集的标签文件路径,将与`dataset_root`拼接形成完整的验证或评估数据的标签文件路径。当数据集为LMDB格式时无需配置 ... loader: shuffle: False batch_size: 96 # 验证或评估批大小 ... ``` -------------------------------- ### Example 8-Device Training Script (Bash) Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/en/tutorials/distribute_train.md A sample bash script for launching distributed training on 8 Ascend devices. It sets environment variables for device count, rank size, and the HCCL table file, then starts multiple training processes in the background. ```bash #!/bin/bash export DEVICE_NUM=8 export RANK_SIZE=8 export RANK_TABLE_FILE="./hccl_8p_01234567_127.0.0.1.json" for ((i = 0; i < ${RANK_SIZE}; i++)); do export DEVICE_ID=$i export RANK_ID=$i echo "Launching rank: ${RANK_ID}, device: ${DEVICE_ID}" if [ $i -eq 0 ]; then echo 'i am 0' python -u tools/train.py --config configs/rec/crnn/crnn_resnet34_zh.yaml &> ./train.log & else echo 'not 0' python -u tools/train.py --config configs/rec/crnn/crnn_resnet34_zh.yaml &> /dev/null & fi done ``` -------------------------------- ### Install Open MPI v4.0.3 Source: https://github.com/mindspore-lab/mindocr/blob/main/examples/license_plate_detection_and_recognition/README_CN.md Steps to download, configure, compile, and install Open MPI v4.0.3. This is required for distributed training and evaluation. Includes steps for configuring environment variables and testing the installation. ```shell tar -xvf openmpi-4.0.3.tar.gz cd openmpi-4.0.0/ ./configure --prefix=/安装目录/openmpi make make install vim /etc/profile ##openmpi## export PATH=$PATH:/安装目录/openmpi/bin export LD_LIBRARY_PAHT=/安装目录/openmpii/lib source /etc/profile cd /安装目录/openmpi/examples make ./hello_c ``` -------------------------------- ### Install MindOCR Source: https://github.com/mindspore-lab/mindocr/blob/main/examples/license_plate_detection_and_recognition/README_CN.md Instructions to clone the MindOCR repository, checkout a specific version (v0.3.2), and install its dependencies using pip. This sets up the MindOCR environment. ```shell git clone https://github.com/mindspore-lab/mindocr.git git checkout v0.3.2 cd mindocr pip install -r requirements.txt pip install -e . ``` -------------------------------- ### Rank Table Configuration Example Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/zh/tutorials/distribute_train.md Example JSON structure for the rank table file. It defines server configurations, including IP addresses and device mappings (device_id, device_ip, rank_id) for distributed training. ```json { "version": "1.0", "server_count": "1", "server_list": [ { "server_id": "127.0.0.1", "device": [ { "device_id": "4", "device_ip": "192.168.100.100", "rank_id": "0" }, { "device_id": "5", "device_ip": "192.168.101.100", "rank_id": "1" }, { "device_id": "6", "device_ip": "192.168.102.100", "rank_id": "2" }, { "device_id": "7", "device_ip": "192.168.103.100", "rank_id": "3" } ], "host_nic_ip": "reserve" } ], "status": "completed" } ``` -------------------------------- ### Install Visual C++ Build Tools for Windows Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/zh/tutorials/frequently_asked_questions.md Provides a link to download and install the necessary Microsoft Visual C++ Build Tools, which are required to compile C++ extensions like 'lanms-neo' on Windows. ```bash https://visualstudio.com/zh-hans/visual-cpp-build-tools/ ``` -------------------------------- ### Install MindOCR Dependencies Source: https://github.com/mindspore-lab/mindocr/blob/main/README.md Installs all required Python packages for MindOCR by reading from the requirements.txt file. This is a prerequisite for building or running the project. ```shell pip install -r requirements.txt ``` -------------------------------- ### Install from Source Source: https://github.com/mindspore-lab/mindocr/blob/main/README_CN.md Clones the MindOCR repository from GitHub and installs it in an editable mode, allowing for direct code modifications and testing. ```shell git clone https://github.com/mindspore-lab/mindocr.git cd mindocr pip install -e . ``` -------------------------------- ### Install lanms-neo on Windows (Build Error) Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/zh/tutorials/frequently_asked_questions.md Details the process of attempting to install 'lanms-neo' as a Windows-compatible alternative to 'lanms'. The build process fails with an error requiring Microsoft Visual C++ Build Tools. ```bash Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Collecting lanms-neo==1.0.2 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7b/fe/beff7e7e4455cb9f69c5734897ca8552a57f6423b062ec86b2ebc1d79c0d/lanms_neo-1.0.2.tar.gz (39 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: lanms-neo Building wheel for lanms-neo (pyproject.toml) ... error error: subprocess-exited-with-error × Building wheel for lanms-neo (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [10 lines of output] running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-cpython-37 creating build\lib.win-amd64-cpython-37\lanms copying lanms\__init__.py -> build\lib.win-amd64-cpython-37\lanms running build_ext building 'lanms._C' extension error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.com/visual-cpp-build-tools/ [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for lanms-neo Failed to build lanms-neo ERROR: Could not build wheels for lanms-neo, which is required to install pyproject.toml-based projects ``` -------------------------------- ### Install MindOCR from Source Source: https://github.com/mindspore-lab/mindocr/blob/main/README.md Clones the MindOCR repository from GitHub and installs the package in editable mode. This method is recommended for development as it allows for direct code modifications and immediate reflection. ```shell git clone https://github.com/mindspore-lab/mindocr.git cd mindocr pip install -e . ``` -------------------------------- ### FileNotFoundError during MindOCR Installation Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/en/tutorials/frequently_asked_questions.md An example of a `FileNotFoundError` that can occur during the MindOCR installation on Windows. This error typically indicates that a required system file or path cannot be found by the installation process. ```bash FileNotFoundError: [WinError 2] the system cannot find the file specified. ``` -------------------------------- ### Test Open MPI Installation Source: https://github.com/mindspore-lab/mindocr/blob/main/examples/license_plate_detection_and_recognition/README.md Compiles and executes the 'hello_c' example program from the Open MPI installation directory to verify that Open MPI is correctly installed and functional. ```shell cd /installation_directory/openmpi/examples make ./hello_c ``` -------------------------------- ### Install MindOCR via Docker Source: https://github.com/mindspore-lab/mindocr/blob/main/README.md Provides steps to pull MindOCR Docker images for specific configurations (e.g., 910, 910*) and create/run a container with necessary device mappings and volume mounts. It also includes instructions for entering the container and setting up the environment variables. ```bash docker pull swr.cn-central-221.ovaijisuan.com/mindocr/mindocr_dev_910_ms_2_2_10_cann7_0_py39:v1 docker pull swr.cn-central-221.ovaijisuan.com/mindocr/mindocr_dev_ms_2_2_10_cann7_0_py39:v1 ``` ```bash docker_name="temp_mindocr" # 910 image_name="swr.cn-central-221.ovaijisuan.com/mindocr/mindocr_dev_910_ms_2_2_10_cann7_0_py39:v1" # 910* # image_name="swr.cn-central-221.ovaijisuan.com/mindocr/mindocr_dev_ms_2_2_10_cann7_0_py39:v1" docker run --privileged --name ${docker_name} \ --tmpfs /tmp \ --tmpfs /run \ -v /sys/fs/cgroup:/sys/fs/cgroup:ro \ --device=/dev/davinci1 \ --device=/dev/davinci2 \ --device=/dev/davinci3 \ --device=/dev/davinci4 \ --device=/dev/davinci5 \ --device=/dev/davinci6 \ --device=/dev/davinci7 \ --device=/dev/davinci_manager \ --device=/dev/hisi_hdc \ --device=/dev/devmm_svm \ -v /etc/localtime:/etc/localtime \ -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \ -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \ --shm-size 800g \ --cpus 96 \ --security-opt seccomp=unconfined \ --network=bridge -itd ${image_name} bash ``` ```bash # set docker id container_id="your docker id" docker exec -it --user root $container_id bash ``` ```bash source env_setup.sh ``` -------------------------------- ### MindSpore Lite Installation and Environment Setup Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/en/inference/environment.md Provides instructions to install MindSpore Lite by unpacking the inference toolkit archive and setting environment variables for runtime and tools. It also includes installing the Python wheel package. ```shell tar -xvf mindspore-lite-2.2.14-linux-{arch}.tar.gz cd mindspore-lite-2.2.14-linux-{arch}/ export LITE_HOME=${PWD} # The actual path after extracting the tar package export LD_LIBRARY_PATH=$LITE_HOME/runtime/lib:$LITE_HOME/runtime/third_party/dnnl:$LITE_HOME/tools/converter/lib:$LD_LIBRARY_PATH export PATH=$LITE_HOME/tools/converter/converter:$LITE_HOME/tools/benchmark:$PATH ``` ```shell pip install mindspore_lite-2.2.14-{python_version}-linux_{arch}.whl ``` -------------------------------- ### Find liboptiling.so Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/en/tutorials/frequently_asked_questions.md Command to locate the liboptiling.so file within the CANN installation directory. This file is crucial for optimal tiling operations on Ascend devices. ```bash find ./ -name liboptiling.so ``` -------------------------------- ### Install Git Hook Scripts Source: https://github.com/mindspore-lab/mindocr/blob/main/CONTRIBUTING_CN.md Installs pre-commit hooks to automatically run linting and code style checks on each commit. This automates code quality enforcement. ```shell pre-commit install ``` -------------------------------- ### YOLOv8 Layout Analysis Configuration Example Source: https://github.com/mindspore-lab/mindocr/blob/main/configs/layout/yolov8/README_CN.md Example configuration file for YOLOv8 layout analysis, detailing parameters for system settings, common options, training, and evaluation. Key parameters include distributed training, batch size, and checkpoint paths. ```yaml system: distribute: &distribute True # 分布式训练为True,单卡训练为False amp_level: 'O0' amp_level_infer: "O0" seed: 42 val_while_train: False # 边训练边验证 drop_overflow_update: False common: ... batch_size: 16 # 训练批大小 annotations_path: publaynet/val.json ... train: ckpt_save_dir: './tmp_layout' # 训练结果(包括checkpoint、每个epoch的性能和曲线图)保存目录 dataset_sink_mode: False dataset: type: PublayNetDataset dataset_path: publaynet/train.txt # 训练数据集路径 ... eval: ckpt_load_path: './tmp_layout/best.ckpt' # checkpoint文件路径 dataset_sink_mode: False dataset: type: PublayNetDataset dataset_path: publaynet/val.txt # 验证数据集路径 ... loader: shuffle: False batch_size: 16 # 验证批大小 ... ``` -------------------------------- ### Start Distributed Training Script Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/zh/tutorials/distribute_train.md Bash script to initiate distributed training on multiple devices. It sets environment variables like RANK_SIZE and RANK_TABLE_FILE, then launches training processes for each rank, mapping them to specific devices. ```bash #!/bin/bash export DEVICE_NUM=8 export RANK_SIZE=4 export RANK_TABLE_FILE="./hccl_4p_4567_127.0.0.1.json" for ((i = 0; i < ${RANK_SIZE}; i++)); do export DEVICE_ID=$((i+4)) export RANK_ID=$i echo "Launching rank: ${RANK_ID}, device: ${DEVICE_ID}" if [ $i -eq 0 ]; then echo 'i am 0' python -u tools/train.py --config configs/rec/crnn/crnn_resnet34_zh.yaml &> ./train.log & else echo 'not 0' python -u tools/train.py --config configs/rec/crnn/crnn_resnet34_zh.yaml &> /dev/null & fi done ``` -------------------------------- ### Example HCCL Rank Table Configuration Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/en/tutorials/distribute_train.md An example JSON structure for the HCCL rank table file. This configuration details server information, device IDs, IPs, and rank IDs essential for distributed training setup. ```json { "version": "1.0", "server_count": "1", "server_list": [ { "server_id": "127.0.0.1", "device": [ { "device_id": "4", "device_ip": "192.168.100.100", "rank_id": "0" }, { "device_id": "5", "device_ip": "192.168.101.100", "rank_id": "1" }, { "device_id": "6", "device_ip": "192.168.102.100", "rank_id": "2" }, { "device_id": "7", "device_ip": "192.168.103.100", "rank_id": "3" } ], "host_nic_ip": "reserve" } ], "status": "completed" } ``` -------------------------------- ### Failed seqeval Installation with pip Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/en/tutorials/frequently_asked_questions.md This snippet illustrates the error message received when attempting to install dependencies from a requirements file, specifically failing on the 'seqeval' package. The root cause is identified as the inability to find the 'setuptools_scm' distribution, which is required during the package's setup process. ```bash pip install -r requirements.txt ``` ```python Collecting seqeval>=1.2.2 (from -r requirements.txt (line 19)) Downloading http://mirrors.aliyun.com/pypi/packages/9d/2d/233c79d5b4e5ab1dbf111242299153f3caddddbb691219f363ad55ce783d/seqeval-1.2.2.tar.gz (43 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.6/43.6 kB 181.0 kB/s eta 0:00:00 Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [48 lines of output] /home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) WARNING: The repository located at mirrors.aliyun.com is not a trusted or secure host and is being ignored. If this repository is available via HTTPS we recommend you use HTTPS instead, otherwise you may silence this warning and allow it anyway with '--trusted-host mirrors.aliyun.com'. ERROR: Could not find a version that satisfies the requirement setuptools_scm (from versions: none) ERROR: No matching distribution found for setuptools_scm Traceback (most recent call last): File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/installer.py", line 101, in _fetch_build_egg_no_warn subprocess.check_call(cmd) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/home/ma-user/anaconda3/envs/MindSpore/bin/python3.9', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpusgt0k69', '--quiet', 'setuptools_scm']' returned non-zero exit status 1. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 2, in File "", line 34, in File "/tmp/pip-install-m2kqztlz/seqeval_da00f708dc0e483b92cd18083513d5e7/setup.py", line 27, in setup( File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/__init__.py", line 102, in setup _install_setup_requires(attrs) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires _fetch_build_eggs(dist) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/__init__.py", line 80, in _fetch_build_eggs dist.fetch_build_eggs(dist.setup_requires) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/dist.py", line 636, in fetch_build_eggs return _fetch_build_eggs(self, requires) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/installer.py", line 38, in _fetch_build_eggs resolved_dists = pkg_resources.working_set.resolve( File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/pkg_resources/__init__.py", line 829, in resolve dist = self._resolve_dist( File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/pkg_resources/__init__.py", line 865, in _resolve_dist dist = best[req.key] = env.best_match( File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1135, in best_match return self.obtain(req, installer) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1147, in obtain return installer(requirement) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/setuptools/installer.py", line 103, in _fetch_build_egg_no_warn raise DistutilsError(str(e)) from e distutils.errors.DistutilsError: Command '['/home/ma-user/anaconda3/envs/MindSpore/bin/python3.9', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpusgt0k69', '--quiet', 'setuptools_scm']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. ``` -------------------------------- ### Install via Docker Source: https://github.com/mindspore-lab/mindocr/blob/main/README_CN.md Provides instructions for pulling a pre-configured Docker image, running a container with necessary device mappings and volume mounts, and entering the container to set up the environment. ```shell # Pull Docker image (example for 910) docker pull swr.cn-central-221.ovaijisuan.com/mindocr/mindocr_dev_910_ms_2_2_10_cann7_0_py39:v1 # Run container (example for 910) docker_name="temp_mindocr" image_name="swr.cn-central-221.ovaijisuan.com/mindocr/mindocr_dev_910_ms_2_2_10_cann7_0_py39:v1" docker run --privileged --name ${docker_name} \ --tmpfs /tmp \ --tmpfs /run \ -v /sys/fs/cgroup:/sys/fs/cgroup:ro \ --device=/dev/davinci1 \ --device=/dev/davinci2 \ --device=/dev/davinci3 \ --device=/dev/davinci4 \ --device=/dev/davinci5 \ --device=/dev/davinci6 \ --device=/dev/davinci7 \ --device=/dev/davinci_manager \ --device=/dev/hisi_hdc \ --device=/dev/devmm_svm \ -v /etc/localtime:/etc/localtime \ -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \ -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \ --shm-size 800g \ --cpus 96 \ --security-opt seccomp=unconfined \ --network=bridge -itd ${image_name} bash # Enter container # Replace 'your docker id' with the actual container ID docker exec -it --user root your_docker_id bash # Set environment variables inside container source env_setup.sh ``` -------------------------------- ### Distributed Training Command for EAST Source: https://github.com/mindspore-lab/mindocr/blob/main/configs/det/east/README.md Command to start distributed training for the EAST model, specifying worker numbers for multi-NPU or multi-machine setups. ```shell # worker_num is the total number of Worker processes participating in the distributed task.\ # local_worker_num is the number of Worker processes pulled up on the current node.\ # The number of processes is equal to the number of NPUs used for training. In the case of single-machine multi-card worker_num and local_worker_num must be the same.\ msrun --worker_num=8 --local_worker_num=8 python tools/train.py --config configs/det/east/east_r50_icdar15.yaml ``` -------------------------------- ### YAML Model Configuration (with Neck) Source: https://github.com/mindspore-lab/mindocr/blob/main/mindocr/models/README_CN.md Example YAML configuration for defining a model architecture that includes a backbone, neck, and head. This format allows for quick modification of the base architecture. ```yaml model: # R type: det backbone: # R name: det_resnet50 # R, backbone specification function name pretrained: False neck: # R name: FPN # R, neck class name out_channels: 256 # D, neck class __init__ arg #use_asf: True head: # R, head class name name: ConvHead # D, head class __init__ arg out_channels: 2 k: 50 ``` -------------------------------- ### Precision Mode Configuration (config.txt) Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/en/inference/convert_tutorial.md Configuration file example for setting the model's precision mode. Options include enforce_fp16, enforce_fp32, preferred_fp32, and enforce_origin. Defaults to enforce_fp16 if not specified. ```text [ascend_context] input_format=NCHW input_shape=x:[1,3,736,1280] precision_mode=enforce_fp32 ``` -------------------------------- ### YAML Model Configuration (without Neck) Source: https://github.com/mindspore-lab/mindocr/blob/main/mindocr/models/README_CN.md Example YAML configuration for defining a model architecture that omits the neck component, directly connecting the backbone to the head. This is common for certain model types. ```yaml model: # R type: rec backbone: # R name: resnet50 # R pretrained: False head: # R name: ConvHead # R out_channels: 30 # D ``` -------------------------------- ### Start Distributed Training Source: https://github.com/mindspore-lab/mindocr/blob/main/docs/en/tutorials/training_recognition_custom_dataset.md Initiate distributed training across multiple devices by setting `system.distribute` to True in the configuration and using `mpirun` to launch the training script. ```shell # To perform distributed training on 4 Ascend devices mpirun -n 4 python tools/train.py --config configs/rec/crnn/crnn_resnet34_ch.yaml ```