### Install pycld3 on CentOS/RHEL from Source

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Installs build tools and Protobuf from source on CentOS/RHEL, then installs pycld3 via pip. This guide is for CentOS 8 and may require adjustments for earlier versions.

```bash
sudo su -
set -ex
pushd /opt
PROTOBUF_VERSION='3.11.4'
yum update -y
yum install -y autoconf automake gcc-c++ glibc-headers gzip libtool make python3-devel zlib-devel
curl -Lo /opt/protobuf.tar.gz \
    "https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protobuf-cpp-${PROTOBUF_VERSION}.tar.gz"
tar -xzvf protobuf.tar.gz
rm -f protobuf.tar.gz
pushd "protobuf-${PROTOBUF_VERSION}"
./configure --with-zlib --disable-debug && make && make install && ldconfig --verbose
popd && rm -rf "protobuf-${PROTOBUF_VERSION}" && popd && set +ex

python -m pip install -U pycld3
```

--------------------------------

### Install pycld3 on Alpine Linux

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Installs build dependencies for pycld3 on Alpine Linux, which does not support PyPI wheels, and then installs the package via pip.

```bash
apk --update add g++ protobuf protobuf-dev
python -m pip install -U pycld3
```

--------------------------------

### Install pycld3 on MacOS with Homebrew

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Updates Homebrew and installs or upgrades the protobuf package before installing pycld3 via pip.

```bash
brew update
brew upgrade protobuf || brew install -v protobuf
python -m pip install -U pycld3
```

--------------------------------

### Install pycld3 on Debian/Ubuntu

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Installs necessary build tools and libraries for pycld3 on Debian/Ubuntu systems before installing the package via pip.

```bash
sudo apt-get update -y
sudo apt-get install -y --no-install-recommends \
    g++ \
    protobuf-compiler \
    libprotobuf-dev
python -m pip install -U pycld3
```

--------------------------------

### Install pycld3 using pip

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Install pycld3 using pip. This command is suitable for supported Python versions and platforms where wheels are available. No external dependencies like protoc or libprotobuf are typically needed when using wheels.

```bash
python -m pip install -U pycld3
```

--------------------------------

### Test pycld3 Installation with Chinese Text

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Verify your pycld3 installation by running this command, which detects the language of a Chinese string. Ensure that the 'libprotobuf' shared object is correctly found by Python.

```bash
export LD_LIBRARY_PATH="$(dirname $(find /usr -name 'libprotoc.so' '(' -type l -o -type f ') )):$LD_LIBRARY_PATH"
python -c 'import cld3; print(cld3.get_language("影響包含對氣候的變化以及自然資源的枯竭程度"))'
```

--------------------------------

### Get Language Prediction with pycld3

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Use this snippet to detect the language of a given text. Ensure pycld3 is installed.

```python
>>> import cld3
>>> cld3.get_language("影響包含對氣候的變化以及自然資源的枯竭程度")
LanguagePrediction(language='zh', probability=0.999969482421875, is_reliable=True, proportion=1.0)
```

--------------------------------

### Get frequent language predictions

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Uses the `get_frequent_languages` function to identify multiple languages within a text and their proportions. Allows specifying the number of languages to return.

```python
import cld3

for lang in cld3.get_frequent_languages(
    "This piece of text is in English. Този текст е на Български.",
    num_langs=3
):
    print(lang)
```

--------------------------------

### Get single language prediction

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Uses the `get_language` function to predict the language of a given text string. Returns a LanguagePrediction object.

```python
import cld3

cld3.get_language("影響包含對氣候的變化以及自然資源的枯竭程度")
cld3.get_language("This is a test")
```

--------------------------------

### Project and Minimum CMake Version

Source: https://github.com/bsolomon1124/pycld3/blob/master/CMakeLists.txt

Initializes the CMake project and sets the minimum required version for CMake. This ensures compatibility with newer CMake features.

```cmake
project(cld3)

# Old versions of cmake dont search/find protobuf lite
cmake_minimum_required(VERSION 3.9)
```

--------------------------------

### Build and Run CLD3 Model

Source: https://github.com/bsolomon1124/pycld3/blob/master/CLD3_README.md

Commands to build and run the CLD3 language detection model after checking out the Chromium repository and copying the code.

```shell
gn gen out/Default
ninja -C out/Default third_party/cld_3/src/src:language_identifier_main
out/Default/language_identifier_main
```

--------------------------------

### Preprocess input text by removing URLs

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Demonstrates preprocessing input text to remove URLs before passing it to cld3, as cld3 performs minimal cleaning.

```python
import re
import cld3

```

--------------------------------

### Find Protobuf Package

Source: https://github.com/bsolomon1124/pycld3/blob/master/CMakeLists.txt

Locates and loads the Protobuf library, which is a required dependency for CLD3. It also prints status messages about the found Protobuf version and libraries.

```cmake
find_package(Protobuf REQUIRED)
message(STATUS "Protobuf_FOUND= ${Protobuf_FOUND}")
message(STATUS "Protobuf_VERSION= ${Protobuf_VERSION}")
message(WARNING "Protobuf 2.5 and CLD3 seems happy together. This script does NOT check if your verison of protobuf is compatible.")
message(STATUS "Protobuf_LIBRARIES= ${Protobuf_LIBRARIES}")
message(STATUS "Protobuf_LITE_LIBRARIES= ${Protobuf_LITE_LIBRARIES}") # Usually /usr/lib64/libprotobuf-lite.so
```

--------------------------------

### Compiler Definitions and Include Directories

Source: https://github.com/bsolomon1124/pycld3/blob/master/CMakeLists.txt

Sets compiler flags for Position Independent Code (PIC), disables the C++11 ABI for compatibility, and enables C++11 standard. It also adds the build directory to include paths for generated headers.

```cmake
add_definitions(-fPIC) # Position Independant Code
add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0)
add_definitions(-std=c++11) # Needed for std::to_string(), ...

include_directories(${CMAKE_CURRENT_BINARY_DIR}) # needed to include generated pb headers
```

--------------------------------

### Custom Protobuf Code Generation

Source: https://github.com/bsolomon1124/pycld3/blob/master/CMakeLists.txt

Includes a custom CMake script for generating C++ source and header files from Protocol Buffer definitions. This is necessary because CLD3 expects generated files in a specific directory.

```cmake
# By default, protobuf_generate_cpp generates pb.* files directy in the cmake build dir. 
# But CLD3 sources have been coded using hard coded pathes to cld_3/protos/*.pb.h. 
# So *.pb.h must be output to cld_3/protos.
# For that, let's use a custom my_protobuf_generate_cpp:
include(${CMAKE_CURRENT_SOURCE_DIR}/misc/myprotobuf.cmake)
my_protobuf_generate_cpp(cld_3/protos PROTO_SRCS PROTO_HDRS src/feature_extractor.proto src/sentence.proto src/task_spec.proto)
message(STATUS "PROTO_HDRS= ${PROTO_HDRS}")
```

--------------------------------

### Link Unit Test Executables

Source: https://github.com/bsolomon1124/pycld3/blob/master/CMakeLists.txt

Defines and links the necessary libraries for the unit test executables. This ensures that the tests can be compiled and run, linking against the CLD3 library and Protobuf Lite.

```cmake
# unit tests exec:
add_executable(language_identifier_main src/language_identifier_main.cc)
target_link_libraries(language_identifier_main cld3 ${Protobuf_LITE_LIBRARIES})

add_executable(getonescriptspan_test src/script_span/getonescriptspan_test.cc)
target_link_libraries(getonescriptspan_test cld3 ${Protobuf_LITE_LIBRARIES})

add_executable(language_identifier_features_test src/language_identifier_features_test.cc)
target_link_libraries(language_identifier_features_test cld3 ${Protobuf_LITE_LIBRARIES})
```

--------------------------------

### Add CLD3 Static Library

Source: https://github.com/bsolomon1124/pycld3/blob/master/CMakeLists.txt

Defines the static library for CLD3, including all its source files, generated protocol buffer files, and script span related files. This library is the core component of the project.

```cmake
add_library(${PROJECT_NAME} 
	${PROTO_SRCS} ${PROTO_HDRS}
	src/base.cc
	src/embedding_feature_extractor.cc
	src/embedding_network.cc
	src/feature_extractor.cc
	src/feature_extractor.h
	src/feature_types.cc
	src/fml_parser.cc
	src/language_identifier_features.cc
	src/lang_id_nn_params.cc 
	src/nnet_language_identifier.cc
	src/registry.cc
	src/relevant_script_feature.cc
	src/sentence_features.cc
	src/task_context.cc
	src/task_context_params.cc
	src/unicodetext.cc
	src/utils.cc
	src/workspace.cc
	
	src/script_span/generated_entities.cc
	src/script_span/getonescriptspan.cc
	src/script_span/getonescriptspan.h
	src/script_span/getonescriptspan_test.cc
	src/script_span/utf8statetable.cc
	src/script_span/offsetmap.cc
	src/script_span/text_processing.cc
	src/script_span/text_processing.h
	src/script_span/fixunicodevalue.cc
	)
```

--------------------------------

### get_language

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Detects the language of a given text string.

```APIDOC
## get_language

### Description
Detects the language of a given text string and returns a LanguagePrediction object.

### Method
```python
cld3.get_language(text: str) -> LanguagePrediction
```

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
import cld3

language_prediction = cld3.get_language("This is a test")
print(language_prediction)
```

### Response
#### Success Response (LanguagePrediction)
- **language** (str) - The predicted language code (e.g., 'en', 'zh').
- **probability** (float) - The probability of the predicted language.
- **is_reliable** (bool) - Indicates if the prediction is considered reliable.
- **proportion** (float) - The proportion of the text that is in the predicted language.

#### Response Example
```json
{
  "language": "en",
  "probability": 0.9999980926513672,
  "is_reliable": true,
  "proportion": 1.0
}
```
```

--------------------------------

### Detect Language Excluding URLs with pycld3

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Use this snippet to detect the language of a string after removing URL components. This requires importing the 're' module for regular expression operations.

```python
import re
import cld3

s = "Je veux que: https://site.english.com/this/is/a/url/path/component#fragment"
cld3.get_language(s)
```

```python
url_re = r"\b(?:https?://|www\.)[a-z0-9-]+(\.[a-z0-9-]+)+(?:[/?].*)?"
new_s = re.sub(url_re, "", s)
cld3.get_language(new_s)
```

--------------------------------

### get_frequent_languages

Source: https://github.com/bsolomon1124/pycld3/blob/master/README.md

Detects the most frequent languages in a given text string.

```APIDOC
## get_frequent_languages

### Description
Detects the most frequent languages in a given text string and returns a list of LanguagePrediction objects, sorted by frequency.

### Method
```python
cld3.get_frequent_languages(text: str, num_langs: int = 3) -> List[LanguagePrediction]
```

### Parameters
#### Path Parameters
None

#### Query Parameters
- **num_langs** (int) - Optional. The maximum number of languages to return. Defaults to 3.

#### Request Body
None

### Request Example
```python
import cld3

for lang in cld3.get_frequent_languages("This piece of text is in English. Този текст е на Български.", num_langs=3):
    print(lang)
```

### Response
#### Success Response (List[LanguagePrediction])
- A list of LanguagePrediction objects, each containing:
  - **language** (str) - The predicted language code.
  - **probability** (float) - The probability of the predicted language.
  - **is_reliable** (bool) - Indicates if the prediction is considered reliable.
  - **proportion** (float) - The proportion of the text that is in the predicted language.

#### Response Example
```json
[
  {
    "language": "bg",
    "probability": 0.9173890948295593,
    "is_reliable": true,
    "proportion": 0.5853658318519592
  },
  {
    "language": "en",
    "probability": 0.9999790191650391,
    "is_reliable": true,
    "proportion": 0.4146341383457184
  }
]
```
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.