### Install Protocol Buffers compiler Source: https://github.com/adsabs/adspipelinemsg/blob/master/README.md Commands to install the Protocol Buffers compiler from source on Linux. Requires autoconf, automake, libtool, curl, make, g++, and unzip. ```bash sudo apt-get install autoconf automake libtool curl make g++ unzip wget https://github.com/google/protobuf/releases/download/v3.17.3/protobuf-python-3.17.3.tar.gz tar -zxvf protobuf-python-3.17.3.tar.gz cd protobuf-3.17.3/ ./configure make sudo make install sudo ldconfig # refresh shared library cache. ``` -------------------------------- ### Build docker container Source: https://github.com/adsabs/adspipelinemsg/blob/master/README.md Build and run the docker container for the project. docker-compose will automatically build the image if it does not exist. ```bash docker-compose up -d --force-recreate ``` -------------------------------- ### Release to test pypi Source: https://github.com/adsabs/adspipelinemsg/blob/master/README.md Commands to build and upload the package to the test PyPI repository. Requires Python 3 and creates a virtual environment. ```bash python3 -m venv ./venv source venv/bin/activate pip install --upgrade setuptools wheel rm -rf dist/ python3 setup.py sdist python3 setup.py bdist_wheel --universal pip install --upgrade twine twine upload --repository-url https://test.pypi.org/legacy/ dist/* ``` -------------------------------- ### Compile protocol buffers from specs Source: https://github.com/adsabs/adspipelinemsg/blob/master/README.md Compile .proto files from the specs directory into Python modules, then fix imports to be relative. The sed command is required because protoc does not express imports in a relative form. ```bash protoc --python_out=../adsmsg/protobuf filename.proto # protoc does not express imports in a relative form, which messes up with our package structure # thus we force relative imports by changing the generated files with the next sed command: # https://github.com/protocolbuffers/protobuf/issues/1491#issuecomment-369324250 sed -i 's/^import \([^ ]*\)_pb2 as \([^ ]*\)$/from . import \1_pb2 as \2/' ../adsmsg/protobuf/*_pb2.py ``` -------------------------------- ### Manually build docker image Source: https://github.com/adsabs/adspipelinemsg/blob/master/README.md Manually build the docker image using docker-compose. The --no-cache flag can be added to stop docker from using the cache. ```bash docker-compose build ``` -------------------------------- ### Upload to official pypi Source: https://github.com/adsabs/adspipelinemsg/blob/master/README.md Upload the package to the official PyPI repository after verifying the test release. ```bash twine upload dist/* ``` -------------------------------- ### Run rebuild script Source: https://github.com/adsabs/adspipelinemsg/blob/master/README.md Run the rebuild.sh script to rebuild the docker container. ```bash ./rebuild.sh ``` -------------------------------- ### Run tests Source: https://github.com/adsabs/adspipelinemsg/blob/master/README.md Run the test suite manually using py.test. ```bash py.test ``` -------------------------------- ### MacOS sed replacement Source: https://github.com/adsabs/adspipelinemsg/blob/master/README.md Alternative sed command for MacOS users due to differences in BSD sed vs GNU sed. Use this instead of the Linux version. ```bash sed -i '' -e 's/^import \([^ ]*\)_pb2 as \([^ ]*\)$/from . import \1_pb2 as \2/' ../adsmsg/protobuf/*_pb2.py ``` -------------------------------- ### Build docker image with no cache Source: https://github.com/adsabs/adspipelinemsg/blob/master/README.md Build the docker image with the --no-cache flag to avoid using cached layers. ```bash docker-compose up --build ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.