### Setting up a Pandas DataFrame for pyJanitor examples Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/then.md This snippet imports necessary libraries (pandas, janitor) and initializes a sample pandas DataFrame (`data_dict`) which will be used as input for subsequent `df.then()` examples. It demonstrates the basic setup required before applying pyJanitor's fluent API. ```python import pandas as pd import janitor data_dict = { "a": [1.23, 2.45, 3.23] * 3, "Bell__Chart": [1, 2, 3] * 3, "decorated-elephant": [1, 2, 3] * 3, "animals": ["rabbit", "leopard", "lion"] * 3, "cities": ["Cambridge", "Shanghai", "Basel"] * 3, } ``` -------------------------------- ### Setting Up Pyjanitor Conda Development Environment - Bash Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/devguide.md These commands guide you through setting up the pyjanitor development environment using Conda. It involves navigating into the cloned repository, creating the environment from a YAML file, installing pyjanitor in development mode, and registering it as a Jupyter Python kernel for notebook development. ```bash cd pyjanitor/ # Activate the pyjanitor conda environment source activate pyjanitor-dev # Create your conda environment conda env create -f environment-dev.yml # Install PyJanitor in development mode python setup.py develop # Register current virtual environment as a Jupyter Python kernel python -m ipykernel install --user --name pyjanitor-dev --display-name "PyJanitor development" ``` -------------------------------- ### Running Fast Tests for pyjanitor Environment Setup (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/CONTRIBUTING.md This command executes a subset of pyjanitor's tests, specifically excluding those marked with 'turtle', to quickly verify that your development environment is correctly set up. Passing these tests indicates readiness for contribution. ```bash python -m pytest -m "not turtle" ``` -------------------------------- ### Installing Pyjanitor Pre-commit Hooks - Bash Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/devguide.md These commands update your Conda environment to include pre-commit and then install the pre-commit hooks. These hooks automate code formatting checks, ensuring consistency before changes are committed to Git. ```bash # Update your environment to install pre-commit conda env update -f environment-dev.yml # Install pre-commit hooks pre-commit install ``` -------------------------------- ### Installing Pre-commit Hooks for Pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/CONTRIBUTING.md These commands ensure that pre-commit hooks are installed and updated in your development environment. Pre-commit hooks automate code formatting checks before each Git commit, helping maintain code quality and consistency across the project. ```Bash conda env update -f environment-dev.yml # Install pre-commit hooks pre-commit install ``` -------------------------------- ### Setting Up Conda Development Environment for Pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/CONTRIBUTING.md These commands guide you through setting up the pyjanitor development environment using Conda. It involves navigating into the cloned repository, activating/creating the dedicated conda environment, installing pyjanitor in development mode, and registering the environment as a Jupyter Python kernel for notebook development. ```Bash cd pyjanitor/ # Activate the pyjanitor conda environment source activate pyjanitor-dev # Create your conda environment conda env create -f environment-dev.yml # Install PyJanitor in development mode python setup.py develop # Register current virtual environment as a Jupyter Python kernel python -m ipykernel install --user --name pyjanitor-dev --display-name "PyJanitor development" ``` -------------------------------- ### Cloning Pyjanitor Repository Locally - Bash Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/devguide.md This command clones your forked pyjanitor repository from GitHub to your local machine. It's the initial step for manual development setup after you've forked the main repository. ```bash git clone git@github.com:/pyjanitor.git ``` -------------------------------- ### Installing pyjanitor with Pipenv Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/README.md Illustrates installing pyjanitor using Pipenv, a dependency management tool, with the --pre flag to enable prerelease dependencies. ```bash pipenv install --pre pyjanitor ``` -------------------------------- ### Installing Pyjanitor with Pipenv Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/index.md This command installs the `pyjanitor` library using the `pipenv` environment manager. The `--pre` flag is required to enable installation of prerelease dependencies, which might be necessary for `pyjanitor`. ```bash pipenv install --pre pyjanitor ``` -------------------------------- ### Serving Pyjanitor Documentation Locally - Bash Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/devguide.md This command allows you to build and serve the pyjanitor documentation locally using mkdocs. Running this enables you to preview any changes made to the documentation in your web browser before submitting them. ```bash python -m mkdocs serve ``` -------------------------------- ### Installing pyjanitor with Pip Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/README.md Demonstrates how to install the pyjanitor library using the pip package manager. This is the standard Python package installation method. ```bash pip install pyjanitor ``` -------------------------------- ### Setting up Data for pyjanitor row_to_names Examples (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/row_to_names.md This snippet imports the necessary pandas and janitor libraries and initializes a sample DataFrame dictionary (`data_dict`) to be used in subsequent examples for demonstrating the `row_to_names` method. This `data_dict` will be converted into a pandas DataFrame for method application. ```python import pandas as pd import janitor data_dict = { "a": [1, 2, 3] * 3, "Bell__Chart": [1, 2, 3] * 3, "decorated-elephant": [1, 2, 3] * 3, "animals": ["rabbit", "leopard", "lion"] * 3, "cities": ["Cambridge", "Shanghai", "Basel"] * 3, } ``` -------------------------------- ### Setting up DataFrame for Currency Conversion - Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/make_currency_column_numeric.md Initializes a Pandas DataFrame with mixed data types, including a column 'a' containing currency strings, empty strings, and other non-numeric strings, to demonstrate the `make_currency_column_numeric` method. This setup is a prerequisite for the subsequent examples. ```python import pandas as pd import janitor data = { "a": ["-$1.00", "", "REPAY"] * 2 + ["$23.00", "", "Other Account"], "Bell__Chart": [1.234_523_45, 2.456_234, 3.234_612_5] * 3, "decorated-elephant": [1, 2, 3] * 3, "animals@#$%^": ["rabbit", "leopard", "lion"] * 3, "cities": ["Cambridge", "Shanghai", "Basel"] * 3, } df = pd.DataFrame(data) ``` -------------------------------- ### Installing Pyjanitor with Pip Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/index.md This command installs the `pyjanitor` library using the `pip` package manager. It's the standard way to install Python packages from PyPI. ```bash pip install pyjanitor ``` -------------------------------- ### Setting up Data for Currency Conversion in Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/convert_currency.md This snippet imports necessary libraries (pandas, janitor, datetime) and initializes a sample pandas DataFrame (`data_dict`) to be used for currency conversion examples. It prepares the data structure for subsequent operations. ```python import pandas as pd import janitor from datetime import date data_dict = { "a": [1.23452345, 2.456234, 3.2346125] * 3, "Bell__Chart": [1/3, 2/7, 3/2] * 3, "decorated-elephant": [1/234, 2/13, 3/167] * 3, "animals": ["rabbit", "leopard", "lion"] * 3, "cities": ["Cambridge", "Shanghai", "Basel"] * 3, } ``` -------------------------------- ### Serving Pyjanitor Documentation Locally Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/CONTRIBUTING.md This command allows you to build and serve the pyjanitor project's documentation locally using mkdocs. Running this command will typically open a local server, enabling you to preview documentation changes in your web browser before committing them. ```Bash python -m mkdocs serve ``` -------------------------------- ### Installing Pyjanitor with Conda Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/index.md This command installs the `pyjanitor` library using the `conda` package manager from the `conda-forge` channel. This is suitable for users managing environments with Anaconda or Miniconda. ```bash conda install pyjanitor -c conda-forge ``` -------------------------------- ### Cloning Pyjanitor Repository Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/CONTRIBUTING.md This command is used to clone your forked pyjanitor repository from GitHub to your local machine. Replace `` with your actual GitHub username to ensure you clone your personal fork. ```Bash git clone git@github.com:/pyjanitor.git ``` -------------------------------- ### Loading Initial DataFrame for Step-by-Step Cleaning (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/pyjanitor_intro.ipynb This snippet reloads the 'dirty_data.xlsx' file into a DataFrame, serving as the starting point for a step-by-step demonstration of pyjanitor's cleaning operations. It allows for observing the incremental changes applied to the DataFrame in subsequent steps. ```python df = pd.read_excel('dirty_data.xlsx', engine='openpyxl') df ``` -------------------------------- ### Importing Libraries for Text Processing - Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/process_text.ipynb This snippet imports the `re` module for regular expression operations and the `pandas` library for data manipulation, which are prerequisites for the text processing examples. ```Python import re import pandas as pd ``` -------------------------------- ### Setting Up Sample Data for Data Cleaning in Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/README.md This snippet imports necessary libraries (numpy, pandas, janitor) and defines a sample dictionary `company_sales`. This dictionary serves as the raw data input for subsequent data cleaning operations, demonstrating a typical starting point for data manipulation tasks. ```Python # Libraries import numpy as np import pandas as pd import janitor # Sample Data curated for this example company_sales = { 'SalesMonth': ['Jan', 'Feb', 'Mar', 'April'], 'Company1': [150.0, 200.0, 300.0, 400.0], 'Company2': [180.0, 250.0, np.nan, 500.0], 'Company3': [400.0, 500.0, 600.0, 675.0] } ``` -------------------------------- ### Creating DataFrame for Lambda Function Examples Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/case_when.ipynb Prepares a new pandas DataFrame with 'age1' and 'age2' columns, serving as input for demonstrating `case_when` with anonymous (lambda) functions. ```python # https://stackoverflow.com/q/43391591/7175713 raw_data = {'age1': [23,45,21],'age2': [10,20,50]} df = pd.DataFrame(raw_data, columns = ['age1','age2']) df ``` -------------------------------- ### Installing pyjanitor with Conda Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/README.md Shows how to install pyjanitor using the Conda package manager from the conda-forge channel. Conda is often used in data science environments. ```bash conda install pyjanitor -c conda-forge ``` -------------------------------- ### Running a Subset of pyjanitor Tests (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/CONTRIBUTING.md This command allows developers to run specific test files or modules, such as 'tests.test_functions', rather than the entire test suite. This is useful for focused testing during development, saving time and resources. ```bash pytest tests.test_functions ``` -------------------------------- ### Creating DataFrame for Condition Precedence Example Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/case_when.ipynb Generates a DataFrame named 'df' containing a single column 'odd' with numbers from 3 to 27, incrementing by 3. This setup is used to illustrate how `case_when` prioritizes the first matching condition. ```python df = range(3, 30, 3) df = pd.DataFrame(df, columns = ['odd']) df ``` -------------------------------- ### Generating Combinations with Dictionary Input using expand_grid Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/expand_grid.ipynb This example demonstrates the basic usage of `expand_grid` by providing a dictionary where keys represent column names and values are lists of elements. The function generates a DataFrame containing all possible combinations of these elements. ```python data = {"x":[1,2,3], "y":[1,2]} result = expand_grid(others = data) result ``` -------------------------------- ### Running Fast Tests for pyjanitor Environment Check Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/devguide.md This command executes the pyjanitor test suite using `pytest`, specifically excluding tests marked with 'turtle'. It's used to quickly verify that the development environment is correctly set up and that the core tests pass, ensuring readiness for contribution. ```bash python -m pytest -m "not turtle" ``` -------------------------------- ### Setting up DataFrame and importing Janitor in Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/limit_column_characters.md This snippet imports the necessary pandas and janitor libraries and initializes a sample pandas DataFrame named `data_dict`. This DataFrame serves as the input for demonstrating the `limit_column_characters` method in subsequent examples. ```python import pandas as pd import janitor data_dict = { "really_long_name_for_a_column": range(10), "another_really_long_name_for_a_column": [2 * item for item in range(10)], "another_really_longer_name_for_a_column": list("lllongname"), "this_is_getting_out_of_hand": list("longername") } ``` -------------------------------- ### Combining Multiple Column Selection Methods in pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/select_columns.ipynb This example illustrates the flexibility of `select_columns` by combining different selection criteria: an exact string match, a glob pattern, and a slice, all in a single call. ```python df.select_columns("id", "code*", slice("code", "code2")) ``` -------------------------------- ### Committing and Pushing Changes to pyjanitor Branch Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/devguide.md This sequence of Git commands stages all modified and new files, commits them with a descriptive message, and then pushes the committed changes to your remote feature or bugfix branch on GitHub. This prepares your work for a pull request. ```bash git add . git commit -m "Your detailed description of your changes." git push origin ``` -------------------------------- ### Running a Subset of pyjanitor Tests Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/devguide.md This command uses `pytest` to execute tests specifically from the `tests.test_functions` module. It's useful for developers who want to run only relevant tests for the changes they've made, speeding up the testing process during development. ```bash pytest tests.test_functions ``` -------------------------------- ### Creating Sample Pandas DataFrame for Pyjanitor Examples Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/index.md This Python code defines a dictionary `company_sales` containing sample data and then uses it to create a Pandas DataFrame. This DataFrame serves as the input for demonstrating `pyjanitor`'s API functionalities. It includes `NaN` values to showcase cleaning capabilities. ```python import pandas as pd import numpy as np company_sales = { 'SalesMonth': ['Jan', 'Feb', 'Mar', 'April'], 'Company1': [150.0, 200.0, 300.0, 400.0], 'Company2': [180.0, 250.0, np.nan, 500.0], 'Company3': [400.0, 500.0, 600.0, 675.0] } ``` -------------------------------- ### Applying limit_column_characters with Multiple Parameters in pyjanitor (Typo in Original Doc) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/round_to_fraction.md This example initializes `example_dataframe2` from `data_dict` and then calls `limit_column_characters` on column 'a' with a limit of 3 and an additional parameter of 4. Similar to the previous example, the section title implies rounding to the nearest third and specific decimal places, but the code snippet actually demonstrates the `limit_column_characters` method, suggesting a typo in the original documentation. ```python example_dataframe2 = pd.DataFrame(data_dict) example_dataframe2.limit_column_characters('a', 3, 4) ``` -------------------------------- ### Selecting Columns by Glob Pattern in pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/select_columns.ipynb This snippet illustrates selecting columns using shell-like glob patterns (e.g., `*`). It will select all columns whose names start with 'type'. ```python df.select_columns("type*") ``` -------------------------------- ### Selecting Columns by Exact String in pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/select_columns.ipynb This example demonstrates how to select a single column from a DataFrame by providing its exact name as a string argument to the `select_columns` method. ```python df.select_columns("id") ``` -------------------------------- ### Loading Local CSV Data (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/medium_franchise.ipynb This Python snippet loads the raw media franchise data from a local CSV file into a pandas DataFrame. The file is a pre-saved version of the Wikipedia table, making the example reproducible without live web scraping. It then displays the first three rows of the DataFrame. ```Python fileurl = '../data/medium_franchise_raw_table.csv' df_raw = pd.read_csv(fileurl) df_raw.head(3) ``` -------------------------------- ### Rounding a DataFrame Column to Nearest Half in pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/round_to_fraction.md This example demonstrates the intended use of the `round_to_fraction` method. It initializes `example_dataframe` from `data_dict` and then applies `round_to_fraction` to the 'a' column, rounding values to the nearest half by specifying a denominator of 2. ```python example_dataframe = pd.DataFrame(data_dict) example_dataframe.round_to_fraction('a', 2) ``` -------------------------------- ### Initializing DataFrame for Basic Example Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/groupby_agg.ipynb Creates a Pandas DataFrame named `df` with sample data including 'item', 'MRP', and 'number_sold' columns. This DataFrame serves as the input for demonstrating the basic usage of the `groupby_agg` method. ```python df = pd.DataFrame( { "item": ["shoe", "shoe", "bag", "shoe", "bag"], "MRP": [220, 450, 320, 200, 305], "number_sold": [100, 40, 56, 38, 25] } ) df ``` -------------------------------- ### Activating Pyjanitor Development Conda Environment - Bash Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/devguide.md This command activates the pyjanitor-dev Conda environment. It's provided as a troubleshooting step, particularly if you encounter module import errors when trying to serve the documentation locally, ensuring the correct environment is active. ```bash source activate pyjanitor-dev || conda activate pyjanitor-dev ``` -------------------------------- ### Creating Initial DataFrame for `complete` Example Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/complete.ipynb Initializes a pandas DataFrame with `Year`, `Taxon`, and `Abundance` columns. This DataFrame serves as the base for demonstrating the `complete` function, highlighting an implicitly missing 'Year 2000 and Agarum' pairing. ```python # from http://imachordata.com/2016/02/05/you-complete-me/ df = pd.DataFrame( { "Year": [1999, 2000, 2004, 1999, 2004], "Taxon": [ "Saccharina", "Saccharina", "Saccharina", "Agarum", "Agarum", ], "Abundance": [4, 5, 2, 1, 8], } ) df ``` -------------------------------- ### Method Chaining expand_grid to an Existing DataFrame Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/expand_grid.ipynb This example shows how `expand_grid` can be chained directly to an existing Pandas DataFrame. A `df_key` parameter is required when chaining to a DataFrame, which is used to prefix the original DataFrame's column names in the resulting expanded grid. ```python df = pd.DataFrame({"x":[1,2], "y":[2,1]}) data = {"z":[1,2,3]} result = df.expand_grid(df_key="df",others = data) result ``` -------------------------------- ### Configuring janitor.chemistry Module Documentation Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/api/chemistry.md This snippet illustrates how to configure the automatic documentation generation for the `janitor.chemistry` module using a Sphinx/MkDocs directive. It applies a filter `!^_` to exclude any members whose names start with an underscore, typically used to hide private or internal attributes and methods from the public documentation. ```Sphinx Directive ::: janitor.chemistry options: filters: - "!^_" ``` -------------------------------- ### Generating Combinations for Measurement Data using expand_grid Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/expand_grid.ipynb This example applies `expand_grid` to a dictionary containing lists of numerical and categorical measurement data (height, weight, sex). It creates a DataFrame that includes every possible combination of these attributes, useful for experimental design or data generation. ```python data = {'height': [60, 70], 'weight': [100, 140, 180], 'sex': ['Male', 'Female']} measurements = expand_grid(others = data) measurements ``` -------------------------------- ### Converting Currency in Pandas DataFrame using pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/convert_currency.md This example demonstrates how to use the `convert_currency` method from the pyjanitor library to convert a column ('a') from USD to EUR, specifically using historical exchange rates from January 1, 2018. It shows the method's application on a prepared DataFrame. ```python example_dataframe = pd.DataFrame(data_dict) example_dataframe.convert_currency( 'a', from_currency='USD', to_currency='EUR', historical_date=date(2018, 1, 1) ) ``` -------------------------------- ### Initializing IPython/Jupyter Environment Configuration - Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/talks/scipy2019/slides.ipynb Configures the IPython/Jupyter environment by loading the `autoreload` extension for automatic code reloading, setting `matplotlib` to display plots inline, and configuring the `InlineBackend` to use 'retina' format for high-resolution figures. This is typically used at the start of a notebook for development convenience. ```Python %load_ext autoreload %autoreload 2 %matplotlib inline %config InlineBackend.figure_format = 'retina' ``` -------------------------------- ### Activating Pyjanitor Conda Development Environment Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/CONTRIBUTING.md This command activates the 'pyjanitor-dev' conda environment. It uses an OR (||) operator to provide compatibility with different conda activation methods, ensuring the environment is correctly loaded for subsequent development tasks. ```Bash source activate pyjanitor-dev || conda activate pyjanitor-dev ``` -------------------------------- ### Viewing Import Timing Log with Tuna CLI Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/development/lazy_imports.md This command uses the `tuna` command-line interface tool to open and visualize the `timing.log` file. `tuna` provides a web-based UI to analyze import performance and identify bottlenecks, requiring prior installation via `pip`. ```bash tuna timing.log ``` -------------------------------- ### Creating New Feature/Bugfix Branch - Bash Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/devguide.md This Git command creates a new branch for your local development, based off the latest 'dev' branch. It's a best practice for isolating your changes for a bug fix or new feature before submitting a pull request. ```bash git checkout -b dev ``` -------------------------------- ### Performing Data Cleaning with pyjanitor Method Chaining Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/pyjanitor_intro.ipynb This snippet illustrates the `pyjanitor` approach to data cleaning, emphasizing method chaining and verb-based function names. It performs the same sequence of operations as the `pandas` example (removing columns, dropping nulls, renaming, adding a column, resetting index) but in a single, chained expression, enhancing readability and conciseness. ```python df = ( pd.DataFrame(...) .remove_columns(['column1']) .dropna(subset=['column2', 'column3']) .rename_column('column2', 'unicorns') .rename_column('column3', 'dragons') .add_column('new_column', ['iterable', 'of', 'items']) .reset_index(drop=True) ) ``` -------------------------------- ### Creating Sample DataFrame for pivot_wider (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/Pivot Data from Long to Wide Form.ipynb This code initializes a pandas DataFrame from a list of dictionaries. It serves as a sample dataset to illustrate the `pivot_wider` functionality in subsequent examples, particularly focusing on scenarios where the final column order is important. ```Python df = [{'Salesman': 'Knut', 'Height': 6, 'product': 'bat', 'price': 5}, {'Salesman': 'Knut', 'Height': 6, 'product': 'ball', 'price': 1}, {'Salesman': 'Knut', 'Height': 6, 'product': 'wand', 'price': 3}, {'Salesman': 'Steve', 'Height': 5, 'product': 'pen', 'price': 2}] df = pd.DataFrame(df) df ``` -------------------------------- ### Committing and Pushing Changes to pyjanitor Feature Branch (Bash) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/CONTRIBUTING.md This sequence of Git commands stages all modified files, commits them with a descriptive message, and then pushes the committed changes to your remote feature branch on GitHub. This is a crucial step before submitting a pull request. ```bash git add . git commit -m "Your detailed description of your changes." git push origin ``` -------------------------------- ### Filtering DataFrame by Date Range in Pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/filter_date.md This example demonstrates how to filter the `example_dataframe` to include rows where the 'DATE' column falls within a specified `start` and `end` date. It uses the `filter_date` method with string representations of dates. ```python start = "01/29/19" end = "01/30/19" example_dataframe.filter_date('DATE', start=start, end=end) ``` -------------------------------- ### Creating a New Feature Branch for Pyjanitor Contribution Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/CONTRIBUTING.md This Git command creates a new branch for your local development, based off the latest 'dev' branch. It's crucial for isolating your changes and following best practices for contributing bug fixes or new features to the pyjanitor project. ```Bash git checkout -b dev ``` -------------------------------- ### Selecting Columns by Custom Lambda Callable in pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/select_columns.ipynb This example uses a custom lambda function as a callable to select columns. It selects columns whose names start with 'code' or end with '1', demonstrating flexible, programmatic selection. ```python df.select_columns(lambda x: x.name.startswith("code") or x.name.endswith("1")) ``` -------------------------------- ### Performing Data Cleaning with pyjanitor Method Chaining in Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/README.md This example showcases pyjanitor's approach to data cleaning using its verb-named methods in a chained fashion. It performs operations like removing columns, dropping NaNs, renaming columns, and adding a new column, demonstrating pyjanitor's explicit and readable API for constructing data processing DAGs. ```Python df = ( pd.DataFrame.from_dict(company_sales) .remove_columns(["Company1"]) .dropna(subset=["Company2", "Company3"]) .rename_column("Company2", "Amazon") .rename_column("Company3", "Facebook") .add_column("Google", [450.0, 550.0, 800.0]) ) ``` -------------------------------- ### Updating Feature Branch with pyjanitor dev Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/devguide.md This command sequence fetches the latest changes from the 'dev' branch of the 'origin' remote and then reapplies your local commits on top of the updated 'dev' branch. This helps to keep your feature branch current and minimizes merge conflicts when submitting a pull request. ```bash git fetch origin dev git rebase origin/dev ``` -------------------------------- ### Creating a DataFrame with More Nulls for Default Value Example (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/coalesce.ipynb This snippet creates another sample pandas DataFrame with columns 's1' and 's2', specifically designed to have more initial null values. This setup is used to illustrate how the `default_value` parameter of `coalesce` handles remaining missing entries. ```python df = pd.DataFrame({'s1':[np.nan,np.nan,6,9,9], 's2':[np.nan,8,7,9,9]}) df ``` -------------------------------- ### Elevating Specific Row to Column Names and Removing It with Rows Above in pandas (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/row_to_names.md This example illustrates using `row_to_names` to elevate the row at index 2 to column names. It also demonstrates removing both the elevated row and all rows preceding it from the DataFrame by setting `remove_row=True` and `remove_rows_above=True`, resulting in a DataFrame starting from the new header. ```python example_dataframe = pd.DataFrame(data_dict) example_dataframe.row_to_names(2, remove_row=True, remove_rows_above=True) ``` -------------------------------- ### Defining Raw Data String - Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/Row_to_Names.ipynb This snippet defines a multi-line string variable `data` containing comma-separated values. This string simulates raw CSV data that will be used to create a Pandas DataFrame, demonstrating a typical input format for data processing examples. ```python data = '''shoe, 220, 100 shoe, 450, 40 item, retail_price, cost shoe, 200, 38 bag, 305, 25 ''' ``` -------------------------------- ### Performing Data Cleaning with pyjanitor Method Chaining - Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/index.md This example showcases `pyjanitor`'s method chaining capabilities for data cleaning. It demonstrates how `pyjanitor` provides verb-named methods like `remove_columns`, `rename_column`, and `add_column` that integrate seamlessly with pandas DataFrames, offering a more explicit and readable syntax for common data manipulation tasks compared to native pandas methods. ```python df = ( pd.DataFrame.from_dict(company_sales) .remove_columns(["Company1"]) .dropna(subset=["Company2", "Company3"]) .rename_column("Company2", "Amazon") .rename_column("Company3", "Facebook") .add_column("Google", [450.0, 550.0, 800.0]) ) # Output looks like this: # Out[15]: # SalesMonth Amazon Facebook Google # 0 Jan 180.0 400.0 450.0 # 1 Feb 250.0 500.0 550.0 ``` -------------------------------- ### Importing Libraries and Preparing Sample Data - Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/mkdocs/index.md This snippet imports necessary libraries (numpy, pandas, janitor) and defines a sample dictionary `company_sales` to be used as input for DataFrame creation. It sets up the initial data structure for subsequent data cleaning operations. ```python # Libraries import numpy as np import pandas as pd import janitor # Sample Data curated for this example company_sales = { 'SalesMonth': ['Jan', 'Feb', 'Mar', 'April'], 'Company1': [150.0, 200.0, 300.0, 400.0], 'Company2': [180.0, 250.0, np.nan, 500.0], 'Company3': [400.0, 500.0, 600.0, 675.0] } ``` -------------------------------- ### Viewing Initial Rows of Raw Light Data Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/bird_call.ipynb Displays the first five rows of the `raw_light` DataFrame, providing a quick overview of its structure and content after loading. ```python raw_light.head() ``` -------------------------------- ### Initializing DataFrame with pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/add_column.md This snippet sets up a pandas DataFrame and imports the `janitor` library, preparing the environment for demonstrating the `add_column` method. It defines sample data and creates a DataFrame instance. ```python import pandas as pd import janitor data = { "a": [1, 2, 3] * 3, "Bell__Chart": [1, 2, 3] * 3, "decorated-elephant": [1, 2, 3] * 3, "animals": ["rabbit", "leopard", "lion"] * 3, "cities": ["Cambridge", "Shanghai", "Basel"] * 3, } df = pd.DataFrame(data) ``` -------------------------------- ### Loading and Displaying Initial DataFrame (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/pyjanitor_intro.ipynb This snippet loads an Excel file named 'dirty_data.xlsx' into a Pandas DataFrame using the 'openpyxl' engine. It then displays the DataFrame, allowing for an initial inspection of the raw, uncleaned data to identify issues like inconsistent column names, missing values, and empty rows/columns. ```python df = pd.read_excel('dirty_data.xlsx', engine='openpyxl') df ``` -------------------------------- ### Defining str_slice Method for Pandas DataFrame (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/anime.ipynb This method wraps `df.str.slice` to extract a substring from a DataFrame column based on start and stop indices. It registers as a DataFrame method, enabling direct chaining. It takes the DataFrame, column name, optional start, and stop indices. ```python @pf.register_dataframe_method def str_slice( df, column_name: str, start: int = None, stop: int = None, *args, **kwargs ): """ Wrapper around `df.str.slice """ df[column_name] = df[column_name].str[start:stop] return df ``` -------------------------------- ### Loading Web Data (R) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/medium_franchise.ipynb This R snippet demonstrates how to load data from a Wikipedia URL. It uses `read_html()` to parse the HTML content, `html_table()` to extract tables, and `.[[2]]` to select the second table, which contains the relevant media franchise data. ```R url <- "https://en.wikipedia.org/wiki/List_of_highest-grossing_media_franchises" df <- url %>% read_html() %>% html_table(fill = TRUE) %>% .[[2]] ``` -------------------------------- ### Performing Data Cleaning with Pandas Imperative Style Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/pyjanitor_intro.ipynb This snippet demonstrates a traditional imperative-style approach to data cleaning using `pandas`. It shows how to create a DataFrame, delete a column, drop rows based on null values in specific columns, rename columns, add a new column, and reset the index. Each operation is performed as a separate statement, often requiring reassignment of the DataFrame. ```python df = pd.DataFrame(...) # create a pandas DataFrame somehow. del df['column1'] # delete a column from the dataframe. df = df.dropna(subset=['column2', 'column3']) # drop rows that have empty values in column 2 and 3. df = df.rename({'column2': 'unicorns', 'column3': 'dragons'}) # rename column2 and column3 df['new_column'] = ['iterable', 'of', 'items'] # add a new column. df.reset_index(inplace=True, drop=True) # reset index to account for the missing row we removed above ``` -------------------------------- ### Setting up a Pandas DataFrame with Dates for Pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/filter_date.md This snippet initializes a pandas DataFrame named `example_dataframe` with two columns, 'AMOUNT' and 'DATE', containing sample data. It imports `pandas` and `janitor` to prepare the environment for demonstrating the `filter_date` method. ```python import pandas as pd import janitor date_list = [ [1, "01/28/19"], [2, "01/29/19"], [3, "01/30/19"], [4, "01/31/19"], [5, "02/01/19"], [6, "02/02/19"], [7, "02/03/19"], [8, "02/04/19"], [9, "02/05/19"], [10, "02/06/19"], [11, "02/07/20"], [12, "02/08/20"], [13, "02/09/20"], [14, "02/10/20"], [15, "02/11/20"], [16, "02/12/20"], [17, "02/07/20"], [18, "02/08/20"], [19, "02/09/20"], [20, "02/10/20"], [21, "02/11/20"], [22, "02/12/20"], [23, "03/08/20"], [24, "03/09/20"], [25, "03/10/20"], [26, "03/11/20"], [27, "03/12/20"]] example_dataframe = pd.DataFrame(date_list, columns=['AMOUNT', 'DATE']) ``` -------------------------------- ### Selecting Columns by Slice in pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/select_columns.ipynb This example shows how to select a range of columns using a `slice` object. It selects all columns from 'code1' up to and including 'type1' based on their order in the DataFrame. ```python df.select_columns(slice("code1", "type1")) ``` -------------------------------- ### Filtering DataFrame by Specific Year in Pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/filter_date.md This example shows how to filter the `example_dataframe` to include only rows from a specific year. It passes a list of years to the `years` parameter of the `filter_date` method. ```python years = [2019] example_dataframe.filter_date('DATE', years=years) ``` -------------------------------- ### Inverting Column Selection in pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/select_columns.ipynb This example shows how to select the complement of the specified columns by setting `invert=True`. It returns all columns *except* those matched by the provided string, glob, and slice criteria. ```python df.select_columns("id", "code*", slice("code", "code2"), invert = True) ``` -------------------------------- ### Adding Column with Single Value in Pandas using pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/add_column.md This example demonstrates how to use `df.add_column()` to add a new column named 'city_pop' to the DataFrame, assigning a single constant value (100000) to all rows. ```python df.add_column("city_pop", 100000) ``` -------------------------------- ### Setting up a Pandas DataFrame with pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/round_to_fraction.md This snippet imports the necessary pandas and janitor libraries and initializes a sample pandas DataFrame named `data_dict`. This DataFrame is designed to be used for demonstrating various data manipulation methods, including the `round_to_fraction` method, and contains a mix of float, fractional, and string data types. ```python import pandas as pd import janitor data_dict = { "a": [1.23452345, 2.456234, 3.2346125] * 3, "Bell__Chart": [1/3, 2/7, 3/2] * 3, "decorated-elephant": [1/234, 2/13, 3/167] * 3, "animals": ["rabbit", "leopard", "lion"] * 3, "cities": ["Cambridge", "Shanghai", "Basel"] * 3, } ``` -------------------------------- ### Viewing Initial Rows of Raw Birds Data Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/bird_call.ipynb Displays the first five rows of the `raw_birds` DataFrame, providing a quick overview of its structure and content after loading. ```python raw_birds.head() ``` -------------------------------- ### Selecting Columns by Regular Expression in pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/select_columns.ipynb This example demonstrates selecting columns using a compiled regular expression. It selects columns whose names contain one or more digits, providing powerful pattern-based matching. ```python df.select_columns(re.compile("\\d+")) ``` -------------------------------- ### Viewing Initial Rows of Raw Call Data Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/bird_call.ipynb Displays the first five rows of the `raw_call` DataFrame, providing a quick overview of its structure and content after loading. ```python raw_call.head() ``` -------------------------------- ### Creating Sample DataFrame for Conditional Logic Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/case_when.ipynb Initializes a pandas DataFrame with two columns, 'col1' and 'col2', containing sample string data. This DataFrame is used to demonstrate `case_when` functionality. ```python # https://stackoverflow.com/q/19913659/7175713 df = pd.DataFrame({'col1': list('ABBC'), 'col2': list('ZZXY')}) df ``` -------------------------------- ### Converting Currency Column with Specific String Replacement - Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/make_currency_column_numeric.md Shows how to use the `cast_non_numeric` parameter to replace specific non-numeric string values with a desired numeric value during currency column conversion. In this example, 'REPAY' is cast to 22. ```python cast_non_numeric = {"REPAY": 22} df.make_currency_column_numeric("a", cast_non_numeric=cast_non_numeric) ``` -------------------------------- ### Truncating column names with custom separator in Pandas (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/limit_column_characters.md This example showcases how to use `limit_column_characters` with a custom separator. Column names are truncated to 7 characters, and a period ('.') is used instead of the default underscore to distinguish duplicated column names. ```python example_dataframe2 = pd.DataFrame(data_dict) example_dataframe2.limit_column_characters(7, ".") ``` -------------------------------- ### Adding Column Based on Other Columns' Mutation in Pandas using pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/add_column.md This example illustrates adding a new column 'city_pop' whose values are derived from a mathematical operation on existing columns ('Bell__Chart' and 'a'). This showcases how `add_column` can take a Series as input. ```python df.add_column("city_pop", df.Bell__Chart - 2 * df.a) ``` -------------------------------- ### Creating Sample Pandas DataFrame - Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/process_text.ipynb This snippet initializes a pandas DataFrame named `df` with two columns, 'text' and 'code', containing sample string data that will be used to demonstrate text processing functionalities. ```Python df = pd.DataFrame({'text': ['Ragnar', 'sammywemmy', 'ginger'], 'code': [1, 2, 3]}) df ``` -------------------------------- ### Loading and Initial Inspection of Raw Teacher Data (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/teacher_pupil.ipynb This snippet imports `pandas` and `pandas_flavor`, then loads the raw international teacher data from a specified URL into a pandas DataFrame. It concludes by displaying the first few rows of the DataFrame to provide an initial overview of its structure and content. ```Python import pandas as pd import pandas_flavor as pf dirty_csv = ( "https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2019/2019-05-07/EDULIT_DS_06052019101747206.csv" ) dirty_df = pd.read_csv(dirty_csv) dirty_df.head() ``` -------------------------------- ### Truncating column names with default separator in Pandas (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/limit_column_characters.md This example demonstrates the basic usage of `limit_column_characters` on `example_dataframe`. It truncates all column names to a maximum of 7 characters. Duplicate names are handled by appending numbers using the default underscore ('_') separator. ```python example_dataframe = pd.DataFrame(data_dict) example_dataframe.limit_column_characters(7) ``` -------------------------------- ### Creating Sample DataFrame for pyjanitor Column Selection Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/select_columns.ipynb This code initializes a pandas DataFrame with diverse data types, including integers, strings, floats, categories, and datetime objects, to serve as a sample dataset for demonstrating various column selection methods. ```python df = pd.DataFrame( { "id": [0, 1], "Name": ["ABC", "XYZ"], "code": [1, 2], "code1": [4, np.nan], "code2": ["8", 5], "type": ["S", "R"], "type1": ["E", np.nan], "type2": ["T", "U"], "code3": pd.Series(["a", "b"], dtype="category"), "type3": pd.to_datetime([np.datetime64("2018-01-01"), datetime.datetime(2018, 1, 1)]), } ) df ``` -------------------------------- ### Filtering DataFrame by Year and Day in Pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/filter_date.md This example illustrates filtering the DataFrame by a combination of year and a range of days. It utilizes the `years` and `days` parameters of the `filter_date` method to select entries within a specific year and day range. ```python years = [2020] days = range(10, 12) example_dataframe.filter_date('DATE', years=years, days=days) ``` -------------------------------- ### Loading Initial DataFrame for Cleaning (Python) Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/anime.ipynb This snippet loads a CSV file from a URL into a Pandas DataFrame, serving as the initial dataset for subsequent cleaning and transformation operations. It uses `pandas.read_csv` to fetch the data and includes necessary imports for `pandas` and `pyjanitor`. ```python import pandas as pd import pyjanitor as pf filename = "https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2019/2019-04-23/raw_anime.csv" df = pd.read_csv(filename) ``` -------------------------------- ### Initializing Pandas DataFrame with Sales Data - Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/sort_columns.ipynb This snippet initializes a pandas DataFrame named `df` with sample sales data. It includes columns for 'SalesMonth', 'Company2', and 'Company3', demonstrating the structure before sorting. ```python df = pd.DataFrame([{'SalesMonth': 'Jan', 'Company2': 180.0, 'Company3': 400.0}, {'SalesMonth': 'Feb', 'Company2': 250.0, 'Company3': 500.0}, {'SalesMonth': 'Feb', 'Company2': 250.0, 'Company3': 500.0}, {'SalesMonth': 'Mar', 'Company2': nan, 'Company3': 600.0}, {'SalesMonth': 'April', 'Company2': 500.0, 'Company3': 675.0}] ) df ``` -------------------------------- ### Filtering DataFrame with Custom Date Format in Pyjanitor Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/filter_date.md This snippet illustrates how to filter the DataFrame using a custom date format for the `start` or `end` parameters. It specifies a non-standard date string and provides the corresponding `format` argument to `filter_date` for correct parsing. ```python end = "01$$$30$$$19" format = "%m$$$%d$$$%y" example_dataframe.filter_date('DATE', end=end, format=format) ``` -------------------------------- ### Selecting Metadata Columns in Python Source: https://github.com/pyjanitor-devs/pyjanitor/blob/dev/examples/notebooks/medium_franchise.ipynb This Python snippet creates a new DataFrame `df_metadata` by selecting specific columns ('franchise', 'revenue_category', 'original_media', 'creators') from the `df_clean` DataFrame. This prepares the metadata for a subsequent join operation. ```python df_metadata = df_clean[ ['franchise', 'revenue_category', 'original_media', 'creators'] ] df_metadata.head(3) ```