### MultiIndex.get_indexer Example Setup Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v1.1.0.rst Initializes a DataFrame with a MultiIndex and another MultiIndex for reindexing, demonstrating the setup for `MultiIndex.get_indexer` behavior changes. ```python df = pd.DataFrame({ 'a': [0, 0, 0, 0], 'b': [0, 2, 3, 4], 'c': ['A', 'B', 'C', 'D'], }).set_index(['a', 'b']) mi_2 = pd.MultiIndex.from_product([[0], [-1, 0, 1, 3, 4, 5]]) ``` -------------------------------- ### DataFrame setup for groupby min() examples Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/cookbook.rst Initializes a DataFrame to be used in subsequent examples demonstrating how to find minimums within groups while retaining other columns. ```python df = pd.DataFrame( {"AAA": [1, 1, 1, 2, 2, 2, 3, 3], "BBB": [2, 1, 3, 4, 5, 1, 2, 3]} ) df ``` -------------------------------- ### Setup for Attribute Access Examples Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/indexing.rst Initializes a pandas Series and a copy of the DataFrame for demonstrating attribute-based indexing. ```python sa = pd.Series([1, 2, 3], index=list('abc')) dfa = df.copy() ``` -------------------------------- ### Install pandas with optional dependencies using pip extras Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v2.0.0.rst This command demonstrates how to install pandas along with specific sets of optional dependencies, such as 'performance' and 'aws', using pip extras. This allows users to include additional functionalities without installing the entire 'all' extra. The available extras are listed in the pandas installation guide. ```bash pip install "pandas[performance, aws]>=2.0.0" ``` -------------------------------- ### Setup DataFrame for Pandas GroupBy Piping Examples Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/groupby.rst Initializes a Pandas DataFrame with columns for 'Store', 'Product', 'Revenue', and 'Quantity'. This dataset serves as a common example for demonstrating the `groupby().pipe()` functionality to perform complex group-wise calculations. ```python n = 1000 df = pd.DataFrame( { "Store": np.random.choice(["Store_1", "Store_2"], n), "Product": np.random.choice(["Product_1", "Product_2"], n), "Revenue": (np.random.random(n) * 50 + 10).round(2), "Quantity": np.random.randint(1, 10, size=n), } ) df.head(2) ``` -------------------------------- ### Doctest Tips for Pandas Examples Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This section provides tips for writing examples that pass doctests in pandas documentation. It covers importing libraries, using random data with a fixed seed, handling multi-line code snippets, showing exceptions, and using ellipses for variable parts of the output. It ensures examples are correctly formatted for testing. -------------------------------- ### Install airspeed-velocity (asv) for benchmarking Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_codebase.rst Install the `asv` benchmarking tool directly from its GitHub repository. This is required to run performance benchmarks for pandas. ```bash pip install git+https://github.com/airspeed-velocity/asv ``` -------------------------------- ### Initialize DataFrame for Compressed Pickling Examples Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/io.rst Creates a sample DataFrame with 1000 rows and mixed data types. This DataFrame is used as input for subsequent examples demonstrating compressed pickling. ```python df = pd.DataFrame( { "A": np.random.randn(1000), "B": "foo", "C": pd.date_range("20130101", periods=1000, freq="s") } ) df ``` -------------------------------- ### Creating a DataFrame for positional indexing examples Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/indexing.rst Initializes a DataFrame with random data and integer indices/columns for subsequent `.iloc` examples. ```python df1 = pd.DataFrame(np.random.randn(6, 4), index=list(range(0, 12, 2)), columns=list(range(0, 8, 2))) df1 ``` -------------------------------- ### Using the `get` method for Series Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/indexing.rst Illustrates the `get` method for Series to retrieve values with a default option. ```python s = pd.Series([1, 2, 3], index=['a', 'b', 'c']) s.get('a') # equivalent to s['a'] s.get('x', default=-1) ``` -------------------------------- ### Install pre-commit hooks Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_codebase.rst Installs pre-commit hooks from the repository root to automatically run style checks before each commit. ```bash pre-commit install ``` -------------------------------- ### Example pandas Startup Script for IPython Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/options.rst A sample Python script to be placed in an IPython startup directory, demonstrating how to import pandas and set global options like `display.max_rows` and `display.precision` automatically. ```python import pandas as pd pd.set_option("display.max_rows", 999) pd.set_option("display.precision", 5) ``` -------------------------------- ### Define a Python Function with a Good Docstring Summary Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This example demonstrates a correctly formatted docstring for a Python function. It includes a concise, single-line summary starting with an infinitive verb, followed by an extended description providing further details. ```python def astype(dtype): """ Cast Series type. This section will provide further details. """ pass ``` -------------------------------- ### Illustrate Bad Docstring: Incomplete Parameters Section in Python Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This example shows an incomplete docstring for a Python method. While it starts with a short and extended summary, the 'Parameters' section is not fully documented, specifically missing the description for the 'color' parameter. ```python class Series: def plot(self, kind, **kwargs): """ Generate a plot. Render the data in the Series as a matplotlib plot of the specified kind. ``` -------------------------------- ### Illustrate Bad Docstring Summary: Missing Infinitive Verb in Python Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This example highlights a docstring summary that is considered bad practice because it does not start with an infinitive verb. The summary begins with a noun phrase ('Method to cast') rather than an action verb. ```python def astype(dtype): """ Method to cast Series type. Does not start with verb. """ pass ``` -------------------------------- ### Example Pandas RangeIndex Metadata Descriptor (Python) Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/developer.rst Demonstrates how a `pandas.RangeIndex` is represented purely through metadata in the Parquet format, without requiring serialization as a data column. The descriptor specifies the index's kind, name, start, stop, and step values, enabling efficient reconstruction. ```python index = pd.RangeIndex(0, 10, 2) { "kind": "range", "name": index.name, "start": index.start, "stop": index.stop, "step": index.step, } ``` -------------------------------- ### Create DataFrame for indexing examples Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v0.20.0.rst Initialize a simple DataFrame with columns A and B and string index for demonstrating indexing methods. ```python df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]}, index=list('abc')) ``` -------------------------------- ### Clean Up ORC Example File (Python) Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/io.rst Removes the generated ORC example file from the filesystem. ```python os.remove("example_pa.orc") ``` -------------------------------- ### Default IPython Startup Directory Path Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/options.rst This shows the typical path for the default IPython profile's startup directory, where custom scripts can be placed to configure the environment. ```none $IPYTHONDIR/profile_default/startup ``` -------------------------------- ### Define Custom Business Hours with Specific Start Time and Weekmask Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/timeseries.rst Configure `CustomBusinessHour` to specify a custom start time and a `weekmask`. This example demonstrates skipping Mondays and starting business hours from 10:00 AM on other weekdays. ```python bhour_mon = pd.offsets.CustomBusinessHour(start="10:00", weekmask="Tue Wed Thu Fri") # Monday is skipped because it's a holiday, business hour starts from 10:00 dt + bhour_mon * 2 ``` -------------------------------- ### Install pandas with optional Excel dependencies Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/getting_started/install.rst This command installs pandas along with additional dependencies required for reading Excel files. ```shell pip install "pandas[excel]" ``` -------------------------------- ### Define helper functions and DataFrame for `pipe` examples Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/basics.rst Set up two functions, `extract_city_name` and `add_country_name`, and a sample DataFrame `df_p` to demonstrate `pipe` method chaining. ```python def extract_city_name(df): """ Chicago, IL -> Chicago for city_name column """ df["city_name"] = df["city_and_code"].str.split(",").str.get(0) return df def add_country_name(df, country_name=None): """ Chicago -> Chicago-US for city_name column """ col = "city_name" df["city_and_country"] = df[col] + country_name return df df_p = pd.DataFrame({"city_and_code": ["Chicago, IL"]}) ``` -------------------------------- ### Initialize sample DataFrame for aggregation Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v0.20.0.rst Create a DataFrame with columns A, B, and C to demonstrate various groupby and aggregation behaviors. ```python df = pd.DataFrame({'A': [1, 1, 1, 2, 2], 'B': range(5), 'C': range(5)}) df ``` -------------------------------- ### Pandas Plotting Example Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This code demonstrates how to include plots in pandas documentation examples using the ``.. plot::`` directive. It shows how to generate a plot with Series data and provides an example of using the directive with the `:context: close-figs` option. This allows plots to be rendered automatically during documentation building. ```python class Series: def plot(self): """ Generate a plot with the ``Series`` data. Examples -------- .. plot:: :context: close-figs >>> ser = pd.Series([1, 2, 3]) >>> ser.plot() """ pass ``` -------------------------------- ### Generate DatetimeIndex with various parameter combinations (Python) Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/timeseries.rst Demonstrates generating date ranges using `start`, `end`, `periods`, and `freq` parameters. Start and end dates are strictly inclusive. ```python pd.date_range(start, end, freq="BME") pd.date_range(start, end, freq="W") pd.bdate_range(end=end, periods=20) pd.bdate_range(start=start, periods=20) ``` -------------------------------- ### Sampling with and without replacement Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/indexing.rst Demonstrates how to use the `replace` option in `sample()` to perform sampling with or without replacement. ```python s = pd.Series([0, 1, 2, 3, 4, 5]) # Without replacement (default): s.sample(n=6, replace=False) # With replacement: s.sample(n=6, replace=True) ``` -------------------------------- ### Convert Q-MAR Quarterly Period to Daily Frequency Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/timeseries.rst Converts a Period with Q-MAR frequency to daily frequency, showing how to get the start and end of the period. ```python p = pd.Period("2011Q4", freq="Q-MAR") p.asfreq("D", "s") ``` ```python p.asfreq("D", "e") ``` -------------------------------- ### Initialize DataFrame for IO Performance Benchmarking Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/io.rst Sets up a large pandas DataFrame and displays its structure, serving as the common dataset for subsequent IO performance tests. ```ipython In [1]: sz = 1000000 In [2]: df = pd.DataFrame({'A': np.random.randn(sz), 'B': [1] * sz}) In [3]: df.info() RangeIndex: 1000000 entries, 0 to 999999 Data columns (total 2 columns): A 1000000 non-null float64 B 1000000 non-null int64 dtypes: float64(1), int64(1) memory usage: 15.3 MB ``` -------------------------------- ### Convert Q-DEC Quarterly Period to Daily Frequency Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/timeseries.rst Converts a Period with Q-DEC frequency to daily frequency, showing how to get the start and end of the period. ```python p = pd.Period("2012Q1", freq="Q-DEC") p.asfreq("D", "s") ``` ```python p.asfreq("D", "e") ``` -------------------------------- ### Create sample DataFrame for performance testing Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/enhancingperf.rst Initialize a DataFrame with random numeric and string data for use in performance benchmarking examples. ```python df = pd.DataFrame( { "a": np.random.randn(1000), "b": np.random.randn(1000), "N": np.random.randint(100, 1000, (1000), dtype="int64"), "x": "x", } ) df ``` -------------------------------- ### Access Series Values as NumPy Array or ExtensionArray in Python Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/migration.rst Series.values returns NumPy array for object dtype but ExtensionArray for string dtype. Use Series.to_numpy() to get NumPy array or Series.array to get ExtensionArray for consistent behavior across dtypes. ```python ser = pd.Series(["a", "b", np.nan], dtype="object") type(ser.values) ``` ```python ser = pd.Series(["a", "b", pd.NA], dtype="str") ser.values ['a', 'b', nan] Length: 3, dtype: str ``` ```python ser = pd.Series(["a", "b", pd.NA], dtype="str") ser.to_numpy() ['a' 'b' nan] ``` -------------------------------- ### Create DataFrame for pandas.pivot_table examples Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/reshaping.rst Initializes a DataFrame with various categorical and numerical columns, including datetime objects, for demonstrating `pivot_table` functionality. ```python import datetime df = pd.DataFrame( { "A": ["one", "one", "two", "three"] * 6, "B": ["A", "B", "C"] * 8, "C": ["foo", "foo", "foo", "bar", "bar", "bar"] * 4, "D": np.random.randn(24), "E": np.random.randn(24), "F": [datetime.datetime(2013, i, 1) for i in range(1, 13)] + [datetime.datetime(2013, i, 15) for i in range(1, 13)], } ) df ``` -------------------------------- ### Create sample CSV for date parsing Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/io.rst This snippet creates a `foo.csv` file with a 'date' column and other data, serving as an example input for `read_csv`'s date handling features. ```python with open("foo.csv", mode="w") as f:\n f.write("date,A,B,C\n20090101,a,1,2\n20090102,b,3,4\n20090103,c,4,5") ``` -------------------------------- ### Attempting to Get Option with Ambiguous Regex Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/options.rst This example demonstrates how an ambiguous regular expression pattern can fail when used with `pd.get_option`, as it matches multiple option names. ```python pd.get_option("max") ``` -------------------------------- ### Test Series Getitem with List-like Integers Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_codebase.rst This example demonstrates testing `Series.__getitem__` and `Series.loc.__getitem__` using list-like integer indexing. It illustrates how a single test can cover similar methods, with the test's primary location determined by the underlying method or the source of a bug fix. ```python import pandas as pd import pandas._testing as tm def test_getitem_listlike_of_ints(): ser = pd.Series(range(5)) result = ser[[3, 4]] expected = pd.Series([2, 3]) tm.assert_series_equal(result, expected) result = ser.loc[[3, 4]] tm.assert_series_equal(result, expected) ``` -------------------------------- ### Pandas DataFrame Method Example Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This code defines a sample DataFrame method. It provides guidance on writing examples, including importing necessary libraries, using meaningful data, and avoiding positional arguments. It also suggests how to present different parameter behaviors. ```python def method(foo=None, bar=None): """ A sample DataFrame method. Do not import NumPy and pandas. Try to use meaningful data, when it makes the example easier to understand. Try to avoid positional arguments like in ``df.method(1)``. They can be all right if previously defined with a meaningful name, like in ``present_value(interest_rate)``, but avoid them otherwise. When presenting the behavior with different parameters, do not place all the calls one next to the other. Instead, add a short sentence explaining what the example shows. Examples -------- >>> import numpy as np >>> import pandas as pd >>> df = pd.DataFrame(np.random.randn(3, 3), ... columns=('a', 'b', 'c')) >>> df.method(1) 21 >>> df.method(bar=14) 123 """ pass ``` -------------------------------- ### Create a pandas DataFrame Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v3.0.0.rst Initializes a DataFrame with sample data to demonstrate column expression building. ```python df = pd.DataFrame({'a': [1, 1, 2], 'b': [4, 5, 6]}) ``` -------------------------------- ### Create a pandas Series with DatetimeIndex Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/timeseries.rst Initializes a pandas Series with integer values and a daily `DatetimeIndex` starting from '2000-01-01' for five periods. This series is used in subsequent non-inclusive slicing examples. ```python ts_example = pd.Series( range(5), index=pd.date_range("2000-01-01", periods=5, freq="D"), ) ts_example ``` -------------------------------- ### View remote repository configuration Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing.rst Example output showing both origin (your fork) and upstream (main pandas repository) remotes with their respective fetch and push URLs. ```shell origin git@github.com:yourname/pandas.git (fetch) origin git@github.com:yourname/pandas.git (push) upstream git://github.com/pandas-dev/pandas.git (fetch) upstream git://github.com/pandas-dev/pandas.git (push) ``` -------------------------------- ### Define Custom Weighted Mean Function Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/window.rst Defines a Python function intended for use with Rolling.apply to calculate a weighted mean. This function is a partial example, showing the start of a custom aggregation logic. ```python def weighted_mean(x): arr = np.ones((1, x.shape[1])) ``` -------------------------------- ### Bad Example of Python Docstring Formatting Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This snippet illustrates common mistakes in Python docstring formatting, serving as a 'bad' example. It highlights issues such as blank lines after the function signature, incorrect placement of the summary line, blank lines between the docstring and the first line of code, and improper placement of the closing quotes. ```python def func(): """Some function. With several mistakes in the docstring. It has a blank line after the signature ``def func():``. The text 'Some function' should go in the line after the opening quotes of the docstring, not in the same line. There is a blank line between the docstring and the first line of code ``foo = 1``. The closing quotes should be in the next line, not in this one.""" foo = 1 bar = 2 return foo + bar ``` -------------------------------- ### Prepare Fixed-Width Data for read_fwf (Python) Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/io.rst This snippet creates a sample fixed-width data string and writes it to a file, demonstrating the format expected by `read_fwf`. ```python data1 = ( "id8141 360.242940 149.910199 11950.7\n" "id1594 444.953632 166.985655 11788.4\n" "id1849 364.136849 183.628767 11806.2\n" "id1230 413.836124 184.375703 11916.8\n" "id1948 502.953953 173.237159 12468.3" ) with open("bar.csv", "w") as f: f.write(data1) ``` -------------------------------- ### Illustrative Python Docstring for a Simple Function Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This snippet provides a complete example of a Python docstring for a simple 'add' function. It demonstrates the standard sections like Parameters, Returns, See Also, and Examples, following the NumPy docstring convention, to clearly document a function's purpose, arguments, and return values. ```python def add(num1, num2): """ Add up two integer numbers. This function simply wraps the ``+`` operator, and does not do anything interesting, except for illustrating what the docstring of a very simple function looks like. Parameters ---------- num1 : int First number to add. num2 : int Second number to add. Returns ------- int The sum of ``num1`` and ``num2``. See Also -------- subtract : Subtract one integer from another. Examples -------- >>> add(2, 2) 4 >>> add(25, 0) 25 >>> add(10, -10) 0 """ return num1 + num2 ``` -------------------------------- ### Setting Values with at and iat Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/indexing.rst Shows how to set scalar values using `at` and `iat` indexers. ```python df.at[dates[5], 'A'] = 7 df.iat[3, 0] = 7 ``` -------------------------------- ### Create Sample DataFrames for Merge Examples in Pandas Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/getting_started/comparison/includes/merge_setup.rst Initializes two pandas DataFrames with 'key' and 'value' columns. df1 contains keys A-D with random values, while df2 contains keys B, D, D, E with random values. These DataFrames are used to demonstrate various merge operations and join types in pandas. ```python df1 = pd.DataFrame({"key": ["A", "B", "C", "D"], "value": np.random.randn(4)}) df1 df2 = pd.DataFrame({"key": ["B", "D", "D", "E"], "value": np.random.randn(4)}) df2 ``` -------------------------------- ### Create Series with integer and datetime indexes Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v1.1.0.rst Setup for demonstrating label lookup behavior changes. Creates two Series with different index types for comparison. ```python ser1 = pd.Series(range(3), index=[0, 1, 2]) ser2 = pd.Series(range(3), index=pd.date_range("2020-02-01", periods=3)) ``` -------------------------------- ### Activate pandas virtual environment with virtualenvwrapper Source: https://github.com/pandas-dev/pandas/wiki/Git-Workflows Activates the pandas virtual environment using virtualenvwrapper. This command switches to the pandas development environment with all dependencies installed. ```shell workon pandas ``` -------------------------------- ### Setting Values with New Keys using loc Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/indexing.rst Illustrates using `loc` to set values with new keys, as `at`'s in-place enlargement behavior is deprecated. ```python df.loc[dates[-1] + pd.Timedelta('1 day'), 0] = 7 df ``` -------------------------------- ### Example Fully-Formed Pandas Parquet Metadata Structure (Text) Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/developer.rst Presents a comprehensive example of the complete metadata structure for a pandas DataFrame stored in Apache Parquet. This includes the `index_columns`, `column_indexes` (with details like `pandas_type`, `numpy_type`, and `encoding`), and `columns` sections, illustrating how various data types and their specific metadata are represented. ```text {'index_columns': ['__index_level_0__'], 'column_indexes': [ {'name': None, 'field_name': 'None', 'pandas_type': 'unicode', 'numpy_type': 'object', 'metadata': {'encoding': 'UTF-8'}} ], 'columns': [ {'name': 'c0', 'field_name': 'c0', 'pandas_type': 'int8', 'numpy_type': 'int8', 'metadata': None}, {'name': 'c1', 'field_name': 'c1', 'pandas_type': 'bytes', 'numpy_type': 'object', ``` -------------------------------- ### Run ASV continuous benchmark with default environment Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_codebase.rst Execute a continuous ASV benchmark comparing the current `HEAD` against `upstream/main`. The `-f 1.1` flag reports benchmarks that changed by more than 10%. ```bash asv continuous -f 1.1 upstream/main HEAD ``` -------------------------------- ### Handling Missing Optional Dependencies for Pandas Connectors (Python) Source: https://github.com/pandas-dev/pandas/blob/main/web/pandas/pdeps/0009-io-extensions.md This snippet demonstrates the `ImportError` raised when an optional dependency for a pandas I/O connector (e.g., `pandas-gbq`) is not installed. It shows the typical traceback message, which guides the user to install the required package using `pip` or `conda` to enable the functionality. ```python >>> pandas.read_gbq(query) Traceback (most recent call last): ... ImportError: Missing optional dependency 'pandas-gbq'. pandas-gbq is required to load data from Google BigQuery. See the docs: https://pandas-gbq.readthedocs.io. Use pip or conda to install pandas-gbq. ``` -------------------------------- ### Pandas Series mean() Method - Example Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst Computes the mean (average) of numeric values in a Series. This is a simple aggregation function that returns a single scalar value representing the arithmetic mean of all elements. ```python ser = pd.Series([1, 2, 3]) ser.mean() # Output: 2 ``` -------------------------------- ### Initialize DataFrame for Custom Indexer Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/window.rst Prepare a DataFrame and a boolean list to demonstrate custom windowing logic. This setup is used to define how windows expand or remain fixed. ```python use_expanding = [True, False, True, False, True] use_expanding df = pd.DataFrame({"values": range(5)}) df ``` -------------------------------- ### Extract Substring by Position in Stata Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/getting_started/comparison/comparison_with_stata.rst This Stata snippet illustrates how to extract a portion of a string based on its starting position and length. It uses the `substr` function to get the first character from the `sex` column. ```stata generate short_sex = substr(sex, 1, 1) ``` -------------------------------- ### Create and Activate pyenv virtualenv on Unix/macOS Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_environment.rst Create and activate a `pyenv` virtual environment for a specific Python version, then install build dependencies. ```bash # Create a virtual environment # Use an ENV_DIR of your choice. We'll use ~/.pyenv/versions/pandas-dev pyenv virtualenv 3.11 pandas-dev # Activate the virtualenv pyenv activate pandas-dev # Now install the build dependencies in the cloned pandas repo python -m pip install -r requirements-dev.txt ``` -------------------------------- ### Python Docstring for Pandas Series.head with See Also Section Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This Python code snippet demonstrates the structure of a docstring for a pandas `Series.head` method, specifically highlighting the 'See Also' section. It shows how to reference related methods like `Series.tail` and `Series.iloc`, providing concise explanations for their relevance and differences. This example serves as a guide for cross-referencing functionalities within pandas documentation. ```python class Series: def head(self): """ Return the first 5 elements of the Series. This function is mainly useful to preview the values of the Series without displaying the whole of it. Returns ------- Series Subset of the original series with the 5 first values. See Also -------- Series.tail : Return the last 5 elements of the Series. Series.iloc : Return a slice of the elements in the Series, which can also be used to return the first or last n. """ return self.iloc[:5] ``` -------------------------------- ### Get First Observation by Group in Pandas Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/getting_started/comparison/comparison_with_stata.rst Retrieves the first observation for each group defined by sex and smoker status using pandas groupby operation. This is the pandas equivalent to the Stata bysort filtering example. ```python tips.groupby(["sex", "smoker"]).first() ``` -------------------------------- ### Basic positional indexing with .iloc on Series Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/indexing.rst Shows how to create a Series and select elements using integer-based slicing and single integer indexing with `.iloc`. ```python s1 = pd.Series(np.random.randn(5), index=list(range(0, 10, 2))) s1 s1.iloc[:3] s1.iloc[3] ``` -------------------------------- ### Illustrate Bad Docstring Summary: Overly Verbose in Python Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This example illustrates a docstring summary that is too verbose and exceeds the recommended single-line length. A good summary should be concise and provide a high-level overview without excessive detail. ```python def astype(dtype): """ Cast Series type from its current type to the new type defined in the parameter dtype. Summary is too verbose and doesn't fit in a single line. """ pass ``` -------------------------------- ### View Series data with head and tail methods Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/basics.rst Demonstrates how to use `head()` to view the first five elements and `tail()` with a custom number to view the last elements of a Series. ```python long_series = pd.Series(np.random.randn(1000)) long_series.head() long_series.tail(3) ``` -------------------------------- ### DataFrame initialization for performance testing Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/indexing.rst Initializes a DataFrame for demonstrating `DataFrame.query()` performance. ```python df = pd.DataFrame(np.random.randn(8, 4), index=dates, columns=['A', 'B', 'C', 'D']) df2 = df.copy() ``` -------------------------------- ### Initialize Pandas Series for Upsampling Example Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v0.18.0.rst Creates a pandas Series with a quarterly frequency, which serves as the base data for demonstrating upsampling operations. This setup is crucial for understanding how `resample` handles increasing data frequency. ```ipython s = pd.Series(np.arange(5, dtype='int64'), index=pd.date_range('2010-01-01', periods=5, freq='Q')) ``` -------------------------------- ### Pandas Chained Assignment (Pre-CoW Behavior) Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/migration.rst This example demonstrates a chained assignment operation that would have worked before Copy-on-Write (CoW). It updates a column based on a condition, but this pattern violates CoW principles and will raise a `ChainedAssignmentError` with CoW enabled. ```python df = pd.DataFrame({"foo": [1, 2, 3], "bar": [4, 5, 6]}) df["foo"][df["bar"] > 5] = 100 df ``` -------------------------------- ### Create DataFrame for pandas.pivot example Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/reshaping.rst Initializes a DataFrame with 'value', 'variable', and 'date' columns, suitable for demonstrating data pivoting operations. ```python data = { "value": range(12), "variable": ["A"] * 3 + ["B"] * 3 + ["C"] * 3 + ["D"] * 3, "date": pd.to_datetime(["2020-01-03", "2020-01-04", "2020-01-05"] * 4) } df = pd.DataFrame(data) ``` -------------------------------- ### Pandas Inplace Modification Without CoW Trigger Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/migration.rst This example demonstrates an inplace modification of a Pandas DataFrame where no Copy-on-Write (CoW) is triggered. Since the DataFrame `df` does not share data with any other object, direct modification of its values does not necessitate a copy. ```python df = pd.DataFrame({"foo": [1, 2, 3], "bar": [4, 5, 6]}) df.iloc[0, 0] = 100 df ``` -------------------------------- ### List configured remote repositories Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing.rst Display all configured remote repositories for your local repository, showing fetch and push URLs. ```shell git remote -v ``` -------------------------------- ### Illustrate Bad Docstring Summary: Third-Person Verb in Python Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This snippet shows an example of an incorrectly formatted docstring summary. The summary uses a third-person verb ('Casts') instead of an infinitive verb, which violates the specified docstring guidelines. ```python def astype(dtype): """ Casts Series type. Verb in third-person of the present simple, should be infinitive. """ pass ``` -------------------------------- ### Define a Python Function with an Extended Docstring Summary Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst This example showcases a docstring with both a concise short summary and a detailed extended summary. The extended summary provides context and use cases for the function, separated by a blank line from the short summary. ```python def unstack(): """ Pivot a row index to columns. When using a MultiIndex, a level can be pivoted so each value in the index becomes a column. This is especially useful when a subindex is repeated for the main index, and data is easier to visualize as a pivot table. The index level will be automatically removed from the index when added as columns. """ pass ``` -------------------------------- ### Save and Load Series with Default Bzip2 Compression using Pandas Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/io.rst This example demonstrates saving a Series to a pickle file, with bzip2 compression inferred from the '.bz2' extension, and then loading it back. ```python df["A"].to_pickle("s1.pkl.bz2") rt = pd.read_pickle("s1.pkl.bz2") rt ``` -------------------------------- ### Prepare CSV Data with Implicit Index Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/io.rst Creates a sample CSV string and writes it to a file, demonstrating data where the first column might be implicitly used as an index. ```python data = "A,B,C\n20090101,a,1,2\n20090102,b,3,4\n20090103,c,4,5" print(data) with open("foo.csv", "w") as f: f.write(data) ``` -------------------------------- ### Pandas `to_numpy()` Returns View for Single Dtype Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/migration.rst This example demonstrates that `to_numpy()` returns a view (shares data) when the Pandas DataFrame consists of only one homogeneous NumPy array (i.e., all columns have the same data type). This array will be read-only by default under Copy-on-Write. ```python df = pd.DataFrame({"a": [1, 2], "b": [3, 4]}) df.to_numpy() ``` -------------------------------- ### Document multiple return values with tuple in Python Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_docstring.rst Shows proper docstring format for functions returning multiple values as a tuple. Each return value is named and documented with its type and description. This example returns both an integer length and a string of random letters. ```python import string def random_letters(): """ Generate and return a sequence of random letters. The length of the returned string is also random, and is also returned. Returns ------- length : int Length of the returned string. letters : str String of random letters. """ length = np.random.randint(1, 10) letters = ''.join(np.random.choice(string.ascii_lowercase) for i in range(length)) return length, letters ``` -------------------------------- ### Setting values with positional indexing Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/indexing.rst Demonstrates how to set values in a Series using integer-based slicing with `.iloc`. ```python s1.iloc[:3] = 0 s1 ``` -------------------------------- ### Create DataFrame with categorical grouper and NA values Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v1.5.1.rst Creates a pandas DataFrame with a categorical column containing NA values and numeric categories. This example demonstrates the setup for testing groupby behavior with categorical groupers and the dropna parameter. ```python df = pd.DataFrame( { "x": pd.Categorical([1, None], categories=[1, 2, 3]), "y": [3, 4], } ) df ``` -------------------------------- ### Get Dummies Data Type Inspection in Pandas Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v0.19.0.rst Demonstrates the new behavior of pd.get_dummies() function which converts categorical variables into dummy/indicator variables. This example shows how to inspect the data types of the resulting dummy variables created from a list of categorical values. ```python pd.get_dummies(["a", "b", "a", "c"]).dtypes ``` -------------------------------- ### Initialize HDFStore and prepare DataFrames for table format Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/io.rst Initializes an HDFStore instance and prepares two DataFrames (`df1`, `df2`) by splitting an existing `df` for subsequent table format operations. ```python store = pd.HDFStore("store.h5") df1 = df[0:4] df2 = df[4:] ``` -------------------------------- ### Build and Upload Website Source: https://github.com/pandas-dev/pandas/wiki/Release-Checklist Builds the website HTML from source files and uploads to the documentation server. Requires SSH key authentication for the docs server access. ```bash make html make upload ``` -------------------------------- ### Reset Index with Copy-on-Write Triggering Copy Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/migration.rst Demonstrates how resetting index on a DataFrame and then modifying it triggers an unnecessary copy when the original DataFrame reference is retained. This example shows the inefficient pattern where two objects share data, causing a copy operation during item assignment. ```python df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}) df2 = df.reset_index(drop=True) df2.iloc[0, 0] = 100 ``` -------------------------------- ### Float indexing coercion with label-based indexers Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v0.18.0.rst Label-based indexers (.loc and []) coerce float scalars to integers for non-Float64Index objects without raising errors. This example shows how float indexing now works for getting and setting values using .loc and direct indexing. ```python s = pd.Series([1, 2, 3], index=[4, 5, 6]) s[5.0] # 2 s.loc[5.0] # 2 s_copy = s.copy() s_copy[5.0] = 10 s_copy # 4 1 # 5 10 # 6 3 # dtype: int64 ``` -------------------------------- ### Demonstrate positional, label, and generic slicing Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/cookbook.rst Shows examples of positional slicing using `iloc`, label-based slicing using `loc`, and generic slicing which adapts based on index type. ```python df.iloc[0:3] # Positional df.loc["bar":"kar"] # Label # Generic df[0:3] df["bar":"kar"] ``` -------------------------------- ### Pandas Correct Chained Assignment with `loc` and CoW Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/migration.rst This example demonstrates the correct way to perform conditional assignment in Pandas when Copy-on-Write (CoW) is enabled. Using `.loc` for both row and column selection ensures a single, unambiguous assignment, adhering to CoW principles and avoiding errors. ```python df.loc[df["bar"] > 5, "foo"] = 100 ``` -------------------------------- ### Setting values with .loc on Series Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/indexing.rst Demonstrates setting values in a Series using `.loc` with a slice. ```python s1.loc['c':] = 0 s1 ``` -------------------------------- ### Handle `allow_fill` and `fill_value` in `Index.take` Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v0.18.1.rst Demonstrates consistent handling of `allow_fill` and `fill_value` parameters in `Index.take` for filling missing values. ```python idx = pd.Index([1.0, 2.0, 3.0, 4.0], dtype="float") # default, allow_fill=True, fill_value=None idx.take([2, -1]) idx.take([2, -1], fill_value=True) ``` -------------------------------- ### Pandas Making NumPy Array Writeable for Modification Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/migration.rst This example demonstrates how to explicitly make a NumPy array obtained from a Pandas DataFrame writeable, allowing in-place modification. While this circumvents Copy-on-Write rules, it should be used with caution as it can lead to unexpected side-effects if the array still shares data with other objects. ```python arr = df.to_numpy() arr.flags.writeable = True arr[0, 0] = 100 arr ``` -------------------------------- ### Run specific 'GroupByMethods' benchmark with asv Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_codebase.rst Use a dot '.' as a separator with the '-b' flag to run a specific group of benchmarks, like 'GroupByMethods', from a file. ```bash asv continuous -f 1.1 upstream/main HEAD -b groupby.GroupByMethods ``` -------------------------------- ### Rewriting Chained Assignment with DataFrame.replace (CoW Compliant) Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/user_guide/migration.rst This example demonstrates a Copy-on-Write compliant way to modify DataFrame values, replacing a chained assignment. By calling `replace` directly on the DataFrame with a dictionary mapping columns to replacements, modifications are applied correctly without violating CoW rules. ```python df = pd.DataFrame({"foo": [1, 2, 3], "bar": [4, 5, 6]}) df.replace({"foo": {1: 5}}, inplace=True) df ``` -------------------------------- ### Comprehensive Pytest example with parametrization and fixtures Source: https://github.com/pandas-dev/pandas/blob/main/doc/source/development/contributing_codebase.rst Illustrates a self-contained test file demonstrating `pytest.mark.parametrize` for various data types, `pytest.mark.skip`, `pytest.mark.xfail`, and custom fixtures for pandas Series testing. ```python import pytest import numpy as np import pandas as pd @pytest.mark.parametrize('dtype', ['int8', 'int16', 'int32', 'int64']) def test_dtypes(dtype): assert str(np.dtype(dtype)) == dtype @pytest.mark.parametrize( 'dtype', ['float32', pytest.param('int16', marks=pytest.mark.skip), pytest.param('int32', marks=pytest.mark.xfail( reason='to show how it works'))]) def test_mark(dtype): assert str(np.dtype(dtype)) == 'float32' @pytest.fixture def series(): return pd.Series([1, 2, 3]) @pytest.fixture(params=['int8', 'int16', 'int32', 'int64']) def dtype(request): return request.param def test_series(series, dtype): # GH result = series.astype(dtype) assert result.dtype == dtype expected = pd.Series([1, 2, 3], dtype=dtype) tm.assert_series_equal(result, expected) ```