Python Data Science Handbook

https://jakevdp.github.io/PythonDataScienceHandbook
Admin
A comprehensive guide covering IPython, NumPy, Pandas, Matplotlib, and machine learning techniques...
Tokens:206,849
Snippets:1,703
Trust Score:8.3
Update:10 months ago
Show doc for...
Tokens:
Raw
### Launch Jupyter Notebook Server

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.00-ipython-beyond-normal-python

This command starts the Jupyter Notebook application from your system's command line. It launches a local web server and provides a URL to access the notebook interface in your browser. Ensure Jupyter is installed in your environment.

```shell
jupyter notebook
```

--------------------------------

### Install line_profiler Package

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.07-timing-and-profiling

Installs the 'line_profiler' package using pip, which is required for line-by-line profiling in IPython.

```bash
$ pip install line_profiler

```

--------------------------------

### Example Jupyter Notebook Server Log Output

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.00-ipython-beyond-normal-python

This is an example of the log output you might see after running the 'jupyter notebook' command. It indicates the directory from which notebooks are being served and the URL where the notebook server is accessible. This output is for informational purposes and shows the server's status.

```shell
[NotebookApp] Serving notebooks from local directory: /Users/jakevdp/PythonDataScienceHandbook
[NotebookApp] 0 active kernels 
[NotebookApp] The IPython Notebook is running at: http://localhost:8888/
[NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
```

--------------------------------

### Install memory_profiler

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.07-timing-and-profiling

Installs the memory_profiler package using pip, which is required to use the %memit and %mprun IPython magic functions for memory profiling.

```bash
$ pip install memory_profiler
```

--------------------------------

### Setup Matplotlib and NumPy Environment

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.01-simple-line-plots

Initializes the Matplotlib backend for inline plotting and imports necessary libraries, Matplotlib.pyplot and NumPy. This sets up the environment for creating visualizations.

```python
import matplotlib.pyplot as plt
plt.style.use('seaborn-whitegrid')
import numpy as np
```

--------------------------------

### Import NumPy and Check Version

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.00-introduction-to-numpy

This snippet demonstrates how to import the NumPy library and display its installed version. It's a common first step when working with NumPy to ensure it's correctly installed and to note the version for compatibility.

```python
import numpy
numpy.__version__
```

--------------------------------

### Install Basemap using Conda

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.13-geographic-data-with-basemap

This command installs the Basemap package using the Conda package manager. It's a straightforward way to get the necessary library for geographic plotting in Python.

```bash
$ conda install basemap
```

--------------------------------

### Draw Map Boundaries with Different Resolutions (Python)

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.13-geographic-data-with-basemap

This example demonstrates how to draw land/sea boundaries using Basemap with different resolution settings ('l' for low and 'h' for high). It shows the impact of resolution on map detail and performance, advising to start with low resolution and increase as needed. Dependencies include Matplotlib and Basemap.

```python
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap

fig, ax = plt.subplots(1, 2, figsize=(12, 8))

for i, res in enumerate(['l', 'h']):
    m = Basemap(projection='gnom', lat_0=57.3, lon_0=-6.2,
                width=90000, height=120000, resolution=res, ax=ax[i])
    m.fillcontinents(color="#FFDDCC", lake_color='#DDEEFF')
    m.drawmapboundary(fill_color="#DDEEFF")
    m.drawcoastlines()
    ax[i].set_title("resolution='{0}'".format(res));

```

--------------------------------

### Python Plotting and Data Setup

Source: https://jakevdp.github.io/PythonDataScienceHandbook/06.00-figure-code

Initializes plotting environment and imports essential libraries for data manipulation and visualization. This is a common setup for many Python data science scripts.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns

```

--------------------------------

### Install Core Python Data Science Packages using Conda

Source: https://jakevdp.github.io/PythonDataScienceHandbook/00.00-preface

This command installs essential Python libraries for data science, including numerical computation (numpy), data manipulation (pandas), machine learning (scikit-learn), plotting (matplotlib, seaborn), and an interactive environment (jupyter). It uses the `conda` package manager, which is part of the Miniconda or Anaconda distributions.

```bash
[~]$ conda install numpy pandas scikit-learn matplotlib seaborn jupyter

```

--------------------------------

### Install NetCDF4 Library

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.13-geographic-data-with-basemap

Installs the NetCDF4 library, which is necessary for reading NetCDF files in Python. This command is typically run in a Conda environment.

```bash
$ conda install netcdf4
```

--------------------------------

### Setup for Matplotlib Customization

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.10-customizing-ticks

Initializes Matplotlib plotting environment with classic style and enables inline display of plots. Imports necessary libraries for data manipulation and plotting.

```python
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
import numpy as np
```

--------------------------------

### Setup Notebook for Plotting (Python)

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.02-simple-scatter-plots

Initializes the Jupyter Notebook environment for plotting with Matplotlib. It sets the backend to display plots inline and imports necessary libraries like matplotlib.pyplot and numpy.

```Python
%matplotlib inline
import matplotlib.pyplot as plt
plt.style.use('seaborn-whitegrid')
import numpy as np

```

--------------------------------

### Listing All Available Magic Functions with %magic

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.03-magic-commands

The %magic command provides a general description of all available IPython magic functions, often including examples. It helps users discover and understand the capabilities of IPython's magic system.

```ipython
In [11]: %magic
```

--------------------------------

### Static Typing Example in C

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.01-understanding-data-types

Demonstrates the explicit declaration of data types for variables in the C programming language, which is a characteristic of statically-typed languages.

```c
/* C code */
int result = 0;
for(int i=0; i<100; i++){
    result += i;
}

```

--------------------------------

### NumPy Broadcasting Compatible Arrays Example

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.05-computation-on-arrays-broadcasting

Demonstrates how two NumPy arrays with compatible shapes can be broadcasted for element-wise addition. It shows the initial shapes, how they are adjusted according to broadcasting rules, and the resulting array.

```python
import numpy as np

a = np.arange(3).reshape((3, 1))
b = np.arange(3)

# Shapes before broadcasting
# a.shape = (3, 1)
# b.shape = (3,)

# Shapes after broadcasting rules are applied
# a.shape -> (3, 3)
# b.shape -> (3, 3)

result = a + b
print(result)
```

--------------------------------

### Setup Matplotlib and NumPy for Plotting

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.07-customizing-colorbars

Imports necessary libraries, Matplotlib for plotting and NumPy for numerical operations. Sets a classic plotting style and enables inline plotting for Jupyter notebooks.

```python
import matplotlib.pyplot as plt
plt.style.use('classic')

%matplotlib inline
import numpy as np
```

--------------------------------

### Python: Control Exception Reporting with %xmode

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.06-errors-and-debugging

Demonstrates how to use IPython's `%xmode` magic function to alter the verbosity of exception tracebacks. It includes examples of defining functions, triggering a ZeroDivisionError, and observing the output with 'Context' (default), 'Plain', and 'Verbose' modes.

```python
def func1(a, b):
    return a / b

def func2(x):
    a = x
    b = x - 1
    return func1(a, b)

# Example usage that causes an error:
# func2(1)
```

```ipython
%xmode Plain
```

```ipython
%xmode Verbose
```

--------------------------------

### Dynamic Typing Example in Python

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.01-understanding-data-types

Illustrates dynamic typing in Python, where variable types are inferred at runtime, allowing for flexibility in assigning different data types to the same variable.

```python
# Python code
result = 0
for i in range(100):
    result += i

```

```python
# Python code
x = 4
x = "four"

```

--------------------------------

### Accessing Object Method Documentation with '?' in IPython

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.01-help-and-documentation

Illustrates using the '?' character in IPython to get documentation for object methods, such as the 'insert' method of a list object. It displays the method's type, string form, and its specific docstring detailing its usage.

```Python
In [3]: L = [1, 2, 3]
In [4]: L.insert?
Type:        builtin_function_or_method
String form: <built-in method insert of list object at 0x1024b8ea8>
Docstring:   L.insert(index, object) -- insert object before index


```

--------------------------------

### Broadcasting Example 1: 2D and 1D Array Addition

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.05-computation-on-arrays-broadcasting

Walks through broadcasting a 1D array to a 2D array, explaining the shape transformations according to broadcasting rules.

```python
M = np.ones((2, 3))
a = np.arange(3)

# M.shape -> (2, 3)
# a.shape -> (3,)

# Rule 1 padding: a.shape -> (1, 3)
# Rule 2 stretching: a.shape -> (2, 3)

M + a

```

--------------------------------

### Load and Inspect Planets Dataset (Python)

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.14-visualization-with-seaborn

Loads the 'planets' dataset using Seaborn and displays the first few rows. This is a prerequisite for creating visualizations with this dataset. It requires the Seaborn library to be installed.

```python
import seaborn as sns

planets = sns.load_dataset('planets')
planets.head()
```

--------------------------------

### Download 20 Newsgroups Data and Get Target Names

Source: https://jakevdp.github.io/PythonDataScienceHandbook/05.05-naive-bayes

Fetches the 20 Newsgroups dataset from scikit-learn and retrieves the list of available target category names. This is the first step in preparing data for text classification.

```python
from sklearn.datasets import fetch_20newsgroups

data = fetch_20newsgroups()
data.target_names
```

--------------------------------

### Pandas Series Initialization Example

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.10-working-with-strings

Initializes a Pandas Series with a list of strings, including a `None` value. This setup is used to demonstrate Pandas' vectorized string operations, particularly their ability to handle missing data.

```python
import pandas as pd
data = ['peter', 'Paul', None, 'MARY', 'gUIDO']
names = pd.Series(data)
names
```

--------------------------------

### Access Specific Element in Structured Array

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.09-structured-data-numpy

Shows how to access a single element within a structured array by combining row indexing and field name. This example gets the 'name' from the last row (index -1). The output is the string value of the requested element.

```python
# Get the name from the last row
data[-1]['name']
```

--------------------------------

### Create a Python Module for %mprun

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.07-timing-and-profiling

Creates a Python file named 'mprun_demo.py' using the %%file magic command in IPython. This file contains the 'sum_of_lists' function, which will be profiled line by line using %mprun.

```python
%%file mprun_demo.py
def sum_of_lists(N):
    total = 0
    for i in range(5):
        L = [j ^ (j >> i) for j in range(N)]
        total += sum(L)
        del L # remove reference to L
    return total
```

--------------------------------

### Load and Visualize Handwritten Digits Data

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.07-customizing-colorbars

This code loads a dataset of handwritten digits (0-5) from Scikit-Learn and visualizes several example images. It uses `load_digits` to get the data and `plt.subplots` to create a grid of plots for displaying the 8x8 pixel images with a binary colormap.

```python
# load images of the digits 0 through 5 and visualize several of them
from sklearn.datasets import load_digits
digits = load_digits(n_class=6)

fig, ax = plt.subplots(8, 8, figsize=(6, 6))
for i, axi in enumerate(ax.flat):
    axi.imshow(digits.images[i], cmap='binary')
    axi.set(xticks=[], yticks=[])
```

--------------------------------

### Pandas String Methods: Finding Names with Regex Constraints

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.10-working-with-strings

Finds all occurrences within each string element that match a regular expression pattern. This example finds names starting and ending with consonants using regex anchors and character sets. It returns a Series of lists.

```python
monte.str.findall(r'^[^AEIOU].*[^aeiou]$')
```

--------------------------------

### Create NumPy Arrays from Scratch

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.01-understanding-data-types

Illustrates various methods for creating NumPy arrays from scratch, including arrays filled with zeros, ones, specific values, and arrays populated with values from a linear sequence.

```python
import numpy as np

# Array of zeros:
np.zeros(10, dtype=int)

# Array of ones:
np.ones((3, 5), dtype=float)

# Array filled with a specific value:
np.full((3, 5), 3.14)

# Array with a linear sequence:
np.arange(0, 20, 2)
```

--------------------------------

### Accessing Function Documentation with '?' in IPython

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.01-help-and-documentation

Demonstrates how to use the '?' character in IPython as a shorthand for the built-in help() function to access documentation for Python objects, including built-in functions like 'len'. This provides details such as type, string representation, namespace, and the docstring.

```Python
In [1]: help(len)
Help on built-in function len in module builtins:

len(...)
    len(object) -> integer

    Return the number of items of a sequence or mapping.


```

```Python
In [2]: len?
Type:        builtin_function_or_method
String form: <built-in function len>
Namespace:   Python builtin
Docstring:
len(object) -> integer

Return the number of items of a sequence or mapping.


```

--------------------------------

### Create and Access Pandas Series Data

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.02-data-indexing-and-selection

Demonstrates the creation of a Pandas Series with custom index and accessing elements using dictionary-like key lookup. Also shows how to check for key existence and retrieve keys/items.

```python
import pandas as pd
data = pd.Series([0.25, 0.5, 0.75, 1.0],
                 index=['a', 'b', 'c', 'd'])
print(data)

print(data['b'])

print('a' in data)

print(data.keys())

print(list(data.items()))
```

--------------------------------

### Setup GridSearchCV for Hyperparameter Tuning in Python

Source: https://jakevdp.github.io/PythonDataScienceHandbook/05.03-hyperparameters-and-model-validation

This snippet demonstrates how to set up Scikit-Learn's GridSearchCV to find the optimal hyperparameters for a model. It defines a parameter grid and initializes the GridSearchCV object with a specified cross-validation strategy. This is useful for automating the search for the best model configuration.

```python
from sklearn.grid_search import GridSearchCV

param_grid = {'polynomialfeatures__degree': np.arange(21),
              'linearregression__fit_intercept': [True, False],
              'linearregression__normalize': [True, False]}

grid = GridSearchCV(PolynomialRegression(), param_grid, cv=7)


```

--------------------------------

### Accessing Object Documentation with '?' in IPython

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.01-help-and-documentation

Shows how to use the '?' character in IPython to retrieve documentation for an object itself, like a list. This provides information about the object's type, its string representation, length, and its general docstring.

```Python
In [5]: L?
Type:        list
String form: [1, 2, 3]
Length:      3
Docstring:
list() -> new empty list
list(iterable) -> new list initialized from iterable's items


```

--------------------------------

### Load line_profiler Extension in IPython

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.07-timing-and-profiling

Loads the 'line_profiler' IPython extension, enabling the use of the %lprun magic command.

```ipython
%load_ext line_profiler

```

--------------------------------

### IPython Reverse-Search Command History (Ctrl-r)

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.02-shell-keyboard-shortcuts

Demonstrates how to use the Ctrl-r shortcut in IPython to perform a reverse-incremental search through command history. This feature allows users to quickly find and reuse previous commands by typing matching characters. The search narrows down as more characters are entered. Pressing Enter executes the found command.

```shell
In [1]: 
(reverse-i-search)`': 
```

```shell
In [1]: 
(reverse-i-search)`sqa': square??

```

```shell
In [1]: 
(reverse-i-search)`sqa': def square(a):
    """Return the square of a"""
    return a ** 2

```

```shell
In [1]: def square(a):
    """Return the square of a"""
    return a ** 2

In [2]: square(2)
Out[2]: 4

```

--------------------------------

### Pandas String Methods: Starts With Check

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.10-working-with-strings

Checks if each string element in a Pandas Series starts with a specified prefix. Returns a Boolean Series indicating the result for each element. This mirrors Python's `startswith()` method.

```python
monte.str.startswith('T')
```

--------------------------------

### Up-sampling Time Series Data with Different Fill Methods

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.11-working-with-time-series

Demonstrates up-sampling time series data to a daily frequency using `asfreq()`. It shows the default behavior (leaving non-business days as NA) and illustrates forward-fill ('ffill') and backward-fill ('bfill') methods for imputing missing values. The example uses the first 10 days of Google stock data.

```python
import matplotlib.pyplot as plt
import pandas as pd

# Assuming 'goog' is a pandas Series with time series data
# fig, ax = plt.subplots(2, sharex=True)
# data = goog.iloc[:10]

# data.asfreq('D').plot(ax=ax[0], marker='o')
# data.asfreq('D', method='bfill').plot(ax=ax[1], style='-o')
# data.asfreq('D', method='ffill').plot(ax=ax[1], style='--o')
# ax[1].legend(["back-fill", "forward-fill"])
# plt.show()
```

--------------------------------

### Standard Imports for Data Science and Visualization

Source: https://jakevdp.github.io/PythonDataScienceHandbook/05.14-image-features

Imports necessary libraries for plotting, data manipulation, and machine learning visualization. Ensures that plots are displayed inline within the notebook.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns; sns.set()
import numpy as np

```

--------------------------------

### Create evenly spaced array with NumPy linspace

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.01-understanding-data-types

Generates an array of a specified number of values evenly spaced between a start and end point. This is useful for creating sequences for plotting or numerical analysis. The function takes the start, stop, and number of samples as arguments.

```python
import numpy as np
# Create an array of five values evenly spaced between 0 and 1
np.linspace(0, 1, 5)
```

--------------------------------

### Load and Display Image using Matplotlib and Scikit-learn

Source: https://jakevdp.github.io/PythonDataScienceHandbook/05.11-k-means

Loads a sample image ('china.jpg') using scikit-learn and displays it using Matplotlib. Requires the 'pillow' package for image loading.

```python
from sklearn.datasets import load_sample_image
import matplotlib.pyplot as plt

# Note: this requires the ``pillow`` package to be installed
china = load_sample_image("china.jpg")
ax = plt.axes(xticks=[], yticks=[])
ax.imshow(china);
```

--------------------------------

### Multi-Dimensional NumPy Array Slicing Examples

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.02-the-basics-of-numpy-arrays

Demonstrates slicing in multi-dimensional NumPy arrays, showing how to select specific rows and columns using comma-separated slice notation. It includes examples of selecting a subset of rows and columns, and reversing dimensions.

```python
import numpy as np

x2 = np.array([[12,  5,  2,  4],
               [ 7,  6,  8,  8],
               [ 1,  6,  7,  7]])

# Two rows, three columns
print(x2[:2, :3])

# All rows, every other column
print(x2[:3, ::2])

# Reversed rows and columns
print(x2[::-1, ::-1])
```

--------------------------------

### Create Subplots Manually with plt.axes

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.08-multiple-subplots

Demonstrates creating axes objects manually using plt.axes. The first example creates a standard axes object filling the figure. The second example creates an inset axes in the top-right corner by specifying its position and dimensions within the figure coordinates.

```python
ax1 = plt.axes()  # standard axes
ax2 = plt.axes([0.65, 0.65, 0.2, 0.2])

```

--------------------------------

### Set up Matplotlib and Seaborn for Plotting

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.06-boolean-arrays-and-masks

Initializes Matplotlib for inline plotting in Jupyter notebooks and sets the plot style using Seaborn. Requires matplotlib and seaborn libraries.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn; seaborn.set()
```

--------------------------------

### Define Function for Profiling

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.07-timing-and-profiling

Defines a Python function that performs calculations involving list comprehensions and sums. This function serves as an example for profiling.

```python
def sum_of_lists(N):
    total = 0
    for i in range(5):
        L = [j ^ (j >> i) for j in range(N)]
        total += sum(L)
    return total

```

--------------------------------

### Basic Histogram Plotting with Matplotlib

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.05-histograms-and-binnings

Generates a basic histogram from random data using Matplotlib. Requires NumPy for data generation and Matplotlib for plotting. Outputs a visual representation of data distribution.

```python
import numpy as np
import matplotlib.pyplot as plt
plt.style.use('seaborn-white')

data = np.random.randn(1000)
```

```python
plt.hist(data);
```

--------------------------------

### Displaying Command History with %history (IPython)

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.04-input-output-history

The `%history` magic command in IPython allows users to access and display a batch of previous inputs. This example demonstrates how to print the first four inputs, including line numbers. For more options and detailed information, users can consult the help documentation by typing `%history?`.

```python
In [16]: %history -n 1-4
   1: import math
   2: math.sin(2)
   3: math.cos(2)
   4: print(In)

```

--------------------------------

### Visualize Negative Patches (Python)

Source: https://jakevdp.github.io/PythonDataScienceHandbook/05.14-image-features

Displays a grid of negative image patches to visually inspect their characteristics. This helps in understanding the nature of the non-face samples generated.

```python
import matplotlib.pyplot as plt

fig, ax = plt.subplots(6, 10)
for i, axi in enumerate(ax.flat):
    axi.imshow(negative_patches[500 * i], cmap='gray')
    axi.axis('off')

```

--------------------------------

### Import Pandas and NumPy Libraries

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.03-operations-in-pandas

Initializes the necessary libraries for data manipulation and numerical operations. This is a prerequisite for most subsequent code examples.

```python
import pandas as pd
import numpy as np

```

--------------------------------

### Generate Daily Date Range with Pandas

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.11-working-with-time-series

Creates a sequence of daily timestamps between a start and end date. By default, the frequency is set to 'D' (daily).

```python
import pandas as pd

print(pd.date_range('2015-07-03', '2015-07-10'))
```

--------------------------------

### Plot KDE Model Performance and Best Parameters

Source: https://jakevdp.github.io/PythonDataScienceHandbook/05.13-kernel-density-estimation

Visualizes the cross-validation accuracy against bandwidth for the KDE model and prints the best found parameters and the corresponding best accuracy score.

```python
plt.semilogx(bandwidths, scores)
plt.xlabel('bandwidth')
plt.ylabel('accuracy')
plt.title('KDE Model Performance')
print(grid.best_params_)
print('accuracy =', grid.best_score_)
```

--------------------------------

### Get DataFrame Shape

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.10-working-with-strings

Retrieves the dimensions (number of rows and columns) of a Pandas DataFrame. Returns a tuple representing (rows, columns).

```python
recipes.shape
```

--------------------------------

### C Type Mismatch Error

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.01-understanding-data-types

Shows an example of a type mismatch error in C, where an attempt to assign a string literal to an integer variable will result in a compilation error.

```c
/* C code */
int x = 4;
x = "four";  // FAILS

```

--------------------------------

### IPython Command History Navigation (Ctrl-p, Ctrl-n)

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.02-shell-keyboard-shortcuts

Explains the use of Ctrl-p (or up arrow) and Ctrl-n (or down arrow) for navigating command history in IPython. These shortcuts allow stepping through previous commands. A key distinction is that these shortcuts only match commands that begin with the typed characters, unlike the more flexible reverse search.

```shell
# Example: Type 'def' then press Ctrl-p to find the most recent command starting with 'def'

```

--------------------------------

### NumPy Broadcasting with logaddexp ufunc

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.05-computation-on-arrays-broadcasting

Demonstrates the application of broadcasting rules with a universal function (ufunc) other than addition, specifically `np.logaddexp`. It uses the reshaped array from the previous example.

```python
import numpy as np

M = np.ones((3, 2))
a = np.arange(3)

a_reshaped = a[:, np.newaxis]

result = np.logaddexp(M, a_reshaped)
print(result)
```

--------------------------------

### Line-by-Line Memory Profiling with %mprun

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.07-timing-and-profiling

Imports the 'sum_of_lists' function from 'mprun_demo.py' and profiles its memory usage line by line using the %mprun magic function. The '-f' flag specifies the function to profile.

```python
from mprun_demo import sum_of_lists
%mprun -f sum_of_lists sum_of_lists(1000000)
```

--------------------------------

### Select Categories and Load Training/Testing Data

Source: https://jakevdp.github.io/PythonDataScienceHandbook/05.05-naive-bayes

Filters the 20 Newsgroups dataset to include only specified categories and then downloads the training and testing subsets. This prepares the data for model training and evaluation.

```python
categories = ['talk.religion.misc', 'soc.religion.christian',
              'sci.space', 'comp.graphics']
train = fetch_20newsgroups(subset='train', categories=categories)
test = fetch_20newsgroups(subset='test', categories=categories)
```

--------------------------------

### Generate and Plot a Histogram

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.11-settings-and-stylesheets

Generates random data using NumPy and plots a basic histogram using Matplotlib. This serves as a starting point for demonstrating plot customization.

```python
x = np.random.randn(1000)
plt.hist(x);

```

--------------------------------

### IPython Miscellaneous Shortcuts

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.02-shell-keyboard-shortcuts

Covers essential miscellaneous shortcuts in IPython for efficient session management. This includes clearing the terminal screen with Ctrl-l, interrupting a running Python command with Ctrl-c (useful for long-running tasks), and exiting the IPython session with Ctrl-d.

```shell
# Ctrl-l: Clear terminal screen
# Ctrl-c: Interrupt current Python command
# Ctrl-d: Exit IPython session

```

--------------------------------

### Create NumPy Advanced Compound Type with Array

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.09-structured-data-numpy

Example of creating an advanced compound data type in NumPy where one field contains a matrix. This is useful for mapping to C structures.

```python
import numpy as np

tp = np.dtype([('id', 'i8'), ('mat', 'f8', (3, 3))])
X = np.zeros(1, dtype=tp)
print(X[0])
print(X['mat'][0])
```

--------------------------------

### Generate Timedelta Range with Pandas

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.11-working-with-time-series

Creates a sequence of timedeltas (durations) starting from zero and increasing by an hour for a specified number of periods. The frequency is set to 'H' (hourly).

```python
import pandas as pd

print(pd.timedelta_range(0, periods=10, freq='H'))
```

--------------------------------

### Create and Inspect Python Lists

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.01-understanding-data-types

Demonstrates creating Python lists with integers, strings, and mixed data types. It shows how to check the type of elements within these lists.

```python
L = list(range(10))
L
type(L[0])

L2 = [str(c) for c in L]
L2
type(L2[0])

L3 = [True, "2", 3.0, 4]
[type(item) for item in L3]
```

--------------------------------

### Apply Dark Background Matplotlib Style

Source: https://jakevdp.github.io/PythonDataScienceHandbook/04.11-settings-and-stylesheets

This example shows how to apply the 'dark_background' stylesheet. This style is particularly useful for presentations, as it provides a dark background for figures, making them stand out on screen.

```python
with plt.style.context('dark_background'):
    hist_and_lines()
```

--------------------------------

### Pandas Many-to-One Join Example

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.07-merge-and-join

Demonstrates a many-to-one join operation in Pandas where one DataFrame has duplicate keys. The result preserves these duplicates as appropriate.

```python
import pandas as pd

df4 = pd.DataFrame({'group': ['Accounting', 'Engineering', 'HR'],
                    'supervisor': ['Carly', 'Guido', 'Steve']})
display('df3', 'df4', 'pd.merge(df3, df4)')


```

--------------------------------

### Generate 'HELLO' Shape Data for Manifold Learning

Source: https://jakevdp.github.io/PythonDataScienceHandbook/05.10-manifold-learning

Creates a dataset of 2D points shaped like the word 'HELLO' by rendering text to an image and sampling points from it. This function is useful for visualizing the effects of manifold learning algorithms on nonlinear data structures. It returns sorted 2D coordinates.

```python
def make_hello(N=1000, rseed=42):
    # Make a plot with "HELLO" text; save as PNG
    fig, ax = plt.subplots(figsize=(4, 1))
    fig.subplots_adjust(left=0, right=1, bottom=0, top=1)
    ax.axis('off')
    ax.text(0.5, 0.4, 'HELLO', va='center', ha='center', weight='bold', size=85)
    fig.savefig('hello.png')
    plt.close(fig)
    
    # Open this PNG and draw random points from it
    from matplotlib.image import imread
    data = imread('hello.png')[::-1, :, 0].T
    rng = np.random.RandomState(rseed)
    X = rng.rand(4 * N, 2)
    i, j = (X * data.shape).astype(int).T
    mask = (data[i, j] < 1)
    X = X[mask]
    X[:, 0] *= (data.shape[0] / data.shape[1])
    X = X[:N]
    return X[np.argsort(X[:, 0])]

```

--------------------------------

### Generate Monthly Period Range with Pandas

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.11-working-with-time-series

Generates a sequence of monthly periods starting from a specified month and continuing for a given number of periods. The frequency is set to 'M' (monthly).

```python
import pandas as pd

print(pd.period_range('2015-07', periods=8, freq='M'))
```

--------------------------------

### Generate Hourly Date Range with Pandas

Source: https://jakevdp.github.io/PythonDataScienceHandbook/03.11-working-with-time-series

Creates a sequence of hourly timestamps starting from a specified date for a given number of periods. The frequency is explicitly set to 'H' (hourly).

```python
import pandas as pd

print(pd.date_range('2015-07-03', periods=8, freq='H'))
```

--------------------------------

### NumPy Ufuncs: Operations on Multi-dimensional Arrays

Source: https://jakevdp.github.io/PythonDataScienceHandbook/02.03-computation-on-arrays-ufuncs

Demonstrates how ufuncs can operate on multi-dimensional NumPy arrays. This example shows exponentiation (2 ** x) applied element-wise to a reshaped 3x3 array.

```python
x = np.arange(9).reshape((3, 3))
2 ** x
```

--------------------------------

### Profile Function Lines with %lprun

Source: https://jakevdp.github.io/PythonDataScienceHandbook/01.07-timing-and-profiling

Uses the IPython magic command %lprun to perform line-by-line profiling of a specified Python function. It requires the line_profiler extension to be loaded.

```ipython
%lprun -f sum_of_lists sum_of_lists(5000)

```
Tokens:
Raw