### Complete Google Search Console Query Example Source: https://joshcarty.github.io/google-searchconsole/getting-started A comprehensive example demonstrating authentication, web property selection, and querying the top 10 search queries from the last 7 days. It also includes formatting the output for readability. ```python import searchconsole # Authenticate account = searchconsole.authenticate(client_config='client_secrets.json', credentials='credentials.json') # Select property webproperty = account['https://www.example.com/'] # Query top queries from the last 7 days report = ( webproperty.query .range('today', days=-7) .dimension('query') .limit(10) .get() ) # Display results print(f"Top 10 queries from the last 7 days:\n") for i, row in enumerate(report, 1): print(f"{i}. {row.query}") print(f" Clicks: {row.clicks}, Impressions: {row.impressions}, " f"CTR: {row.ctr:.2%}, Avg Position: {row.position:.1f}\n") ``` -------------------------------- ### Install Google Search Console Python Library Source: https://joshcarty.github.io/google-searchconsole/getting-started Installs the Google Search Console Python library directly from GitHub using pip. Ensure you have Python 3.8 or higher installed. ```bash pip install git+https://github.com/joshcarty/google-searchconsole ``` -------------------------------- ### Get Top Performing Queries (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Retrieves the top 10 performing search queries for the last 30 days. It requires authentication with the Search Console API and uses the `searchconsole` library to fetch and display query, clicks, impressions, CTR, and average position. ```python import searchconsole account = searchconsole.authenticate( client_config='client_secrets.json', credentials='credentials.json' ) webproperty = account['https://www.example.com/'] # Query top queries report = ( webproperty.query .range('today', days=-30) .dimension('query') .limit(10) .get() ) print("Top 10 Queries (Last 30 Days)\n" + "="*50) for i, row in enumerate(report, 1): print(f"{i:2d}. {row.query}") print(f" Clicks: {row.clicks:>6,} | Impressions: {row.impressions:>8,}") print(f" CTR: {row.ctr:>6.2%} | Avg Position: {row.position:>5.1f}\n") ``` -------------------------------- ### Complete Google Search Console API Interaction Example (Python) Source: https://joshcarty.github.io/google-searchconsole/api-reference A comprehensive example demonstrating the entire workflow: authenticating, selecting a web property, building a complex query with dimensions, filters, and limits, fetching the report, analyzing results, accessing data row by row, and exporting the data to CSV and JSON files. ```python import searchconsole import json # 1. Authenticate account = searchconsole.authenticate( client_config='client_secrets.json', credentials='credentials.json' ) # 2. Select property webproperty = account['https://www.example.com/'] print(f"Permission: {webproperty.permission}") # 3. Build complex query report = ( webproperty.query .range('today', days=-30) .dimension('query', 'page', 'country') .filter('query', 'python', 'contains') .filter('page', '/blog/', 'contains') .filter('country', 'usa', 'equals') .search_type('web') .data_state('all') .limit(1000) .get() ) # 4. Analyze results print(f"Total rows: {len(report)}") print(f"Dimensions: {report.dimensions}") print(f"Metrics: {report.metrics}") # 5. Access data for row in report[:10]: print(f"{row.query} on {row.page}") print(f" Country: {row.country}") print(f" Clicks: {row.clicks}, Impressions: {row.impressions}") print(f" CTR: {row.ctr:.2%}, Position: {row.position:.1f} ") # 6. Export df = report.to_dataframe() df.to_csv('search_console_data.csv', index=False) data = report.to_dict() with open('search_console_data.json', 'w') as f: json.dump(data, f, indent=2) ``` -------------------------------- ### Immutable Query Building Example (Python) Source: https://joshcarty.github.io/google-searchconsole/api-reference Demonstrates the immutable nature of the Query object, where methods return new instances, allowing for safe reuse of the base query. This is crucial for generating multiple reports from a single starting point. ```python base = webproperty.query.range('today', days=-7) queries_report = base.dimension('query').get() pages_report = base.dimension('page').get() # base remains unchanged ``` -------------------------------- ### Set Up Development Environment Source: https://joshcarty.github.io/google-searchconsole/contributing Steps to set up the local development environment using uv for dependency management or pip. This includes installing necessary development dependencies. ```bash # Install dependencies uv sync # Or using pip python -m venv .venv source .venv/bin/activate # On Windows: .venv\Scripts\activate pip install -e ".[dev]" ``` -------------------------------- ### Month-over-Month Performance Comparison (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Compares the current month's performance (clicks and impressions) against the previous month. It calculates the percentage change in clicks and provides a summary of both months' metrics. ```python import datetime # Current month data this_month_start = datetime.date.today().replace(day=1) this_month_report = ( webproperty.query .range(this_month_start, 'today') .dimension('query') .get() ) # Previous month data last_month_end = this_month_start - datetime.timedelta(days=1) last_month_start = last_month_end.replace(day=1) last_month_report = ( webproperty.query .range(last_month_start, last_month_end) .dimension('query') .get() ) # Convert to DataFrames this_month_df = this_month_report.to_dataframe() last_month_df = last_month_report.to_dataframe() # Compare totals this_month_clicks = this_month_df['clicks'].sum() last_month_clicks = last_month_df['clicks'].sum() change = ((this_month_clicks - last_month_clicks) / last_month_clicks * 100) print(f"Month-over-Month Performance\n{'='*50}") print(f"\nThis Month:") print(f" Clicks: {this_month_clicks:,}") print(f" Impressions: {this_month_df['impressions'].sum():,}") print(f"\nLast Month:") print(f" Clicks: {last_month_clicks:,}") print(f" Impressions: {last_month_df['impressions'].sum():,}") print(f"\nChange: {change:+.1f}%") ``` -------------------------------- ### Executing Queries to Get All Results Source: https://joshcarty.github.io/google-searchconsole/api-reference Illustrates the `get()` method for executing a query and retrieving all matching rows. This method automatically handles pagination for large result sets. ```python # Build and execute report = ( webproperty.query .range('today', days=-7) .dimension('query') .get() ) ``` ```python # With multiple operations report = ( webproperty.query .range('today', days=-30) .dimension('query', 'page') .filter('query', 'python', 'contains') .limit(1000) .get() ) ``` -------------------------------- ### Install google-searchconsole Python Package Source: https://joshcarty.github.io/google-searchconsole/index Installs the google-searchconsole Python library using pip from a Git repository. This is the initial step to use the library's functionalities. ```bash pip install git+https://github.com/joshcarty/google-searchconsole ``` -------------------------------- ### Compare Geographic Performance (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Compares website performance across different countries over the last 30 days. It retrieves country-level data, sorts it by clicks, and displays the top 10 countries, along with calculating the click share of the top 5. ```python report = ( webproperty.query .range('today', days=-30) .dimension('country') .get() ) df = report.to_dataframe() # Sort by clicks df = df.sort_values('clicks', ascending=False) print("Performance by Country\n") print(df.head(10).to_string(index=False)) # Calculate percentage of total df['click_share'] = (df['clicks'] / df['clicks'].sum() * 100).round(2) top_5 = df.head(5) print(f"\nTop 5 Countries Account for {top_5['click_share'].sum():.1f}% of Clicks") ``` -------------------------------- ### List and Select Google Search Console Web Properties Source: https://joshcarty.github.io/google-searchconsole/getting-started Shows how to list all available web properties associated with your Google Search Console account and how to select a specific web property by its URL or index. ```python # View all properties for property in account.webproperties: print(f"URL: {property.url}, Permission: {property.permission}") # Select by URL webproperty = account['https://www.example.com/'] # Or by index webproperty = account[0] # First property ``` -------------------------------- ### Calculating Date Ranges with Offsets (Python) Source: https://joshcarty.github.io/google-searchconsole/api-reference Shows how to use the `daterange` utility function to calculate a start and end date based on specified offsets (days, months) from a given start or stop date. Returns a tuple of ISO format date strings. ```python from searchconsole.utils import daterange daterange(start=None, stop=None, days=0, months=0) # Last 7 days start, end = daterange('today', days=-7) print(start, end) # ('2024-11-08', '2024-11-15') ``` -------------------------------- ### Filtering Data in Google Search Console API Queries Source: https://joshcarty.github.io/google-searchconsole/getting-started Shows how to apply filters to API queries to narrow down results based on specific criteria. Examples include filtering by queries containing a keyword, matching a specific page URL, and combining multiple filter conditions for precise data extraction. ```python # Filter by query containing "python" report = ( webproperty.query .range('today', days=-7) .dimension('query') .filter('query', 'python', 'contains') .get() ) # Filter by specific page report = ( webproperty.query .range('today', days=-7) .dimension('query') .filter('page', 'https://www.example.com/blog/', 'equals') .get() ) # Multiple filters report = ( webproperty.query .range('today', days=-7) .dimension('query') .filter('page', '/blog/', 'contains') .filter('country', 'usa', 'equals') .get() ) ``` -------------------------------- ### Load Google Credentials from Environment Variable Source: https://joshcarty.github.io/google-searchconsole/authentication Demonstrates how to securely load Google Search Console API credentials from environment variables. This is a recommended security practice for production environments, avoiding hardcoding sensitive information. It requires the 'os' module. ```python import os credentials = os.environ.get('GOOGLE_CREDENTIALS') ``` -------------------------------- ### Analyze Blog Post Performance (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Retrieves data for pages containing '/blog/' within the last 30 days and analyzes their performance. It calculates total clicks, impressions, and identifies the top 10 performing blog posts based on clicks. The output includes extracted titles, URLs, and performance metrics. ```python # Get all pages containing '/blog/' report = ( webproperty.query .range('today', days=-30) .dimension('page') .filter('page', '/blog/', 'contains') .get() ) df = report.to_dataframe() print(f"Total blog posts: {len(df)}") print(f"Total blog clicks: {df['clicks'].sum():,}") print(f"Total blog impressions: {df['impressions'].sum():,}") # Top performing blog posts top_blogs = df.nlargest(10, 'clicks') print("\nTop 10 Blog Posts:\n") for idx, row in top_blogs.iterrows(): # Extract title from URL (customize as needed) title = row['page'].split('/')[-2].replace('-', ' ').title() print(f"{title}") print(f" URL: {row['page']}") print(f" Clicks: {row['clicks']:,} | Impressions: {row['impressions']:,}") print(f" CTR: {row['ctr']:.2%} | Position: {row['position']:.1f}\n") ``` -------------------------------- ### Year-over-Year Performance Comparison (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Compares the performance of the last 30 days to the same 30-day period in the previous year. This helps in understanding year-over-year growth or decline in metrics like clicks and impressions. ```python import datetime # This year (last 30 days) this_year_report = ( webproperty.query .range('today', days=-30) .dimension('date') .get() ) # Last year (same 30-day period) days_ago_365 = datetime.date.today() - datetime.timedelta(days=365) days_ago_395 = days_ago_365 - datetime.timedelta(days=30) last_year_report = ( webproperty.query .range(days_ago_395, days_ago_365) .dimension('date') .get() ) # Note: The code to process and compare 'this_year_report' and 'last_year_report' # would follow here, similar to the month-over-month comparison. ``` -------------------------------- ### Find Underperforming Pages by CTR (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Identifies pages with high impressions (>= 100) but low click-through rates (CTR) within the last 30 days. It sorts pages by CTR in ascending order to find underperformers and suggests potential optimizations. ```python report = ( webproperty.query .range('today', days=-30) .dimension('page') .get() ) df = report.to_dataframe() # Filter for pages with significant impressions significant_pages = df[df['impressions'] >= 100].copy() # Sort by CTR (ascending) to find underperformers underperformers = significant_pages.nsmallest(10, 'ctr') print("Top 10 Underperforming Pages (High Impressions, Low CTR)\n") for idx, row in underperformers.iterrows(): print(f"{row['page']}") print(f" Impressions: {row['impressions']:,} | Clicks: {row['clicks']}") print(f" CTR: {row['ctr']:.2%} | Position: {row['position']:.1f}") print(f" 💡 Opportunity: Improve meta description or title tag\n") ``` -------------------------------- ### Analyze Google Discover Performance Source: https://joshcarty.github.io/google-searchconsole/examples Analyzes Google Discover traffic performance over the last 30 days, focusing on clicks, impressions, and CTR. It also identifies the top 10 pages driving the most clicks within Discover. Note that Discover data does not include position metrics. Requires a 'webproperty' object with a query interface. ```python # Note: Discover doesn't return position data discover_report = ( webproperty.query .search_type('discover') .range('today', days=-30) .dimension('page') .get() ) df = discover_report.to_dataframe() print("Google Discover Performance (Last 30 Days)\n") print(f"Total clicks: {df['clicks'].sum():,}") print(f"Total impressions: {df['impressions'].sum():,}") print(f"Average CTR: {df['ctr'].mean():.2%}") # Top Discover pages top_discover = df.nlargest(10, 'clicks') print("\nTop 10 Pages in Discover:\n") for idx, row in top_discover.iterrows(): print(f"{row['page']}") print(f" Clicks: {row['clicks']:,} | CTR: {row['ctr']:.2%}\n") ``` -------------------------------- ### Executing a Single Query Request Source: https://joshcarty.github.io/google-searchconsole/api-reference Explains the `execute()` method, which runs a query and returns only a single page of results. This is typically used when `get()` is not preferred. ```python query.execute() ``` -------------------------------- ### Analyze Image Search Performance Source: https://joshcarty.github.io/google-searchconsole/examples Analyzes performance specifically for image search over the last 30 days. It retrieves data for image searches, calculates total clicks and impressions, average CTR, and identifies the top 10 performing image pages based on clicks. Requires a 'webproperty' object with a query interface. ```python # Get image search data image_report = ( webproperty.query .search_type('image') .range('today', days=-30) .dimension('page') .get() ) df = image_report.to_dataframe() print("Image Search Performance\n") print(f"Total clicks: {df['clicks'].sum():,}") print(f"Total impressions: {df['impressions'].sum():,}") print(f"Average CTR: {df['ctr'].mean():.2%}") # Top performing image pages top_images = df.nlargest(10, 'clicks') print("\nTop 10 Pages for Image Search:\n") for idx, row in top_images.iterrows(): print(f"{row['page']}") print(f" Clicks: {row['clicks']:,} | Impressions: {row['impressions']:,}\n") ``` -------------------------------- ### Track Specific Keywords Performance (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Monitors the performance of a list of specific keywords over the past 90 days. For each keyword, it retrieves date-level data and calculates total clicks, impressions, and average position. ```python keywords = ['python tutorial', 'learn python', 'python for beginners'] for keyword in keywords: report = ( webproperty.query .range('today', days=-90) .dimension('date') .filter('query', keyword, 'equals') .get() ) df = report.to_dataframe() print(f"\n{keyword.upper()}") print(f"Total clicks: {df['clicks'].sum():,}") print(f"Total impressions: {df['impressions'].sum():,}") print(f"Average position: {df['position'].mean():.1f}") ``` -------------------------------- ### Iterating Through Report Rows (Python) Source: https://joshcarty.github.io/google-searchconsole/api-reference Provides an example of iterating directly over a report object to access each row, similar to iterating over a list. ```python for row in report: print(row.query, row.clicks) ``` -------------------------------- ### Authenticate with Google Search Console API (Initial) Source: https://joshcarty.github.io/google-searchconsole/getting-started Authenticates with the Google Search Console API for the first time using a `client_secrets.json` file. This process is interactive and will open a browser window for user login and permission granting. ```python import searchconsole # First-time authentication (interactive) account = searchconsole.authenticate(client_config='client_secrets.json') ``` -------------------------------- ### Accessing All Report Columns Source: https://joshcarty.github.io/google-searchconsole/api-reference Shows how to get a combined list of all column names (dimensions and metrics) from a `Report` object. This provides a comprehensive view of the data available in the report. ```python report.columns # List[str] ``` ```python print(report.columns) # ['query', 'page', 'clicks', 'impressions', 'ctr', 'position'] ``` -------------------------------- ### Authenticate and Save Google Search Console Credentials Source: https://joshcarty.github.io/google-searchconsole/getting-started Demonstrates how to authenticate with the Google Search Console API and then save the credentials to a file for future use, avoiding the interactive OAuth flow on subsequent runs. ```python # First time: authenticate and save account = searchconsole.authenticate(client_config='client_secrets.json') account.serialize_credentials('credentials.json') # Future runs: load saved credentials account = searchconsole.authenticate(client_config='client_secrets.json', credentials='credentials.json') ``` -------------------------------- ### Compare Performance Across Search Types Source: https://joshcarty.github.io/google-searchconsole/examples Compares performance metrics (clicks, impressions, CTR) across different search types (web, image, video, news) over the last 30 days. It aggregates the data into a pandas DataFrame and displays it as a string. It also generates a bar chart visualizing clicks and impressions by search type. Requires a 'webproperty' object with a query interface. ```python search_types = ['web', 'image', 'video', 'news'] results = [] for search_type in search_types: try: report = ( webproperty.query .search_type(search_type) .range('today', days=-30) .dimension('page') .get() ) df = report.to_dataframe() results.append({ 'search_type': search_type, 'clicks': df['clicks'].sum(), 'impressions': df['impressions'].sum(), 'ctr': df['ctr'].mean() }) except Exception as e: print(f"No data for {search_type}: {e}") # Display comparison import pandas as pd comparison_df = pd.DataFrame(results) print("Search Type Comparison\n") print(comparison_df.to_string(index=False)) # Visualize comparison_df.plot( x='search_type', y=['clicks', 'impressions'], kind='bar', title='Clicks and Impressions by Search Type' ) ``` -------------------------------- ### Limiting Query Results Source: https://joshcarty.github.io/google-searchconsole/api-reference Demonstrates the `limit` method for controlling the number of rows returned. It can be used to fetch a specific number of rows or to paginate through results by specifying a starting position. ```python # First 10 rows query.limit(10) ``` ```python # First 100 rows query.limit(100) ``` ```python # Rows 100-199 (pagination) query.limit(100, 100) ``` ```python # Rows 200-299 query.limit(100, 200) ``` ```python # Large limit (auto-paginated) query.limit(50000) # Will make multiple API calls ``` -------------------------------- ### Multiple Dimension Queries for Google Search Console API Source: https://joshcarty.github.io/google-searchconsole/getting-started Illustrates how to retrieve data segmented by multiple dimensions, such as queries by page, queries by date, and pages broken down by country and device. This allows for a more granular analysis of search traffic and user behavior. ```python # Queries by page report = ( webproperty.query .range('today', days=-7) .dimension('query', 'page') .get() ) # Queries by date report = ( webproperty.query .range('today', days=-30) .dimension('query', 'date') .get() ) # Pages by country and device report = ( webproperty.query .range('today', days=-7) .dimension('page', 'country', 'device') .get() ) ``` -------------------------------- ### Compare Year-over-Year Performance Metrics Source: https://joshcarty.github.io/google-searchconsole/examples Compares key performance metrics (Clicks, Impressions, CTR, Position) between the current year and the last year over the last 30 days. It calculates the percentage change for each metric and prints a formatted summary. Requires 'this_year_report' and 'last_year_report' objects that can be converted to dataframes. ```python this_year_df = this_year_report.to_dataframe() last_year_df = last_year_report.to_dataframe() metrics = { 'Clicks': ('clicks', this_year_df['clicks'].sum(), last_year_df['clicks'].sum()), 'Impressions': ('impressions', this_year_df['impressions'].sum(), last_year_df['impressions'].sum()), 'CTR': ('ctr', this_year_df['ctr'].mean(), last_year_df['ctr'].mean()), 'Position': ('position', this_year_df['position'].mean(), last_year_df['position'].mean()), } print("Year-over-Year Comparison (Last 30 Days)\n") for metric_name, (col, this_year, last_year) in metrics.items(): change = ((this_year - last_year) / last_year * 100) if last_year > 0 else 0 print(f"{metric_name}:") print(f" This Year: {this_year:, .2f}") print(f" Last Year: {last_year:, .2f}") print(f" Change: {change:+.1f}% ") ``` -------------------------------- ### Content Gap Analysis for Queries (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Finds queries where the website ranks in the top 10 positions but has a low click-through rate (CTR), indicating potential content optimization opportunities. It filters for queries with at least 50 impressions and suggests improvements for featured snippets or titles. ```python # Get queries with position in top 10 but low CTR report = ( webproperty.query .range('today', days=-30) .dimension('query') .get() ) df = report.to_dataframe() # Filter for top 10 positions with at least 50 impressions top_10 = df[(df['position'] <= 10) & (df['impressions'] >= 50)].copy() # Sort by CTR to find opportunities opportunities = top_10.nsmallest(20, 'ctr') print("Ranking Opportunities (Top 10 Position, Low CTR)\n") for idx, row in opportunities.iterrows(): print(f"Query: {row['query']}") print(f" Position: {row['position']:.1f} | CTR: {row['ctr']:.2%}") print(f" Clicks: {row['clicks']} | Impressions: {row['impressions']}") print(f" 💡 Consider: Optimizing for featured snippets or improving title\n") ``` -------------------------------- ### Analyze Mobile Traffic from USA for Blog Posts Source: https://joshcarty.github.io/google-searchconsole/examples Analyzes website traffic specifically for mobile users from the USA visiting blog posts over the last 30 days. It calculates total clicks, unique queries, and unique pages, and then identifies the top 10 queries driving the most clicks within this segment. Requires a 'webproperty' object with a query interface. ```python report = ( webproperty.query .range('today', days=-30) .dimension('query', 'page') .filter('page', '/blog/', 'contains') .filter('country', 'usa', 'equals') .filter('device', 'mobile', 'equals') .get() ) df = report.to_dataframe() print("Mobile Traffic (USA) for Blog Posts\n") print(f"Total clicks: {df['clicks'].sum():,}") print(f"Unique queries: {df['query'].nunique()}") print(f"Unique pages: {df['page'].nunique()}") # Top queries for this segment top_queries = df.groupby('query').agg({ 'clicks': 'sum', 'impressions': 'sum' }).nlargest(10, 'clicks') print("\nTop 10 Mobile Queries (USA, Blog):\n") print(top_queries) ``` -------------------------------- ### OAuth2 Authentication using Configuration Dictionaries Source: https://joshcarty.github.io/google-searchconsole/authentication Allows OAuth2 authentication by passing client configuration and credentials directly as Python dictionaries, instead of file paths. This is useful for integrating with secret management systems or environment variables. Requires client_id, client_secret, auth_uri, token_uri, redirect_uris, and token/refresh_token details. ```python client_config = { "installed": { "client_id": "your_client_id.apps.googleusercontent.com", "client_secret": "your_client_secret", "auth_uri": "https://accounts.google.com/o/oauth2/auth", "token_uri": "https://oauth2.googleapis.com/token", "redirect_uris": ["http://localhost"] } } credentials = { "token": "ya29.a0AfH6SMBx...", "refresh_token": "1//0gZ9K8W...", "token_uri": "https://oauth2.googleapis.com/token", "client_id": "your_client_id.apps.googleusercontent.com", "client_secret": "your_client_secret", "scopes": ["https://www.googleapis.com/auth/webmasters.readonly"] } account = searchconsole.authenticate(client_config=client_config, credentials=credentials) ``` -------------------------------- ### Analyze Click-Through Rate by Position Range (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Analyzes click-through rate (CTR) by grouping queries into different position ranges. It fetches query-level data, categorizes it using pandas.cut, and then aggregates metrics like clicks, impressions, mean CTR, and mean position. ```python import pandas as pd # Get query-level data report = ( webproperty.query .range('today', days=-30) .dimension('query') .get() ) df = report.to_dataframe() # Categorize by position df['position_range'] = pd.cut( df['position'], bins=[0, 3, 5, 10, 20, 100], labels=['1-3', '4-5', '6-10', '11-20', '20+'] ) # Analyze by position range analysis = df.groupby('position_range').agg({ 'clicks': 'sum', 'impressions': 'sum', 'ctr': 'mean', 'position': 'mean' }).round(4) print(analysis) ``` -------------------------------- ### Plot Performance Trends Over Time (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Tracks and visualizes website performance trends (clicks, impressions, CTR) over the last 90 days using matplotlib. It fetches daily data, processes it with pandas, and generates plots for each metric. ```python import matplotlib.pyplot as plt # Get daily data for the last 90 days report = ( webproperty.query .range('today', days=-90) .dimension('date') .get() ) df = report.to_dataframe() df['date'] = pd.to_datetime(df['date']) df = df.sort_values('date') # Plot trends fig, axes = plt.subplots(2, 2, figsize=(15, 10)) # Clicks trend axes[0, 0].plot(df['date'], df['clicks'], marker='o') axes[0, 0].set_title('Clicks Over Time') axes[0, 0].set_xlabel('Date') axes[0, 0].set_ylabel('Clicks') # Impressions trend axes[0, 1].plot(df['date'], df['impressions'], marker='o', color='orange') axes[0, 1].set_title('Impressions Over Time') axes[0, 1].set_xlabel('Date') axes[0, 1].set_ylabel('Impressions') # CTR trend axes[1, 0].plot(df['date'], df['ctr'] * 100, marker='o', color='green') axes[1, 0].set_title('CTR Over Time') axes[1, 0].set_xlabel('Date') axes[1, 0].set_ylabel('CTR (%)') ``` -------------------------------- ### Analyze Device Breakdown (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Analyzes website traffic performance broken down by device type (desktop, mobile, tablet) over the last 30 days. It displays clicks, impressions, CTR, and average position for each device, and calculates the click share per device. ```python report = ( webproperty.query .range('today', days=-30) .dimension('device') .get() ) df = report.to_dataframe() print("Performance by Device\n") for _, row in df.iterrows(): print(f"{row['device'].upper()}") print(f" Clicks: {row['clicks']:,}") print(f" Impressions: {row['impressions']:,}") print(f" CTR: {row['ctr']:.2%}") print(f" Avg Position: {row['position']:.1f}\n") # Calculate device share df['device_share'] = (df['clicks'] / df['clicks'].sum() * 100) print("Click Share by Device:") for _, row in df.iterrows(): print(f" {row['device']}: {row['device_share']:.1f}%") ``` -------------------------------- ### Converting Report to List of Dictionaries (Python) Source: https://joshcarty.github.io/google-searchconsole/api-reference Demonstrates the `to_dict()` method for converting a report into a list of dictionaries, where each dictionary represents a row. Includes an example of saving the data to a JSON file. ```python report.to_dict() data = report.to_dict() print(data[0]) # { # 'query': 'python tutorial', # 'clicks': 150, # 'impressions': 2500, # 'ctr': 0.06, # 'position': 5.2 # } # Save to JSON import json with open('data.json', 'w') as f: json.dump(data, f, indent=2) ``` -------------------------------- ### Converting Report to Pandas DataFrame (Python) Source: https://joshcarty.github.io/google-searchconsole/api-reference Shows how to convert a report into a pandas DataFrame using the `to_dataframe()` method. Requires the pandas library. Includes examples of DataFrame operations and saving to CSV or Excel. ```python report.to_dataframe() df = report.to_dataframe() print(df.head()) print(df.info()) print(df.describe()) # Pandas operations print(df['clicks'].sum()) print(df.groupby('country')['clicks'].sum()) # Save to CSV df.to_csv('report.csv', index=False) # Save to Excel df.to_excel('report.xlsx', index=False) ``` -------------------------------- ### Build Search Analytics Queries with WebProperty Source: https://joshcarty.github.io/google-searchconsole/api-reference Initiates the creation of Search Analytics queries for a specific web property. The `query` attribute of a WebProperty object serves as the starting point for defining query parameters like date range and dimensions. ```python report = webproperty.query.range('today', days=-7).dimension('query').get() ``` -------------------------------- ### Find Top Pages by Traffic (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Identifies the top 20 pages driving the most traffic over the last 30 days. The results are converted to a pandas DataFrame for easier sorting and display of page-level click and impression data. ```python report = ( webproperty.query .range('today', days=-30) .dimension('page') .limit(20) .get() ) # Convert to DataFrame for easier analysis df = report.to_dataframe() # Sort by clicks top_pages = df.nlargest(10, 'clicks') print("Top 10 Pages by Clicks\n") for idx, row in top_pages.iterrows(): print(f"{row['page']}") print(f" {row['clicks']:,} clicks, {row['impressions']:,} impressions\n") ``` -------------------------------- ### Python Unit Test Structure Source: https://joshcarty.github.io/google-searchconsole/contributing An example of how to structure unit tests in Python using the `unittest` framework. It demonstrates setting up a test class, individual test methods, and asserting expected outcomes, including exception handling. ```python import unittest from searchconsole import Query class TestNewFeature(unittest.TestCase): def test_feature_works(self): """Test that the new feature works as expected.""" # Setup query = Query(...) # Execute result = query.new_feature() # Assert self.assertEqual(result.something, expected_value) def test_feature_edge_case(self): """Test edge case handling.""" query = Query(...) with self.assertRaises(ValueError): query.new_feature(invalid_input) ``` -------------------------------- ### Saving and Loading OAuth2 Credentials for Search Console API Source: https://joshcarty.github.io/google-searchconsole/authentication Demonstrates how to save OAuth2 credentials after the initial authentication to avoid repeated browser flows in subsequent sessions. It also shows how to load these saved credentials. Requires 'client_secrets.json' and saves to 'credentials.json'. ```python # First authentication - opens browser account = searchconsole.authenticate(client_config='client_secrets.json') # Save credentials for future use account.serialize_credentials('credentials.json') # Load saved credentials - no browser needed account = searchconsole.authenticate( client_config='client_secrets.json', credentials='credentials.json', ) ``` -------------------------------- ### Query Google Search Console Data (Python) Source: https://joshcarty.github.io/google-searchconsole/index Shows how to query data from Google Search Console using a fluent API. It includes examples of specifying date ranges, dimensions, filters, and retrieving data. ```python # Basic query for today and last 7 days, by query dimension report = webproperty.query.range('today', days=-7).dimension('query').get() # Build complex queries with method chaining report = ( webproperty.query .range('today', days=-30) .dimension('query', 'page') .filter('query', 'python', 'contains') .filter('page', '/blog/', 'contains') .limit(1000) .get() ) # Query different search types image_data = webproperty.query.search_type('image').range('today', days=-7).get() discover_data = webproperty.query.search_type('discover').range('today', days=-7).get() video_data = webproperty.query.search_type('video').range('today', days=-7).get() # Apply sophisticated filters # Contains filter report = query.filter('query', 'python', 'contains').get() # Regex filter (RE2 syntax) report = query.filter('page', r'/blog/\d{4}/\d{2}/', 'includingRegex').get() # Multiple filters report = ( query .filter('country', 'usa', 'equals') .filter('device', 'mobile', 'equals') .get() ) ``` -------------------------------- ### Filter Pages Using Regex for Date-Based URLs Source: https://joshcarty.github.io/google-searchconsole/examples Filters pages to find URLs matching a specific date-based pattern (e.g., /blog/YYYY/MM/) over the last 30 days. It then extracts the year and month from these URLs and analyzes the performance (clicks, impressions, number of posts) by publication month. Requires a 'webproperty' object with a query interface and the 're' module for regex operations. ```python # Find URLs matching pattern /blog/YYYY/MM/ report = ( webproperty.query .range('today', days=-30) .dimension('page') .filter('page', r'/blog/\d{4}/\d{2}/', 'includingRegex') .get() ) df = report.to_dataframe() print(f"Found {len(df)} blog posts with date-based URLs\n") # Extract year and month import re def extract_date(url): match = re.search(r'/blog/(\d{4})/(\d{2})/', url) if match: return f"{match.group(1)}-{match.group(2)}" return None df['year_month'] = df['page'].apply(extract_date) # Analyze by year-month monthly_performance = df.groupby('year_month').agg({ 'clicks': 'sum', 'impressions': 'sum', 'page': 'count' }).rename(columns={'page': 'num_posts'}) print("Performance by Publication Month:\n") print(monthly_performance.sort_index(ascending=False)) ``` -------------------------------- ### Plot Average Position Over Time (Python) Source: https://joshcarty.github.io/google-searchconsole/examples Generates a plot showing the average search position over time using matplotlib. It requires a pandas DataFrame with 'date' and 'position' columns. The plot is saved as a PNG file. ```python import matplotlib.pyplot as plt # Assuming 'axes' and 'df' are already defined and populated axes[1, 1].plot(df['date'], df['position'], marker='o', color='red') axes[1, 1].set_title('Average Position Over Time') axes[1, 1].set_xlabel('Date') axes[1, 1].set_ylabel('Position') axes[1, 1].invert_yaxis() # Lower position is better plt.tight_layout() plt.savefig('search_console_trends.png') print("Trend chart saved to search_console_trends.png") ``` -------------------------------- ### Query Google Search Console Data (Last 7 Days) Source: https://joshcarty.github.io/google-searchconsole/getting-started Builds and executes a query to retrieve search query data for the last 7 days, including dimensions like query, clicks, impressions, CTR, and position. Results are then printed. ```python # Build the query query = webproperty.query.range('today', days=-7).dimension('query') # Execute and get results report = query.get() # Print results print(f"Found {len(report)} rows") for row in report: print(f"Query: {row.query}") print(f" Clicks: {row.clicks}") print(f" Impressions: {row.impressions}") print(f" CTR: {row.ctr:.2%}") print(f" Position: {row.position:.1f}") print() ``` -------------------------------- ### Date Range Queries for Google Search Console API Source: https://joshcarty.github.io/google-searchconsole/getting-started Demonstrates how to define various date ranges for API queries, including recent periods (last 7/30 days, last 3 months, yesterday) and specific custom date ranges. These queries are essential for analyzing search performance over different timeframes. ```python # Last 7 days query = webproperty.query.range('today', days=-7) # Last 30 days query = webproperty.query.range('today', days=-30) # Specific date range query = webproperty.query.range('2024-01-01', '2024-01-31') # Yesterday only query = webproperty.query.range('yesterday') # Last 3 months query = webproperty.query.range('today', months=-3) ``` -------------------------------- ### Interactive OAuth2 Authentication with Search Console API Source: https://joshcarty.github.io/google-searchconsole/authentication Initiates an interactive OAuth2 flow for authenticating with the Google Search Console API. It opens a browser for user login and permission granting, returning an authenticated Account object. Requires a 'client_secrets.json' file. ```python import searchconsole # First-time authentication account = searchconsole.authenticate(client_config='client_secrets.json') ``` -------------------------------- ### Console Flow OAuth2 Authentication for Remote Environments Source: https://joshcarty.github.io/google-searchconsole/authentication Enables OAuth2 authentication in environments without direct browser access, such as Google Colab. It prints an authorization URL to the console, requiring manual copy-pasting of the URL and the redirect URI. Requires 'client_secrets.json'. ```python account = searchconsole.authenticate(client_config='client_secrets.json', flow='console') ``` -------------------------------- ### Console Flow Authentication for Google Search Console API Source: https://joshcarty.github.io/google-searchconsole/getting-started Provides a Python code snippet for authenticating with the Google Search Console API using the console flow. This method is useful when the browser doesn't automatically open during the OAuth process, offering an alternative by printing a URL for manual browser access. ```python account = searchconsole.authenticate(client_config='client_secrets.json', flow='console') ``` -------------------------------- ### Filter Internal Pages in Google Search Console (Python) Source: https://joshcarty.github.io/google-searchconsole/examples This Python code snippet uses the Google Search Console client library to retrieve a report of web pages. It filters out pages containing '/admin/', 'staging.', or '/test/' in their URL. The results are then converted to a Pandas DataFrame to display the count of public pages and the sum of clicks. ```python # Get all pages except admin, staging, or test pages report = ( webproperty.query .range('today', days=-30) .dimension('page') .filter('page', '/admin/', 'notContains') .filter('page', 'staging.', 'notContains') .filter('page', '/test/', 'notContains') .get() ) df = report.to_dataframe() print(f"Public pages: {len(df)}") print(f"Total clicks: {df['clicks'].sum():,}") ``` -------------------------------- ### Python Filter Function Example Source: https://joshcarty.github.io/google-searchconsole/contributing An example Python function demonstrating how to filter results by dimension values. It includes type hints, a docstring explaining parameters and return values, and usage examples. ```python def filter(dimension=expression, operator="equals", group_type="and"): """ Filter results by dimension values. Args: dimension: Dimension to filter on (query, page, date, country, device, searchAppearance) expression: Value to filter by operator: Filter operator (equals, contains, notContains, etc.) group_type: How to combine multiple filters (default: "and") Returns: New Query object with filter applied Example: >>> query.filter('country', 'usa', 'equals') >>> query.filter('query', 'python', 'contains') """ # Implementation pass ``` -------------------------------- ### Getting the Number of Rows in a Report (Python) Source: https://joshcarty.github.io/google-searchconsole/api-reference Demonstrates how to get the total number of rows in a report using the built-in `len()` function. ```python num_rows = len(report) print(f"Report has {num_rows} rows") ``` -------------------------------- ### Preview Documentation Locally Source: https://joshcarty.github.io/google-searchconsole/contributing Command to serve the documentation locally using Zensical, allowing for previewing changes before committing. The documentation is built from source files in the `docs/` directory. ```bash zensical serve ``` -------------------------------- ### Building Query Parameters Without Execution Source: https://joshcarty.github.io/google-searchconsole/api-reference Shows how to use the `build()` method to construct the query parameters as a dictionary without actually executing the query. This is useful for inspecting or modifying the query before execution. ```python params = ( webproperty.query .range('today', days=-7) .dimension('query') .filter('page', '/blog/', 'contains') .build() ) print(params) # { # 'startDate': '2024-11-08', # 'endDate': '2024-11-15', # 'dimensions': ['query'], # 'dimensionFilterGroups': [{ # 'filters': [{ # 'dimension': 'page', # 'expression': '/blog/', # 'operator': 'contains' # }] # }], # 'startRow': 0, # 'rowLimit': 25000 # } ``` -------------------------------- ### Query Google Search Console Data Source: https://joshcarty.github.io/google-searchconsole/authentication Fetches a report from Google Search Console for a specified date range. It assumes an authenticated 'account' object and a 'webproperty' object are already available. This method is straightforward for basic data retrieval. ```python webproperty = account['https://www.example.com/'] report = webproperty.query.range('today', days=-7).get() ``` -------------------------------- ### Set Up Test Authentication Credentials Source: https://joshcarty.github.io/google-searchconsole/contributing Environment variables required for running tests, including client configuration, credentials, and web property URI. Alternatively, these can be set in a .env file. ```bash export SEARCHCONSOLE_CLIENT_CONFIG='{"installed": {...}}' export SEARCHCONSOLE_CREDENTIALS='{"token": "...", ...}' export SEARCHCONSOLE_WEBPROPERTY_URI='https://www.example.com/' ``` ```dotenv # .env SEARCHCONSOLE_CLIENT_CONFIG={"installed": {...}} SEARCHCONSOLE_CREDENTIALS={"token": "...", ...} SEARCHCONSOLE_WEBPROPERTY_URI=https://www.example.com/ ```