### Start Media Download Example

Source: https://addownloader.readthedocs.io/en/latest/_sources/media_download.rst.txt

Initiates the media download process for a specified number of ads within a project. Requires project name, number of ads, and the data containing ad information.

```python
>>> start_media_download(project_name = "test1", nr_ads = 20, data = data)
```

--------------------------------

### Initialize Media Download Folders

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/media_download.html

Initializes output folders for images and videos if they do not already exist. This setup is crucial before starting any media download operations.

```python
# initialize folders for the images and videos of current category
folder_path_img = f"output/{project_name}/ads_images"
folder_path_vid = f"output/{project_name}/ads_videos"

# check if the folders exist
if not os.path.exists(folder_path_img):
    os.makedirs(folder_path_img)

if not os.path.exists(folder_path_vid):
    os.makedirs(folder_path_vid)
```

--------------------------------

### AdLibAPI.start_download

Source: https://addownloader.readthedocs.io/en/latest/adlib_api.html

Starts the download process from the Meta Ad Library API based on the provided parameters. Returns a pandas DataFrame containing the downloaded ad data.

```APIDOC
## AdLibAPI.start_download Method

### Description
Start the download process from the Meta Ad Library API based on the provided parameters.

### Method Signature
`AdLibAPI.start_download(_params =None_)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
* **params** (_dict_) – The parameters for the API request. Default is None, parameters are retrieved from the created AdLibApi object.

### Returns
A dataframe containing the downloaded and processed ad data from the Meta Online Ad Library.

### Return Type
pandas.Dataframe

### Example
```python
>>> data = ads_api.start_download()
>>> print(data.head(1))
      id ad_delivery_start_time ad_delivery_stop_time  ... unknown_45-54 unknown_55-64 unknown_65+
0  11111             2023-06-30            2024-01-09  ...          21.0           5.0        11.0
```
```

--------------------------------

### Run AdDownloader Analytics Dashboard

Source: https://addownloader.readthedocs.io/en/latest/_sources/index.rst.txt

Start the graphical user interface (GUI) for AdDownloader analytics. This will open a Dash dashboard in your web browser.

```python
from AdDownloader.start_app import start_gui # takes some time to load...
start_gui()
```

--------------------------------

### Accept Cookies Example

Source: https://addownloader.readthedocs.io/en/latest/_sources/media_download.rst.txt

Initializes a Chrome WebDriver, navigates to a given URL, and then accepts cookies using the accept_cookies function. This is typically used at the start of a session.

```python
>>> driver = webdriver.Chrome()
>>> driver.get(data['ad_snapshot_url'][0]) # start from here to accept cookies
>>> accept_cookies(driver)
```

--------------------------------

### Download Media Example

Source: https://addownloader.readthedocs.io/en/latest/_sources/media_download.rst.txt

Downloads media content by finding an element, extracting its source URL, and then calling the download_media function. Ensure the driver is initialized and navigated to the correct URL before execution.

```python
>>> driver.get(data['ad_snapshot_url'][0])
>>> img_element = driver.find_element(By.XPATH, img_xpath)
>>> media_url = img_element.get_attribute('src')
>>> media_type = 'image'
>>> download_media(media_url, media_type, str(data['id'][i]), folder_path_img)
```

--------------------------------

### Start Data Download and Display Head

Source: https://addownloader.readthedocs.io/en/latest/_sources/adlib_api.rst.txt

Initiate the data download process and display the first row of the resulting data. This is useful for quickly inspecting the downloaded data structure.

```python
>>> data = ads_api.start_download()
>>> print(data.head(1))
```

--------------------------------

### Start Media Download Process

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/media_download.html

Initializes the media download process for a project, configuring logging and validating input parameters. It handles cases where no data is provided or the requested number of ads exceeds available data.

```python
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import TimeoutException
import requests
import os
import cv2
from AdDownloader.helpers import configure_logging, close_logger

chrome_opts = Options()
chrome_opts.add_argument("--disable-gpu")
chrome_opts.add_argument("--no-sandbox")
chrome_opts.add_argument("--enable-unsafe-swiftshader")
chrome_opts.add_argument("--log-level=4") # suppress logs
chrome_opts.add_argument("--disable-notifications") 



[docs]
def start_media_download(project_name, nr_ads, data=[]):
    """
    Start media content download for a given project and desired number of ads. 
    The ads media are saved in the output folder with the project_name.

    :param project_name: The name of the current project.
    :type project_name: str
    :param nr_ads: The desired number of ads for which media content should be downloaded.
    :type nr_ads: int
    :param data: A dataframe containing an `ad_snapshot_url` column.
    :type data: pandas.DataFrame
    """

    # configure logger
    logger = configure_logging(project_name)

    # check if data was provided
    if data is None or len(data) == 0:
        logger.error("No data was provided for media download. Please try again.")
        return(print("No data was provided for media download. Please try again."))

    # check if the nr of ads to download is within the length of the data
    if nr_ads > len(data):
        print(f'More ad media requested than available in the data. Downloading the maximum number ({len(data)}).')
        logger.warning(f'More ads requested than available in the data. Downloading the maximum number ({len(data)}).')
        nr_ads = len(data)
        
    print(f"Downloading media content for project {project_name}.")
    logger.info(f'Downloading media content for project {project_name}.')
    nr_ads_processed = 0
    nr_ads_failed = 0
```

--------------------------------

### Generate Line Graph for Ad Start Time

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/analysis.html

Creates a line graph showing total reach or impressions by ad delivery start time. Use 'impressions_avg' for political ads and 'eu_total_reach' for others.

```python
if pol_ads: # political ads
        ad_start_cohort = data.groupby("ad_delivery_start_time")['impressions_avg'].sum().reset_index()
        fig1 = px.line(ad_start_cohort, x='ad_delivery_start_time', y='impressions_avg', 
                    title='Total Average Impressions by Ad Delivery Start Date',
                    labels={'ad_delivery_start_time': 'Ad Campaign Start', 'impressions_avg': 'Total Average Impressions'})

    else: # all ads
        ad_start_cohort = data.groupby("ad_delivery_start_time")['eu_total_reach'].sum().reset_index()
        fig1 = px.line(ad_start_cohort, x='ad_delivery_start_time', y='eu_total_reach', 
                   title='EU Total Reach by Ad Delivery Start Date',
                   labels={'ad_delivery_start_time': 'Ad Campaign Start', 'eu_total_reach': 'EU Total Reach'})
```

--------------------------------

### Start Ad Download with AdLibAPI

Source: https://addownloader.readthedocs.io/en/latest/adlib_api.html

Initiates the download of ad data from the Meta Ad Library API. Call this method without parameters to use default settings.

```python
>>> data = ads_api.start_download()
>>> print(data.head(1))
      id ad_delivery_start_time ad_delivery_stop_time  ... unknown_45-54 unknown_55-64 unknown_65+
0  11111             2023-06-30            2024-01-09  ...          21.0           5.0        11.0


```

--------------------------------

### start_download

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/adlib_api.html

Starts the download process from the Meta Ad Library API based on the provided parameters. It handles fetching data for search terms or page IDs and then processes the downloaded JSON files into a pandas DataFrame.

```APIDOC
## start_download

### Description
Start the download process from the Meta Ad Library API based on the provided parameters. This method fetches data and processes it into a DataFrame.

### Method
`start_download(self, params=None)`

### Parameters
- **params** (dict, optional): The parameters for the API request. If None, parameters are retrieved from the AdLibApi object. Defaults to None.

### Returns
- **pandas.DataFrame**: A dataframe containing the downloaded and processed ad data from the Meta Online Ad Library, or None if no data was downloaded.
```

--------------------------------

### Extract Image Files from Directory

Source: https://addownloader.readthedocs.io/en/latest/_sources/index.rst.txt

Lists image files (jpg, png, jpeg) from a specified directory. This is a common setup step for image analysis.

```python
   images_path = f"output/test2/ads_images"
   image_files = [f for f in os.listdir(images_path) if f.endswith(('jpg', 'png', 'jpeg'))]
```

--------------------------------

### Start Comprehensive Text Analysis

Source: https://addownloader.readthedocs.io/en/latest/_sources/analysis.rst.txt

Initiates a full text analysis pipeline, optionally including topic modeling. This function consolidates multiple analysis steps.

```python
# without topic modeling
tokens, freq_dist, textblb_sent, nltk_sent = start_text_analysis(data)

# with topic modeling
tokens, freq_dist, textblb_sent, nltk_sent, lda_model, topics, coherence_lda, perplexity, log_likelihood, avg_similarity, topics_df = start_text_analysis(data)
# for output see all examples from above
```

--------------------------------

### Start Ad Download Process

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/adlib_api.html

Initiates the download of ad data from the Meta Ad Library API. Handles both search terms and page IDs, processing data in batches for page IDs. Returns a pandas DataFrame of processed ad data or None if no data is downloaded.

```python
def start_download(self, params = None):
        """
        Start the download process from the Meta Ad Library API based on the provided parameters.

        :param params: The parameters for the API request. Default is None, parameters are retrieved from the created AdLibApi object.
        :type params: dict
        :returns: A dataframe containing the downloaded and processed ad data from the Meta Online Ad Library.
        :rtype: pandas.Dataframe
        """

        if params is None:
            params = self.request_parameters

        if params["search_terms"] is not None:
            self.fetch_data(url = self.base_url, params = params, page_number = 1)
        
        if params["search_page_ids"] is not None:
            search_page_ids_list = params["search_page_ids"]
            for i in range(0, len(search_page_ids_list), 10):
                end_index = min(i + 9, len(search_page_ids_list) - 1)
                print(f"Fetching data starting for indexes [{i},{end_index+1}]")
                params["search_page_ids"] = str(search_page_ids_list[i:end_index+1])

                # call the function with the initial API endpoint and parameters
                self.fetch_data(self.base_url, params, page_ids = f"[{i},{end_index}]", page_number = 1)
        
        if not os.path.exists(f"output/{self.project_name}/json"):
            print("JSON files were not downloaded. Try a new request.")
            self.logger.info("JSON files were not downloaded. Try a new request.")
            return None
            
        nr_json_files = len([file for file in os.listdir(f"output/{self.project_name}/json") if file.endswith('.json')])
        print(f"Done downloading {nr_json_files} json files for the given parameters.")
        self.logger.info(f'Done downloading {nr_json_files} json files for the given parameters.')
        print("Data processing will start now.")
        self.logger.info('Data processing will start now.')

        # process into excel files:
        try:
            final_data = transform_data(self.project_name, country = params["ad_reached_countries"], ad_type = params["ad_type"])
            total_ads = len(final_data)
            print(f"Done processing and saving ads data for {total_ads} ads for project {self.project_name}.")
            self.logger.info(f'Done processing and saving ads data for {total_ads} ads for project {self.project_name}.')

            return(final_data)

        except Exception:
            print("No data was downloaded. Please try a new request.")
            self.logger.warning('No data was downloaded. Please try a new request.')
            
        # close the logger
        close_logger(self.logger)
```

--------------------------------

### Get Graphs from Data

Source: https://addownloader.readthedocs.io/en/latest/_sources/analysis.rst.txt

Generates multiple graphs from input data. The resulting figures can be displayed or saved locally.

```python
fig1, fig2, fig3, fig4, fig5, fig6, fig7, fig8, fig9, fig10 = get_graphs(data)
>>> fig1.show() # will open a webpage with the graph, which can also be saved locally
```

--------------------------------

### Manual Download for Political Ads

Source: https://addownloader.readthedocs.io/en/latest/_sources/index.rst.txt

Download political and issue ads data and media using AdLibAPI. This example demonstrates setting specific parameters for political ads, including country, date range, ad type, and audience size.

```python
# the same can be done for POLITICAL_AND_ISSUE_ADS:
plt_ads_api = adlib_api.AdLibAPI(access_token, project_name = "test2")

plt_ads_api.add_parameters(ad_reached_countries = 'US', ad_delivery_date_min = "2023-02-01", ad_delivery_date_max = "2023-03-01", 
                           ad_type = "POLITICAL_AND_ISSUE_ADS", ad_active_status = "ALL", estimated_audience_size_max = 10000, languages = 'es', search_terms = "Biden")
               
# check the parameters
plt_ads_api.get_parameters()

# start the ad data download
plt_data = plt_ads_api.start_download()

# start the media download
start_media_download(project_name = "test2", nr_ads = 20, data = plt_data)
```

--------------------------------

### Extract Frames Example

Source: https://addownloader.readthedocs.io/en/latest/_sources/media_download.rst.txt

Extracts frames from a video file at a specified interval for a given project. Requires the video file path, project name, and the interval in seconds.

```python
>>> extract_frames(video = "test_video.mp4", project_name = "test1", interval = 3)
```

--------------------------------

### Start Text Analysis Function

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/analysis.html

Initiates text analysis, including preprocessing, word frequency calculation, and sentiment analysis. Can optionally perform topic modeling. Ensure the input DataFrame has a column with text data, specified by `column_name`.

```python
def start_text_analysis(text_data, column_name = "ad_creative_bodies", topics = False):
    """
    Perform text analysis including preprocessing, word frequency calculation, sentiment analysis, and topic modeling.
    If `topics = False`, the function will only return the tokens, frequency distribution, word cloud, and text sentiment, otherwise it will additionally return the LDA model, coherence and a dataframe with assigned topics to each ad.

    :param data: A pandas DataFrame containing an `ad_creative_bodies` column with ad captions.
    :type data: pandas.DataFrame
    :param column_name: The name of the column in the Data Frame that contains the text data (default is "ad_creative_bodies").
    :type column_name: str
    :param topics: If True, topic modelling will be performed in addition to the text and sentiment analysis.
    :type topics: bool
    :return: A tuple containing the tokens, word frequency distribution, sentiment scores using TextBlob and Vader, the LDA model and its metrics.
    :rtype: tuple
    """
    try:
        text_data = text_data.dropna(subset = column_name).copy()
    except Exception as e:
        print(f"Error occured when processing the text: {e}.")
    
    try:
        text_data.loc[:, column_name] = text_data[column_name].apply(lambda x: ast.literal_eval(x)[0])
    except:
        pass # don't need to remove the square brackets
    
    tokens = text_data[column_name].apply(preprocess)
    freq_dist = get_word_freq(tokens)

    textblb_sent, nltk_sent = get_sentiment(text_data[column_name])

    if topics:
        lda_model, topics, coherence_lda, perplexity, log_likelihood, avg_similarity, topics_df = get_topics(tokens)
        return tokens, freq_dist, textblb_sent, nltk_sent, lda_model, topics, coherence_lda, perplexity, log_likelihood, avg_similarity, topics_df
    else: 
        return tokens, freq_dist, textblb_sent, nltk_sent
```

--------------------------------

### AdDownloader.cli.intro_messages()

Source: https://addownloader.readthedocs.io/en/latest/cli.html

Displays introductory messages and gathers user input, including a Meta developer access token, for selected tasks. It then proceeds to run the appropriate function based on the user's choice.

```APIDOC
## AdDownloader.cli.intro_messages()

### Description
Display introductory messages and gather user input and Meta developer access token for selected task. After selecting the task, the respective function will be run.

### Method
N/A (Function Call)

### Endpoint
N/A
```

--------------------------------

### AdLibAPI.start_download Method

Source: https://addownloader.readthedocs.io/en/latest/_sources/adlib_api.rst.txt

Initiates the download of ad data based on the configured parameters.

```APIDOC
## AdLibAPI.start_download

### Description
Starts the process of downloading ad data.

### Returns
- **data** (DataFrame) - A pandas DataFrame containing the downloaded ad data.

### Request Example
```python
>>> data = ads_api.start_download()
>>> print(data.head(1))
      id ad_delivery_start_time ad_delivery_stop_time  ... unknown_45-54 unknown_55-64 unknown_65+
0  11111             2023-06-30            2024-01-09  ...          21.0           5.0        11.0
```
```

--------------------------------

### start_media_download Function

Source: https://addownloader.readthedocs.io/en/latest/_sources/media_download.rst.txt

Initiates the media download process for a specified project, downloading a given number of ads. It requires the project name and the number of ads to process.

```APIDOC
## start_media_download Function

### Description
Initiates the media download process for a specified project, downloading a given number of ads. It requires the project name and the number of ads to process.

### Function Signature
start_media_download(project_name: str, nr_ads: int, data: dict)

### Parameters
* **project_name** (str) - The name of the project.
* **nr_ads** (int) - The number of ads to download.
* **data** (dict) - A dictionary containing ad data.

### Example
```python
# Assuming 'data' is defined
start_media_download(project_name = "test1", nr_ads = 20, data = data)
```
```

--------------------------------

### Get API Request Parameters

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/adlib_api.html

Retrieves the parameters currently set for the API request, excluding the access token. Useful for inspecting or reusing request configurations.

```python
def get_parameters(self):
        """
        Get the parameters used for the API request (without the access token).

        :returns: A dictionary containing the parameters for the API request.
        :rtype: dict
        """
        params = self.request_parameters.copy()
        # remove the access_token from the copy
        params.pop("access_token", None)

        return(params)
```

--------------------------------

### Introductory Messages and Task Selection

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/cli.html

Displays welcome messages, prompts for user input including the task and access token, and initiates the corresponding task based on user selection. It handles different task flows, including opening a dashboard.

```python
questions = [
        inquirer3.List(
            "task", 
            message="Welcome to the AdDownloader! Select the task you want to perform: ", 
            choices=['A - download ads data only', 'B - download ads media content only', 'C - download both ads data and media content',
                     'D - open dashboard (using existing data)']
        ),
        inquirer3.Password(
            name="access_token", 
            message='Please provide a valid access token',
            ignore=lambda answers: answers['task'] == 'D - open dashboard (using existing data)',
        ),
        inquirer3.Confirm("start", message="Are you sure you want to proceed?", default=True),
    ]

    answers = inquirer3.prompt(questions, theme=default_style)

    rprint(f"[green bold]You have chosen task {answers['task']}.[green bold]")

    if answers['task'] == 'A - download ads data only':
        rprint("[yellow]Please enter a name for your project. All created folders will use this name:[yellow]")
        project_name = input()
        run_task_A(project_name, answers)
    
    elif answers['task'] == 'B - download ads media content only':
        rprint("[yellow]Please enter the project_name you have ads data for.\n The data needs to be in the output/<project_name>/ads_data folder.[yellow]")
        project_name = input()
        run_task_B(project_name, answers)

    elif answers['task'] == 'C - download both ads data and media content':
        rprint("[yellow]Please enter a name for your project. All created folders will use this name:[yellow]")
        project_name = input()
        run_task_A(project_name, answers)
        run_task_B(project_name, answers)

    elif answers['task'] == 'D - open dashboard (using existing data)':
        rprint("[yellow]The link to open the dashboard will appear below. Click Ctrl+C to close the dashboard.[yellow]")
        from AdDownloader.start_app import start_gui # takes some time to load...
        start_gui()
        

    rprint("[yellow]=============================================[yello]")
    rprint("Finished.")
```

--------------------------------

### Get AdLibAPI Request Parameters

Source: https://addownloader.readthedocs.io/en/latest/adlib_api.html

Retrieves the current parameters configured for API requests, excluding the access token. This is useful for inspecting or modifying request settings.

```python
>>> ads_api.get_parameters()
{'fields': 'id, ad_delivery_start_time, ad_delivery_stop_time, ad_creative_bodies, ad_creative_link_captions, ad_creative_link_descriptions, ad_creative_link_titles, ad_snapshot_url, page_id, page_name, target_ages, target_gender, target_locations, eu_total_reach, age_country_gender_reach_breakdown', 'ad_reached_countries': 'BE', 'search_page_ids': None, 'search_terms': 'pizza', 'ad_delivery_date_min': '2023-09-01', 'ad_delivery_date_max': '2023-09-02', 'limit': '300', 'access_token': 'XX'}

```

--------------------------------

### Get Available Ad Library Fields

Source: https://addownloader.readthedocs.io/en/latest/_sources/adlib_api.rst.txt

Retrieve a string listing all available fields for the Ad Library API. You can optionally specify the ad_type to filter the fields.

```python
>>> ads_api.get_fields(ad_type = "ALL")
'id, ad_delivery_start_time, ad_delivery_stop_time, ad_creative_bodies, ad_creative_link_captions, ad_creative_link_descriptions, ad_creative_link_titles, ad_snapshot_url, 
page_id, page_name, target_ages, target_gender, target_locations, eu_total_reach, age_country_gender_reach_breakdown'
```

--------------------------------

### AdDownloader.cli.run_task_A(_project_name, _answers)

Source: https://addownloader.readthedocs.io/en/latest/cli.html

Executes Task A, which involves downloading ad data from the Meta Online Ad Library using parameters provided by the user.

```APIDOC
## AdDownloader.cli.run_task_A(_project_name, _answers)

### Description
Run task A: Download ads data from the Meta Online Ad Library based on user-provided parameters.

### Method
N/A (Function Call)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Parameters
- **project_name** (str) - The name of the current project.
- **answers** (dict) - User’s answers from the initial questions for tasks A/C regarding desired parameters.
```

--------------------------------

### Get Sentiment Scores for Captions

Source: https://addownloader.readthedocs.io/en/latest/analysis.html

Retrieves sentiment scores for ad captions using both TextBlob and Vader (NLTK). This function requires a pandas Series of captions as input.

```python
>>> textblb_sent, nltk_sent = get_sentiment(data["ad_creative_bodies"])
>>> nltk_sent.head(3)
0     {'neg': 0.0, 'neu': 0.859, 'pos': 0.141, 'comp...
1     {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...
2     {'neg': 0.098, 'neu': 0.644, 'pos': 0.258, 'co...
>>> textblb_sent.head(3)
0     0.125000
1     0.112500
2     0.142857


```

--------------------------------

### AdDownloader CLI: Run Task A - Download Ads Data

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/cli.html

Initiates the download of ads data from the Meta Online Ad Library based on user-specified parameters. It configures the AdLibAPI with access token and project name, sets download parameters, and reports the download duration.

```python
def run_task_A(project_name, answers):
    """
    Run task A: Download ads data from the Meta Online Ad Library based on user-provided parameters.

    :param project_name: The name of the current project.
    :type project_name: str
    :param answers: User's answers from the initial questions for tasks A/C regarding desired parameters.
    :type answers: dict
    """
    ads = AdLibAPI(f"{answers['access_token']}", project_name = project_name)
    # ask for search parameters
    add_answers = request_params_task_AC()
    search_by = add_answers['search_by']
    ad_type = add_answers['ad_type']
    ads.add_parameters(
        ad_reached_countries = add_answers['ad_reached_countries'], 
        ad_delivery_date_min = add_answers['ad_delivery_date_min'], 
        ad_delivery_date_max = add_answers['ad_delivery_date_max'], 
        search_page_ids = add_answers['pages_id_path'] if search_by == 'Pages ID' else None,
        search_terms = add_answers['search_terms'] if search_by == 'Search Terms' else None,
        ad_type = "ALL" if ad_type == 'All' else "POLITICAL_AND_ISSUE_ADS"
    )
        
    rprint("[yellow]Let's check the parameters you provided:[yellow]")
    rprint(f"[green bold]{ads.get_parameters()}.[green bold]")

    rprint("[yellow]Ad data download will begin now.[yellow]")
    start_time = time.time()
    ads.start_download()
    end_time = time.time()
    elapsed_time = end_time - start_time
    minutes = int(elapsed_time // 60)
    seconds = int(elapsed_time % 60)
    print(f"Download finished in {minutes} minutes and {seconds} seconds.")
```

--------------------------------

### Get Sentiment from Captions

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/analysis.html

Retrieves sentiment scores for ad captions using both NLTK's Vader sentiment analyzer and TextBlob. This provides a comprehensive sentiment analysis.

```python
from math import sqrt, ceil
import numpy as np
import pandas as pd
import os
import re
import random
from collections import Counter
from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
from nltk.sentiment import SentimentIntensityAnalyzer
from nltk.stem import WordNetLemmatizer, SnowballStemmer
import nltk
import gensim.corpora as corpora
from gensim.models.coherencemodel import CoherenceModel
from textblob import TextBlob
import plotly.express as px
from PIL import Image
from sklearn.cluster import KMeans
from skimage import color as sk_color
from skimage.feature import canny, corner_harris, corner_peaks
import cv2
from transformers import BlipProcessor, BlipForConditionalGeneration, BlipForQuestionAnswering
from sklearn.decomposition import LatentDirichletAllocation
from sklearn.feature_extraction.text import CountVectorizer
import ast
from itertools import combinations
import logging

#nltk.download('omw-1.4')s
nltk.download('stopwords', quiet=True)
nltk.download('punkt', quiet=True)
nltk.download('vader_lexicon', quiet=True)
nltk.download('wordnet', quiet=True)
# disable gensim logging
logging.getLogger('gensim').setLevel(logging.WARNING)

DATE_MIN = 'ad_delivery_start_time'
DATE_MAX = 'ad_delivery_stop_time'

GENDERS = ['female', 'male', 'unknown']
AGE_RANGES = ['13-17', '18-24', '25-34', '35-44', '45-54', '55-64', '65+']

AD_PATTERN = re.compile(r"ad_([0-9]+)_(img|frame[0-9]+)\.(png|jpeg|jpg)")

stop_words = set(stopwords.words('english')) # need to add more languages



[docs]
def get_sentiment(captions):
    """
    Retrieve the sentiment of the ad captions using two libraries: Vader from NLTK and TextBlob.

    :param captions: A list of ad captions (strings).
    :type captions: list
    :return: A dictionary containing sentiment scores (compound, positive, negative, neutral for Vader; polarity, subjectivity for TextBlob) for each caption.
    :rtype: dict
    """
    sia = SentimentIntensityAnalyzer()
    sentiments = {'vader': [], 'textblob': []}
    for caption in captions:
        try:
            # Vader sentiment analysis
            vader_score = sia.polarity_scores(caption)
            sentiments['vader'].append(vader_score)

            # TextBlob sentiment analysis
            blob = TextBlob(caption)
            textblob_score = {'polarity': blob.sentiment.polarity, 'subjectivity': blob.sentiment.subjectivity}
            sentiments['textblob'].append(textblob_score)
        except Exception as e:
            print(f"Exception {e} occured for caption: {caption}")
            sentiments['vader'].append({'compound': 0, 'neg': 0, 'neu': 1, 'pos': 0})
            sentiments['textblob'].append({'polarity': 0, 'subjectivity': 0})

    return sentiments

```

--------------------------------

### Get Ad Fields

Source: https://addownloader.readthedocs.io/en/latest/adlib_api.html

Retrieves the default fields for an API request for 'ALL' ad types. This method is useful for understanding the available data points for ad retrieval.

```python
>>> ads_api.get_fields(ad_type = "ALL")
'id, ad_delivery_start_time, ad_delivery_stop_time, ad_creative_bodies, ad_creative_link_captions, ad_creative_link_descriptions, ad_creative_link_titles, ad_snapshot_url,
page_id, page_name, target_ages, target_gender, target_locations, eu_total_reach, age_country_gender_reach_breakdown'
```

--------------------------------

### AdDownloader.cli.run_analysis()

Source: https://addownloader.readthedocs.io/en/latest/cli.html

The main function to initiate and control the AdDownloader tool's operation, running continuously until the user decides to stop the analysis.

```APIDOC
## AdDownloader.cli.run_analysis()

### Description
Main function to run the AdDownloader tool until the user stops the analysis.

### Method
N/A (Function Call)

### Endpoint
N/A
```

--------------------------------

### Initialize AdLibAPI

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/adlib_api.html

Initializes the AdLibAPI object with an access token and optional version and project name. Configures logging based on the project name.

```python
from AdDownloader.adlib_api import AdLibAPI

api = AdLibAPI(access_token="YOUR_ACCESS_TOKEN")
```

--------------------------------

### Get Current AdLibAPI Parameters

Source: https://addownloader.readthedocs.io/en/latest/_sources/adlib_api.rst.txt

Retrieve the current set of parameters configured for the Ad Library API request. This includes fields, search terms, dates, and access token.

```python
>>> ads_api.get_parameters()
{'fields': 'id, ad_delivery_start_time, ad_delivery_stop_time, ad_creative_bodies, ad_creative_link_captions, ad_creative_link_descriptions, ad_creative_link_titles, ad_snapshot_url, page_id, page_name, target_ages, target_gender, target_locations, eu_total_reach, age_country_gender_reach_breakdown', 'ad_reached_countries': 'BE', 'search_page_ids': None, 'search_terms': 'pizza', 'ad_delivery_date_min': '2023-09-01', 'ad_delivery_date_max': '2023-09-02', 'limit': '300', 'access_token': 'XX'}
```

--------------------------------

### AdDownloader.cli.run_task_B(_project_name, _answers, _file_name)

Source: https://addownloader.readthedocs.io/en/latest/cli.html

Executes Task B, responsible for downloading ad media content based on user selections. It can utilize a specified Excel file for data or default to 'original_data.xlsx'.

```APIDOC
## AdDownloader.cli.run_task_B(_project_name, _answers, _file_name)

### Description
Run task B: Download ads media content based on user’s choices.

### Method
N/A (Function Call)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Parameters
- **project_name** (str) - The name of the current project.
- **answers** (dict) - User’s answers from the initial questions for tasks A/C regarding desired parameters.
- **file_name** (str) - Name of the Excel file containing ads data. If none is provided data is taken from ‘output/project_name/ads/data/original_data.xlsx’
```

--------------------------------

### Initialize AdLibAPI

Source: https://addownloader.readthedocs.io/en/latest/_sources/adlib_api.rst.txt

Instantiate the AdLibAPI class with your Facebook access token and a project name. Ensure you have a valid access token.

```python
>>> from AdDownloader import adlib_api
>>> access_token = input() # your fb-access-token-here
>>> ads_api = adlib_api.AdLibAPI(access_token, project_name = "test1")
```

--------------------------------

### Perform Text Analysis on Ad Captions

Source: https://addownloader.readthedocs.io/en/latest/_sources/index.rst.txt

Analyzes ad creative bodies to extract keywords, display word clouds, check sentiment, and identify topics using LDA. Requires setup for text analysis libraries.

```python
   # perform text analysis of the ad captions
   tokens, freq_dist, word_cl, textblb_sent, nltk_sent = start_text_analysis(data['ad_creative_bodies'])
   print(f"Most common 10 keywords: {freq_dist.most_common(10)}")

   # show the word cloud
   plt.imshow(word_cl, interpolation='bilinear')
   plt.axis("off")
   plt.show()

   # check the sentiment
   textblb_sent.head(20) # or textblb_sent

   # get the topics
   lda_model, coherence, topics_df = get_topics(tokens, num_topics = 5)

   # print the top 5 words for each topic and the coherence score
   for idx, topic in lda_model.print_topics(num_words=5):
      print("Topic: {} \nWords: {}".format(idx + 1, topic))

   print('Coherence Score:', coherence)
```

--------------------------------

### AdDownloader CLI: Run Task B - Download Ads Media

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/cli.html

Handles the execution of Task B, which involves downloading media content for ads based on user selections. This function is designed to be called with project name and initial user answers.

```python
def run_task_B(project_name, answers):
    """
    Run task B: Download ads media content based on user's choices.

```

--------------------------------

### Load Ad Data from Excel

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/analysis.html

Loads ad data from an Excel file into a pandas DataFrame. It also calculates the campaign duration based on ad delivery start and stop times. Handles potential errors during file loading.

```python
from math import sqrt, ceil
import numpy as np
import pandas as pd
import os
import re
import random
from collections import Counter
from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
from nltk.sentiment import SentimentIntensityAnalyzer
from nltk.stem import WordNetLemmatizer, SnowballStemmer
import nltk
import gensim.corpora as corpora
from gensim.models.coherencemodel import CoherenceModel
from textblob import TextBlob
import plotly.express as px
from PIL import Image
from sklearn.cluster import KMeans
from skimage import color as sk_color
from skimage.feature import canny, corner_harris, corner_peaks
import cv2
from transformers import BlipProcessor, BlipForConditionalGeneration, BlipForQuestionAnswering
from sklearn.decomposition import LatentDirichletAllocation
from sklearn.feature_extraction.text import CountVectorizer
import ast
from itertools import combinations
import logging

#nltk.download('omw-1.4')s
nltk.download('stopwords', quiet=True)
nltk.download('punkt', quiet=True)
nltk.download('vader_lexicon', quiet=True)
nltk.download('wordnet', quiet=True)
# disable gensim logging
logging.getLogger('gensim').setLevel(logging.WARNING)

DATE_MIN = 'ad_delivery_start_time'
DATE_MAX = 'ad_delivery_stop_time'

GENDERS = ['female', 'male', 'unknown']
AGE_RANGES = ['13-17', '18-24', '25-34', '35-44', '45-54', '55-64', '65+']

AD_PATTERN = re.compile(r"ad_([0-9]+)_(img|frame[0-9]+)\.(png|jpeg|jpg)")

stop_words = set(stopwords.words('english')) # need to add more languages



[docs]
def load_data(data_path):
    """
    Load ads data from an Excel file into a dataframe given a valid path.

    :param data_path: A valid path to an Excel file containing ad data.
    :type data_path: str
    :returns: A dataframe containing ads data and an additional campaign duration column.
    :rtype: pandas.DataFrame
    """
    try:
        data = pd.read_excel(data_path)
        data[DATE_MIN] = pd.to_datetime(data[DATE_MIN])
        data[DATE_MAX] = pd.to_datetime(data[DATE_MAX])

        data['campaign_duration'] = np.where(
            data['ad_delivery_stop_time'].isna(),
            (pd.Timestamp('today') - data['ad_delivery_start_time']).dt.days,
            (data['ad_delivery_stop_time'] - data['ad_delivery_start_time']).dt.days
        )
        return data
    except Exception as e:
        print(f"Unable to load data. Error: {e}")
        return None

```

--------------------------------

### Configure Project Logging

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/helpers.html

Sets up a logger that writes messages to a file named 'logs.log' within a project-specific directory. It prevents duplicate handlers and ensures logs are appended.

```python
import logging
import os

def configure_logging(project_name):
    """
    Configures and returns a logger with a file handler set to write logs to a specified project's log file. 
    This function creates a log file named 'logs.log' within a directory named after the `project_name` under the 'output' directory.
    It checks if the logger already has handlers to prevent adding multiple handlers that do the same thing, ensuring 
    that each message is logged only once.

    :param project_name: The name of the project for which logging is being configured.
    :type project_name: str
    :returns: A configured logger object that logs messages to 'output/<project_name>/logs.log'.
    :rtype: logging.Logger
    """

    log_path = f"output/{project_name}"
    if not os.path.exists(log_path):
        os.makedirs(log_path)

    logger = logging.getLogger(project_name)
    logger.propagate = False
    logger.setLevel(logging.INFO)

    log_file = logging.FileHandler(os.path.join(log_path, "logs.log"), 'a')  # append mode
    formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s', datefmt='%d/%m/%Y %H:%M:%S')
    log_file.setFormatter(formatter)

    # check if the logger already has handlers to prevent adding multiple handlers that do the same thing
    if not logger.handlers:
        logger.addHandler(log_file)

    return logger
```

--------------------------------

### Get Ad Fields for API Request

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/adlib_api.html

Returns a string of comma-separated fields for the API request, tailored to the specified ad type ('ALL' or 'POLITICAL'). Consult the Meta Ad Library API documentation for a complete list of available fields.

```python
def get_fields(self, ad_type):
        """
        Get the default fields for the API request, depends on the type of ads to be retrieved (All or Political). For available fields visit https://www.facebook.com/ads/library/api 

        :param ad_type: The type of the ads to be retrieved.
        :type ad_type: str
        :returns: A string containing the fields for the API request.
        :rtype: str
        """
        if ad_type == "ALL":
            return("id, ad_delivery_start_time, ad_delivery_stop_time, ad_creative_bodies, ad_creative_link_captions, ad_creative_link_descriptions, ad_creative_link_titles, ad_snapshot_url, beneficiary_payers, languages, page_id, page_name, target_ages, target_gender, target_locations, eu_total_reach, age_country_gender_reach_breakdown")
        else:
            return("id, ad_delivery_start_time, ad_delivery_stop_time, ad_creative_bodies, ad_creative_link_captions, ad_creative_link_descriptions, ad_creative_link_titles, ad_snapshot_url, bylines, currency, delivery_by_region, demographic_distribution, estimated_audience_size, impressions, languages, spend, page_id, page_name, target_ages, target_gender, target_locations")
```

--------------------------------

### AdDownloader CLI: Request Ad Parameters

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/cli.html

Prompts the user for parameters to download ad data from the Meta Ad Library for tasks A and C. Includes validation for country codes, dates, and Excel file paths.

```python
def request_params_task_AC():
    """
    Prompt user for additional parameters for the API request in tasks A and C that involve ad data download from the Meta Ad Library.

    :returns: User-provided parameters.
    :rtype: dict
    """
    add_questions = [
        inquirer3.List(
            "ad_type", 
            message="What type of ads do you want to search?", 
            choices=['All', 'Political/Elections'],
        ),
        inquirer3.Text(
            "ad_reached_countries",
            message="What reached countries do you want? (Provide the code, default is 'NL')",
            validate=CountryValidator.validate_country,
        ),
        inquirer3.Text(
            "ad_delivery_date_min",
            message="What is the minimum ad delivery date you want? (default is '2023-01-01')",
            validate=DateValidator.validate_date,
        ),
        inquirer3.Text(
            "ad_delivery_date_max",
            message="What is the maximum ad delivery date you want? (default is today\'s date)",
            validate=DateValidator.validate_date,
        ),
        inquirer3.List(
            "search_by", 
            message="Do you want to search by pages ID or by search terms?", 
            choices=['Pages ID', 'Search Terms'],
        ),
        inquirer3.Text(
            "pages_id_path",
            message="Please provide the name of your Excel file with pages ID (needs to be inside the data folder)",
            ignore=lambda answers: answers['search_by'] == 'Search Terms',
            validate=ExcelValidator.validate_excel,
        ),
        inquirer3.Text(
            "search_terms",
            message="Please provide one or more search terms, separated by a comma",
            ignore=lambda answers: answers['search_by'] == 'Pages ID',
        )
    ]

    answers = inquirer3.prompt(add_questions, theme=default_style)
    return answers
```

--------------------------------

### Main Analysis Runner

Source: https://addownloader.readthedocs.io/en/latest/_modules/AdDownloader/cli.html

The main function that orchestrates the AdDownloader tool. It continuously runs the introductory messages and task selection process until the user decides to stop the analysis.

```python
app = typer.Typer() # create the app




@app.command("run-analysis")
def run_analysis():
    """
    Main function to run the AdDownloader tool until the user stops the analysis.
    """
    
    while True:
        intro_messages()

        # ask if the user wants to perform another analysis
        rerun = typer.confirm("Do you want to perform a new analysis?")
        if not rerun:
            rprint("[yellow]=============================================[yello]")
            rprint("[yellow]Analysis completed. Thank you for using AdDownloader! [yello]")
            break




# need this to run the app
if __name__ == "__main__":
    app()
```

--------------------------------

### AdLibAPI Initialization

Source: https://addownloader.readthedocs.io/en/latest/_sources/adlib_api.rst.txt

Initializes the AdLibAPI class with an access token and an optional project name.

```APIDOC
## AdLibAPI.__init__

### Description
Initializes the AdLibAPI class.

### Parameters
- **access_token** (str) - Required - The Facebook access token.
- **project_name** (str) - Optional - The name of the project.

### Request Example
```python
>>> from AdDownloader import adlib_api
>>> access_token = input() # your fb-access-token-here
>>> ads_api = adlib_api.AdLibAPI(access_token, project_name = "test1")
```
```

--------------------------------

### Automated Ad Download via CLI

Source: https://addownloader.readthedocs.io/en/latest/_sources/index.rst.txt

Run the AdDownloader analysis process using the command-line interface. This function automates the entire ad download process.

```python
from AdDownloader.cli import run_analysis
run_analysis()
```

--------------------------------

### AdLibAPI Initialization

Source: https://addownloader.readthedocs.io/en/latest/adlib_api.html

Initializes the AdLibAPI object with an access token and optional version and project name.

```APIDOC
## AdLibAPI.__init__ Method

### Description
Initialize the AdLibAPI object by providing a valid Meta developer token and a project name.

### Parameters
#### Path Parameters
- **access_token** (str) - Required - The access token for authentication.
- **version** (str) - Optional - The version of the Meta Ad Library API. Default is “v18.0”.
- **project_name** (str) - Optional - The name of the project. Default is the current date and time.

### Request Example
```python
>>> from AdDownloader import adlib_api
>>> access_token = input() # your fb-access-token-here
>>> ads_api = adlib_api.AdLibAPI(access_token, project_name = "test1")
```
```

--------------------------------

### Manual Ad Data and Media Download

Source: https://addownloader.readthedocs.io/en/latest/_sources/index.rst.txt

Download ad data and media content programmatically using AdLibAPI and start_media_download. Requires a Meta developer access token. Media download can be initiated immediately or from previously saved data.

```python
from AdDownloader import adlib_api
from AdDownloader.media_download import start_media_download
import pandas as pd

access_token = input() # your fb-access-token-here
# initialize the AdLibAPI object
ads_api = adlib_api.AdLibAPI(access_token, project_name = "test1")

# either search_terms OR search_pages_ids
ads_api.add_parameters(ad_reached_countries = 'BE', ad_delivery_date_min = "2023-09-01", ad_delivery_date_max = "2023-09-02", search_terms = "pizza")

# check the parameters
ads_api.get_parameters()

# start the ad data download
data = ads_api.start_download()

# if you want to download media right away
start_media_download(project_name = "test1", nr_ads = 20, data = data)

# if you want to download media from an earlier project
data_path = 'path/to/your/data.xlsx'
new_data = pd.read_excel(data_path)

start_media_download(project_name = "test2", nr_ads = 20, data = new_data)

# you can find all the output in the 'output/your-project-name' folder
```

--------------------------------

### Configure Logging

Source: https://addownloader.readthedocs.io/en/latest/helpers.html

Sets up a logger to write messages to a project-specific log file. It prevents duplicate handlers to ensure each message is logged only once.

```python
AdDownloader.helpers.configure_logging(_project_name_)
```