UAV Obstacle Avoiding DRL

https://github.com/zyunfeii/uav_obstacle_avoiding_drl
Admin
This is a project about deep reinforcement learning autonomous obstacle avoidance algorithm for UAV.
Tokens:1,965
Snippets:12
Trust Score:9.3
Update:1 year ago
Show doc for...
Tokens:
Raw
### Core Method Utilities (Python)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

This file, 'Method.py', encapsulates various essential methods used throughout the project, particularly focusing on how to calculate the reward for agents in the reinforcement learning setup. It provides foundational utilities for agent training.

```python
# Method.py

import numpy as np

def calculate_reward(agent_state, action, next_state, done, collision):
    """Calculates the reward for an agent's action.

    Args:
        agent_state: Current state of the agent.
        action: The action taken by the agent.
        next_state: The state after taking the action.
        done: Boolean indicating if the episode has terminated.
        collision: Boolean indicating if a collision occurred.

    Returns:
        The calculated reward value.
    """
    reward = 0
    # Reward logic based on distance to goal, collision, etc.
    if collision:
        reward -= 100
    elif done:
        reward += 50 # Reached goal
    else:
        # Reward for progress towards goal, penalize for staying still etc.
        reward += (np.linalg.norm(agent_state[:2] - next_state[:2])) * -1 # Penalize distance moved
        reward += 1 # Small positive reward for existing
    
    return reward

# Other utility methods would be defined here...
```

--------------------------------

### DRL Training Configuration (Python)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

The 'config.py' file centralizes the configuration parameters for the training process of the DRL algorithms. This includes settings like the maximum number of episodes (MAX_EPISODE) and batch size.

```python
# config.py

MAX_EPISODE = 1000
BATCH_SIZE = 64
LEARNING_RATE = 0.001
GAMMA = 0.99
LAMBDA = 0.95
EPS_START = 0.9
EPS_END = 0.05
EPS_DECAY = 200
TARGET_UPDATE_FREQ = 10

# Environment specific parameters
ENV_WIDTH = 100
ENV_HEIGHT = 100
NUM_OBSTACLES = 5
```

--------------------------------

### D Star Algorithm Implementation (C++)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

Provides a C++ implementation of the D star algorithm, a pathfinding algorithm suitable for dynamic environments. This is part of the traditional methods for UAV path planning.

```cpp
// D star algorithm implementation in C++ would be located here.
// Example structure:
// class DStar {
// public:
//   DStar(Grid* grid);
//   Path findPath(Point start, Point goal);
// };
```

--------------------------------

### Static Obstacle Environment Definition (Python)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

Defines the parameters and characteristics of static obstacle environments used for training and testing UAVs. This file contains environment-specific settings relevant to the simulation.

```python
# static_obstacle_environment.py

class StaticEnvironment:
    def __init__(self, width, height, num_obstacles):
        self.width = width
        self.height = height
        self.num_obstacles = num_obstacles
        self.obstacles = self.generate_obstacles()

    def generate_obstacles(self):
        # Logic to generate static obstacles (e.g., positions, sizes)
        obstacles = []
        # ... implementation ...
        return obstacles

    def reset(self):
        # Reset environment state
        pass

    def step(self, action):
        # Execute action and return next state, reward, done
        pass

# Example Usage:
# env = StaticEnvironment(width=100, height=100, num_obstacles=5)
```

--------------------------------

### Reward Curve Drawing Utility (Python)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

A Python script containing a 'Painter' class designed to visualize and draw the reward curves generated during the training process of various DRL methods. This aids in evaluating algorithm performance.

```python
import matplotlib.pyplot as plt
import seaborn as sns

class Painter:
    def __init__(self):
        sns.set_style("darkgrid")

    def draw_reward_curve(self, rewards, title="Reward Curve"):
        plt.figure(figsize=(10, 6))
        plt.plot(rewards)
        plt.xlabel("Episode")
        plt.ylabel("Reward")
        plt.title(title)
        plt.show()

# Example Usage:
# painter = Painter()
# sample_rewards = [i*10 for i in range(100)]
# painter.draw_reward_curve(sample_rewards, "Sample Training Reward")
```

--------------------------------

### Dynamic Obstacle Environment Definition (Python)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

Defines the parameters and characteristics of dynamic obstacle environments for UAV training. This includes settings for moving obstacles and their behavior within the simulation.

```python
# dynamic_obstacle_environment.py

class DynamicEnvironment:
    def __init__(self, width, height, num_dynamic_obstacles):
        self.width = width
        self.height = height
        self.num_dynamic_obstacles = num_dynamic_obstacles
        self.dynamic_obstacles = self.generate_dynamic_obstacles()

    def generate_dynamic_obstacles(self):
        # Logic to generate dynamic obstacles (positions, velocities, patterns)
        dynamic_obstacles = []
        # ... implementation ...
        return dynamic_obstacles

    def reset(self):
        # Reset environment state, including dynamic obstacles
        pass

    def step(self, action):
        # Update dynamic obstacle positions and execute agent action
        # Return next state, reward, done
        pass

# Example Usage:
# env = DynamicEnvironment(width=200, height=200, num_dynamic_obstacles=3)
```

--------------------------------

### Artificial Potential Field Algorithm Implementation (Python)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

Provides Python implementations for the Artificial Potential Field (APF) algorithm in both two-dimensional and three-dimensional environments. This serves as a method for obstacle avoidance planning.

```python
from APF.APFPy2 import APF as APF2D
from APF.APFPy3 import APF as APF3D
from APF.ApfAlgorithm import APFAlgorithm

# Example Usage (Conceptual)
# apf_2d = APF2D()
# apf_3d = APF3D()
# apf_algo = APFAlgorithm()
```

--------------------------------

### Multi-Obstacle Dynamic Environment Test (Python)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

A script specifically designed to test the trained DRL model within a dynamic environment featuring multiple obstacles. It allows for evaluating performance in more complex scenarios.

```python
# Multi_obstacle_environment_test.py

# from dynamic_obstacle_environment import DynamicEnvironment
# from your_drl_model import YourDRLModel

def test_multi_obstacle_environment():
    # env = DynamicEnvironment(width=200, height=200, num_dynamic_obstacles=4)
    # model = YourDRLModel()
    # model.load_pretrained_weights('path/to/weights')
    
    # num_episodes = 10
    # for episode in range(num_episodes):
    #     state = env.reset()
    #     done = False
    #     total_reward = 0
    #     while not done:
    #         action = model.select_action(state)
    #         next_state, reward, done, _ = env.step(action)
    #         total_reward += reward
    #         state = next_state
    #     print(f"Episode {episode+1}: Total Reward = {total_reward}")
    pass

# if __name__ == "__main__":
#     test_multi_obstacle_environment()
```

--------------------------------

### Traditional UAV Path Planning Algorithms (MATLAB)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

Contains MATLAB implementations of traditional UAV path planning algorithms including A* search, Rapidly-exploring Random Trees (RRT), and Ant Colony Optimization (ACO). These are used for performance comparison against DRL methods.

```matlab
% Astarbenchmark.m: Implements A* search algorithm.
% RRTbenchmark.m: Implements RRT algorithm.
% AntColonybenchmark.m: Implements Ant Colony Optimization algorithm.
```

--------------------------------

### Performance Metric Calculation (MATLAB)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

MATLAB scripts for calculating performance indices related to UAV routes. 'calGs.m' calculates the Gs index, and 'calLs.m' calculates the Ls index, both used to assess route efficiency.

```matlab
% calGs.m: Calculates the Gs index for route performance.
% function Gs = calGs(route)
%     % Implementation...
% end

% calLs.m: Calculates the Ls index for route performance.
% function Ls = calLs(route)
%     % Implementation...
% end
```

--------------------------------

### Artificial Potential Field Algorithm Implementation (MATLAB)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

Offers a MATLAB implementation for the Artificial Potential Field (APF) algorithm, specifically for two-dimensional environments. This is used for obstacle avoidance path planning.

```matlab
% Code for APF_matlab folder would go here.
% Example: 
% function [path] = APF_matlab(start, goal, obstacles)
%     % Implementation details...
% end
```

--------------------------------

### IFDS and IIFDS Algorithm Implementation (MATLAB)

Source: https://github.com/zyunfeii/uav_obstacle_avoiding_drl/blob/master/readme.md

This section details the MATLAB realization of the Interfered Fluid Dynamic System (IFDS) and Interfered Interfered Fluid Dynamic System (IIFDS) algorithms. These are flow field-based obstacle avoidance planning algorithms.

```matlab
% Code for IIFDS_and_IFDS folder would go here.
% Example: 
% function [path] = IFDS_IIFDS_matlab(start, goal, environment)
%     % Implementation details...
% end
```
Tokens:
Raw