multirotor - Quickstart

This tutorial shows how to use multirotor to simulate, control, and tune a drone. It has five parts:

  1. A first flight. Create a vehicle, tune a controller, and fly waypoints with logging. This is the example in the README, with explanations.

  2. Motors and a battery. Simulate the electrical side of the same flight.

  3. Trajectories. Fly a longer path and add a wind disturbance.

  4. Environments. Use the vehicle as a Gymnasium environment for reinforcement learning.

  5. Fleets. Simulate and control many vehicles together.

This document can be run as a Jupyter notebook using the jupytext extension. Link to raw markdown file is here.

%matplotlib inline
from pprint import pprint
import numpy as np
np.set_printoptions(precision=3, suppress=True)
import matplotlib.pyplot as plt
plt.rcParams.update({'font.size': 8})

1. A first flight

Create a vehicle

A vehicle is represented by a multirotor.simulation.Multirotor class, which is responsible for the physics calculations. The Multirotor is parametrized by VehicleParams and SimulationParams.

VehicleParams itself is parametrized by PropellerParams for each propeller. A propeller may optionally have MotorParams characterizing the electrical properties. If not, any changes to propeller speeds are instant. Part 2 adds motors. Let’s create a simple vehicle without them:

from multirotor.vehicle import PropellerParams, SimulationParams
pp = PropellerParams(
    k_thrust=0.01,
    k_drag=1e-2,
    moment_of_inertia=1e-3
)
sp = SimulationParams(
    dt=1e-2 # simulation time-step in seconds
)

The vertical thrust in Newtons \(F_z\) is governed by the thrust coefficient. For propeller velocity (radians per second) \(\omega\),

\[ F_z = k_{thrust} \cdot \omega^2 \]

Similarly, the torque produced about the vertical axis of the multirotor due to propeller drag, \(T_z\) is governed by the drag coefficient,

\[ T_z = k_{drag} \cdot \omega^2 \]

multirotor has a convenience function for creating a vehicle from simple geometries:

from multirotor.helpers import vehicle_params_factory
from multirotor.simulation import Multirotor
vp = vehicle_params_factory(
    n=4,          # number of propellers, equally spaced
    m_prop=1e-2,  # mass of propeller / kg
    d_prop=0.3,   # distance of propeller from center of mass /m
    params=pp,    # each propeller's parameters
    m_body=2      # mass of body
)
vehicle = Multirotor(params=vp, simulation=sp)

For other layouts, create VehicleParams directly. It takes the propellers, their angles and distances, the mass and the inertia matrix. multirotor.models has ready-made parameters for two vehicles.

The Multirotor provides a host of methods to manipulate the vehicle. The two main methods are:

  1. step_dynamics. It takes a 6D vector of forces and torques in the body frame of the vehicle, and returns the state.

  2. step_speeds. It takes a nD vector of propeller speeds, and returns the state.

Note: The data type of the vectors should match vehicle.dtype.

The vehicle is described by its .state attribute. It is a 12-dimensional array containing the position, velocity, orientation, and angular velocity. Individual state components can be accessed by their relevant attributes:

# Step using forces/torques
state = vehicle.step_dynamics(np.asarray([1,2,vehicle.weight,0,0,1], vehicle.dtype))
# Step using speeds (rad/s)
state = vehicle.step_speeds(np.asarray([100, 100, 100, 100], dtype=vehicle.dtype))

print('State:\n', state)
print('\nConsists of:')
print(f'{"Position": >32}', vehicle.position)
print(f'{"Velocity (body-frame)": >32}', vehicle.velocity)
print(f'{"Orientation": >32}', vehicle.orientation)
print(f'{"Angular Rate": >32}', vehicle.angular_rate)

print('\nAdditional properties:')
print(f'{"Velocity (inertial-frame)": >32}', vehicle.inertial_velocity)
print(f'{"Acceleration": >32}', vehicle.acceleration)
print(f'{"Euler rate": >32}', vehicle.euler_rate)
State:
 [ 0.     0.     0.009  0.005  0.01   1.863  0.     0.     0.013 -0.
  0.     0.862]

Consists of:
                        Position [0.    0.    0.009]
           Velocity (body-frame) [0.005 0.01  1.863]
                     Orientation [0.    0.    0.013]
                    Angular Rate [-0.     0.     0.862]

Additional properties:
       Velocity (inertial-frame) [0.005 0.01  1.863]
                    Acceleration [  0.008  -0.004 186.268]
                      Euler rate [0.    0.    0.862]

Visualize

multirotor.visualize.VehicleDrawing class is a wrapper around matplotlib. It can interactively visualize the vehicle in 3D. In a notebook, use the %matplotlib widget backend (from ipympl) for an interactive figure. Here, the figure shows the path after 10 seconds:

from multirotor.visualize import VehicleDrawing
vehicle.reset() # resets time/position back to 0
drawing = VehicleDrawing(vehicle, trace=True)

for j in range(1000):
    vehicle.step_dynamics(np.asarray(
        [0.4, 2 * np.sin(j*np.pi/100), vehicle.params.mass*sp.g +  np.cos(j*2*np.pi/1000),
         0,0,0],
        dtype=vehicle.dtype))
    if j % 10 == 0:
        drawing.update()
drawing.axis.set_title(f'pos:{vehicle.position}')
Text(0.5, 0.92, 'pos:[9.804 3.12  0.025]')
_images/d887ee86ecaba524e5d94763b2f3b42c0b09eec421e97cb42faa23cd92cf3ef6.png

Control the vehicle

Controlling a vehicle is another challenge. multirotor provides a multirotor.controller.Controller class, which uses cascaded PID control to navigate. multirotor.optimize.get_controller makes one with default gains:

from multirotor.optimize import get_controller

vehicle.reset()
ctrl = get_controller(vehicle)

A controller has a step() method, which takes the reference position and yaw, and outputs the dynamics needed to achieve that.

\[ F_z,\tau_x,\tau_y,\tau_z = \texttt{step(}x,y,z,\psi\texttt{)} \]

Let’s say we want the vehicle to go up to \(z=10\) and \(x=20\)

dynamics = ctrl.step(reference=[20,0,10,0], persist=False)
print('F_z=%.3f, T_x==%.3f, T_y=%.3f, T_z=%.3f' % (dynamics[0], dynamics[1], dynamics[2], dynamics[3]))
F_z=122.012, T_x==-0.000, T_y=0.196, T_z=0.000

Now that we have the prescribed dynamics, we must convert them into prescribed speeds (radians / s) for the vehicle propellers. That’s where control allocation comes in:

speeds = vehicle.allocate_control(dynamics[0], dynamics[1:4])
print(speeds)
[55.02  55.438 55.438 55.02 ]

And finally, the speeds can be applied to the vehicle:

state = vehicle.step_speeds(speeds)
print(state)
[ 0.    -0.     0.003 -0.     0.     0.5    0.     0.001 -0.     0.
  0.2   -0.   ]

This can then be looped over and over again. DataLog records the vehicle and controller at each step, and plot_datalog plots the recording:

from multirotor.helpers import DataLog
from multirotor.visualize import plot_datalog

vehicle.reset()
ctrl.reset()
log = DataLog(vehicle, ctrl)
for i in range(500):
    dynamics = ctrl.step(reference=[20,0,1,0], persist=True)
    speeds = vehicle.allocate_control(dynamics[0], dynamics[1:4])
    state = vehicle.step_speeds(speeds)
    log.log()
log.done_logging() # converts to numpy arrays
plot_datalog(log, figsize=(6,4));
_images/9fc0fd8686a36f65d35e30b26e2fa9e50965b2f54f941fa9a757422f3a99d073.png

Tune the controller

The PID controller has many tunable parameters. Searching for an adequate parametrization for a vehicle is a complex search problem. multirotor.optimize provides a convenience function called optimize() to search for the best parameters. Parameter search is done using the optuna library, which is installed as a dependency.

Each trial of the search sets the gains of the controller, then flies 60 scenarios from different start positions to the origin. Every trial flies the same scenarios, so trials differ in score only because their gains differ. The start positions are spread over near and far distances. The score is the mean total reward of the scenarios. The search takes about a minute for 100 trials:

from multirotor.optimize import optimize, apply_params
study = optimize(vp, sp, ntrials=100)
print('Best score:', study.best_value)
print('Best parameters:')
pprint(study.best_params, compact=True)
[W 2026-11-05 23:33:12,368] Trial 36 failed with parameters: {'r_pitch_roll.k_p': 55.18976550571697, 'r_pitch_roll.k_i': 0.26673943215960316, 'r_pitch_roll.k_d': 0.3766596952703432, 'r_pitch_roll.max_acceleration': 13.315366522089771, 'r_yaw.k_p': 0.8664790937566078, 'r_yaw.k_i': 0.9728612797268383, 'r_yaw.k_d': 0.2335277092429375, 'r_yaw.max_acceleration': 0.8809528364814145, 'a_pitch_roll.k_p': 14.393974043731749, 'a_pitch_roll.k_i': 0.49844327647632936, 'a_pitch_roll.k_d': 0.5963826655553521, 'a_yaw.k_p': 4.900984771116885, 'a_yaw.k_i': 0.9858272480723522, 'a_yaw.k_d': 0.10346629431871102, 'p.k_p': 22.294568370609305, 'p.k_i': 0.24512760529283506, 'p.k_d': 1.710123360503095, 'v.k_p': 0.9897592180667448, 'v.k_i': 0.39993551483704526, 'v.k_d': 0.6027206994031784, 'z.k_p': 1.342640424021454, 'z.k_i': 0.1062247718594915, 'z.k_d': 0.034737011552851596, 'vz.k_p': 6.881437332014457, 'vz.k_i': 0.5721789277071578, 'vz.k_d': 0.24513414991228968} because of the following error: KeyboardInterrupt().
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/multirotor/envs/latest/lib/python3.13/site-packages/optuna/study/_optimize.py", line 206, in _run_trial
    value_or_values = func(trial)
  File "/home/docs/checkouts/readthedocs.org/user_builds/multirotor/checkouts/latest/multirotor/optimize.py", line 451, in objective
    return run_scenarios(env, trial_ctrl, starts)
  File "/home/docs/checkouts/readthedocs.org/user_builds/multirotor/checkouts/latest/multirotor/optimize.py", line 398, in run_scenarios
    rewards.append(run_sim(env, traj, ctrl).reward.sum())
                   ~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/multirotor/checkouts/latest/multirotor/optimize.py", line 98, in run_sim
    _, r, terminated, truncated, _ = env.step(dynamics)
                                     ~~~~~~~~^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/multirotor/checkouts/latest/multirotor/env.py", line 199, in step
    state = self.state
            ^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/multirotor/checkouts/latest/multirotor/env.py", line 49, in state
    @property
    
KeyboardInterrupt
[W 2026-11-05 23:33:12,375] Trial 36 failed with value None.
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
Cell In[11], line 2
      1 from multirotor.optimize import optimize, apply_params
----> 2 study = optimize(vp, sp, ntrials=100)
      3 print('Best score:', study.best_value)
      4 print('Best parameters:')

File ~/checkouts/readthedocs.org/user_builds/multirotor/checkouts/latest/multirotor/optimize.py:511, in optimize(vp, sp, ctrl, ntrials, args, seed, study_name, n_jobs, timeout, verbosity)
    509 study.set_user_attr('scenarios', scenarios)
    510 train, heldout = make_scenario_suites(args, seed, sp.dtype or np.float32)
--> 511 study.optimize(
    512     make_objective(vp, sp, ctrl=ctrl, args=args, starts=train),
    513     n_trials=ntrials,
    514     show_progress_bar=True,
    515     n_jobs=n_jobs,
    516     timeout=timeout
    517 )
    518 if any(t.state == optuna.trial.TrialState.COMPLETE for t in study.trials):
    519     env = make_env(vp, sp, args)

File ~/checkouts/readthedocs.org/user_builds/multirotor/envs/latest/lib/python3.13/site-packages/optuna/study/study.py:502, in Study.optimize(self, func, n_trials, timeout, n_jobs, catch, callbacks, gc_after_trial, show_progress_bar)
    400 def optimize(
    401     self,
    402     func: ObjectiveFuncType,
   (...)
    409     show_progress_bar: bool = False,
    410 ) -> None:
    411     """Optimize an objective function.
    412 
    413     Optimization is done by choosing a suitable set of hyperparameter values from a given
   (...)
    500             If nested invocation of this method occurs.
    501     """
--> 502     _optimize(
    503         study=self,
    504         func=func,
    505         n_trials=n_trials,
    506         timeout=timeout,
    507         n_jobs=n_jobs,
    508         catch=tuple(catch) if isinstance(catch, Iterable) else (catch,),
    509         callbacks=callbacks,
    510         gc_after_trial=gc_after_trial,
    511         show_progress_bar=show_progress_bar,
    512     )

File ~/checkouts/readthedocs.org/user_builds/multirotor/envs/latest/lib/python3.13/site-packages/optuna/study/_optimize.py:68, in _optimize(study, func, n_trials, timeout, n_jobs, catch, callbacks, gc_after_trial, show_progress_bar)
     66 try:
     67     if n_jobs == 1:
---> 68         _optimize_sequential(
     69             study,
     70             func,
     71             n_trials,
     72             timeout,
     73             catch,
     74             callbacks,
     75             gc_after_trial,
     76             reseed_sampler_rng=False,
     77             time_start=None,
     78             progress_bar=progress_bar,
     79         )
     80     else:
     81         if n_jobs == -1:

File ~/checkouts/readthedocs.org/user_builds/multirotor/envs/latest/lib/python3.13/site-packages/optuna/study/_optimize.py:165, in _optimize_sequential(study, func, n_trials, timeout, catch, callbacks, gc_after_trial, reseed_sampler_rng, time_start, progress_bar)
    162         break
    164 try:
--> 165     frozen_trial_id = _run_trial(study, func, catch)
    166 finally:
    167     # The following line mitigates memory problems that can be occurred in some
    168     # environments (e.g., services that use computing containers such as GitHub Actions).
    169     # Please refer to the following PR for further details:
    170     # https://github.com/optuna/optuna/pull/325.
    171     if gc_after_trial:

File ~/checkouts/readthedocs.org/user_builds/multirotor/envs/latest/lib/python3.13/site-packages/optuna/study/_optimize.py:263, in _run_trial(study, func, catch)
    256         assert False, "Should not reach."
    258 if (
    259     updated_state == TrialState.FAIL
    260     and func_err is not None
    261     and not isinstance(func_err, catch)
    262 ):
--> 263     raise func_err
    264 return trial._trial_id

File ~/checkouts/readthedocs.org/user_builds/multirotor/envs/latest/lib/python3.13/site-packages/optuna/study/_optimize.py:206, in _run_trial(study, func, catch)
    204 with get_heartbeat_thread(trial._trial_id, study._storage):
    205     try:
--> 206         value_or_values = func(trial)
    207     except exceptions.TrialPruned as e:
    208         # TODO(mamu): Handle multi-objective cases.
    209         state = TrialState.PRUNED

File ~/checkouts/readthedocs.org/user_builds/multirotor/checkouts/latest/multirotor/optimize.py:451, in make_objective.<locals>.objective(trial, ctrl)
    449 trial_ctrl = make_trial_controller(env, ctrl, args)
    450 trial_ctrl.set_params(**ctrl_params)
--> 451 return run_scenarios(env, trial_ctrl, starts)

File ~/checkouts/readthedocs.org/user_builds/multirotor/checkouts/latest/multirotor/optimize.py:398, in run_scenarios(env, ctrl, starts)
    396     ctrl.reset()
    397     traj = Trajectory(env.vehicle, np.asarray([[0, 0, 0]]), proximity=0.1)
--> 398     rewards.append(run_sim(env, traj, ctrl).reward.sum())
    399 return float(np.mean(rewards))

File ~/checkouts/readthedocs.org/user_builds/multirotor/checkouts/latest/multirotor/optimize.py:98, in run_sim(env, traj, ctrl)
     96 # Send speeds to environment
     97 dynamics = np.asarray([0,0,*action], env.vehicle.dtype) # (Fx, Fy, Fz, Tx, Ty, Tz)
---> 98 _, r, terminated, truncated, _ = env.step(dynamics)
     99 log.log(reward=r)
    100 if terminated or truncated:

File ~/checkouts/readthedocs.org/user_builds/multirotor/checkouts/latest/multirotor/env.py:199, in DynamicsMultirotorEnv.step(self, action, disturb_forces, disturb_torques)
    197 action[:3] += disturb_forces
    198 action[3:] += disturb_torques
--> 199 state = self.state
    200 nstate = self.vehicle.step_dynamics(u=action)
    201 reward = self.reward(state, action, nstate)

File ~/checkouts/readthedocs.org/user_builds/multirotor/checkouts/latest/multirotor/env.py:49, in BaseMultirotorEnv.state(self)
     40     self.observation_space = gym.spaces.Box(
     41         low=-np.inf,
     42         high=np.inf,
     43         dtype=np.float32,
     44         shape=(12,)
     45     )
     46     self.vehicle = vehicle
---> 49 @property
     50 def state(self) -> np.ndarray:
     51     return self.vehicle.state
     52 @state.setter
     53 def state(self, x: np.ndarray):

KeyboardInterrupt: 

Then, the best parameters can be applied to a controller:

tuned = get_controller(vehicle)
apply_params(tuned, study.best_params);

A score on the scenarios that the search flew is optimistic, because the search chose the gains that scored best on them. At the end, optimize flies the best gains through 150 other scenarios. Compare that score with the default gains on the same scenarios:

from multirotor.optimize import make_env, make_scenario_suites, run_scenarios

_, heldout = make_scenario_suites(seed=0, dtype=vehicle.dtype)
env = make_env(vp, sp)
default = get_controller(env.vehicle)
print('Default gains, held-out score:', round(run_scenarios(env, default, heldout), 1))
print('Tuned gains, held-out score:  ', round(study.user_attrs['heldout_score'], 1))

A reached origin scores about 450 and a crash about −400, so a difference of a few tens is a few more or fewer successful flights. The default gains are a good starting point. The search helps most when the default gains fly your vehicle poorly.

Fly waypoints

multirotor.trajectories.Trajectory takes a list of waypoints and breaks them into smaller segments for the controller. It moves to the next segment when the vehicle is within proximity of the current one. resolution is the length of a segment:

from multirotor.trajectories import Trajectory

vehicle.reset()
tuned.reset()
log = DataLog(vehicle, tuned)
traj = Trajectory(vehicle, points=[[5,0,5], [5,5,5], [0,0,2]], proximity=0.5, resolution=1)
for i, (ref, feed_forward_velocity) in enumerate(traj):
    dynamics = tuned.step(reference=[*ref, 0], feed_forward_velocity=feed_forward_velocity) # yaw=0 in this case
    speeds = vehicle.allocate_control(dynamics[0], dynamics[1:4])
    vehicle.step_speeds(speeds)
    log.log()
    if i == 6000: # stop if a waypoint is not reached
        break
log.done_logging()
print(f'Reached the end after {vehicle.t:.1f} s at {vehicle.position}')
plot_datalog(log, figsize=(6,4));

2. Motors and a battery

So far, propeller speeds change instantly. A real propeller is driven by a motor, and the motor by a battery. Add a MotorParams to each propeller to simulate the motor’s electrical and rotational dynamics. Add a BatteryParams to the vehicle to simulate the battery.

This example uses the parameters of an octorotor with 18-inch propellers from multirotor.models.tarott18, and gives it a 6-cell lithium-polymer battery. The open-circuit voltage of the battery falls with its state of charge, and its terminal voltage is lower by internal_resistance times the current. If the terminal voltage falls below min_voltage, the battery cuts off the motors.

from copy import deepcopy
from dataclasses import replace
from multirotor.models.tarott18 import MP, PP, VP, SP
from multirotor.vehicle import BatteryParams

cells = 6
cell_soc = [0., 0.05, 0.1, 0.2, 0.4, 0.6, 0.8, 0.9, 1.]
cell_ocv = [3.3, 3.55, 3.65, 3.72, 3.78, 3.85, 3.97, 4.06, 4.2]
battery = BatteryParams(
    capacity=16.,              # Ah
    internal_resistance=0.01,  # ohm
    ocv_soc=cell_soc,
    ocv_voltage=cells * np.asarray(cell_ocv),
    min_voltage=cells * 3.5
)
propeller = replace(PP, motor=MP)  # the propeller is driven by this motor
vp_motors = replace(VP, propellers=[deepcopy(propeller) for _ in range(8)], battery=battery)
octo = Multirotor(vp_motors, SP)
print(f'Mass: {vp_motors.mass} kg, weight: {octo.weight:.1f} N')

The actions of step_speeds stay the same: speeds in rad/s. Each motor converts its speed into a voltage using MotorParams.speed_voltage_scaling, and the motor accelerates the propeller. The controller and the allocation do not change. One extra step is needed: spinning up the motors from rest draws a large current, which can sag the battery below min_voltage. So the motors start at hover speed:

octo_ctrl = get_controller(octo)
octo.reset()
octo_ctrl.reset()

hover_speeds = octo.allocate_control(octo.weight, np.zeros(3))
for propeller, speed in zip(octo.propellers, hover_speeds):
    propeller.motor.speed = speed

log = DataLog(octo, octo_ctrl, other_vars=('soc', 'voltage', 'current'))
traj = Trajectory(octo, points=[[0,0,3], [5,0,3], [5,5,3]], proximity=0.5, resolution=1)
for i, (ref, feed_forward_velocity) in enumerate(traj):
    dynamics = octo_ctrl.step(reference=[*ref, 0], feed_forward_velocity=feed_forward_velocity)
    octo.step_speeds(octo.allocate_control(dynamics[0], dynamics[1:4]))
    log.log(soc=octo.battery.soc, voltage=octo.battery.voltage, current=octo.battery_current)
    if i == 6000:
        break
log.done_logging()
print(f'Reached the end after {octo.t:.1f} s at {octo.position}')
print(f'Battery: {100 * octo.battery.soc:.1f} % charge, {float(octo.battery.voltage):.1f} V')

The log has the battery measurements next to the vehicle’s:

fig, axes = plt.subplots(2, 2, figsize=(7, 4.5), sharex=True)
axes[0, 0].plot(log.t, 100 * log.soc)
axes[0, 0].set_ylabel('State of charge /%')
axes[0, 1].plot(log.t, log.voltage)
axes[0, 1].set_ylabel('Battery voltage /V')
axes[1, 0].plot(log.t, log.current)
axes[1, 0].set_ylabel('Battery current /A')
axes[1, 1].plot(log.t, log.z)
axes[1, 1].set_ylabel('Altitude /m')
for ax in axes[1]:
    ax.set_xlabel('Time /s')
fig.tight_layout()

The Detailed Demo flies this vehicle until the battery cuts off.

3. Trajectories

A Trajectory is a sequence of waypoints. Here, the vehicle flies a figure-eight at a height of 3 m. multirotor.trajectories.eight_curve generates the waypoints. A wind disturbance pushes the vehicle: step_speeds takes disturbance forces and torques in the body frame of the vehicle.

from multirotor.coords import direction_cosine_matrix, inertial_to_body
from multirotor.trajectories import eight_curve

def wind(t, vehicle):
    "A gust along x that grows and fades over 20 s. Returns the force in the body frame."
    gust_inertial = np.asarray([2 * np.sin(np.pi * t / 20), 0, 0])
    dcm = direction_cosine_matrix(*vehicle.orientation)
    return inertial_to_body(gust_inertial, dcm)

waypoints = eight_curve(a=8, N=20)
waypoints[:, 2] = 3.
vehicle.reset()
tuned.reset()
log = DataLog(vehicle, tuned)
traj = Trajectory(vehicle, points=waypoints, proximity=1., resolution=2.)
for i, (ref, feed_forward_velocity) in enumerate(traj):
    dynamics = tuned.step(reference=[*ref, 0], feed_forward_velocity=feed_forward_velocity)
    speeds = vehicle.allocate_control(dynamics[0], dynamics[1:4])
    vehicle.step_speeds(speeds, disturb_forces=wind(vehicle.t, vehicle))
    log.log()
    if i == 8000:
        break
log.done_logging()

fig, ax = plt.subplots(figsize=(4, 3.5))
ax.plot(log.x, log.y, label='flown')
ax.plot(waypoints[:, 0], waypoints[:, 1], 'o', ms=3, label='waypoints')
ax.set_xlabel('x /m'); ax.set_ylabel('y /m'); ax.set_aspect('equal'); ax.legend();

The Trajectory class gives the controller one waypoint at a time. multirotor.trajectories.GuidedTrajectory plans smooth S-curve motion between waypoints. It needs the optional py-scurve package. See the Detailed Demo for the other controller and trajectory options.

4. Environments

multirotor.env defines Gymnasium environments for reinforcement learning. The environments, called DynamicsMultirotorEnv and SpeedsMultirotorEnv, take either the dynamics or propeller speeds as actions. The observation is the 12-dimensional state of the vehicle. By default, the reward is for navigating to the origin.

  • reset(seed=..., options=...) starts an episode at a random position inside a bounding box, and returns (observation, info). Use options={'state': x} to start from the state x.

  • step(action) returns (observation, reward, terminated, truncated, info). An episode terminates when the vehicle reaches the origin, leaves the bounding box or tips over. It is truncated when it runs out of time. info says which of reached, out_of_bounds, tipped and out_of_time happened.

from multirotor.env import DynamicsMultirotorEnv

env = DynamicsMultirotorEnv(Multirotor(vp, sp), allocate=True)
observation, info = env.reset(seed=0)
print('Initial position:', env.vehicle.position)
print('Observation space:', env.observation_space)
print('Action space:     ', env.action_space)

With allocate=True, an action is the 6D vector of forces and torques (the force along \(z\) and the three torques matter). The environment allocates propeller speeds for it. Here, the PID controller is the policy. It gets the observation from the vehicle that it was made for:

policy = get_controller(env.vehicle)
total_reward = 0.
for step in range(2000):
    thrust_torques = policy.step(reference=[0, 0, 0, 0]) # go to the origin
    action = np.asarray([0, 0, *thrust_torques], env.vehicle.dtype)
    observation, reward, terminated, truncated, info = env.step(action)
    total_reward += reward
    if terminated or truncated:
        break
print(f'Episode ended after {step + 1} steps with total reward {total_reward:.1f}')
print('Why:', [name for name, happened in info.items() if happened])

A reinforcement learning library calls the same reset() and step() methods with the actions of its own policy. To change the objective, subclass BaseMultirotorEnv or one of its children and override reward(). reward() also sets the flags that end an episode, so call the parent’s method and adjust its result:

class SmoothEnv(DynamicsMultirotorEnv):
    "Penalizes the angular rate of the vehicle."

    def reward(self, state, action, nstate):
        reward = super().reward(state, action, nstate)
        return reward - 0.01 * np.linalg.norm(nstate[9:12])

5. Fleets

multirotor.fleet.Fleet simulates many vehicles together. It holds the parameters and the state of all vehicles in arrays, and steps through all vehicles together. It exposes the same core API as Multirotor: step_speeds(), step_dynamics(), allocate_control(). Every input and output has one row per vehicle. A Fleet is made from a list of VehicleParams. Vehicles must have a homogeneous configuration (the same propeller, motor and battery setup), but parameter values can vary.

from multirotor.fleet import Fleet
from multirotor.optimize import get_fleet_controller

# Eight vehicles whose bodies weigh between 2 and 3.75 kg
fleet_params = [
    vehicle_params_factory(n=4, m_prop=1e-2, d_prop=0.3, params=pp, m_body=2 + 0.25 * k)
    for k in range(8)
]
fleet = Fleet(fleet_params, sp)
print(len(fleet), 'vehicles. State shape:', fleet.state.shape)

FleetController is a batch of controllers, one for each vehicle in a fleet. get_fleet_controller gives each vehicle the default gains. Each vehicle gets its own target, one row per vehicle: \(x, y, z, \psi\). Here the vehicles fly to points on a circle:

fleet_ctrl = get_fleet_controller(fleet)
angles = np.linspace(0, 2 * np.pi, len(fleet), endpoint=False)
targets = np.stack([4 * np.cos(angles), 4 * np.sin(angles), np.full_like(angles, 3), np.zeros_like(angles)], axis=1)

for i in range(1500):
    dynamics = fleet_ctrl.step(targets)                                      # shape (K, 4)
    speeds = fleet.allocate_control(dynamics[:, 0], dynamics[:, 1:4])        # shape (K, n)
    fleet.step_speeds(speeds)

fig, ax = plt.subplots(figsize=(3.5, 3.5))
ax.plot(targets[:, 0], targets[:, 1], 'o', ms=8, mfc='none', label='targets')
ax.plot(fleet.position[:, 0], fleet.position[:, 1], 'x', label='vehicles')
ax.set_aspect('equal'); ax.legend();

The gains (i.e. the controllers’ PID parameters) can differ from vehicle to vehicle. set_params takes the index of the vehicle (and so of its controller) whose gains to change, then the same nested dictionary as Controller.set_params. Here it sets the rate-stage gains of vehicle 3:

fleet_ctrl.set_params(3, ctrl_r=dict(k_p=np.asarray([20., 20., 0.])))
print(fleet_ctrl.get_params(3)['ctrl_r']['k_p'], fleet_ctrl.get_params(2)['ctrl_r']['k_p'])

Note: Fleet simulation can differ from individual vehicle simulation by the solver’s tolerance. All controllers in the fleet must have identical structure and properties (for example square_root_scaling and leashing), but can have their own gains (i.e. PID parameters). A fleet needs quadratic thrust; the blade-element model is not supported. Fleets with motors and batteries work the same way.