paper reading-perception, planning, control and coordination for autonomous vehicles

Lidar Perception

  • point cloud based approach, directly use the raw sensor data for further processing. usually applied a voxel-based filtering to reduce the number of points

  • feature based approach, extract parametric features out of the point cloud, and represent the environment usign extracted features (out of date)

  • grid based approach

segmentation algorithms

cluster points into multiple homogeneous groups.

  • edge based method
  • region based method, cluster neighborhood points based on certain criteria
  • model-based / parametric method,
  • graph based method

detection algorithms

categorize each cluster into different objects, the information in each cluster is mainly from spatial relationship and Lidar intensity of the points.

Vision

usually deal with road detection and on-road object detection.

Road Detection

lane mark detection

  • lane line feature extraction
    basically identify the pixels that belong to lane line marks.

  • fitting the pixels into different models

  • estimate the vehicle pose(lateral position and moving orientation) based on the fitted model.

road surface detection

inform the self-driving car on the location of free space where it can drive without collision. usually three ways:

  • feature based detection

first identify the feature points or patches in the original image; based on the identified features, either model fitting or segmentation kind of algorithms will be applied to identify the road surfaces.

  • feature based learning
    first extract a set of features associated to pixels or image patches, then train a classifier based on the features to assign a road or non-road label to the pixels or patches
  • deep learning

on-road object detection

mainly concerns vehicle and pedestrain object classes, and mainly with deep learning based approaches, whose pipeline usually like:

1) the proposal bounding boxes needs to be generated around the input image

2) each proposal box will be sent through the CNN network to determine a classification and fine tune its bounding box location

Fusion

sensor fusion betweeen Lidar and camera is necessary to make the best use of these devices and achive a robust environment perception result.

Localization

the problem of estimating the ego vehicle’s pose, can divided into 2 sub-problems: pose fixing problem and dead reckoning problem.

  • pose fixing problem is to predict a measurement given a pose, e.g. a map.

  • dead reckoning problem, the state is related to the observation by a set of differential equations, and these equations has to be integrated in order to navigate.

map aided localization algorithm use local features to achieve highly precise localization. e.g. SLAM.

a key event in smoothing based SLAM is loop closure, basically when features that have not been seen for a while are observed again from the sensor readings. when a loop closure is detected, the error caused by imperfect odometry can then be removed, and a substantial portion of the map can be updated.

approaches to semantic mapping can be categorized into three ways: object based, appearance based, and activity based.

  • appearance based semantic mapping, interpret sensor readings to construct semantic information of the environment

  • object based semantic mapping, use the occurrence of key objects to build a semantic understanding of the environments, where object recognition and classification is important.

  • activity based semantic mapping, relies on information about the activities of the agents around the ego vehicle.

Planning

mission planning

performed through graph search over a directed graph network, which reflects road/path network connectivity.

behavioral planning

decision making to ensure the vehicle follows any road rules and interacts with other agents in a conventional, safe manner.

motion planning

the process of deciding on a sequence of actions to reach a specified goal, typically based on avoiding colisions on a sequence of actions to reach a specified goal.

  • combinatorial planning, builds a discrete representation of the real environment, and finds a complete solution.
  • sampling-based planning, utilizes a collision checking module to conduct discrete searching over samples drawn from the configuration space, which rely on random sampling of continuous spaces and the generation of a feasible trajectory graph where feasibility is verified through collision checking of nodes and edges to connect these nodes .

planning in dynamic environments

  • decision making structures for obstacle avoidance

    to monitor regions along the intended path for potential obstacle collisions, where these regions are labeled as “critial zones”, and checking against the trajectories of all nearby vehicles to determine a “time to collision”.

  • planning in space-time
  • control space obstacle represnetations

    rather than checking for collisions directly in robot’s configuration space, directly plan in the control space by prohibiting certain control actions which are predicted to lead to collision.

incremental planning and replanning

a means of incrementally generating sub-goals, a new plan is generated as often as a new sub-goal is defined. iteratively replanning to generate new solution trajectories presents a potential opportunity to carry over knowledge from previous planning iterations to subsequent planning iterations.

Control

feedback control

e.g. proportional-integral-derivative(PID) controller, the limitation of feedback-only control, is has delayed response to errors.

model predictive control

trajectory generation

  • combined trajectory generation and tracking

  • separate trajectory generation and tracking

    1) trajectory generation, to find an entire control input, which corresponds to some desired state trajectory. can be dealed as a two point boundary value problem, with a starting state and a final state.

    1.1) sensor based trajectory generation

    1.2) dynamic based trajectory generation

trajectory tracking

  • geometric methods

    pursues a point along the path that is located at a certain lookahead distance away from the vehicle’s current position. the input is waypoints, rather than smooth curves.

  • model based methods

    kinematic model based controllers performs well at low speed applications, but the error increase as the vehicle speed and curvature rate of path increases.

    dynamic model based controllers performs well for higher speed driving applications

summary

this is an overview of self-driving car system. From a job-hunting view, the algorithms details are more important and better with some practial experience.

paper link