Lidar Perception
point cloud based approach, directly use the raw sensor data for further processing. usually applied a voxel-based filtering to reduce the number of points
feature based approach, extract parametric features out of the point cloud, and represent the environment usign extracted features (out of date)
grid based approach
segmentation algorithms
cluster points into multiple homogeneous groups.
- edge based method
- region based method, cluster neighborhood points based on certain criteria
- model-based / parametric method,
- graph based method
detection algorithms
categorize each cluster into different objects, the information in each cluster is mainly from spatial relationship and Lidar intensity of the points.
Vision
usually deal with road detection and on-road object detection.
Road Detection
lane mark detection
lane line feature extraction
basically identify the pixels that belong to lane line marks.fitting the pixels into different models
- estimate the vehicle pose(lateral position and moving orientation) based on the fitted model.
road surface detection
inform the self-driving car on the location of free space where it can drive without collision. usually three ways:
- feature based detection
first identify the feature points or patches in the original image; based on the identified features, either model fitting or segmentation kind of algorithms will be applied to identify the road surfaces.
- feature based learning
first extract a set of features associated to pixels or image patches, then train a classifier based on the features to assign a road or non-road label to the pixels or patches
- deep learning
on-road object detection
mainly concerns vehicle and pedestrain object classes, and mainly with deep learning based approaches, whose pipeline usually like:
1) the proposal bounding boxes needs to be generated around the input image
2) each proposal box will be sent through the CNN network to determine a classification and fine tune its bounding box location
Fusion
sensor fusion betweeen Lidar and camera is necessary to make the best use of these devices and achive a robust environment perception result.
Localization
the problem of estimating the ego vehicle’s pose, can divided into 2 sub-problems: pose fixing problem and dead reckoning problem.
pose fixing problem is to predict a measurement given a pose, e.g. a map.
dead reckoning problem, the state is related to the observation by a set of differential equations, and these equations has to be integrated in order to navigate.
map aided localization algorithm use local features to achieve highly precise localization. e.g. SLAM.
a key event in smoothing based SLAM is loop closure, basically when features that have not been seen for a while are observed again from the sensor readings. when a loop closure is detected, the error caused by imperfect odometry can then be removed, and a substantial portion of the map can be updated.
approaches to semantic mapping can be categorized into three ways: object based, appearance based, and activity based.
appearance based semantic mapping, interpret sensor readings to construct semantic information of the environment
object based semantic mapping, use the occurrence of key objects to build a semantic understanding of the environments, where object recognition and classification is important.
activity based semantic mapping, relies on information about the activities of the agents around the ego vehicle.
Planning
mission planning
performed through graph search over a directed graph network, which reflects road/path network connectivity.
behavioral planning
decision making to ensure the vehicle follows any road rules and interacts with other agents in a conventional, safe manner.
motion planning
the process of deciding on a sequence of actions to reach a specified goal, typically based on avoiding colisions on a sequence of actions to reach a specified goal.
- combinatorial planning, builds a discrete representation of the real environment, and finds a complete solution.
- sampling-based planning, utilizes a collision checking module to conduct discrete searching over samples drawn from the configuration space, which rely on random sampling of continuous spaces and the generation of a feasible trajectory graph where feasibility is verified through collision checking of nodes and edges to connect these nodes .
planning in dynamic environments
decision making structures for obstacle avoidance
to monitor regions along the intended path for potential obstacle collisions, where these regions are labeled as “critial zones”, and checking against the trajectories of all nearby vehicles to determine a “time to collision”.
- planning in space-time
control space obstacle represnetations
rather than checking for collisions directly in robot’s configuration space, directly plan in the control space by prohibiting certain control actions which are predicted to lead to collision.
incremental planning and replanning
a means of incrementally generating sub-goals, a new plan is generated as often as a new sub-goal is defined. iteratively replanning to generate new solution trajectories presents a potential opportunity to carry over knowledge from previous planning iterations to subsequent planning iterations.
Control
feedback control
e.g. proportional-integral-derivative(PID) controller, the limitation of feedback-only control, is has delayed response to errors.
model predictive control
trajectory generation
combined trajectory generation and tracking
separate trajectory generation and tracking
1) trajectory generation, to find an entire control input, which corresponds to some desired state trajectory. can be dealed as a two point boundary value problem, with a starting state and a final state.
1.1) sensor based trajectory generation
1.2) dynamic based trajectory generation
trajectory tracking
geometric methods
pursues a point along the path that is located at a certain lookahead distance away from the vehicle’s current position. the input is waypoints, rather than smooth curves.
model based methods
kinematic model based controllers performs well at low speed applications, but the error increase as the vehicle speed and curvature rate of path increases.
dynamic model based controllers performs well for higher speed driving applications
summary
this is an overview of self-driving car system. From a job-hunting view, the algorithms details are more important and better with some practial experience.