Lidar Perception

• point cloud based approach, directly use the raw sensor data for further processing. usually applied a voxel-based filtering to reduce the number of points

• feature based approach, extract parametric features out of the point cloud, and represent the environment usign extracted features (out of date)

• grid based approach

segmentation algorithms

cluster points into multiple homogeneous groups.

• edge based method
• region based method, cluster neighborhood points based on certain criteria
• model-based / parametric method,
• graph based method

detection algorithms

categorize each cluster into different objects, the information in each cluster is mainly from spatial relationship and Lidar intensity of the points.

Vision

usually deal with road detection and on-road object detection.

Road Detection

lane mark detection

• lane line feature extraction
  basically identify the pixels that belong to lane line marks.

• fitting the pixels into different models

• estimate the vehicle pose(lateral position and moving orientation) based on the fitted model.

road surface detection

inform the self-driving car on the location of free space where it can drive without collision. usually three ways:

• feature based detection

first identify the feature points or patches in the original image; based on the identified features, either model fitting or segmentation kind of algorithms will be applied to identify the road surfaces.

• feature based learning
  first extract a set of features associated to pixels or image patches, then train a classifier based on the features to assign a road or non-road label to the pixels or patches

• deep learning

on-road object detection

mainly concerns vehicle and pedestrain object classes, and mainly with deep learning based approaches, whose pipeline usually like:

1) the proposal bounding boxes needs to be generated around the input image

2) each proposal box will be sent through the CNN network to determine a classification and fine tune its bounding box location

Fusion

sensor fusion betweeen Lidar and camera is necessary to make the best use of these devices and achive a robust environment perception result.

Localization

the problem of estimating the ego vehicle’s pose, can divided into 2 sub-problems: pose fixing problem and dead reckoning problem.

• pose fixing problem is to predict a measurement given a pose, e.g. a map.

• dead reckoning problem, the state is related to the observation by a set of differential equations, and these equations has to be integrated in order to navigate.

map aided localization algorithm use local features to achieve highly precise localization. e.g. SLAM.

a key event in smoothing based SLAM is loop closure, basically when features that have not been seen for a while are observed again from the sensor readings. when a loop closure is detected, the error caused by imperfect odometry can then be removed, and a substantial portion of the map can be updated.

approaches to semantic mapping can be categorized into three ways: object based, appearance based, and activity based.

• appearance based semantic mapping, interpret sensor readings to construct semantic information of the environment

• object based semantic mapping, use the occurrence of key objects to build a semantic understanding of the environments, where object recognition and classification is important.

• activity based semantic mapping, relies on information about the activities of the agents around the ego vehicle.

Planning

mission planning

performed through graph search over a directed graph network, which reflects road/path network connectivity.

behavioral planning

decision making to ensure the vehicle follows any road rules and interacts with other agents in a conventional, safe manner.

motion planning

the process of deciding on a sequence of actions to reach a specified goal, typically based on avoiding colisions on a sequence of actions to reach a specified goal.

• combinatorial planning, builds a discrete representation of the real environment, and finds a complete solution.

• sampling-based planning, utilizes a collision checking module to conduct discrete searching over samples drawn from the configuration space, which rely on random sampling of continuous spaces and the generation of a feasible trajectory graph where feasibility is verified through collision checking of nodes and edges to connect these nodes .

planning in dynamic environments

• decision making structures for obstacle avoidance

  to monitor regions along the intended path for potential obstacle collisions, where these regions are labeled as “critial zones”, and checking against the trajectories of all nearby vehicles to determine a “time to collision”.

• planning in space-time

• control space obstacle represnetations

  rather than checking for collisions directly in robot’s configuration space, directly plan in the control space by prohibiting certain control actions which are predicted to lead to collision.

incremental planning and replanning

a means of incrementally generating sub-goals, a new plan is generated as often as a new sub-goal is defined. iteratively replanning to generate new solution trajectories presents a potential opportunity to carry over knowledge from previous planning iterations to subsequent planning iterations.

Control

feedback control

e.g. proportional-integral-derivative(PID) controller, the limitation of feedback-only control, is has delayed response to errors.

model predictive control

trajectory generation

• combined trajectory generation and tracking

• separate trajectory generation and tracking

  1) trajectory generation, to find an entire control input, which corresponds to some desired state trajectory. can be dealed as a two point boundary value problem, with a starting state and a final state.

  1.1) sensor based trajectory generation

  1.2) dynamic based trajectory generation

trajectory tracking

• geometric methods

  pursues a point along the path that is located at a certain lookahead distance away from the vehicle’s current position. the input is waypoints, rather than smooth curves.

• model based methods

  kinematic model based controllers performs well at low speed applications, but the error increase as the vehicle speed and curvature rate of path increases.

  dynamic model based controllers performs well for higher speed driving applications

summary

this is an overview of self-driving car system. From a job-hunting view, the algorithms details are more important and better with some practial experience.

paper link

一些方案

• 基于gps/imu融合的定位
• 基于视觉的定位
• 基于点云的定位
• 基于视觉的物体识别与跟踪
• 基于lidar的物体识别与跟踪

基于点云的定位

简化概率问题：已知t_0 时刻的点云信息，以及t_1时刻，无人车位置的先验概率分布，求无人车位置的概率分布。

贝叶斯法则:

$$ P(X_t) = P(Z_t | X_t ) \cdot \ \vec(P(X_t)) $$

$\vec(P(X_t))$ 是汽车当前位置的概率分布； $P(Z_t | X_t)$ 是当前位置下观察的点云概率分布。

ROS based system

ROS是基于消息传递通信的、分布式多进程框架。Topic发布、接受是一种异步通信方式； Service服务是一种利用同步通信请求/回复交互的分布式系统。

传感器

Lidar

• 环境感知： 通过雷达扫描汽车周围的环境3d模型。运用相关算法比对上一帧和下一帧，从而匹配环境中的其他车辆或行人
• slam定位：实时扫描地图，与高精地图中的特征物比对，实现导航及精准定位。
• 供应商： Velodyne

毫米波雷达

• 可用工作频段 24ghz、77ghz, 波长 1 ～ 10 mm。77ghz物体分辨率较24ghz提高2～4倍，测速和测距精度提高3~5倍。电磁波频率越高，距离和速度的检测解析度越高。

• 供应商

  射频芯片：24ghz成熟供应（博世、飞思卡尔）； 77ghz没有对中国开放

  雷达数据处理芯片：恩智浦，意行半导体

摄像头

• 高动态
• 中低像素
• 适合温度范围 -40 ～ 80 度
• 防磁抗振
• 寿命长

计算平台

• 计算单元与计算负载

  GPU执行卷积任务最有效，DSP执行特征提取最有效。

• 移动端soc架构

  I/O子系统与前端传感器交互；DSP负责图像处理流以进行特征提取；由GPU进行目标识别和其他深度学习任务；由一个多核CPU完成规划、控制和互动的子任务；由FPGA进行动态重构以分时共享的方式完成传感器数据压缩上传，物体跟踪和流量预测等。计算部件和I/O部件之间通过共享内存进行数据通信。

系统安全

安全问题：强磁场干扰IMU；假大功率的gps信号；干扰激光雷达，在无人处周围放置强反光物；干扰高精地图的更新； ros系统劫持、通信修改。obd-2入侵， 充电桩入侵，车载cd入侵，蓝牙入侵。

Spark与ROS的分布式模拟平台

基于合成数据的模拟；基于真实数据回放的模拟。

高精地图

深圳之行

来深圳前只说见见朋友。而工作在这里的年轻人，都在讲述着让我陷入焦虑的故事。对国内的年轻人、行业选择、工作地点有了接近真实的了解。可能是朋友的分布在各个地方不同，深圳的年轻人最明显的感受是干劲儿足，而且对未来充满了希望。没有白走的路。

在美帝的最后一段时间有接触面试：阿里巴巴人工智能实验室、美团无人车、硅谷的自动驾驶创业团队， zoox, cruise, tosimple, drive.ai，当时只是隐约感觉不适应。工作内容、技能要求，都不在我的雷达里。

过去来讲，是选择性忽略行业常识。对互联网人、金融人的自信、高薪比较回避，也因为在国外相对生活无忧、高新安逸的环境。而这些行业领头平台对个人成长的价值，也被忽略了。当用常识去看世界，对互联网平台的成长是非常感兴趣。

实际上，平均2～3年，在华为、腾讯等国内领军互联网平台的年轻人可以独立带项目了。相比，在国外工作的头三年，大部分还在为身份焦虑，或者假期哪里玩而晒朋友圈。

站在快速迭代的行业ceo角度讲，比如互联网、金融行业，都越来越校招，使用应届毕业生，其创造的价值和投入的成本确实更划算。除非有突出和无法复制的核心。所以，工作中一定要保持积累，否则离开了平台，又缺少技能积累，很快失去竞争力。甚至一些“曲线救国”的规划，会抓紧到大平台“偷学”完成，然后迅速跳更期望的位置。

回国的朋友说，国内同龄年轻人，对行业技能的积累，比国外回来的要高一两个层次。记得两年前的一个中午，站在上海张江高科园区，看到黑压压的年轻人涌出办公楼，那一刻的感受是蛮绝望的。当时我兴庆不在国内工作。没意识到危机，就先被危险吓到～

行业分布

在美帝的几年，对美帝的产业分布会有些认识。比如，汽车制造业在密西根；石油能源行业在德州；互联网、智能、医疗等在硅谷湾区、波士顿等。对国内的行业分布，也是这次来深圳才慢慢听到的。之前对地域的认识只有南北方区别。北京是互联网、外企聚集地；上海和周边也是外企聚集地，特别汽车行业的产业链全覆盖；深圳广州从加工制造起身，电子硬件产业链完整，也有腾讯、华为等互联网链条。成都、重庆、武汉也集中一批软件产业和汽车制造业。

为什么一定要到行业top的公司去实践，想必是一个常识。虽然平台是带不走的，这里面的管理文化、优秀的同事、解决的行业问题、做的产品、见到的客户等等都会烙印在个人身上。

为什么一定要去top的行业去实践，也是同样的常识。对于没有明确兴趣和职业方向的年轻人，top的行业是能帮忙迅速完成鉴定的。这就比如年轻人当初一定要去一线大城市，就是想去见识下，不为别的。是骡子是马，在top行业溜一圈就有底了。

产品方向

过去几年在技术岗没有明显成就，所以快30岁就考虑转产品岗位了。这真是一个危险的信号。以前的解读是，反正技术不会干一辈子，总要横向扩展。实际上，相比单纯的技术方向如果不出众，转产品岗只会越走越低。因此，年纪的危机感也加重了。不再像一个初出茅庐的年轻人愿意去挑战，而是求安稳的心态。越缺少专注，越容易被不重要的问题困扰。记得刚回来找工作，我对薪资待遇是念念不忘，而不在能做成什么事。现在要保持一个学习者同时给企业创造价值的心态。

很多朋友了解我不到30岁，都觉得有些问题考虑的过早了。应该继续放手搏一搏。长线来讲，现在还是积累的年纪。选择回避，去一个非技术的岗位、去一个二线城市、甚至换一个不知深浅的行业，都不是明智的。

也不要思考这种弱智问题，诸如，35岁以后干什么呢？ 以前很擅长谈vision, 没有行业积累，不能低下去踏实做事情，会是回国就业的障碍。

阿里巴巴，2018年统计员工9万，其中85%月薪资在20k ～ 50k

腾讯，2017年统计员工人数4万，其中88%月薪资在20k~50k

百度，2017年统计员工人数近4万，其中70%月薪资在20k~50k

滴滴，2019年统计员工人数为1.3万，其中71%月薪资在20k～50k

华为，2017年统计员工人数为17万，其中55%月薪资在20k~50k

国内top行业和行业top公司的薪资情况，half million的样子。回归常识吧。

参考，OEM自动驾驶研发工程师薪资（年）

长城（保定）  30万 + 
长安（重庆）  40万 + 
吉利（杭州）  40万 +
广汽（广州）  35万 + 

公司性质

主机厂、外资供应商、国内领头互联网企业、创业团队，也都有接触到了。从最开始很happy有主机厂的offer，到慢慢听到猎头讲，国内oem的尿性。领头互联网内部也是小股作战，同时又有平台的优势；优秀的创业团队比较吸引资本，都非常棒。相比，外资供应商是体系成熟、个人自由、但核心内容不一定在大陆，所以适合做跳板，迅速完成积累然后跳出。

如果衣食无忧、每天和快活的人在一起，谁还去努力奋斗呢。 – by me

从北京回来写下： 在一线城市的优势就是对政策的敏感，能够提早做布局。相比，三四线更多是跟随和被影响。

上层可以很自然向下扩展，但下层很难上去。不论是产品、概念、服务。比如，提到英语培训机构，会想到 新东方，而不会是 钟祥少儿英语。 新东方从北京发展到武汉市自然而然的；但定位在钟祥的少儿英语，甚至不会逆向到荆门。其他包括互联网服务业、造车新势力等等。

起点决定了发展上升空间。

底层会有很多直接面向终端人群的利弊。比如一个家电品牌在某村的供销商，必然跟当地人很熟。只是在中国只要发展到一定阶段，就会碰到“天花板”。造成这些地方（下层）的生意，处于一个“长不大也饿不死”的阶段。

底层的生意也容易受壁垒保护，毕竟除了概念、体制可以低成本渗透，企业运营管理要做到三四线是相当沉重的。所以，三四线可以肆无忌惮地复制品牌，或者同类小生意林立。比如，服装、餐饮等等。

之前考虑，为什么中国没有像Mc Donald’s, Wendy’s的全国连锁店，只看到了模式复制的成本。其实更深应该是两国体制不同。美国整体运营环境较为平坦；而中国是有上下层的。政治模式决定经济模式，所以，美国的品牌容易横向扩展，标准在哪里都可以建立；而中国的品牌都需要从上到下，游戏规则只会在上层建立。

一些不安分的理想青年，大约不想只参与游戏，被割韭菜，所以才要努力在现有的格局上捅窟窿。昌兄说，有朝一日，中国如美国，都是百年老店了，年轻人也就会逆来顺受，接受命运的安排，兴许人生也更轻松。

在三四线做点小生意，不求闻达于诸侯，苟且偷生不好吗？ 作为一个无产者，犯不上考虑这些问题。作为有产者，如何保证财产安全，就是底层忧虑的根源。对个人产权缺乏保护是任何奋斗且成功的青年都会担心的。而唯一的途径，就是通过更多途径赚钱，积累财富。雪球越滚越大，像一只无形的鞭子。受益者也许是国家，兴百姓苦，亡百姓苦。

决策规划, 根据导航信息及车辆的当前状态，在有限时间内，计算出一条合适的轨迹。

Lattice Planner

• Sample candidate trajectories
• Assign costs
• Select lowest-cost trajectory, to satisfy constraint & collision check in each cycle
• Output

Frenet 坐标

定义沿路面的一条光滑参考线（路面中心线）， 汽车位置坐标由纵向偏移量，横向偏移量表示。纵向坐标，由汽车质心在参考线上的投影点，到参考线起点的长度s表示；横向坐标， 由投影点到汽车质心的距离l表示。

在Frenet坐标下，汽车的朝向、速度、加速度可由横、纵向偏移量的一阶导、二阶导表示。另外，由车辆动力学控制方程可知，横向的偏移量变化率与纵向速度相关，即横向运动是由纵向运动诱发的。

生成轨迹

分别由初始（当前）横向、纵向的状态信息（位置、速度、加速度），和下一时刻的横向、纵向状态信息（位置、速度、加速度），可以得到Frenet坐标下的一个轨迹点。通过一系列的时间点t0，t1, … tn, 可以得到一系列的轨迹点p0, p1, … pn， 即形成一条轨迹。

轨迹集合采样

轨迹采样，本质上是通过在解空间随机布点，然后贪婪搜索，筛选最低cost function value的点，即为可行轨迹。

横向偏移量由动力学方程可由纵向偏移量唯一决定，即从控制方程上，每个scenario可以变成由纵向偏移量当唯一自变量的控制方程，从而保证有唯一最优解或者无解。但实际上，由纵向偏移量唯一表示的控制方程，需要满足不同scenario的约束条件，即一个带约束条件的优化问题。随机算法是一个鲁棒性很强的方法，而且大部分情况下，至少可以得到一个可行解。apollo中横向轨迹、纵向轨迹分别采样。

apollo的横向轨迹的采样，由内部自定义三种横向偏移量： -0.5， 0， 0.5； 同时，设计到达这些横向偏移量的纵向位移： 10， 20， 40， 80. 两层循环即可得到一个轨迹集合。由此定义的横向轨迹不一定符合运动学、动力学约束。

apollo的纵向轨迹的采样，考虑巡航、跟车、超车、停车等scenario。

比如，巡航场景，通过两层循环采样。外层循环将速度从零到速度上限按等间距均匀遍历，内层循环由1s-8s均匀遍历。对于停车的场景，末状态的速度、加速度是0.

ST-Graph

横坐标是时刻，纵坐标是障碍车的车头、车尾的位置。st图将描绘障碍车进入当前车道到离开当前车道所占据的位置区域。自驾车必须确保与障碍车所占据位置有重合。跟车，即在被占区域的下方，或超车，即在被占区域的上方。

由跟车或超车，即可采样各个时刻的纵向偏移量，然后同理横向偏移量采样，形成纵向轨迹集合。

轨迹cost

轨迹规划的约束条件： 达到目的、 横向偏移代价（尽量沿着道路中心行驶）、碰撞代价、符合交规、平稳舒适（横向加速度代价）、 纵向加速度代价（激烈加速）、向心加速度代价（转弯场景）。

对于换道场景，对当前车道、目标车道的参考线做一次采样，并找到最低cost的轨迹。可以设计换道cost 。

EM planner

expectation maximum(EM) ， 最大期望算法，在概率模型中寻找参数最大似然估计或最大后验估计，其概率模型依赖于无法观测的隐性变量。

首先，计算期望（e)， 利用对隐藏变量的现有估计值，计算其最大似然估计值；然后，最大值m， 最大化在e步上求的的最大似然值来计算参数的值。step-2 的参数被用于下一个 step-1 的计算。

em planner 会迭代对路径path、速度velocity进行优化。基于当前步的path和对其他运动物体的预测，优化当前步的速度；然后基于这个新的速度，再优化path。迭代直到收敛。

EM Planner 相比 Lattice Planner，利用了更多当前时间步的状态信息，在每个local timestep 都得到一个可行解。一系列的局部解，相当于弱形式，如果解空间存在可行解，局部解的迭代最终也将给出系统的可行解

1
2
3
4
5

bool Plan(const common::vehicle_state::VehicleState &vehicle_state, 
const bool is_on_auto_mode,
const double publish_time, 
std::vector<common::TrajectoryPoint>

纵向控制

通过控制刹车、油门实现对车速的控制。由一个级联控制器和标定表构成。级联控制器包括：位置PID闭环控制器， 速度PID闭环控制器。标定表即速度-加速度-刹车-油门命令 标定表。

1
2
3
4
5
6

Status LonController::ComputeControlCommand(
	const localization::LocalizationEstimate *localization,  
	const canbus::Chassis *chassis, 
	const planning::ADCTrajectory *planning_published_trajectory,
	control::ControlCommand *cmd)

其输出是油门\刹车命令cmd.

位置PID闭环控制器

输入变量：期望位置 + 当前实际位置

输出变量：速度补偿量

速度PID闭环控制器

输入变量：速度补偿 + 当前位置-速度偏差

输出变量：加速度补偿量

速度-加速度-刹车/油门命令 标定表

输入变量： 加速度补偿量 + 规划驾速度， 车速

输出变量： 油门／刹车控制量

1
2
3
4

void LonController::ComputeLongitudinalErrors(
	const TrajectoryAnalyzer *trajectory_analyzer, 
	const double preview_time ,
	SimpleLongitudinalDebug *debug)

CarSim/Simulink 与 apollo 连结 ？

横向控制

通过调节方向盘转角实现对航向的控制，由一个前馈控制器和反馈控制器组合，实现车辆动力学模型和lqr 模型。

1
2
3
4
5

Status LatController::ComputeControlCommand(
	const localization::LocalizationEstimate *localization,
	const canbus::Chassis *chassis,
	const planning::ADCTrajectory *planning_published_trajectory,
	control::ControlCommand *cmd)

前馈控制器

输入变量：道路曲率

输出变量：方向盘前馈控制量

前馈控制变量实现补偿道路曲率对稳态误差的影响。

反馈控制器

输入变量： 期望航向角

输出变量： 方向盘反馈控制量

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

// update status matrix 
void LatController::UpdateStateAnalyticalMatching(
	SimpleLateralDebug *debug)
// cal lat error 
double LatController::ComputeLateralErrors(
	const double x, 
	const double y,
	const double theta,
	const double linear_v ,
	const double angular_v, 
	const TrajectoryAnalyzer &trajectory_analyzer,
	SimpleLateralDebug *debug)
void LatController::UpdateMatrix() 
void common::math::SolveLQRProblem()  // gain matrix 
steer_angle =  steer_angle_feedback + steer_angle_feedforward

控制模块更新cmd 后，发送给canbus模块。

MPC 模型

mpc的实现依赖于过程的动态模型。对时域内，每个当前时刻进行优化，求取每个时刻的最优控制解，从而得到整个时域的优化解。

• 基于线性化的预测

实际系统状态-控制方程（系统状态变量、控制变量）具有时域非线性（二次项等）。首先对该系统进行线性近似，从而可以实现通过当前状态变量、控制变量对下一个时刻系统状态和控制的预测。

• 滚动优化

设计符合约束条件的目标函数，为状态、控制的离散的能量二次方程。优化的目标就是在每个时刻，寻找最优控制变量，使得目标函数最小。MPC相当于给出每个时刻的局部最优控制值，整个时域的最优控制就是一个控制序列。

• 控制更新

每个优化的输出即每个时步的控制增量，用于更新当下时步的系统控制变量。

LQR 模型

lqr 给出时域系统的全局最优解， 其目标函数是积分函数。相比较，lqr 相当于加权伽辽金方法，给出弹性体的全局最优解，对应强形式的控制方程，解空间非常狭小，甚至不存在。而mpc 相当于有限元方法，给出弹性体每个单元的局部解，对应弱形式的控制方程，每个单元的解空间span可张。理论上弱形式在极限情况给出的解就是强形式的解。

数值方法上，lqr相当于直接法，mpc相当于迭代法。直接法给出控制方程的唯一真实解；迭代法给出近似解，但近似解可收敛到系统的真实解。

车辆模型

运动学模型给出车辆纵向控制变量，即车辆模型刚体质心参考点及航向角的控制方程；动力学模型给出车辆横向控制变量。

由运动学和动力学模型给出了整车状态-控制方程，采用mpc or lqr 算法求解控制变量，即实现车辆控制。

自动驾驶供应链中，ai算法、芯片、传感器（毫米波、激光、相机）等创业公司不计，传统主机厂、供应商内部相关团队不计。独立的小股团队也不计。直接把自动驾驶解决方案当作团队vision的有将近40家公司，分别来自电子芯片、人工智能、传统主机厂等不同背景。

每年估计烧十几个亿，市场还是有很多热情。2019年也许会死掉很多。另外，对市场时机需要敏感。几个海龟团队都是2018年才开始的。

apollo self-driving car

HD maps

where to define the center-meter fidelity level maps, with landmarks, and even the height dimension as well, which is used as the guiding map and also the global coordinate to locate the vehicle in the world.

how to keep HD maps updated is a big invest. Baidu has hundreds service cars in China to collect the natural highway HD map data, even still to 2020.

Localization

SLAM tech is a robot maping the world at the same time localizing itself in the world. with HD maps prepared first, self-driving car only need localize itself at every timestep. and the common idea is comparing a few local landmarks with the corresponding global landmarks, then transfer the local vehicle position to its global position in HD map.

LiDAR localization

either based on the cloud points matching from continous timestamps; or calculating the error between the LiDAR points with the HD maps points; or based on the Karman filter, which give the highest possibility of the location of vehicle

always accessible but not easy to construct, especially requiring HD maps.

Visual localization

based on particle filters, which give the most likely location of the vehicle.

Perception

detection && classification

where the object located, and classify it. detection CNN to find the object in the image; then use classification CNN to classify it. or use a combined CNN to detect and classify at same time.

tracking

tracking helps when detection failed; also tracking helps to identity the object, so when objects overlapped in the image, still can tell which is which.

Prediction

model based prediction

data driven prediction

after prediction, then generate the trajectory.

Planning

transfer the world to graph, and find routings in the graph. routing is a global path, comparing to trajectory is the local way.

Vehicle control

linear quadratic regulator

PID

model predictive control

things changes faster than I expected, already in China for two weeks. The first few days with family, that was precious, and the last spring festival at home was 5 years ago.

life in China

I am searching jobs in China now, the recruiters are highly efficient. almost in 24hrs, a reply return; and they prefer wechat rather than email, Liepin ranther LinkedIn. on the other hand, I went to tongji hosptial at Wuhan for a doctor appoitment, but stay in the line waiting most time. People are everywhere, even in a third-class city, like my hometown, I can only find two caffe shop to sit down and use the free wifi; the skyline buildings are everywhere, the evening scene feel like in the movie: Matrix.

in USA, I never touch the real China, which is attracting but concerned. As many of my friends in USA, we always talking about China, but never take a move back to China. Till the moment I stand in China, things are real.

In general, if people looking for a stable middle-class, USA definitely is a better choice, with few career pressure, 8 hrs a day, 20 days PTO a year, large house with green grass, and income is about 2 times highly as the same engineering in China, without considering 996 in China.

people back to China must think big, rather than looking for a stable income.

self-driving in China

OEM level: Changcheng, Audi, Volkswagen, Geely, xiaopeng, Beiqi, and many names I few heared before, are investing in new energy, intelligent vehicles, and self-driving cars and expending, the market sounds high to hell. there are about 500 start-ups focusing on component supply chain, e.g. in-vehicle sensors, network communication infrastructure, platform solutions, AI algorithms to chips design, specilized scenario applications(mining, seaport, airport, logistics, warehousing, city bus, city-cleaners)

it was the best of times, it was the worst of times. many start-up companies will die so sure, but the market is the training course for next BIG.

The second half of 2018 is my time to feel everything is accelerated.

when understanding planning in self-driving car, all other modules make sense.

perception layer

with all concepts such as lidar, radar, camera, sensors, GPS, CAN, data fusion, computer vision, SLAM, AI-based detect/tracking algorithms, they are all perception related, which helps the car to understand itself and the surroundings.

perception first helps the car understand itself located in the world; then helps to predict the intensions of other vehicles/pedestrains around.

behavior layer

there are two steps here: behavior predictation, behavior planning.

based on the prediction info of other agents intensions from previous timestep/configuration, the behavior predictation module predicts current behavior of the self-driving car, which usually implemented either by model based methods or data driven method (AI-trained), and which output all the possible feasible maneuvers for current timestep.

the car choose only one maneuver at each timestep/configuration, and the behavior planning module is used to weight all the feasible maneuvers from the behavior predictation, and find the most-likely maneuver, which usually is implemented based on a cost function with constraints.

the all possible manuever is also called trajectory planning, and the choosen manuever is also called motion planning, which is locally-space and time-depended.

control layer

since motion planning, then send the command to vehicle control actor and update the car physically.

end-2-end motion planning

deep learning is also used to demo end2end motion planning. e.g. from camera output to vehicle control output frame to frame, while many situations may not be trained in the model, so not that realisty.

path/routine planning

the motion planning pipleline above is happening every timestep for self-driving car and locally.

at high-level is path/routine planning, bascially given the start point and destination point. there are a few algorithms, like global graph search, random tree, incremental graph search.

reinforcement learing in simulator

previous blog, it is also popular to learn behavior in simulation environement and train with reinforcement learning.

simulation in self-driving

how simulation tool chain can accelerate self-driving development ?

usually simualtion enviornemnt can help to verify and test the perception, behavior, control algorithms.

if working with reinforcement learning, a virtual simulator is required also.

what else ?