CARLA used to support training, prototyping, validation of self-driving models, including perception and control. CARLA is usded to study the performance of three approaches, 1) classic modular pipeline that comprises a vision-based perception module, a rule-based planner, and a maneuver controller; 2)a deep network that maps sensory input to driving commands via imitaion learning; 3) end-to-end reinforcment learning.
all approaches make use of a high-level topological planner. the planner takes the current position of the agent and the location of the goal as input, and use A* algorithm to provide a high-level plan. this plan advises the agent to turn left/right, or keep straight at intersections, but not provide a trajectory neither geometric info. which is a weaker form of common GPS.
simulation engine
CARLA simulates a dynamic world and provide a simple interface between the world and an agent that interacts with the world. CARLA is designed as a server-client system, where the server runs the simulation and renders the scene, the client API is responsible for interaction between the agent and the server via sockets.
the client send commands and meta-commands to the server and receives sensor readings in return. the comands control the vehicle and includes steering, accelerating, and braking. meta-commands controls the behavior of hte server, e.g. resetting hte simulation, modifying the sensor suite.
environment
the static 3D world, such as buildings, traffic signs, and the dynamic objects such as vehicles, pedestrains. the behavior of non-player characters is based on standard UE4 vehicle model(PhysXVehicles), and extended with a basic controller to govern NPC’s behavior: lane following, respecting traffic lights, speed limits, and decision making at intersections.
pedestrains
pedestrains navigate the streets according to a town-specific navigation map, which conveys a location-based cost, which is designed to encourage pedestrains to walk along sidewalkd and marked road crossing, but allows them to cross road at any point.
sensors
camera parameters include 3D location, 3D orientation with respect to the vehicle’s coordinate system, field of view, and depth of field.
the semantic segmentation pseudo-sensor provides 12 semantic classes: road, land-marking, traffic sign, sidewalk, fence, pole, wall, building, vegetation, vehicle, pedestrain, and other.
a range of measurements associated with the state of the agent. ?
measurements concerning traffic rules include the percentage of vehicle’s footprint that impinges on wrong-way lanes or sidewalks, as well as states of the traffic lights and speed limit at the current location of the vehicle.
CARLA provides access to exact location and bounding boxes of all dynamic objects.
autonomous driving
the agent interacts with the environment over discrete time steps. at each time step, the agent gets an observation, which is a tuple of sensory inputs, and must produce an action, which represents steering, throttle, brake.
modular pipeline
the pipeline includes: perception, planning, continuous control. local planning is critical based on visual perception.
perception
using semantic segmentation network based on RefineNet to estimate lanes, road limits, and dynamic objects. and a classification model is used to determine proximity to intersections.
the local planner
coordinates low-level navigation by generating a set of waypoints, near-term goal states that represents the desired position and orientation of the car in near future. the rule-based state: 1) road-following, 2) left-turn, 3) right-turn, 4) intersection-forward, 5) hazard-stop. transitions between states are performed based on estimates provided by the perception module and on topological info provided by the global planner.
continuous controller
using PID controller, which inputs current pose, speed, a list of waypoints, and outputs steering, throttle, and brake.
carla release
9.1
1) enable client to detect collisions and determine lane changes : sensor.other.collision, sensor.other.lane_detector,
2) access to the road network, waypoints nearby current vehicle and define user navigation algorithms: Map
3) support new map created from external RoadRunner/VectorZero, in OpenDriven map standard
9.2
1) simulation of traffic scenarios by Scenario Runner. e.g. following leading vehicle, stationary object crossing, dynamic object crossing, opposite vehicle running red light, vehicle turn right/left etc
2)upgraded ROS bridge
3) vehicle navigation from client side: BasicAgent, navigate to a point given location while dealing with other vehicles and traffic lights safely; RoamingAgent, drives around making random choices when presented to multiple options
9.3
1) new town and new pedestrains
2) no rendering mode, a 2D map visualization tool that display vehicles, traffic lights, speed limits, pedestrains, road, etc. help to improve the server framerate
3) traffic light class in client TrafficLightState
4) new sensors. ObstacleDetector, a simple raycast sensor to detect something in front of the ego vehicle and what is it; and GlobalNavigationSatelliteSystem, attach to ego vehicle and get its geolocation, which is based on the geo-reference define in OpenDriven file associated with each map.
9.4
1) allow client side to change physics properties of vehicle or their components in runtime WheelsPhysicsControl
2) logging and playback system, which includes a camera-following mode to follow a target actor while replaying the simulation, and can replay situations from different viewpoints. and the logging query engine allow users to query different types of events.
3) random streaming port, which makes it possible to stream sensor data in a secondary port
4) import maps, replace maps as tar.gz files in “ExportedMaps” folder