Smart Control of Traffic Lights Using AI
The congestion of urban traffic is becoming one of the critical issues with the increasing population and automobiles in cities. Traffic jams not only cause extra delay and stress for the drivers but also increase fuel consumption, add transportation costs, and increase carbon dioxide and air
pollution. Although it seems to pervade everywhere, megacities are the ones most affected by it. And its ever-increasing nature makes it imperative to know the road traffic density in real-time for better signal control and effective traffic management. There can be different causes of
congestion in traffic, such as insufficient capacity, unrestrained demand, large red light delays, etc. The traffic controller is one of the critical factors affecting traffic flow. The conventional traffic patterns are nonlinear and complex and time-dependent rather than traffic-dependent. Therefore the need for simulating and optimizing traffic control to better accommodate this increasing demand arises.
System Overview
Our proposed system takes an image from the CCTV cameras at traffic junctions as input for real-time traffic density calculation using image processing and object detection. This system can be broken down into 3 modules: Vehicle Detection module, Signal Switching Algorithm, and Simulation module. As shown in the figure below, this image is passed on to the vehicle detection algorithm, which uses YOLO. The number of vehicles of each class, such as car, bike, bus, and truck, is detected, which is to calculate the density of traffic. The signal-switching algorithm uses this density, among some other factors, to set the green signal timer for each lane. The red signal times are updated accordingly. The green signal time is restricted to a maximum and minimum value in order to avoid starvation of a particular lane. A simulation is also developed to demonstrate the system’s effectiveness and compare it with the existing static system. You can view a presentation of this project on YouTube here.
Vehicle Detection Module
- The proposed system uses YOLO (You only look once) for vehicle detection, which provides the desired accuracy and processing time. A custom YOLO model was trained for vehicle detection, which can detect vehicles of different classes like cars, bikes, heavy vehicles (buses and trucks), and rickshaws.
- The dataset for training the model was prepared by scraping images from Google and labeling them manually using LabelIMG, a graphical image annotation tool.
- Then the model was trained using the pre-trained weights downloaded from the YOLO website. The configuration of the .cfg file used for training was changed in accordance with the specifications of our model. The number of output neurons in the last layer was set equal to the number of classes the model is supposed to detect by changing the ‘classes’ variable. In our system, this was 4 viz. Car, Bike, Bus/Truck, and Rickshaw. The number of filters also needs to be changed by the formula 5*(5+number of classes), i.e., 45 in our case.
- After making these configuration changes, the model was trained until the loss was significantly less and no longer seemed to reduce. This marked the end of the training, and the weights were now updated according to our requirements.
- These weights were then imported in code and used for vehicle detection with the help of the OpenCV library. A threshold is set as the minimum confidence required for successful detection. After the model is loaded and an image is fed to the model, it gives the result in a JSON format, i.e., in the form of key-value pairs, in which labels are keys, and their confidence and coordinates are values. Again, OpenCV can be used to draw the bounding boxes on the images from the labels and coordinates received.
Following are some images of the output of the Vehicle Detection Module:
Signal Switching Algorithm
The Signal Switching Algorithm sets the green signal timer according to traffic density returned by the vehicle detection module, and updates the red signal timers of other signals accordingly. It also switches between the signals cyclically according to the timers.
The algorithm takes the information about the vehicles that were detected from the detection module, as explained in the previous section, as input. This is in JSON format, with the label of the object detected as the key and the confidence and coordinates as the values. This input is then parsed to calculate the total number of vehicles of each class. After this, the green signal time for the signal is calculated and assigned to it, and the red signal times of other signals are adjusted accordingly. The algorithm can be scaled up or down to any number of signals at an intersection.
The following factors were considered while developing the algorithm:
- The processing time of the algorithm to calculate traffic density and then the green light duration — this decides at what time the image needs to be acquired
- Number of lanes
- Total count of vehicles of each class like cars, trucks, motorcycles, etc.
- Traffic density calculated using the above factors
- Time added due to lag each vehicle suffers during start-up and the non-linear increase in lag suffered by the vehicles which are at the back
- The average speed of each class of vehicle when the green light starts i.e. the average time required to cross the signal by each class of vehicle
- The minimum and maximum time limit for the green light duration — to prevent starvation
Working of the algorithm
When the algorithm is first run, the default time is set for the first signal of the first cycle and the times for all other signals of the first cycle and all signals of the subsequent cycles are set by the algorithm. A separate thread is started which handles the detection of vehicles for each direction and the main thread handles the timer of the current signal. When the green light timer of the current signal (or the red light timer of the next green signal) reaches 0 seconds, the detection threads take the snapshot of the next direction. The result is then parsed and the timer of the next green signal is set. All this happens in the background while the main thread is counting down the timer of the current green signal. This allows the assignment of the timer to be seamless and hence prevents any lag. Once the green timer of the current signal becomes zero, the next signal becomes green for the amount of time set by the algorithm.
The image is captured when the time of the signal that is to turn green next is 0 seconds. This gives the system a total of 5 seconds (equal to value of yellow signal timer) to process the image, to detect the number of vehicles of each class present in the image, calculate the green signal time, and accordingly set the times of this signal as well as the red signal time of the next signal. To find the optimum green signal time based on the number of vehicles of each class at a signal, the average speeds of vehicles at startup and their acceleration times were used, from which an estimate of the average time each class of vehicle takes to cross an intersection was found. The green signal time is then calculated using the formula below.
where:
- GST is green signal time
- noOfVehiclesOfClass is the number of vehicles of each class of vehicle at the signal as detected by the vehicle detection module,
- averageTimeOfClass is the average time the vehicles of that class take to cross an intersection, and
- noOfLanes is the number of lanes at the intersection.
The average time each class of vehicle takes to cross an intersection can be set according to the location, i.e., region-wise, city-wise, locality-wise, or even intersection-wise based on the characteristics of the intersection, to make traffic management more effective. Data from the respective transport authorities can be analyzed for this.
The signals switch in a cyclic fashion and not according to the densest direction first. This is in accordance with the current system where the signals turn green one after the other in a fixed pattern and does not need the people to alter their ways or cause any confusion. The order of signals is also the same as the current system, and the yellow signals have been accounted for as well.
Following are some images of the output of the Signal Switching Algorithm:
Simulation Module
A simulation was developed from scratch using Pygame to simulate real-life traffic. It assists in visualizing the system and comparing it with the existing static system. It contains a 4-way intersection with 4 traffic signals. Each signal has a timer on top of it, which shows the time remaining for the signal to switch from green to yellow, yellow to red, or red to green. Each signal also has the number of vehicles that have crossed the intersection displayed beside it. Vehicles such as cars, bikes, buses, trucks, and rickshaws come in from all directions. In order to make the simulation more realistic, some of the vehicles in the rightmost lane turn to cross the intersection. Whether a vehicle will turn or not is also set using random numbers when the vehicle is generated. It also contains a timer that displays the time that has elapsed since the start of the simulation. More details about the simulation can be found in this article.
Key steps in the development of simulation
- Took an image of a 4-way intersection as background.
2. Gathered top-view images of car, bike, bus, truck, and rickshaw.
3. Resize them.
4. Rotated them for display along different directions.
5. Gathered images of traffic signals — red, yellow, and green.
6. Code: For rendering the appropriate image of the signal, depending on whether it is red, green, or yellow.
7. Code: For displaying the current signal time i.e., the time left for a green signal to turn yellow or a red signal to turn green or a yellow signal to turn red. The green time of the signals is set according to the algorithm by taking into consideration the number of vehicles at the signal. The red signal times of the other signals are updated accordingly.
8. Generation of vehicles according to direction, lane, vehicle class, and whether it will turn or not, all set by random variables. The distribution of vehicles among the 4 directions can be controlled. A new vehicle is generated and added to the simulation after every 0.75 seconds.
9. Code: For how the vehicles move, each class of vehicle has a different speed, and there is a gap between 2 vehicles, if a car is following a bus, then its speed is reduced so that id does not crash into the bus.
10. Code: For how they react to traffic signals i.e. stop for yellow and red, move for green. If they have passed the stop line, then continue to move if the signal turns yellow.
11. Code: For displaying the number of vehicles that have crossed the signal.
12. Code: For displaying the time elapsed since the start of the simulation.
13. Code: For updating the time elapsed as simulation progresses and exiting when the time elapsed equals the desired simulation time, then printing the data that will be used for comparison and analysis.
14. To make the simulation closer to reality, even though there are just 2 lanes in the image, we add another lane to the left of this, which has only bikes, which is generally the case in many cities.
15. Vehicles turning and crossing the intersection in the simulation to make it more realistic.
Following are some images of the final simulation:
Source Code and Related Content
The source code for this code can be found on Github here.
A detailed guide to building the simulation can be found in these articles:
- Traffic Intersection Simulation using Pygame, Part 1
- Traffic Intersection Simulation using Pygame, Part 2
- Traffic Intersection Simulation using Pygame, Part 3
This simulation was developed as part of a research project titled ‘Smart Control of Traffic Lights using Artificial Intelligence’. Check out its demonstration video here. This research work was presented at the IEEE International Conference on Recent Advances and Innovations in Engineering (ICRAIE) 2020 and published in IEEE Xplore. Read the paper here.
