Building smart cities – using 5G cameras on Melbourne garbage trucks

A research project currently being conducted by Swinburne University of Technology in collaboration with Brimbank City Council is using the council’s garbage trucks as a way of integrating smart technology into our cities to improve the management of roads and council assets

Supported by $1.18 million in funding from the federal government’s Australian 5G Innovation Initiative, the project that commenced in November 2021 will run until the end of September, analysing data from 900km of road within the Brimbank Council area.

The project is equipping 13 garbage trucks operated by Melbourne council of Brimbank with high resolution stereo vision cameras and GPS sensors, with the data sent to a Cloud-based system for analysis to identify assets on the road or road signs that require maintenance. This helps the council create a map of jobs, such as fixing broken road signs, graffitied bus shelters or damaged roads in real-time.

The Initiative provides businesses with funding to trial and test applications that demonstrate 5G’s capability and benefits across a range of industry sectors and locations.

The director of Swinburne’s Factory of the Future and Digital Innovation Lab, Associate Professor Prem Prakash Jayaraman, speaking to Deals on Wheels, says that this particular project is the first of its kind.

“The aim of this project is to get some real-world data of how 5G can support applications like this,” says Jayaraman.

“So, working with Brimbank, who we have partnered with in the past, we started brainstorming how we could apply for this funding from the 5G Innovation Initiative, and that’s when we came up with one of the common problems that councils face, which is maintenance of the local area and citizens’ perception of what the council is doing to improve its community. This includes rubbish dumped on the side of the road, graffiti, damaged road signs and bus shelters, etc.

“We decided to develop a solution to address this fundamental problem, because this has a direct impact on citizens in terms of how their community looks and how it feels for them.”

Information from the cameras is uploaded via 5G to the Cloud for real-time tracking of issues found as the trucks drive their usual routes

Cutting-edge technology

Working alongside Optus, which provided the 5G infrastructure, and AWS, which provided the Cloud services, the Swinburne project team is using its expertise and experience in researching the Internet of Things, Cloud computing, camera technology and sensors to make garbage trucks ‘intelligent’ by installing 3D depth sensing cameras and 5G router on board the trucks and upload data in real-time as they make their usual rubbish collection rounds.

“These cameras are a bit different than your normal phone cameras, which provides you with a standard RGB [red, green and blue] image,” says Jayaraman.

“Depth sensing means the camera knows exactly how far away a particular object is. This is helpful for the system to know how tall a bin is or is there a bin that has fallen over, because it knows the dimensions.

“So, we thought 5G was a very good medium for us to be able to collect data, upload it and, in real time, detect issues such as someone’s dumped a mattress at the side of the road. We can see these and notify the council that there is an issue and raise a job for them to fix it pro-actively.”

The cameras collect 3D perception data at a rate of 900MB/second (average mobile phone download speeds are around 43MB/sec) and build a heat map of the area in front of the truck.

When it comes to why garbage trucks were specifically chosen to trial the system, Jayaraman says that a key feature is their ‘natural mobility’ – the trucks already run on every road within the council’s area each week, so are the perfect mechanism for putting additional eyes on the road to spot issues as they occur.

“You can’t get anything better than garbage trucks because they go to every nook and corner of your council,” he explains.

“And 5G offers a lot of benefits for this kind of scenario in terms of low latency and high bandwidth. And with the trucks always moving at slow speed, they are a great way of testing 5G coverage.

“But, also, it’s a very interesting use case because, if we are successful – and looks like we will be because we’ve been getting very good results out of the initial deployments – you’re looking at significant savings in terms of both time and dollars. And, most importantly, it helps the council move from being reactive to more proactive. Currently, the only way they know rubbish has been dumped is if somebody calls up and says they’ve found a mattress on the side of the road.”

Through applying this approach, time is saved by not having to send staff out to assess potential issues – the data is supplied to them in the office where a decision can be made on what remedial work needs to take place. There is also the added value to people living in the council area – the faster dumped rubbish or a vandalised bus stop is removed or repaired, the higher the satisfaction.

The aim of the project is to see how 5G can be applied to real-world issues such as rubbish being dumped at the side of the road

Thinking smart

This first step of the project is testing the creation of a heat map, which presents potential issues found on the road as the garbage trucks follow their usual routes. The map is built using state-of-the-art AI and machine learning techniques that increase in accuracy over time. As Jayaraman points out, there’s nothing else like this taking place.

“We are actually building this from scratch,” he says.

“So, we need to trial this to build confidence in the system. Once we have enough confidence, the map will highlight something like a dumped mattress and the council will be able to validate that and log jobs.

“Eventually, when we go to a level where the system is very high in terms of confidence, this is fully automated. Then the council can focus on other kinds of problems, such as potholes in the road, flooding after a storm, overhanging trees, etc. For this project we are focusing on the three main problems of road signs, bus shelters and dumped rubbish. However, with the data that we are gathering, we could do a whole lot more.”

So, this is just the beginning – with trucks already patrolling the streets, a whole checklist of items could potentially be programmed into the software. More councils are already interested in installing the system in their own areas, too. But what if your council covers a vast regional area and thousands of kilometres of roads?

The project team. L-R: Dr Felip Marti, Associate Professor Chris McCarthy and Associate Professor Prem Prakash Jayaraman

Expanding out of cities

As many truckies know, rural roads can often be a bumpy ride and, with the largest council area in Australia covering a distance greater than the entire state of Victoria (East Pilbara for those who are wondering), it can often be difficult to regularly assess roads.

This particular project is focusing on a major city council and the capabilities of a high-speed 5G internet connection for data processing and analysis in real time. But Jayaraman says there absolutely is the potential to expand out of urban areas and there is still the potential for this technology to be utilised on rural and regional roads, even if the 5G coverage isn’t there.

“A lot of this can be done while you’re offline and then you upload the data when you’re online,” he says.

“So, the system doesn’t always need to be connected. The real benefit in having the high-speed internet connectivity is to be able to do this in real time. In rural areas, when you’ve got connectivity, you can the jobs out then, it would just be a slower process.

“The beauty about this is that the garbage trucks are already on the road – they’re not something additional that we have to put into the system. The cost of getting the data is nothing – we don’t need to pay someone to drive on roads to specifically collect this data.

“So, in places serviced by garbage trucks, it’s an easy thing to implement. But of course, there are other schemes. For example, you could have discussions with other providers, including toll road operators.”

With all kinds of trucks getting smarter and more connected, we could be on the edge of a breakthrough to finally target bad roads – a plus for operators, councils and residents alike. 

Photography: Swinburne University of Technology

Send this to a friend