Why do we in our company use ROS even though we don’t make robots? Let’s find out.

So the agenda for my talk is to first give an introduction about Vimaan and warehousing to take you back to our roots in the sky, our beginnings with ROS 1, with our first product, our evolution towards smarter solutions nowadays. Then go on to our architecture for near real-time processing some wins which we’ve had with ROS 1. Some challenges we faced, like maybe several of you here and why we are migrating to ROS 2.

They say a picture speaks a thousand words. So here’s 24,000 frames, words per second. To establish some context about the warehousing industry and about what we do and how we play our part:

Warehousing is one of the last industries to digitize, and it shows from slow inefficient and error prone practices across inbound receiving to pallet put away to outdated inventory tracking processes in storage, picking, packing and shipping. Relying on human labor alone means settling for higher headcounts, lower inventory accuracy, lost and damaged goods, delayed shipments and higher costs across the board. Now’s the time for warehouses to automate, digitize, and modernize.

Welcome to Vimaan the first of its kind wall-to-wall inventory tracking solution to support the real-time digital warehouse. Vimaan is computer vision for inventory and warehouse events with sensors positioned throughout a warehouse to automatically read labels, measure dimensions, and track inventory with over 99% accuracy. Let’s step inside.

Incoming pallets drive through PalletSCAN as they pass through the gate. PalletSCAN scans barcodes, reads text counts and even dimensions and looks for damages.

All data is processed and recorded through the ViewDECK web app providing warehouses with actionable insights on all incoming pallets. All four facing sides are scanned and captured through the gate, including the front sides and top.

ParcelSCAN enables warehouses to process five-sided package scanning, which extracts and validates incoming label data.  Once received, PickTrack is used to track the locations of putaway pallets. Once goods are stored, StorTRACK becomes the crucial step in providing the safest, most efficient and longest continually running cycle counting automation capabilities available on the market.

So you saw an overview of all the problems we faced. Our roots were in the sky with the first product, which was StorTRACK Air. Learn more about warehouse drones here. So this was built for the use case of cycle counting, which is where you have goods on racks inside warehouses in shelves. And what you’re trying to do is to basically inspect and what is actually on the racks.

Is it representative of what is in the warehouse records that the company has?

And then generate discrepancies and show that this is where we saw a great opportunity to introduce a framework like ROS.   We introduced it into our firmware and application software. We really love that it had seamless async programming with good Python and C++ support. It was highly modular and flexible so we could compose our application in the form of nodes and services and topics and all the good stuff that you’re used to here as ROS developers. It was amazing that we had introspection tools for inspecting our nodes topics as well as visualizations. And the documentation clearly helps the support for diverse hardware and sensor suites, not unlike what gentleman from analog devices and others have spoken about was also a big reason why we directly jumped onto ROS 1.

What was the system architecture like?

Well, if you see here the input for our system was high resolution cameras for data capture, stereo tracking cameras and depth cameras, which would help with the autonomy side of things that would get processed on an embedded Linux companion computer. And we would also have a flight controller for the low level motor commands and the pit loops which are running to keep it aloft and to help with the data capture. And of course battery management and safety is a must which we have for ensuring safe operation.

What was the software architecture like?

We had autonomy nodes, data recording launch and monitoring nodes and the flight controller interface, which were all running on top of ROS as the middleware, which in turn was on top of embedded Linux with camera drivers and USB serial communication.

But then what were the drawbacks?

There’s cardboard lying on the floor. There is some shrink wrap which is flying off the shelves. And safety being such a big concern, we were always ensuring that there was a pilot to take care in case there is an emergency situation so that they can always ensure safe and harmless operation. So a real algorithm for a drone would actually, not unlike a home cleaning robot, would be like for every aisle in the warehouse while the aisle is not flyable, sit on the base station, wait for the human to clean the aisle, and then think existential thoughts of course. So clearly we could do better. There had to be a better, maybe more effective, efficient way of addressing the same problem. And that’s where our evolution towards smarter solutions store track.

The one you saw in the video is our camera rig, which can be picked up by forklifts, order pickers and similar equipment. It has eight hours of battery life compared to 20 minutes on the drone. So that’s like a great increase. We are able to illuminate the shells, show laser guides to the operator and they can reuse the same equipment to scan the racks and go to the same heights and achieve much better accuracy in a way more efficient fashion. We have ParcelSCAN, which is a product for high volume inventory, which is moving on conveyor belts. We have a screen for our operators to look at and we scan the label on it and not just read the barcode, but also the text to identify country of origin, serial number, other details and all this while reading this in under one second for a very high throughput conveyor belt.

Similarly, like you saw in the video, we have a five-sided scanning solution where we have some cameras on the PickTRACK and a lot on an arch, which is called PalletSCAN for palletized goods. And here too, we can look for damaged goods. We can look for incorrect inventory coming in. We have these multiple cameras with LED lighting at the inbound and outbound gates of warehouses, which can detect issues at the source itself. And of course there’s pack view. We all get boxes with things packed inside. This helps ensure that you don’t get a brick when you order an iPhone. Every is scaned, everything is audited. There is evidence and we help increase the throughput that these stations have from a barcode based operation with manual data entry to something using computer vision and AI that Vimaan has built.

We saw so many products and platforms, no moving parts here. Why do we still use ROS?

This to recall was the system architecture of the drone product we had. This is of our new edge platform. If you notice, we have introduced lidars into the mix. We have dimensioning and localization solutions as well. And we have an MCU or a microcontroller unit for power management and peripheral control. We don’t have motors, but we do have indicator beacons, but the rest of it is fairly the same. Similarly on the software stack, the companion computer on the drone had the autonomy nodes, data recording launch and monitoring flight control. And now on our edge platform we do have localization because it’s not just important to read the labels. You also want to correlate it to the location inside the warehouse, which rack, which shelf, which bin, which inbound gate.

We do that through our localization stack. Similarly, we have an MCU interface for all the fills. We are there on the hardware for indicating beacons and for strobe lights and for a lot of the peripherals we have. So if you come to see it’s literally the same architecture. So this was one of the major reasons why our requirements and our reasons for using ROS never went away.

We do build solutions nowadays, which are camera based, which have a very complex applications stack, which needs near real time processing, running at the edge as well as in servers. And with this distributed compute, we wanted to reduce the time to market, reduce the friction to quickly build a platform which can serve all these use cases with different configurations. And ROS really helped us. It was that reason that we were able to transition from the software stack from the drone to our new edge platform, which can now cater to so many different use cases and SKUs so quickly.

Our distributed computing architecture today looks like a lot of these image and location capture systems that you saw, which are the edge compute units and multiple of them feed into a server computer on the premises where we have our really complex computer vision pipeline. We have a warehouse management system interface to look against the ground truth of what the warehouses have. And then the reason that our customers pay us money is the reports that we generate under the dashboard and the real value that they gain from our solution in the warehouse.

So in all of this, what were the specific wins that we had with ROS 1?

The reuse of the architecture across our entire product portfolio was a major win. We have achieved low latency processing with greater than 4K resolution images, multiple cameras for less than one second turnaround time over a distributed compute between our edge and our server modules.

We have support for both publish subscribe as well as request response kind of raw services for blocking RPCs and applications written in Python C++ JavaScript, we were also able to add our own custom wrappers to integrate custom nodes to leverage the NVIDIA low level APIs to fully exploit all the hardware blocks that we got on our edge nodes from GPUs to hardware, jpeg, decoders, ENC coders, the image signal processors, we really could juice that out to the maximum with the maximum amount of control as well. And like you saw, by having a platform approach and having a common modular architecture, we could really have very easy bring up and provisioning of systems by just making everything very configurable, but always not well. There are some challenges with RO one. There was a single point of over reliability. The ROS master-based architecture, it leads to issues with unreliable network node.

From our experience, there was a lack of prioritization between messages like images and control commands. So let’s say if your bandwidth is being filled, there is no guarantee that one control command you’re sending to transition a beacon or stop a system will actually make it through at the cost of maybe a lower priority message like an image. And I do remember there was several times where we had to go back to the API documentation because there was a lack of homogeneity between Python and C ++ and JavaScript client implementations. So what’s the way forward? No surprises here. ROS one is gone, ROS 2 is in.

So what are the reasons that we at Vimaan are transitioning to ROS 2?

It is time, clearly early robustness. Like I mentioned, we really want to avoid network connectivity issues with a single Ross master. We would love to try out automatic discoverability over the network. We believe it can greatly enhance robustness, reliability, and also ease of configurability, industrial grade certified middleware like DDS. And now also frameworks like Xenu look very exciting to us. So, we’ve been very melodic in our orchestration, ntic in our designs, but I guess it’s time to be humble as we move forward towards greatness.

There is prioritization of topics like I mentioned that we also want to include. Nvidia Nitros is something that we are really looking forward to, which can allow for zero copy.

We do a lot of embedded machine learning on our edge compute units, but we really look forward to having a truly decoupled architecture without any penalties of taking images back and forth between the CPU and the GPU address space. We also love the fact that ROS two is modular and oops or object oriented by design. We do want to enforce higher code quality by design from the framework itself and to create production ready systems out of the box with very good CICD support. And Zeno has shown some really promising results and we really want to try it out and see how we can get all the gains from this new framework as well.