The importance of experience in hackathons
Apr 22, 2018
photo: Brad Larson, private archive
Recently we’ve had the chance to ask Brad Larson some questions, he is the man behind On-UAV
Brad, what made you
We had been developing a product using the Jetson TX1 and
Do you have any prior experience in building AI solutions?
Personally, I’ve been fascinated with machine learning since the late
For the last two years, we have been developing software and custom convolutional network designs to perform accelerated object detection and image classification on power-constrained hardware like mobile devices or embedded systems. We’ve relied heavily on NVIDIA hardware and software in this development, so it was a natural step to extend some of our work to the Jetson TX1 and TX2.
With this much past
Two years ago, Perceptual Labs was spun out of SonoPlot in order to commercialize machine vision technology that had been created in pursuit of improving the latter’s robotic systems. Perceptual Labs also drew in work that had been done on optimizing GPU-based image and video processing on mobile devices. The company was founded to apply GPU-accelerated machine vision techniques and machine learning to solving visual problems on mobile and embedded hardware.
At roughly the same time, Perceptual Labs and Rapid Imaging realized we had
One such application of Rapid Imaging’s software was in disaster recovery, where they were able to provide live augmented reality overlays to UAV pilots that showed locations of streets and other landmarks overlaid on live video from flooded or storm-damaged areas. However, that could only describe what should be there under the
That’s where our work on applying convolutional neural networks to object detection could close the loop. We decided to start by seeing if we could identify and localize storm-damaged buildings in UAV video streams, classify their level of damage, and then georeference their location based on where they were detected in UAV camera video. There are many other places this could be applied, but this was a good starting point for a proof-of-principle.
Rapid Imaging partnered with Embry Riddle Aeronautical University in mid-2017 to gather UAV footage and telemetry from flights over tornado-damaged areas in the central U.S. This recorded video and telemetry was used as training and validation data for a custom object detection convolutional neural network. Once we had demonstrated that the network was able to reliably detect storm-damaged buildings, we combined it with Rapid Imaging’s georeferencing software and worked to get the compound software working efficiently on the Jetson TX1 / TX2.
We chose the Jetson devices as targets because they are rapidly becoming part of standard computing modules and control hardware for UAVs in a variety of applications. Being able to do all this processing on a TX1 or TX2 would allow for live, in-the-field analysis of
Despite all the thorough preparations, were there any unexpected, unpleasant surprises that you’ve encountered?
Even with the
Satisfying the needs of the contest also proved to be more difficult than we had expected, because gathering and editing all the video footage we needed took quite a bit of time. Luckily, we had equipment that allowed us to capture the video output of the TX1 and TX2 boards, which saved us some effort and made for a cleaner presentation.
We were tuning and refining our software and video presentation up until the last minute, but we feel it all came together well.
Using as much time as you could, that’s noteworthy. Now, what would you say was something completely new you’ve learned while working on this project?
We learned quite a bit about the Jetson TX1 / TX2 hardware, the TensorRT accelerated inference library, and overall techniques for deploying convolutional neural networks to hardware like this. The process of porting Rapid Imaging’s georeferencing software to the Jetson TX1 / TX2 also required learning quite a bit about the characteristics of this platform.
And besides that, did you encounter any other possibilities with to develop your skills?
In line with the answer above, we expanded our knowledge and skills for handling deployment of convolutional neural networks to devices such as this and became familiar with the development process on these computers. Performance tuning on this class of hardware was something that we spent quite a bit of time on, and our ability to attend NVIDIA’s GTC as a result of this competition allowed us to get hands-on training with NVIDIA engineers in this area. We were able to get all of our questions answered about applying these techniques to the development of our software.
We’re much more confident now in our ability to bring this out as a commercial product as a result.
With working on this project being sort of an elaborate learning process, could you name the most difficult, and the easiest moments?
Getting initially set up on the Jetson TX1 and TX2 computers was probably the easiest part of the
As I said above, the video production for our demonstration might have been the most difficult part of the process. It surely was one of the most time-consuming aspects because of the footage that needed to be gathered, edited together, and narrated. The product was worth the effort, though.
Now for something different, do you enjoy the idea of hackathons?
That depends on the duration and nature of it. Very short hackathons aren’t my favorite, but a longer-term directed competition like this one
You seem very professional in
It has been a while since I last participated in a short-duration hackathon. I usually don’t feel that I have the ability to work quickly enough to assemble a complete demonstration during the time constraints of your typical weekend-length hackathon.
Would you say it’s possible for you to participate in more hackathons in the future?
If they are structured like this competition, probably. As I said, the shorter hackathons usually don’t give me enough time to assemble something I’m proud to show off. A longer-duration competition with a specific goal is much more interesting to me.
Do you believe hackathons can help you in some way in the future?
If they take place in areas that I’d like to work in or that use new technology, they can be great
As for the NVIDIA Challenge, is there something particularly valuable that you’ve learned, or gained otherwise?
This competition has been hugely useful for everyone on our side. It gave us the structure and motivation to complete a proof-of-principle and taught us a tremendous amount about the Jetson TX1 / TX2 hardware. The ability to attend GTC 2018 exposed us to leading-edge research in our
Beyond all that, the exposure from being a finalist in the competition has been transformative for our companies and is already helping us out in many ways.
And lastly, if you could, what would be your piece of advice to give to all the people willing to participate in similar hackathons in the future?
I’d make sure that you had a concrete plan for your project, know exactly the problem it is solving and how it will do so, and be able to explain that clearly. Put together a plan for your video presentation and start working on your
If you would like to participate in a similar hackathon visit our website with