How does Arity fine-tune those invaluable algorithms?
As Arity continues to grow and expand our core capabilities, we are consistently fine-tuning our technology. Our philosophy of continuous improvement is exemplified by the technology team that maintains and refines Arity’s mobile phone-based collision detection algorithm. Arity compiles customer labels for tens of thousands of detected collisions per year and we actively use this information to tweak the collision algorithm to optimize performance. It has been quite a journey to get to where we are today and we are proud to have been and continue to be a part of the process.
The Ups and Downs of Optimizing Algorithms
With the Arity Driving Engine software development kit (SDK) integrated into our customer’s mobile apps and carefully designed customer feedback channels, we can identify common user activities that are harder to differentiate from real vehicle collisions. One activity that throws the algorithm for a loop is amusement park rides. Roller coasters and similar rides can produce trajectories and accelerations comparable to those generated during real collisions. In the summertime, the number of false positives from these activities was magnified. After we identified the problem our team immediately assembled to find a solution.
To increase the volume of our roller coaster data set and gain additional context about the physics of the problem, we decided it was best to develop some first-hand experience. Luckily, we had no problem finding willing Arity volunteers. We narrowed down the large pool of enthusiastic volunteers to a group of data scientists and data engineers who were willing to take on the twists and turns of this unique field assignment.
We laid out a detailed plan of tests to be conducted at Six Flags in Gurnee, Illinois (in fact, we used integer programming to optimize an NP-hard scheduling problem, but that’s another blog post). In parallel, Arity’s data science team worked very closely with our mobile engineering team to develop special builds for the Arity SDK for the testing team’s test phones. Ahead of the big day, we trained our team of volunteers to manually start their trip in data collection mode, verify successful data collection, among many other things.
After a very challenging day-long effort that featured several rides on several roller coasters and other amusement park rides, our volunteers transferred the data that they collected over to the collision detection team as trained by the mobile engineering team. Using this data set, along with the customer labeled data already captured in the field, the collision detection team rapidly developed the roller coaster filter for the algorithm, which has achieved greater than 95% precision and recalls in separating real collisions from former detected collisions that were actually roller coaster rides.
It’s all downhill from here… NOT!
As summer quickly turned into winter, we started to observe false positives at ski resorts. Indeed, certain skiing/snowboarding/sledding behaviors produce trajectories and accelerations comparable to those generated during some minor, but real collisions. Given the cold-start nature of the problem, similar to our roller coaster challenge, our biggest issue was the lack of a high volume of real and reliable ski ride data. Specific to this case, Arity worked closely with our customers to collect a new set of ski-specific data. During their winter activities, several employees from both Arity and our customers graciously volunteered to collect data during their ski vacations. Using this data, we created a ski filter for the algorithm that has performed with greater than 90% precision and recalls in separating real collisions from the former detected collision that were actually skiing.
Arity’s massive telematics database includes data from nearly one million OBD-II devices and 12 million mobile daily active users. This data is combined with the claims information (i.e., collision labels) for millions of insurance customers and customer labels which we gather in collaboration with Arity’s application partners. Whether we are speeding down a mountain or dropping from the heavens on a roller coaster, these are just two of the many examples of how Arity will go to great lengths to use this data to reach our goal of making transportation smarter, safer, and more useful for everyone.
This post was jointly authored by Venu Tammali and Shanshan Wang
Venu is a Data Scientist on Advanced Sensor Insights team. He has a Ph.D. in Applied Math and is determined to make transportation smarter and safer for everyone.
Shanshan Wang is a Data Scientist on Advanced Sensor Insights team. She has a Ph.D. in Industrial Engineering. She enjoys knowing all the best roller coasters that would give you collision-like experience.