Machine Learning & Augmented Reality in an EPL iOS App
The primary aim of this project was to determine the best method of implementing an image recognition model in an iOS application. Various Machine Learning (ML) techniques were examined before it was determined that a Convolutional Neural Network (CNN) using transfer learning is the best method to incorporate image recognition in an iOS application. The secondary aim of the project was to build an iOS application using the Core ML and ARkit frameworks to combine ML and augmented reality (AR) in a single application using the newest technologies available. A test driven development (TDD) approach was used leading to over 70% test coverage across the whole code base.
Prior to creating the iOS application, a custom model was built in Python using Keras to recognise football crests. Due to the limited amount of images to train the model, image augmentation techniques were used to expand the dataset and improve the accuracy of the model. Once the model was performing with a high accuracy it was converted to Core ML and integrated into the iOS application. The iOS application analyses an image of a football crest and presents the user the analysed team’s stats in AR. With sports broadcasting using AR for sports stats, this application provides football fans with data in the same format as the TV but in a mobile application. All the AR content is interactive and the user can view the team’s next game along with a prediction of the result. The prediction is made on the server-side analysing the team’s past performances and using a Poisson Distribution to generate results. Making the prediction on the server side ensures that all users of the application retrieve the same prediction for each match. The user can also view a team’s players and their stats, by retrieving images and data from the server. In order to generate the AR content the iOS application requests data from a server relevant to the recognised team's crest. The server scrapes the internet to gather data on each Premier League team and stores the data in an SQLite database where it can easily be retrieved by the iOS application. The server application for this project was developed in Python using a Django server. The iOS application is a native iOS application developed in Swift 4.
The study proved that the more images used to train the machine learning model the higher the accuracy of the model. By comparing the custom-built Keras model with the transfer learning trained MobileNet model it was proven that the transfer learning approach performed significantly better on both speed and accuracy. The application’s model provided over 80% crest recognition accuracy. Given that AR is a new area for mobile applications, how to make AR content intuitive was examined by conducting extensive usability testing to determine the ease of learning of the AR content.