A few weeks ago I participated in my first ever machine learning competition. It was a contest initiated by Honeywell, hosted on CrowdAnalytix. I used a data driven approach to predict fuel consumption in aircrafts during different phases of flight, based on flight data recorder (FDR) archives. I’m happy to say I took 1st place! Below is an overview of my approach.
The objective was two-fold:
- Predict fuel flow for every second during different flight phases. The more fuel an airplane contains, the heavier it is. A heavier airplane will have higher fuel consumption. So I’m assuming the predictive model can then be used to predict fuel consumption during future flights, which can help reducing the amount of fuel entered in the fuel tanks before the actual flight.
- Use the model to extract actionable insights and provide these in a report.
The uncompressed data-set was about 14.4 GB of flight data recordings from 1005 different flights. Measurements were done every second, and every measurement contained 224 parameters. Flight data contained measurements during preflight, taxi, take-off, climb, cruise, descent and roll-out. There was also some data belonging to unknown flight-phases.
Understanding the data
I started out by doing a quick crash course on aircraft flight parameters, using lots of Wikipedia and an article explaining flight controls of a Boeing 737 (also check the other articles on the right).
Once I got some basic understanding on it, I categorized the 224 parameters in different categories. This helped me get better understanding and intuition:
- Time: Year, month, day, hour, minute, second
- Location: Latitude and longitude
- Environment: Static air temperature, total air temperature, wind speed & direction, …
- Flight controls: Air-brake position, elevator position, spoiler, rudder, aileron, …
- Flight controls set by pilot: Control column position, rudder pedal position, …
- Position: Pressure altitude, altitude rate, angle of attack, corrected angle of attack, indicated angle of attack, baro correct altitude, drift angle, ground speed, lateral acceleration, longitudinal acceleration, …
I then also did some exploratory analysis by using visualization. This showed that there were several parameters that were very correlated (0.99 and more) during certain flight phases, but not during other flight phases.
The visual inspection also proved very useful to get a feel for which parameters to use to filter for certain speeds or heights, as the data-set contained several parameters for each (e.g. for altitude there was pressure altitude, baro correct altitude, radio altitude, …)
Basically I did all of the above to understand airplane flight controls and environment measurements. Now it was time to get coding… 🙂
Time to predict
I first split the training data in several parts, one for each flight phase. I ran through below steps for each flight phase.
As I found some very correlating variables (see above), I made sure to first remove unnecessary and noisy variables that could potentially confuse the predictive model.
There was also data in the training set where fire was detected. In those cases I simply removed the feature AND the observations, if it wasn’t too much data.
I did quite a lot of manual feature engineering and I have no doubt this is the strength of my approach. In total I added about 40 new features. Most new features were lags.
As an example, I noticed that there was barely any correlation between throttle target and fuel consumption. That doesn’t make any sense, does it? The throttle of the engines SHOULD impact fuel flow. So I noticed that when I took an X seconds lag then suddenly the correlation was quite obvious! This makes sense. If the pilot changes the throttle target then it will take some time for the engines to adapt to it.
This insight proved valuable for several of the pilot’s settings in the cockpit. When he/she changes a setting, it usually takes some time before the airplane responds. Making sure that the machine learning model had the possibility to ‘learn’ this insight greatly increased the predictive power of the model
I chose XGBoost as predicting model and simply went along with its standard settings for training as unfortunately I ran out of time to tune the hyper-parameters. No doubt some tuning could make it predict even better. Time was really an issue for me: I wasn’t able to train my latest and best model on all data (notably my final submission for the cruising phase was based on an older model).
Personally I was really missing an ‘actual engine throttle’ feature. I think this extra feature could make a big contribution to an even better predictive model. However, perhaps predicting would be way too easy with this feature, so I’m thinking maybe Honeywell decided to remove this parameter from their data-set in order to let participants actively search for other, less visible relationships.
In the end I still think I under-utilized the aircraft specific knowledge I obtained. And there was still so much data that I left unused and could have implemented. It was only close to the deadline that I realized that different airports have different altitudes and that this affects air pressure, which can have a considerable impact on fuel used during the take-off and climb. I could have normalized the fuel consumption for different airports to make it easier for the model to predict. But hey, we were trying to do in a week or 2 what some aircraft engineers probably do full-time! 🙂
Over the past few months I’ve had several questions on the above via email. First of all, I’m totally okay with this. That said, if you have a question, then could you please ask me in the comments below? That way all other readers get to see the answer as well. I would appreciate it :). I have now also extended the above article and included things I had answered to people privately.