28 August 2019
Acrea has held an internal hackathon with all consultants participating. The aim was to deal with machine learning in a practical way. It started with tutorials conducted with Microsoft Azure Machine Learning Studio. A sense of achievement quickly set in. The visual tool is easy to use and nicely visualizes the data flow. It as pure cloud solution and free for smaller experiments: highly recommended.
Then the hacking started: A team tried to apply neural networks to make Bitcoin buy/sell recommendations based on historical price data. It was not to be expected to be profitable as historical price data does not contain sufficient data about the future. Yet a lot can be learned about machine learning.
The initially simple models gave way to more and more complex models: more layers were trained and two teams applied it to text classification and fraud detection. Additionally, Google TensorFlow and Keras were used.
Emotions came up in the Bitcoin team: a simulation predicted a tenfold increase of the investment within a year. You could see dollar signs glowing in their eyes. The teams working on fraud detection achieved surprisingly good detection rates. For a practical application however it was not sufficient as each mistake (false positives and false negatives) would incur cost. So what is needed to increase the detection rate: a more sophisticated model or more and more detailed data? A better model quickly achieves improvements; but with limited data attributes you obviously hit a ceiling that even the best model cannot overcome.
Slowly the Bitcoin team realized that the new wealth had to wait and that the supposed increase in wealth was based on faulty procedure. They had correctly used separate data to train and evaluate the model. However for the simulation of the investment strategy they also used the training data. No surprise the predictions were so favorable.
Another team used convolutional neural networks (CNN) for image recognition and experienced first-hand how time-consuming the training of a realistic neural networks can be. For 50,000 tiny images it took more than an hour. Furthermore, the model is complex and shows how much experience and knowledge has matured over the years and must be incorporated into models in order to achieve good results.
The corrected procedure for the Bitcoin prediction did not go without effect. The invested money no longer increased. The search for a profitable investment strategy is not over yet. We will see in the near future who no longer has the financial need to come to work…