Travel just a decade into the past and implement what is considered a basic neural network, you will have gotten a state-of-the-art system ready to be published to a top peer-reviewed artificial intelligence conference. Just the effort of computing gradients and updating weights in a loop to make the model learn something was an achievement. And rightfully so, you needed to know what you were doing. It was a practical challenge for all the buzzwords you hear these days such as loss functions, regression, multi-class classification, cross-entropy and back-propagation.
Jump back into the stark sober today. Forget knowing what a derivative is, just copying and pasting someone else’s code online will give you some results. Libraries such as TensorFlow and PyTorch have made deep learning easier than a walk in the park. Almost anyone with some basic understanding of what computer code is, can read a blog post very much like this one and start training a model. This has spawned an enormous amount of research papers in recent years that use deep learning. Combine this with a recent trend of reproducible research, there is a litter of research code on GitHub. They are absolutely amazing (sarcasm intended). Commented out code, vague function and variable names, no testing, to-dos randomly jotted around and a lack of comments are a common sight for my sore eyes. This mess is what people seem to refer to as research code. I can’t say I am innocent. As the deadlines approach, we all crack. I have been there and done those things…
Lowering the entry level for deep learning is a good thing, it democratises access to technology. But as Uncle Ben pointed out, great power comes with great responsibility.
We’ve seen many cases of bias in trained deep learning models such as Convolutional Neural Networks (CNNs) or a spree of small start-up companies claim to be offering AI services. My fellow friend and colleague very recently summarised the situation one could encounter when dealing with deep learning related research papers, tools and companies:
What they think they did
Both the engineering and mathematical foundations of state-of-the-art models are non-trivial. The people researching, implementing or running an AI company often present their view about what they think is happening. For example, “the model learns to recognise X” is a common manifestation of this phenomenon. There is some clear picture in their mind as to what this system is doing and more often than not how great it is. The cost of lowering the entry level is abstraction. Deep learning libraries abstract away many aspects of messy details of an underlying algorithm such as initialising the weights. Read a 5-minute blog, copy some code and you might have a CNN ready to go. Hence, in research papers, authors think they might have done something but in reality something else might be going on. It is very easy to overlook a small setting and let the deep learning library do something. This situation is more conspicuous on AI company websites selling amazing services with phrases such as cutting-edge, big data, machine learning and my recent favourite AI-ready.
What the maths think they did
It may or may not come to you as surprise that most AI conferences will ask authors to avoid decorative math. It is unfortunately a rife scenario in which authors of papers, people in companies vomit mathematical equations and symbols to make the contribution or the product more substantial. This is when the second observation comes into play. Often those equations do not match the implemented code if they have released the infamous research code. It is possible the mathematics presented tell a different story.
What the code says they actually did
Since we are in the realm of computing (we are indeed, everything we talked about so far is executed on a computer), there is one source of truth we can somewhat rely on and that is what the computer actually did. The source code instructs the computer to perform some actions and deep learning is no new form of magic. At the end of the day, we write some code (hopefully refining any copied code) to build, train and evaluate neural networks. Hence, one can check the released source code to see what on Earth is actually going on. This brings me back to the start of this article, the crippling research code. It will be messy, rushed but it will tell some form of truth. How many layers was this magic CNN from the paper? How were the weights initialised? And many more await in the glorious maze of research source code. Take it in a privately owned, profit hungry AI startup setting and you might never find any answers to any AI-ready future products or services you see on their websites.
As with many things in nature, there is a balance of good and evil when it comes to accessible deep learning. It is a great tool that more should benefit from but respect its limitations and take responsibility. I haven’t even touched upon ethical and moral considerations. I’m sure there is and will be more debate when similar to social media, deep learning starts saturate and law makers catch up.