If you don’t know anything about feature engineering, the term may remind you of some deep technical concept. If you know something about it, you know that it is some deep technical concept. So why should business managers even care?
Turns out, this has a direct relationship to ROI of AI projects. Dollars in + better feature engineering = more dollars out.
At its core, artificial intelligence is software. Like all software, it understands only numbers. As users of this software, we must map the real world to a set of numbers to feed to AI, and then map the numbers from its output back to the real world to make any decisions. This mapping varies from problem to problem and domain to domain.
A feature is any type of input that an AI model must consider to predict something. Consider the facial recognition problem. Each hue (red, green, blue) at each pixel of an image would be a feature. If the image comes with some metatags like time of the day or location, they are additional features. A facial recognition software accepts all these feature and outputs certain unique facial markers that can describe a face. Another system can then match these facial markers to a person.
Feature engineering is the art of mapping real world things in our domain to numbers in the domain of computers.
Any business manager who has dealt with AI knows that their team must first train it by providing it with inputs whose outputs are known. The model compares its predicted output with the known output to learn. In general, a model predicts better if it has been trained on more training data; or if it employs more complex algorithm, provided that there is sufficient data to train for it; or if it uses better feature engineering; among other factors. Getting the right training data is sometimes very expensive. Complexity incurs additional costs in terms for its need for more data and the data science team that must manage exponentially larger set of variables.
Feature engineering comes for free, relatively. All else being equal a better featured model can be less complex, require less data to train, cost less, and, most importantly, be more accurate. There are two aspects of it. I have laid out both, but business managers should especially focus on the second.
1. 8 Proven Ways to Use Chatbots for Marketing (with Real Examples)
2. How to Use Texthero to Prepare a Text-based Dataset for Your NLP Project
For data scientists, feature engineering is a way to make the process of training their models more efficient by using inherent opportunities within the data. In most situations they are dealing with the constraint of available data. So they may use techniques to make learning more data efficient. For example, for facial recognition, results are similar if we take a group of pixels together, rather than each pixel individually. However, such grouping (also called convolution, as in Convolutional Neural Network) reduces the number of features and data requirement exponentially.
Data scientists may rank order features and retain only the top few. If there is plenty of data, they take multiple samples for multiple feature sets and use an ensemble technique to get the best results. For interdependent features they may create new ones to try and resolve the interdependence. Many times data scientists create multiple layers of a machine, where the lower layers are meant to automatically learn features and feed to higher layers. Terms like auto-encoders, attention-mechanisms, ensemble-stacking become relevant for this.
It is important for business managers to understand that such feature engineering is iterative, and is never complete, really. It may be a good idea for them to involve themselves in these discussions if only to understand what progress is being made on various metrics, how does that relate to progress in tangible business terms, and when is the time to stop.
Business managers have a far more important role to play other than monitoring progress of data science teams — it is to transfer their domain knowledge to the learning models. Feature engineering is what enables it.
For facial recognition, the simple knowledge that ear-eye-nose-eye-ear come in a sequence goes a long way in making machines efficient. Once we feed in approximate shapes of various facial features, it becomes even better. The knowledge that mouth is generally below the nose also helps. The data science team may choose to discard the original feature-set of pixels and RBG hues, and instead focus on identifying the facial features and their location in the picture. Not only these features are more useful, they are far fewer in number. Fewer features means less training-data is sufficient and the model can be less complex.
Let’s take something simpler. Say you are trying to model whether to send a discount coupon for a new line of baby-food products. It’s a perfect world and you have all the data you need — ages of children living in every household, annual household income, shopping destinations, payment modes, etc. A starting point for this model could be to take all these features and predict, per household, whether to send a discount coupon.
However you can use certain heuristics — add a flag if the household has a child younger than three; or combine data at a block level; or quantify the level of digitization in the family based on payment modes; etc. It may turn out that the right answer is to promote a sale at the local grocery store in certain zip codes, instead of sending the mailers. Your data science team will be able to get to such insights, but only if you are involved in the feature engineering discussion.
Each domain and each problem has similar insights. As a business manager it is your job to explain these to your data science team, and help them in feature engineering. This is very self-serving, because a job well done means more accuracy, higher scalability and higher ROI.
