Automated Feature Engineering in Machine Learning

In Machine Learning, the supervised labels are used as inputs to Unsupervised algorithms such as Clustering. At times we do have to engineer our features to fit the algorithm such that the algorithm produces consistent results. To customise such implementations, a kernel based method becomes useful for performing predictions on the domain.

Feature Engineering involves:

  • Feature scaling and transformation
  • Feature normalisation and regularisation
  • Model evaluation and validation

Here, an automated feature engineering process is demonstrated using SVM kernel method and Multi-Layer Perceptron.

import torch
import torch.nn as nn
import numpy as np
from sklearn.svm import SVR
def svm_kernel():
return np.matmul(X[:,0:2]**2 + X[:,2:4]**2, \
(np.sqrt(X[:,0:2]) + np.sqrt(X[:,2:4])).transpose())
svm = SVR(C=1.0, kernel=svm_kernel)m = nn.Linear(10*15,4)input_ = torch.from_numpy(np.array([np.random.randn(1,10*15) for l in range(1000)])).to(dtype=torch.float)output_ = [m(ip) for ip in input_]X = np.square(np.random.randn(10*15,4))
Y = np.square(np.rabdom.randn(10*15))
sm.fit(X, Y)svm.predict(output[idx].detach.numpy())

Automating the attributes to be plugged into another algorithm will ensure the results produced have homogeneous behaviour. Ensemble and Gradient Boosting methods are some of the examples of heterogeneous machine learning algorithms. Computer vision algorithms such as Viola Jones and HOG are some of the examples of kernel based methods.

Developer | Scholar | Computing | Data Governance

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store