Automated Feature Engineering in Machine Learning
In Machine Learning, the supervised labels are used as inputs to Unsupervised algorithms such as Clustering. At times we do have to engineer our features to fit the algorithm such that the algorithm produces consistent results. To customise such implementations, a kernel based method becomes useful for performing predictions on the domain.
Feature Engineering involves:
- Feature scaling and transformation
- Feature normalisation and regularisation
- Model evaluation and validation
Here, an automated feature engineering process is demonstrated using SVM kernel method and Multi-Layer Perceptron.
import torch
import torch.nn as nn
import numpy as np
from sklearn.svm import SVRdef svm_kernel():
return np.matmul(X[:,0:2]**2 + X[:,2:4]**2, \
(np.sqrt(X[:,0:2]) + np.sqrt(X[:,2:4])).transpose())svm = SVR(C=1.0, kernel=svm_kernel)m = nn.Linear(10*15,4)input_ = torch.from_numpy(np.array([np.random.randn(1,10*15) for l in range(1000)])).to(dtype=torch.float)output_ = [m(ip) for ip in input_]X = np.square(np.random.randn(10*15,4))
Y = np.square(np.rabdom.randn(10*15))sm.fit(X, Y)svm.predict(output[idx].detach.numpy())
Automating the attributes to be plugged into another algorithm will ensure the results produced have homogeneous behaviour. Ensemble and Gradient Boosting methods are some of the examples of heterogeneous machine learning algorithms. Computer vision algorithms such as Viola Jones and HOG are some of the examples of kernel based methods.
