Accord.MachineLearning
Learning algorithm for s.
The algorithm is mostly intended to be used to create
weak classifiers in the context of an
learning algorithm. Please refer to the class for more examples
on using the classifier in this scenario. A simple example is shown below:
Adapter for models that do not implement a .Decide function.
The type for the weak classifier model.
Gets or sets the weak decision model.
Gets or sets the decision function used by the .
Creates a new Weak classifier given a
classification model and its decision function.
The classifier.
The classifier decision function.
Computes the classifier decision for a given input.
The input vector.
The model's decision label.
Computes a class-label decision for a given .
The input vector that should be classified into
one of the possible classes.
A class-label that best described according
to this classifier.
Simple classifier that based on decision margins that
are perpendicular to one of the space dimensions.
The classifier is mostly intended to be used as a weak classifier
in the context of an learning algorithm. Please refer to the
class for more examples on using the classifier in this scenario.
A simple example is shown below:
-
Wikipedia contributors. "AdaBoost." Wikipedia, The Free Encyclopedia. Wikipedia, The
Free Encyclopedia, 10 Aug. 2017. Web. 7 Sep. 2017
-
M. I. Malinen and P.Fränti, "Balanced K-means for Clustering", Joint Int.Workshop on Structural, Syntactic,
and Statistical Pattern Recognition (S+SSPR 2014), LNCS 8621, 32-41, Joensuu, Finland, August 2014.
-
M. I. Malinen, "New alternatives for k-Means clustering." PhD thesis. Available in:
http://cs.uef.fi/sipu/pub/PhD_Thesis_Mikko_Malinen.pdf
How to perform clustering with Balanced K-Means.
-
Place K points into the space represented by the objects that are
being clustered. These points represent initial group centroids.
-
Form a batch by choosing B objects from the whole input dataset.
For each object in the batch, determine the group that has the closest centroid.
Then, update the centroid with a gradient step.
-
Repeat step 2 until the centroids converge or the maximal number of iterations has been performed.
References:
-
D. Sculley. Web-Scale K-Means Clustering. Available on:
https://www.eecs.tufts.edu/~dsculley/papers/fastkmeans.pdf
-
Quinlan, J. R. C4.5: Programs for Machine Learning. Morgan
Kaufmann Publishers, 1993.
-
Quinlan, J. R. C4.5: Programs for Machine Learning. Morgan
Kaufmann Publishers, 1993.
-
Quinlan, J. R. Improved use of continuous attributes in c4.5. Journal
of Artificial Intelligence Research, 4:77-90, 1996.
-
Mitchell, T. M. Machine Learning. McGraw-Hill, 1997. pp. 55-58.
-
Wikipedia, the free encyclopedia. ID3 algorithm. Available on
http://en.wikipedia.org/wiki/ID3_algorithm
This example shows the simplest way to induce a decision tree with continuous variables.
-
Quinlan, J. R 1986. Induction of Decision Trees.
Mach. Learn. 1, 1 (Mar. 1986), 81-106.
-
Mitchell, T. M. Machine Learning. McGraw-Hill, 1997. pp. 55-58.
-
Wikipedia, the free encyclopedia. ID3 algorithm. Available on
http://en.wikipedia.org/wiki/ID3_algorithm
This example shows the simplest way to induce a decision tree with discrete variables.
-
Lior Rokach, Oded Maimon. The Data Mining and Knowledge Discovery Handbook,
Chapter 9, Decision Trees. Springer, 2nd ed. 2010, XX, 1285 p. 40 illus.
Available at: http://www.ise.bgu.ac.il/faculty/liorr/hbchap9.pdf .
// Suppose you have the following input and output data
// and would like to learn the relationship between the
// inputs and outputs by using a Decision Tree:
double[][] inputs = ...
int[] output = ...
// To prune a decision tree, we need to split your data into
// training and pruning groups. Let's say we have 100 samples,
// and would like to reserve 50 samples for training, and 50
// for pruning:
// Gather the first half for the training set
var trainingInputs = inputs.Submatrix(0, 49);
var trainingOutput = output.Submatrix(0, 49);
// Gather the second hand data for pruning
var pruningInputs = inputs.Submatrix(50, 99);
var pruningOutput = output.Submatrix(50, 99);
// Create the decision tree
DecisionTree tree = new DecisionTree( ... );
// Learn our tree using the training data
C45Learning c45 = new C45Learning(tree);
double error = c45.Run(trainingInputs, trainingOutput);
// Now we can attempt to prune the tree using the pruning groups
ErrorBasedPruning prune = new ErrorBasedPruning(tree, pruningInputs, pruningOutput);
// Gain threshold
prune.Threshold = 0.1;
double lastError;
double error = Double.PositiveInfinity;
do
{
// Now we can start pruning the tree as
// long as the error doesn't increase
lastError = error;
error = prune.Run();
} while (error < lastError);
Initializes a new instance of the class.
The tree to be pruned.
The pruning set inputs.
The pruning set outputs.
Gets or sets the minimum allowed gain threshold
to prune the tree. Default is 0.01.
Computes one pass of the pruning algorithm.
Attempts to prune a node's subtrees.
Whether the current node was changed or not.
Random Forest learning algorithm.
This example shows the simplest way to induce a decision tree with continuous variables.
-
Wikipedia, The Free Encyclopedia. Cross-validation (statistics). Available on:
http://en.wikipedia.org/wiki/Cross-validation_(statistics)
The type of the machine learning model.
The type of the input data.
The type of the output data or labels.
The type of the learning algorithm used to learn .
-
Wikipedia, The Free Encyclopedia. Cross-validation (statistics). Available on:
http://en.wikipedia.org/wiki/Cross-validation_(statistics)
The type of the machine learning model.
The type of the input data.
-
Wikipedia, The Free Encyclopedia. Cross-validation (statistics). Available on:
http://en.wikipedia.org/wiki/Cross-validation_(statistics)
The type of the machine learning model.
The type of the input data.
The type of the output data or labels.
-
Anita Wasilewska, Lecture Notes. Available on
http://www3.cs.stonybrook.edu/~cse634/lecture_notes/07apriori.pdf
>
-
Anita Wasilewska, Lecture Notes. Available on
http://www3.cs.stonybrook.edu/~cse634/lecture_notes/07apriori.pdf
>
-
Wikipedia contributors. "Naive Bayes classifier." Wikipedia, The Free Encyclopedia.
Wikipedia, The Free Encyclopedia, 16 Dec. 2011. Web. 5 Jan. 2012.
This page contains two examples, one using text and another one using normal double vectors.
The first example is the classic example given by Tom Mitchell. If you are not interested
in text or in this particular example, please jump to the second example below.
In the first example, we will be using a mixed-continuous version of the famous Play Tennis
example by Tom Mitchell (1998). In Mitchell's example, one would like to infer if a person
would play tennis or not based solely on four input variables. The original variables were
categorical, but in this example, two of them will be categorical and two will be continuous.
The rows, or instances presented below represent days on which the behavior of the person
has been registered and annotated, pretty much building our set of observation instances for
learning:
-
Wikipedia contributors. "Naive Bayes classifier." Wikipedia, The Free Encyclopedia.
Wikipedia, The Free Encyclopedia, 16 Dec. 2011. Web. 5 Jan. 2012.
In this example, we will be using the famous Play Tennis example by Tom Mitchell (1998).
In Mitchell's example, one would like to infer if a person would play tennis or not
based solely on four input variables. Those variables are all categorical, meaning that
there is no order between the possible values for the variable (i.e. there is no order
relationship between Sunny and Rain, one is not bigger nor smaller than the other, but are
just distinct). Moreover, the rows, or instances presented below represent days on which the
behavior of the person has been registered and annotated, pretty much building our set of
observation instances for learning:
-
Wikipedia, The Free Encyclopedia. Mean-shift. Available on:
http://en.wikipedia.org/wiki/Mean-shift
-
Comaniciu, Dorin, and Peter Meer. "Mean shift: A robust approach toward
feature space analysis." Pattern Analysis and Machine Intelligence, IEEE
Transactions on 24.5 (2002): 603-619. Available at:
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1000236
-
Conrad Lee. Scalable mean-shift clustering in a few lines of python. The
Sociograph blog, 2011. Available at:
http://sociograph.blogspot.com.br/2011/11/scalable-mean-shift-clustering-in-few.html
-
Carreira-Perpinan, Miguel A. "Acceleration strategies for Gaussian mean-shift image
segmentation." Computer Vision and Pattern Recognition, 2006 IEEE Computer Society
Conference on. Vol. 1. IEEE, 2006. Available at:
http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=1640881
The following example demonstrates how to use the Mean Shift algorithm with
a uniform kernel to solve a clustering task:
The resulting image will be:
Gets the clusters found by Mean Shift.
Gets or sets the used to
compute distances between points in the clustering.
Gets or sets the bandwidth (radius, or smoothness)
parameter to be used in the mean-shift algorithm.
Gets or sets the maximum number of neighbors which should be
used to determine the direction of the mean-shift during the
computations. Default is zero (unlimited number of neighbors).
Gets or sets whether the algorithm can use parallel
processing to speedup computations. Enabling parallel
processing can, however, result in different results
at each run.
Gets or sets whether to use the agglomeration shortcut,
meaning the algorithm will stop early when it detects that
a sample is going to follow the same path as another sample
when running in parallel.
Gets or sets whether to use seeding to initialize the algorithm.
With seeding, new points will be sampled from an uniform grid in
the range of the input points to be used as seeds. Otherwise, the
input points themselves will be used as the initial centroids for
the algorithm.
Gets or sets whether cluster labels should be computed
at the end of the learning iteration. Setting to False
might save a few computations in case they are not necessary.
Gets or sets whether cluster proportions should be computed
at the end of the learning iteration. Setting to False
might save a few computations in case they are not necessary.
Gets the dimension of the samples being
modeled by this clustering algorithm.
Gets or sets the maximum number of iterations to
be performed by the method. If set to zero, no
iteration limit will be imposed. Default is 0.
Gets or sets the relative convergence threshold
for stopping the algorithm. Default is 1e-3.
Gets or sets the density kernel to be used in the algorithm.
Default is to use the .
Creates a new algorithm.
Creates a new algorithm.
The bandwidth (also known as radius) to consider around samples.
The density kernel function to use.
Creates a new algorithm.
The dimension of the samples to be clustered.
The bandwidth (also known as radius) to consider around samples.
The density kernel function to use.
Divides the input data into clusters.
The data where to compute the algorithm.
Divides the input data into clusters.
The data where to compute the algorithm.
The weight associated with each data point.
Learns a model that can map the given inputs to the desired outputs.
The model inputs.
The weight of importance for each input sample.
A model that has learned how to produce suitable outputs
given the input data .
Learns a model that can map the given inputs to the desired outputs.
The model inputs.
The weight of importance for each input sample.
A model that has learned how to produce suitable outputs
given the input data .
Barnes-Hutt t-SNE.
The code contained in this class was adapted from Laurens van der Maaten excellent
BH T-SNE code from https://github.com/lvdmaaten/bhtsne/. The original license is
listed below:
Copyright (c) 2014, Laurens van der Maaten (Delft University of Technology)
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. All advertising materials mentioning features or use of this software
must display the following acknowledgement:
This product includes software developed by the Delft University of Technology.
4. Neither the name of the Delft University of Technology nor the names of
its contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY LAURENS VAN DER MAATEN ''AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
EVENT SHALL LAURENS VAN DER MAATEN BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
OF SUCH DAMAGE.
exp( Q( s, a ) / t )
p( s, a ) = -----------------------------
SUM( exp( Q( s, b ) / t ) )
b
where Q(s, a) is action's a estimation (usefulness) at state s and
t is .
Temperature parameter of Boltzmann distribution. Should be greater than 0.
The property sets the balance between exploration and greedy actions.
If temperature is low, then the policy tends to be more greedy.
Initializes a new instance of the class.
Temperature parameter of Boltzmann distribution.
Choose an action.
Action estimates.
Returns selected action.
The method chooses an action depending on the provided estimates. The
estimates can be any sort of estimate, which values usefulness of the action
(expected summary reward, discounted reward, etc).
Epsilon greedy exploration policy.
The class implements epsilon greedy exploration policy. According to the policy,
the best action is chosen with probability 1-epsilon. Otherwise,
with probability epsilon, any other action, except the best one, is
chosen randomly.
According to the policy, the epsilon value is known also as exploration rate.
Epsilon value (exploration rate), [0, 1].
The value determines the amount of exploration driven by the policy.
If the value is high, then the policy drives more to exploration - choosing random
action, which excludes the best one. If the value is low, then the policy is more
greedy - choosing the beat so far action.
Initializes a new instance of the class.
Epsilon value (exploration rate).
Choose an action.
Action estimates.
Returns selected action.
The method chooses an action depending on the provided estimates. The
estimates can be any sort of estimate, which values usefulness of the action
(expected summary reward, discounted reward, etc).
Exploration policy interface.
The interface describes exploration policies, which are used in Reinforcement
Learning to explore state space.
Choose an action.
Action estimates.
Returns selected action.
The method chooses an action depending on the provided estimates. The
estimates can be any sort of estimate, which values usefulness of the action
(expected summary reward, discounted reward, etc).
Roulette wheel exploration policy.
The class implements roulette whell exploration policy. Acording to the policy,
action a at state s is selected with the next probability:
Q( s, a )
p( s, a ) = ------------------
SUM( Q( s, b ) )
b
where Q(s, a) is action's a estimation (usefulness) at state s.
The exploration policy may be applied only in cases, when action estimates (usefulness)
are represented with positive value greater then 0.
Initializes a new instance of the class.
Choose an action.
Action estimates.
Returns selected action.
The method chooses an action depending on the provided estimates. The
estimates can be any sort of estimate, which values usefulness of the action
(expected summary reward, discounted reward, etc).
Tabu search exploration policy.
The class implements simple tabu search exploration policy,
allowing to set certain actions as tabu for a specified amount of
iterations. The actual exploration and choosing from non-tabu actions
is done by base exploration policy.
Base exploration policy.
Base exploration policy is the policy, which is used
to choose from non-tabu actions.
Initializes a new instance of the class.
Total actions count.
Base exploration policy.
Choose an action.
Action estimates.
Returns selected action.
The method chooses an action depending on the provided estimates. The
estimates can be any sort of estimate, which values usefulness of the action
(expected summary reward, discounted reward, etc). The action is choosed from
non-tabu actions only.
Reset tabu list.
Clears tabu list making all actions allowed.
Set tabu action.
Action to set tabu for.
Tabu time in iterations.
Robust circle estimator with RANSAC.
Gets the RANSAC estimator used.
Gets the final set of inliers detected by RANSAC.
Creates a new RANSAC 2D circle estimator.
Inlier threshold.
Inlier probability.
Produces a robust estimation of the circle
passing through the given (noisy) points.
A set of (possibly noisy) points.
The circle passing through the points.
Produces a robust estimation of the circle
passing through the given (noisy) points.
A set of (possibly noisy) points.
The circle passing through the points.
Produces a robust estimation of the circle
passing through the given (noisy) points.
A set of (possibly noisy) points.
The circle passing through the points.
Produces a robust estimation of the circle
passing through the given (noisy) points.
A set of (possibly noisy) points.
The circle passing through the points.
Robust line estimator with RANSAC.
Gets the RANSAC estimator used.
Gets the final set of inliers detected by RANSAC.
Creates a new RANSAC line estimator.
Inlier threshold.
Inlier probability.
Produces a robust estimation of the line
passing through the given (noisy) points.
A set of (possibly noisy) points.
The line passing through the points.
Produces a robust estimation of the line
passing through the given (noisy) points.
A set of (possibly noisy) points.
The line passing through the points.
Produces a robust estimation of the line
passing through the given (noisy) points.
A set of (possibly noisy) points.
The line passing through the points.
Produces a robust estimation of the line
passing through the given (noisy) points.
A set of (possibly noisy) points.
The line passing through the points.
Robust plane estimator with RANSAC.
Gets the RANSAC estimator used.
Gets the final set of inliers detected by RANSAC.
Creates a new RANSAC 3D plane estimator.
Inlier threshold.
Inlier probability.
Produces a robust estimation of the plane
passing through the given (noisy) points.
A set of (possibly noisy) points.
The plane passing through the points.
K-Nearest Neighbor (k-NN) algorithm.
The type of the input data.
The k-nearest neighbor algorithm (k-NN) is a method for classifying objects
based on closest training examples in the feature space. It is amongst the simplest
of all machine learning algorithms: an object is classified by a majority vote of
its neighbors, with the object being assigned to the class most common amongst its
k nearest neighbors (k is a positive integer, typically small).
If k = 1, then the object is simply assigned to the class of its nearest neighbor.
References:
-
Wikipedia contributors. "K-nearest neighbor algorithm." Wikipedia, The
Free Encyclopedia. Wikipedia, The Free Encyclopedia, 10 Oct. 2012. Web.
9 Nov. 2012. http://en.wikipedia.org/wiki/K-nearest_neighbor_algorithm
The first example shows how to create and use a k-Nearest Neighbor algorithm to classify
a set of numeric vectors in a multi-class decision problem involving 3 classes. It also shows
how to compute class decisions for a new sample and how to measure the performance of a classifier.
Please note that class diagrams for each of the inner namespaces are
also available within their own documentation pages.
Common interface for binary support vector machines.
The type of the input data handled by the machine.
Gets or sets the collection of weights used by this machine.
Gets or sets the collection of support vectors used by this machine.
Gets or sets the threshold (bias) term for this machine.
Gets or sets the kernel used by this machine.
Gets whether this machine has been calibrated to
produce probabilistic outputs (through the Probability
method).
If this machine has a linear kernel, compresses all
support vectors into a single parameter vector.
Obsolete.
Obsolete.
Obsolete.
Obsolete.
Obsolete.
Base class for calibration algorithms.
Gets or sets a cancellation token that can be used to
stop the learning algorithm while it is running.
Gets or sets the input vectors for training.
Gets or sets the output labels for each training vector.
Initializes a new instance of the class.
The machine to be calibrated.
Creates an instance of the model to be learned. Inheritors
of this abstract class must define this method so new models
can be created from the training data.
Initializes a new instance of the class.
The machine to be calibrated.
Initializes a new instance of the class.
Gets whether the machine being learned is linear.
Gets the machine's function.
Gets the machine to be taught.
Learns a model that can map the given inputs to the given outputs.
The model inputs.
The desired outputs associated with each inputs.
The weight of importance for each input-output pair (if supported by the learning algorithm).
A model that has learned how to produce given .
Learns a model that can map the given inputs to the given outputs.
The model inputs.
The desired outputs associated with each inputs.
The weight of importance for each input-output pair (if supported by the learning algorithm).
A model that has learned how to produce given .
Learns a model that can map the given inputs to the given outputs.
The model inputs.
The desired outputs associated with each inputs.
The weight of importance for each input-output pair (if supported by the learning algorithm).
A model that has learned how to produce given .
Learns a model that can map the given inputs to the given outputs.
The model inputs.
The desired outputs associated with each inputs.
The weight of importance for each input-output pair (if supported by the learning algorithm).
A model that has learned how to produce given .
Learns a model that can map the given inputs to the given outputs.
The model inputs.
The desired outputs associated with each inputs.
The weight of importance for each input-output pair (if supported by the learning algorithm).
A model that has learned how to produce given .
Runs the learning algorithm.
Obsolete.
Obsolete.
Base class for learning algorithms.
Initializes a new instance of the class.
Obsolete.
Gets or sets the input vectors for training.
Gets or sets the output labels for each training vector.
Complexity (cost) parameter C. Increasing the value of C forces the creation
of a more accurate model that may not generalize well. If this value is not
set and is set to true, the framework
will automatically guess a value for C. If this value is manually set to
something else, then will be automatically
disabled and the given value will be used instead.
The cost parameter C controls the trade off between allowing training
errors and forcing rigid margins. It creates a soft margin that permits
some misclassifications. Increasing the value of C increases the cost of
misclassifying points and forces the creation of a more accurate model
that may not generalize well.
If this value is not set and is set to
true, the framework will automatically guess a suitable value for C by
calling . If this value
is manually set to something else, then the class will respect the new value
and automatically disable .
Gets or sets the positive class weight. This should be a
value higher than 0 indicating how much of the
parameter C should be applied to instances carrying the positive label.
Gets or sets the negative class weight. This should be a
value higher than 0 indicating how much of the
parameter C should be applied to instances carrying the negative label.
Gets or sets the weight ratio between positive and negative class
weights. This ratio controls how much of the
parameter C should be applied to the positive class.
A weight ratio lesser than one, such as 1/10 (0.1) means 10% of C will
be applied to the positive class, while 100% of C will be applied to the
negative class.
A weight ratio greater than one, such as 10/1 (10) means that 100% of C will
be applied to the positive class, while 10% of C will be applied to the
negative class.
Gets or sets a value indicating whether the Complexity parameter C
should be computed automatically by employing an heuristic rule.
Default is true.
true if complexity should be computed automatically; otherwise, false.
Gets or sets whether initial values for some kernel parameters
should be estimated from the data, if possible. Default is true.
Gets or sets a value indicating whether the weight ratio to be used between
values for negative and positive instances should
be computed automatically from the data proportions. Default is false.
true if the weighting coefficient should be computed
automatically from the data; otherwise, false.
Gets or sets the kernel function use to create a
kernel Support Vector Machine. If this property
is set, will be
set to false.
Gets or sets the cost values associated with each input vector.
Creates an instance of the model to be learned. Inheritors
of this abstract class must define this method so new models
can be created from the training data.
Learns a model that can map the given inputs to the given outputs.
The model inputs.
The desired outputs associated with each inputs.
The weight of importance for each input-output pair (if supported by the learning algorithm).
A model that has learned how to produce given .
Runs the main body of the learning algorithm.
Computes the error rate for a given set of input and outputs.
Obsolete.
Obsolete.
Base class for regression learning algorithms.
Gets or sets a cancellation token that can be used to
stop the learning algorithm while it is running.
Gets or sets the cost values associated with each input vector.
Initializes a new instance of the class.
Obsolete.
Complexity (cost) parameter C. Increasing the value of C forces the creation
of a more accurate model that may not generalize well. If this value is not
set and is set to true, the framework
will automatically guess a value for C. If this value is manually set to
something else, then will be automatically
disabled and the given value will be used instead.
The cost parameter C controls the trade off between allowing training
errors and forcing rigid margins. It creates a soft margin that permits
some misclassifications. Increasing the value of C increases the cost of
misclassifying points and forces the creation of a more accurate model
that may not generalize well.
If this value is not set and is set to
true, the framework will automatically guess a suitable value for C by
calling . If this value
is manually set to something else, then the class will respect the new value
and automatically disable .
Insensitivity zone ε. Increasing the value of ε can result in fewer
support vectors in the created model. Default value is 1e-3.
Parameter ε controls the width of the ε-insensitive zone, used to fit the training
data. The value of ε can affect the number of support vectors used to construct the
regression function. The bigger ε, the fewer support vectors are selected. On the
other hand, bigger ε-values results in more flat estimates.
Gets or sets the individual weight of each sample in the training set. If set
to null, all samples will be assumed equal weight. Default is null.
Gets or sets a value indicating whether the Complexity parameter C
should be computed automatically by employing an heuristic rule.
Default is false.
true if complexity should be computed automatically; otherwise, false.
Gets or sets whether initial values for some kernel parameters
should be estimated from the data, if possible. Default is true.
Gets whether the machine to be learned
has a kernel.
Gets or sets the kernel function use to create a
kernel Support Vector Machine. If this property
is set, will be
set to false.
Gets or sets the input vectors for training.
Gets or sets the output values for each calibration vector.
Gets the machine to be taught.
Creates an instance of the model to be learned. Inheritors
of this abstract class must define this method so new models
can be created from the training data.
Learns a model that can map the given inputs to the given outputs.
The model inputs.
The desired outputs associated with each inputs.
The weight of importance for each input-output pair (if supported by the learning algorithm).
A model that has learned how to produce given .
Runs the learning algorithm.
Obsolete.
Obsolete.
Common interface for Support Machine Vector learning algorithms.
Common interface for Support Machine Vector learning algorithms.
Common interface for Support Machine Vector learning algorithms.
Obsolete.
Obsolete.
Common interface for Support Machine Vector learning algorithms.
Gets or sets the support vector machine being learned.
Obsolete.
Obsolete.
Least Squares SVM (LS-SVM) learning algorithm.
References:
-
Suykens, J. A. K., et al. "Least squares support vector machine classifiers: a large scale
algorithm." European Conference on Circuit Theory and Design, ECCTD. Vol. 99. 1999. Available on:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.43.6438
Obsolete.
Initializes a new instance of the class.
Creates an instance of the model to be learned. Inheritors
of this abstract class must define this method so new models
can be created from the training data.
Least Squares SVM (LS-SVM) learning algorithm.
References:
-
Suykens, J. A. K., et al. "Least squares support vector machine classifiers: a large scale
algorithm." European Conference on Circuit Theory and Design, ECCTD. Vol. 99. 1999. Available on:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.43.6438
Creates an instance of the model to be learned. Inheritors
of this abstract class must define this method so new models
can be created from the training data.
Base class for Least Squares SVM (LS-SVM) learning algorithm.
Constructs a new Least Squares SVM (LS-SVM) learning algorithm.
Convergence tolerance. Default value is 1e-6.
The criterion for completing the model training process. The default is 1e-6.
Gets or sets the cache size to partially
stored the kernel matrix. Default is the
same number of input vectors.
Runs the main body of the learning algorithm.
Obsolete.
Averaged Stochastic Gradient Descent (ASGD) for training linear support vector machines.
min_w \sum |wj| + C \sum max(0, 1-yi w^T xi)^2,
Given: x, y, Cp, Cn and eps as the stopping tolerance
See Yuan et al. (2010) and appendix of LIBLINEAR paper, Fan et al. (2008)
The following example shows how to obtain a
from a linear . It contains exactly the same data
used in the documentation page for
.
min_w \sum |wj| + C \sum max(0, 1-yi w^T xi)^2,
Given: x, y, Cp, Cn and eps as the stopping tolerance
See Yuan et al. (2010) and appendix of LIBLINEAR paper, Fan et al. (2008)
Creates an instance of the model to be learned. Inheritors
of this abstract class must define this method so new models
can be created from the training data.
Base class for linear coordinate descent learning algorithm.
Constructs a new coordinate descent algorithm for L1-loss and L2-loss SVM dual problems.
Gets the value for the Lagrange multipliers
(alpha) for every observation vector.
Convergence tolerance. Default value is 0.1.
The criterion for completing the model training process. The default is 0.1.
Runs the learning algorithm.
Obsolete.
Obsolete.
Different categories of loss functions that can be used to learn
support vector machines.
Hinge-loss function.
Squared hinge-loss function.
L2-regularized, L1 or L2-loss dual formulation
Support Vector Machine learning (-s 1 and -s 3).
This class implements a learning algorithm
specifically crafted for linear machines only. It provides a L2-regularized, L1
or L2-loss coordinate descent learning algorithm for optimizing the dual form of
learning. The code has been based on liblinear's method solve_l2r_l1l2_svc
method, whose original description is provided below.
Liblinear's solver -s 1: L2R_L2LOSS_SVC_DUAL and -s 3:
L2R_L1LOSS_SVC_DUAL. A coordinate descent algorithm for L1-loss and
L2-loss SVM problems in the dual.
min_\alpha 0.5(\alpha^T (Q + D)\alpha) - e^T \alpha,
s.t. 0 <= \alpha_i <= upper_bound_i,
where Qij = yi yj xi^T xj and
D is a diagonal matrix
In L1-SVM case:
upper_bound_i = Cp if y_i = 1
upper_bound_i = Cn if y_i = -1
D_ii = 0
In L2-SVM case:
upper_bound_i = INF
D_ii = 1/(2*Cp) if y_i = 1
D_ii = 1/(2*Cn) if y_i = -1
Given: x, y, Cp, Cn, and eps as the stopping tolerance
See Algorithm 3 of Hsieh et al., ICML 2008.
The next example shows how to solve a multi-class problem using a one-vs-one SVM
where the binary machines are learned using the Linear Dual Coordinate Descent algorithm.
min_\alpha 0.5(\alpha^T (Q + D)\alpha) - e^T \alpha,
s.t. 0 <= \alpha_i <= upper_bound_i,
where Qij = yi yj xi^T xj and
D is a diagonal matrix
In L1-SVM case:
upper_bound_i = Cp if y_i = 1
upper_bound_i = Cn if y_i = -1
D_ii = 0
In L2-SVM case:
upper_bound_i = INF
D_ii = 1/(2*Cp) if y_i = 1
D_ii = 1/(2*Cn) if y_i = -1
Given: x, y, Cp, Cn, and eps as the stopping tolerance
See Algorithm 3 of Hsieh et al., ICML 2008.
The next example shows how to solve a multi-class problem using a one-vs-one SVM
where the binary machines are learned using the Linear Dual Coordinate Descent algorithm.
min_\alpha 0.5(\alpha^T (Q + D)\alpha) - e^T \alpha,
s.t. 0 <= \alpha_i <= upper_bound_i,
where Qij = yi yj xi^T xj and
D is a diagonal matrix
In L1-SVM case:
upper_bound_i = Cp if y_i = 1
upper_bound_i = Cn if y_i = -1
D_ii = 0
In L2-SVM case:
upper_bound_i = INF
D_ii = 1/(2*Cp) if y_i = 1
D_ii = 1/(2*Cn) if y_i = -1
Given: x, y, Cp, Cn, and eps as the stopping tolerance
See Algorithm 3 of Hsieh et al., ICML 2008.
The next example shows how to solve a multi-class problem using a one-vs-one SVM
where the binary machines are learned using the Linear Dual Coordinate Descent algorithm.
min_w \sum |wj| + C \sum log(1+exp(-yi w^T xi)),
Given: x, y, Cp, Cn, and eps as the stopping tolerance
See Yuan et al. (2011) and appendix of LIBLINEAR paper, Fan et al. (2008)
Probabilistic SVMs are exactly the same as logistic regression models
trained using a large-margin decision criteria. As such, any linear SVM
learning algorithm can be used to obtain
objects as well.
The following example shows how to obtain a
from a probabilistic linear . It contains
exactly the same data used in the
documentation page for .
min_w \sum |wj| + C \sum log(1+exp(-yi w^T xi)),
Given: x, y, Cp, Cn, and eps as the stopping tolerance
See Yuan et al. (2011) and appendix of LIBLINEAR paper, Fan et al. (2008)
Probabilistic SVMs are exactly the same as logistic regression models
trained using a large-margin decision criteria. As such, any linear SVM
learning algorithm can be used to obtain
objects as well.
The following example shows how to obtain a
from a probabilistic linear . It contains
exactly the same data used in the
documentation page for .
min_\alpha 0.5(\alpha^T Q \alpha) + \sum \alpha_i log (\alpha_i)
+ (upper_bound_i - \alpha_i) log (upper_bound_i - \alpha_i),
s.t. 0 <= \alpha_i <= upper_bound_i,
where Qij = yi yj xi^T xj and
upper_bound_i = Cp if y_i = 1
upper_bound_i = Cn if y_i = -1
Given: x, y, Cp, Cn, and eps as the stopping tolerance
See Algorithm 5 of Yu et al., MLJ 2010.
Probabilistic SVMs are exactly the same as logistic regression models
trained using a large-margin decision criteria. As such, any linear SVM
learning algorithm can be used to obtain
objects as well.
The following example shows how to obtain a
from a probabilistic linear . It contains
exactly the same data used in the
documentation page for .
min_\alpha 0.5(\alpha^T Q \alpha) + \sum \alpha_i log (\alpha_i)
+ (upper_bound_i - \alpha_i) log (upper_bound_i - \alpha_i),
s.t. 0 <= \alpha_i <= upper_bound_i,
where Qij = yi yj xi^T xj and
upper_bound_i = Cp if y_i = 1
upper_bound_i = Cn if y_i = -1
Given: x, y, Cp, Cn, and eps as the stopping tolerance
See Algorithm 5 of Yu et al., MLJ 2010.
Probabilistic SVMs are exactly the same as logistic regression models
trained using a large-margin decision criteria. As such, any linear SVM
learning algorithm can be used to obtain
objects as well.
The following example shows how to obtain a
from a probabilistic linear . It contains
exactly the same data used in the
documentation page for .
-
John C. Platt. 1999. Probabilistic Outputs for Support Vector Machines and Comparisons to
Regularized Likelihood Methods. In ADVANCES IN LARGE MARGIN CLASSIFIERS (1999), pp. 61-74.
-
Hsuan-Tien Lin, Chih-Jen Lin, and Ruby C. Weng. 2007. A note on Platt's probabilistic outputs
for support vector machines. Mach. Learn. 68, 3 (October 2007), 267-276.
The following example shows how to calibrate a SVM that has
been trained to perform a simple XOR function.
-
John C. Platt. 1999. Probabilistic Outputs for Support Vector Machines and Comparisons to
Regularized Likelihood Methods. In ADVANCES IN LARGE MARGIN CLASSIFIERS (1999), pp. 61-74.
-
Hsuan-Tien Lin, Chih-Jen Lin, and Ruby C. Weng. 2007. A note on Platt's probabilistic outputs
for support vector machines. Mach. Learn. 68, 3 (October 2007), 267-276.
The following example shows how to calibrate a SVM that has
been trained to perform a simple XOR function.
-
John C. Platt. 1999. Probabilistic Outputs for Support Vector Machines and Comparisons to
Regularized Likelihood Methods. In ADVANCES IN LARGE MARGIN CLASSIFIERS (1999), pp. 61-74.
-
Hsuan-Tien Lin, Chih-Jen Lin, and Ruby C. Weng. 2007. A note on Platt's probabilistic outputs
for support vector machines. Mach. Learn. 68, 3 (October 2007), 267-276.
The following example shows how to calibrate a SVM that has
been trained to perform a simple XOR function.
min_\beta 0.5\beta^T (Q + diag(lambda)) \beta - p \sum_{i=1}^l|\beta_i| + \sum_{i=1}^l yi\beta_i,
s.t. -upper_bound_i <= \beta_i <= upper_bound_i,
where Qij = yi yj xi^T xj and
D is a diagonal matrix
In L1-SVM case:
upper_bound_i = C
lambda_i = 0
In L2-SVM case:
upper_bound_i = INF
lambda_i = 1/(2*C)
Given: x, y, p, C and eps as the stopping tolerance
See Algorithm 4 of Ho and Lin, 2012.
Obsolete.
Initializes a new instance of the class.
Creates an instance of the model to be learned. Inheritors
of this abstract class must define this method so new models
can be created from the training data.
Coordinate descent algorithm for the L1 or L2-loss linear Support
Vector Regression (epsilon-SVR) learning problem in the dual form
(-s 12 and -s 13).
This class implements a learning algorithm
specifically crafted for linear machines only. It provides a L2-regularized, L1
or L2-loss coordinate descent learning algorithm for optimizing the dual form of
learning. The code has been based on liblinear's method solve_l2r_l1l2_svc
method, whose original description is provided below.
Liblinear's solver -s 12: L2R_L2LOSS_SVR_DUAL and -s 13:
L2R_L1LOSS_SVR_DUAL. A coordinate descent algorithm for L1-loss and
L2-loss linear epsilon-vector regression (epsilon-SVR).
min_\beta 0.5\beta^T (Q + diag(lambda)) \beta - p \sum_{i=1}^l|\beta_i| + \sum_{i=1}^l yi\beta_i,
s.t. -upper_bound_i <= \beta_i <= upper_bound_i,
where Qij = yi yj xi^T xj and
D is a diagonal matrix
In L1-SVM case:
upper_bound_i = C
lambda_i = 0
In L2-SVM case:
upper_bound_i = INF
lambda_i = 1/(2*C)
Given: x, y, p, C and eps as the stopping tolerance
See Algorithm 4 of Ho and Lin, 2012.
Initializes a new instance of the class.
Creates an instance of the model to be learned. Inheritors
of this abstract class must define this method so new models
can be created from the training data.
Base class for Coordinate descent algorithm for the L1 or L2-loss linear Support
Vector Regression (epsilon-SVR) learning problem in the dual form (-s 12 and -s 13).
Constructs a new coordinate descent algorithm for L1-loss and L2-loss SVM dual problems.
Gets or sets the cost function that
should be optimized. Default is
.
Gets the value for the Lagrange multipliers
(alpha) for every observation vector.
Convergence tolerance. Default value is 0.1.
The criterion for completing the model training process. The default is 0.1.
Runs the learning algorithm.
Obsolete.
L2-regularized L2-loss linear support vector regression
(SVR) learning algorithm in the primal formulation (-s 11).
This class implements a L2-regularized L2-loss support vector regression (SVR)
learning algorithm that operates in the primal form of the optimization problem.
This method has been based on liblinear's l2r_l2_svr_fun problem specification,
optimized using a Trust-region Newton method.
Liblinear's solver -s 11: L2R_L2LOSS_SVR. A trust region newton algorithm
for the primal of L2-regularized, L2-loss linear epsilon-vector regression (epsilon-SVR).
Initializes a new instance of the class.
Obsolete.
Creates an instance of the model to be learned. Inheritors
of this abstract class must define this method so new models
can be created from the training data.
L2-regularized L2-loss linear support vector regression
(SVR) learning algorithm in the primal formulation (-s 11).
This class implements a L2-regularized L2-loss support vector regression (SVR)
learning algorithm that operates in the primal form of the optimization problem.
This method has been based on liblinear's l2r_l2_svr_fun problem specification,
optimized using a Trust-region Newton method.
Liblinear's solver -s 11: L2R_L2LOSS_SVR. A trust region newton algorithm
for the primal of L2-regularized, L2-loss linear epsilon-vector regression (epsilon-SVR).
Initializes a new instance of the class.
Creates an instance of the model to be learned. Inheritors
of this abstract class must define this method so new models
can be created from the training data.
Base class for newton method for linear regression learning algorithm.
Constructs a new Newton method algorithm for L2-regularized
support vector regression (SVR-SVMs) primal problems.
Convergence tolerance. Default value is 0.01.
The criterion for completing the model training process. The default is 0.01.
Gets or sets the maximum number of iterations that should
be performed until the algorithm stops. Default is 1000.
Runs the learning algorithm.
Obsolete.
Gets the selection strategy to be used in SMO.
Uses the sequential selection strategy as
suggested by Keerthi et al's algorithm 1.
Always select the worst violation pair
to be optimized first, as suggested in
Keerthi et al's algorithm 2.
Use a second order selection algorithm, using
the same algorithm as LibSVM's implementation.
Sequential Minimal Optimization (SMO) Algorithm
The SMO algorithm is an algorithm for solving large quadratic programming (QP)
optimization problems, widely used for the training of support vector machines.
First developed by John C. Platt in 1998, SMO breaks up large QP problems into
a series of smallest possible QP problems, which are then solved analytically.
This class follows the original algorithm by Platt with additional modifications
by Keerthi et al.
This class can also be used in combination with
or to learn s
using the one-vs-one or one-vs-all multi-class decision strategies, respectively.
References:
-
Wikipedia, The Free Encyclopedia. Sequential Minimal Optimization. Available on:
http://en.wikipedia.org/wiki/Sequential_Minimal_Optimization
-
John C. Platt, Sequential Minimal Optimization: A Fast Algorithm for Training Support
Vector Machines. 1998. Available on: http://research.microsoft.com/en-us/um/people/jplatt/smoTR.pdf
-
S. S. Keerthi et al. Improvements to Platt's SMO Algorithm for SVM Classifier Design.
Technical Report CD-99-14. Available on: http://www.cs.iastate.edu/~honavar/keerthi-svm.pdf
-
J. P. Lewis. A Short SVM (Support Vector Machine) Tutorial. Available on:
http://www.idiom.com/~zilla/Work/Notes/svmtutorial.pdf
The following example shows how to use a SVM to learn a simple XOR function.
-
Wikipedia, The Free Encyclopedia. Sequential Minimal Optimization. Available on:
http://en.wikipedia.org/wiki/Sequential_Minimal_Optimization
-
John C. Platt, Sequential Minimal Optimization: A Fast Algorithm for Training Support
Vector Machines. 1998. Available on: http://research.microsoft.com/en-us/um/people/jplatt/smoTR.pdf
-
S. S. Keerthi et al. Improvements to Platt's SMO Algorithm for SVM Classifier Design.
Technical Report CD-99-14. Available on: http://www.cs.iastate.edu/~honavar/keerthi-svm.pdf
-
J. P. Lewis. A Short SVM (Support Vector Machine) Tutorial. Available on:
http://www.idiom.com/~zilla/Work/Notes/svmtutorial.pdf
The following example shows how to use a SVM to learn a simple XOR function.
-
Wikipedia, The Free Encyclopedia. Sequential Minimal Optimization. Available on:
http://en.wikipedia.org/wiki/Sequential_Minimal_Optimization
-
John C. Platt, Sequential Minimal Optimization: A Fast Algorithm for Training Support
Vector Machines. 1998. Available on: http://research.microsoft.com/en-us/um/people/jplatt/smoTR.pdf
-
S. S. Keerthi et al. Improvements to Platt's SMO Algorithm for SVM Classifier Design.
Technical Report CD-99-14. Available on: http://www.cs.iastate.edu/~honavar/keerthi-svm.pdf
-
J. P. Lewis. A Short SVM (Support Vector Machine) Tutorial. Available on:
http://www.idiom.com/~zilla/Work/Notes/svmtutorial.pdf
The following example shows how to use a SVM to learn a simple XOR function.
-
http://courses.media.mit.edu/2006fall/mas622j/Projects/aisen-project/index.html
-
http://nlp.stanford.edu/IR-book/html/htmledition/multiclass-svms-1.html
The following example shows how to learn a linear, multi-class support vector
machine using the algorithm.
-
http://courses.media.mit.edu/2006fall/mas622j/Projects/aisen-project/index.html
-
http://nlp.stanford.edu/IR-book/html/htmledition/multiclass-svms-1.html
The following example shows how to learn a linear, multi-class support vector
machine using the algorithm.
-
http://courses.media.mit.edu/2006fall/mas622j/Projects/aisen-project/index.html
-
http://nlp.stanford.edu/IR-book/html/htmledition/multiclass-svms-1.html
The following example shows how to learn a linear, multi-class support vector
machine using the algorithm.
-
http://courses.media.mit.edu/2006fall/mas622j/Projects/aisen-project/index.html
-
http://nlp.stanford.edu/IR-book/html/htmledition/multiclass-svms-1.html
Initializes a new instance of the class.
The number of classes in the multi-class classification problem.
The number of inputs (length of the input vectors) accepted by the machine.
The kernel function to be used.
One-against-all Multi-label Kernel Support Vector Machine Classifier.
The Support Vector Machine is by nature a binary classifier. Multiple label
problems are problems in which an input sample is allowed to belong to one
or more classes. A way to implement multi-label classes in support vector
machines is to build a one-against-all decision scheme where multiple SVMs
are trained to detect each of the available classes.
Currently this class supports only Kernel machines as the underlying classifiers.
If a Linear Support Vector Machine is needed, specify a Linear kernel in the
constructor at the moment of creation.
References:
-
http://courses.media.mit.edu/2006fall/mas622j/Projects/aisen-project/index.html
-
http://nlp.stanford.edu/IR-book/html/htmledition/multiclass-svms-1.html
The following example shows how to learn a linear, multi-label (one-vs-rest) support
vector machine using the algorithm.
-
http://courses.media.mit.edu/2006fall/mas622j/Projects/aisen-project/index.html
-
http://nlp.stanford.edu/IR-book/html/htmledition/multiclass-svms-1.html
The following example shows how to learn a linear, multi-label (one-vs-rest) support
vector machine using the algorithm.
-
http://courses.media.mit.edu/2006fall/mas622j/Projects/aisen-project/index.html
-
http://nlp.stanford.edu/IR-book/html/htmledition/multiclass-svms-1.html
The following example shows how to learn a linear, multi-label (one-vs-rest) support
vector machine using the algorithm.
-
http://courses.media.mit.edu/2006fall/mas622j/Projects/aisen-project/index.html
-
http://nlp.stanford.edu/IR-book/html/htmledition/multiclass-svms-1.html
The following example shows how to learn a linear, multi-label (one-vs-rest) support
vector machine using the algorithm.
-
http://en.wikipedia.org/wiki/Support_vector_machine
-
http://www.kernel-machines.org/
The first example shows how to learn a linear SVM. However, since the
problem being learned is not linearly separable, the classifier will
not be able to produce a perfect decision boundary.
-
http://en.wikipedia.org/wiki/Support_vector_machine
-
http://www.kernel-machines.org/
The first example shows how to learn an SVM using a
standard kernel that operates on vectors of doubles.
-
http://en.wikipedia.org/wiki/Support_vector_machine
-
http://www.kernel-machines.org/
The first example shows how to learn an SVM using a
standard kernel that operates on vectors of doubles.
-
Wikipedia, The Free Encyclopedia. Cross-validation (statistics). Available on:
http://en.wikipedia.org/wiki/Cross-validation_(statistics)
-
Wikipedia contributors. "K-nearest neighbor algorithm." Wikipedia, The
Free Encyclopedia. Wikipedia, The Free Encyclopedia, 10 Oct. 2012. Web.
9 Nov. 2012. http://en.wikipedia.org/wiki/K-nearest_neighbor_algorithm
The first example shows how to create and use a k-Nearest Neighbor algorithm to classify
a set of numeric vectors in a multi-class decision problem involving 3 classes. It also shows
how to compute class decisions for a new sample and how to measure the performance of a classifier.
// Example binary data
double[][] inputs =
{
new double[] { -1, -1 },
new double[] { -1, 1 },
new double[] { 1, -1 },
new double[] { 1, 1 }
};
int[] xor = // xor labels
{
-1, 1, 1, -1
};
// Declare the parameters and ranges to be searched
GridSearchRange[] ranges =
{
new GridSearchRange("complexity", new double[] { 0.00000001, 5.20, 0.30, 0.50 } ),
new GridSearchRange("degree", new double[] { 1, 10, 2, 3, 4, 5 } ),
new GridSearchRange("constant", new double[] { 0, 1, 2 } )
};
// Instantiate a new Grid Search algorithm for Kernel Support Vector Machines
var gridsearch = new GridSearch<KernelSupportVectorMachine>(ranges);
// Set the fitting function for the algorithm
gridsearch.Fitting = delegate(GridSearchParameterCollection parameters, out double error)
{
// The parameters to be tried will be passed as a function parameter.
int degree = (int)parameters["degree"].Value;
double constant = parameters["constant"].Value;
double complexity = parameters["complexity"].Value;
// Use the parameters to build the SVM model
Polynomial kernel = new Polynomial(degree, constant);
KernelSupportVectorMachine ksvm = new KernelSupportVectorMachine(kernel, 2);
// Create a new learning algorithm for SVMs
SequentialMinimalOptimization smo = new SequentialMinimalOptimization(ksvm, inputs, xor);
smo.Complexity = complexity;
// Measure the model performance to return as an out parameter
error = smo.Run();
return ksvm; // Return the current model
};
// Declare some out variables to pass to the grid search algorithm
GridSearchParameterCollection bestParameters; double minError;
// Compute the grid search to find the best Support Vector Machine
KernelSupportVectorMachine bestModel = gridsearch.Compute(out bestParameters, out minError);
Gets or sets the parallelization options for this algorithm.
Gets or sets a cancellation token that can be used
to cancel the algorithm while it is running.
Constructs a new Grid search algorithm.
The range of parameters to search.
A function that fits a model using the given parameters.
The range of parameters to consider during search.
Searches for the best combination of parameters that results in the most accurate model.
The best combination of parameters found by the grid search.
The minimum error of the best model found by the grid search.
The best model found during the grid search.
Searches for the best combination of parameters that results in the most accurate model.
The results found during the grid search.
Contains results from the grid-search procedure.
The type of the model to be tuned.
Gets all combination of parameters tried.
Gets all models created during the search.
Gets the error for each of the created models.
Gets the index of the best found model
in the collection.
Gets the best model found.
Gets the best parameter combination found.
Gets the minimum error found.
Gets the size of the grid used in the grid-search.
Initializes a new instance of the class.
Initializes a new instance of the class.
Initialization schemes for clustering algorithms.
Do not perform initialization.
Randomly sample points to become centroids.
Use the kmeans++ seeding algorithm for generating initial centroids.
Use the PAM BUILD algorithm for generating initial centroids.
Lloyd's k-Means clustering algorithm.
In statistics and machine learning, k-means clustering is a method
of cluster analysis which aims to partition n observations into k
clusters in which each observation belongs to the cluster with the
nearest mean.
It is similar to the expectation-maximization algorithm for mixtures
of Gaussians in that they both attempt to find the centers of natural
clusters in the data as well as in the iterative refinement approach
employed by both algorithms.
The algorithm is composed of the following steps:
-
Place K points into the space represented by the objects that are
being clustered. These points represent initial group centroids.
-
Assign each object to the group that has the closest centroid.
-
When all objects have been assigned, recalculate the positions
of the K centroids.
-
Repeat Steps 2 and 3 until the centroids no longer move. This
produces a separation of the objects into groups from which the
metric to be minimized can be calculated.
This particular implementation uses the squared Euclidean distance
as a similarity measure in order to form clusters.
References:
-
Wikipedia, The Free Encyclopedia. K-means clustering. Available on:
http://en.wikipedia.org/wiki/K-means_clustering
-
Matteo Matteucci. A Tutorial on Clustering Algorithms. Available on:
http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/kmeans.html
How to perform clustering with K-Means.
The resulting image will be:
Gets the clusters found by K-means.
Gets or sets the cluster centroids.
Gets the number of clusters.
Gets the dimensionality of the data space.
Gets or sets the distance function used
as a distance metric between data points.
Gets or sets whether covariance matrices for the clusters should
be computed at the end of an iteration. Default is true.
Gets or sets whether the clustering distortion error (the
average distance between all data points and the cluster
centroids) should be computed at the end of the algorithm.
The result will be stored in . Default is true.
Gets or sets the maximum number of iterations to
be performed by the method. If set to zero, no
iteration limit will be imposed. Default is 0.
Gets or sets the relative convergence threshold
for stopping the algorithm. Default is 1e-5.
Gets the number of iterations performed in the
last call to this class' Compute methods.
Gets the cluster distortion error after the
last call to this class' Compute methods.
Gets or sets the strategy used to initialize the
centroids of the clustering algorithm. Default is
.
Initializes a new instance of KMeans algorithm
The number of clusters to divide input data.
The distance function to use. Default is to
use the distance.
Initializes a new instance of the K-Means algorithm
The number of clusters to divide the input data into.
Initializes a new instance of the KMeans algorithm
The number of clusters to divide the input data into.
The distance function to use. Default is to use the
distance.
Randomizes the clusters inside a dataset.
The data to randomize the algorithm.
Learns a model that can map the given inputs to the desired outputs.
The model inputs.
The weight of importance for each input sample.
A model that has learned how to produce suitable outputs
given the input data .
Divides the input data into K clusters.
The data where to compute the algorithm.
Divides the input data into K clusters.
The data where to compute the algorithm.
The weight associated with each data point.
Computes the information about each cluster (covariance, proportions and error).
The data points.
Computes the information about each cluster (covariance, proportions and error).
The data points.
The assigned labels.
Divides the input data into K clusters.
The data where to compute the algorithm.
The weight to consider for each data sample. This is used in weighted K-Means
The total sum of the weights in .
Determines if the algorithm has converged by comparing the
centroids between two consecutive iterations.
The previous centroids.
The new centroids.
Returns if all centroids had a percentage change
less than . Returns otherwise.
Divides the input data into K clusters.
The data where to compute the algorithm.
The average square distance from the
data points to the clusters' centroids.
QLearning learning algorithm.
The class provides implementation of Q-Learning algorithm, known as
off-policy Temporal Difference control.
The following example shows how to learn a model using reinforcement learning through the
Q-learning algorithm. The following code has been inherited from the AForge.NET Framework,
and has not been modified ever since. If you have better ideas on how to improve its
interface, please share it in the project's issue tracker at
https://github.com/accord-net/framework/issues.
If you would like, and if your ideas are feasible and encouraging enough, you can be named an
official contributor of the project. If you would like, you could opt to "inherit" the reinforcement learning
portion of the project such that you could be free to commit, modify and, more importantly, authorship
those modules directly from your own GitHub account without having to wait for Pull Request approvals.
You can be listed as an official author of the Accord.NET Framework, making it possible to list the
creation or shared authorship of the reinforcement learning project in your CV.
-
P. D. Kovesi. MATLAB and Octave Functions for Computer Vision and Image Processing.
School of Computer Science and Software Engineering, The University of Western Australia.
Available in: http://www.csse.uwa.edu.au/~pk/research/matlabfns
-
Wikipedia, The Free Encyclopedia. RANSAC. Available on:
http://en.wikipedia.org/wiki/RANSAC
-
Wikipedia, The Free Encyclopedia. K-d tree. Available on:
http://en.wikipedia.org/wiki/K-d_tree
-
Moore, Andrew W. "An intoductory tutorial on kd-trees." (1991).
Available at: http://www.autonlab.org/autonweb/14665/version/2/part/5/data/moore-tutorial.pdf
// This is the same example found in Wikipedia page on
// k-d trees: http://en.wikipedia.org/wiki/K-d_tree
// Suppose we have the following set of points:
double[][] points =
{
new double[] { 2, 3 },
new double[] { 5, 4 },
new double[] { 9, 6 },
new double[] { 4, 7 },
new double[] { 8, 1 },
new double[] { 7, 2 },
};
// To create a tree from a set of points, we use
KDTree<int> tree = KDTree.FromData<int>(points);
// Now we can manually navigate the tree
KDTreeNode<int> node = tree.Root.Left.Right;
// Or traverse it automatically
foreach (KDTreeNode<int> n in tree)
{
double[] location = n.Position;
Assert.AreEqual(2, location.Length);
}
// Given a query point, we can also query for other
// points which are near this point within a radius
double[] query = new double[] { 5, 3 };
// Locate all nearby points within an euclidean distance of 1.5
// (answer should be be a single point located at position (5,4))
KDTreeNodeCollection<int> result = tree.Nearest(query, radius: 1.5);
// We can also use alternate distance functions
tree.Distance = Accord.Math.Distance.Manhattan;
// And also query for a fixed number of neighbor points
// (answer should be the points at (5,4), (7,2), (2,3))
KDTreeNodeCollection<int> neighbors = tree.Nearest(query, neighbors: 3);
' This is the same example found in Wikipedia page on
' k-d trees: http://en.wikipedia.org/wiki/K-d_tree
' Suppose we have the following set of points:
Dim points =
{
New Double() { 2, 3 },
New Double() { 5, 4 },
New Double() { 9, 6 },
New Double() { 4, 7 },
New Double() { 8, 1 },
New Double() { 7, 2 }
}
' To create a tree from a set of points, we use
Dim tree = KDTree.FromData(Of Integer)(points)
' Now we can manually navigate the tree
Dim node = tree.Root.Left.Right
' Or traverse it automatically
For Each n As KDTreeNode(Of Integer) In tree
Dim location = n.Position
Console.WriteLine(location.Length)
Next
' Given a query point, we can also query for other
' points which are near this point within a radius
'
Dim query = New Double() {5, 3}
' Locate all nearby points within an Euclidean distance of 1.5
' (answer should be a single point located at position (5,4))
'
Dim result = tree.Nearest(query, radius:=1.5)
' We can also use alternate distance functions
tree.Distance = Function(a, b) Accord.Math.Distance.Manhattan(a, b)
' And also query for a fixed number of neighbor points
' (answer should be the points at (5,4), (7,2), (2,3))
'
Dim neighbors = tree.Nearest(query, neighbors:=3)
Creates a new .
The number of dimensions in the tree.
Creates a new .
The number of dimensions in the tree.
The Root node, if already existent.
Creates a new .
The number of dimensions in the tree.
The Root node, if already existent.
The number of elements in the Root node.
The number of leaves linked through the Root node.
Inserts a value in the tree at the desired position.
A double-vector with the same number of elements as dimensions in the tree.
The value to be inserted.
Creates the Root node for a new given
a set of data points and their respective stored values.
The data points to be inserted in the tree.
The values associated with each point.
Return the number of leaves in the Root subtree.
Whether the given vector
can be ordered in place. Passing true will change the original order of
the vector. If set to false, all operations will be performed on an extra
copy of the vector.
The Root node for a new
contained the given .
Convenience class for k-dimensional tree static methods. To
create a new KDTree, specify the generic parameter as in
.
Please check the documentation page for
for examples, usage and actual remarks about kd-trees.
Creates a new .
The number of dimensions in the tree.
Creates a new .
The number of dimensions in the tree.
The root node, if already existent.
Creates a new .
The number of dimensions in the tree.
The root node, if already existent.
The number of elements in the root node.
The number of leaves linked through the root node.
Adds a new point to this tree.
A double-vector with the same number of elements as dimensions in the tree.
Creates a new k-dimensional tree from the given points.
The type of the value to be stored.
The points to be added to the tree.
Whether the given vector
can be ordered in place. Passing true will change the original order of
the vector. If set to false, all operations will be performed on an extra
copy of the vector.
A populated with the given data points.
Creates a new k-dimensional tree from the given points.
The points to be added to the tree.
Whether the given vector
can be ordered in place. Passing true will change the original order of
the vector. If set to false, all operations will be performed on an extra
copy of the vector.
A populated with the given data points.
Creates a new k-dimensional tree from the given points.
The type of the value to be stored.
The points to be added to the tree.
The corresponding values at each data point.
Whether the given vector
can be ordered in place. Passing true will change the original order of
the vector. If set to false, all operations will be performed on an extra
copy of the vector.
A populated with the given data points.
Creates a new k-dimensional tree from the given points.
The points to be added to the tree.
The distance function to use.
Whether the given vector
can be ordered in place. Passing true will change the original order of
the vector. If set to false, all operations will be performed on an extra
copy of the vector.
A populated with the given data points.
Creates a new k-dimensional tree from the given points.
The type of the value to be stored.
The points to be added to the tree.
The corresponding values at each data point.
The distance function to use.
Whether the given vector
can be ordered in place. Passing true will change the original order of
the vector. If set to false, all operations will be performed on an extra
copy of the vector.
A populated with the given data points.
Creates a new k-dimensional tree from the given points.
The type of the value to be stored.
The points to be added to the tree.
The distance function to use.
Whether the given vector
can be ordered in place. Passing true will change the original order of
the vector. If set to false, all operations will be performed on an extra
copy of the vector.
A populated with the given data points.
K-dimensional tree node (for ).
K-dimensional tree node (for ).
Gets or sets the value being stored at this node.
Base class for K-dimensional tree nodes.
The class type for the nodes of the tree.
Gets or sets the position of
the node in spatial coordinates.
Gets or sets the dimension index of the split. This value is a
index of the vector and as such should
be higher than zero and less than the number of elements in .
Returns a that represents this instance.
A that represents this instance.
Compares the current object with another object of the same type.
An object to compare with this object.
A value that indicates the relative order of the objects being compared. The return value has the following meanings: Value Meaning Less than zero This object is less than the parameter.Zero This object is equal to . Greater than zero This object is greater than .
Indicates whether the current object is equal to another object of the same type.
An object to compare with this object.
true if the current object is equal to the parameter; otherwise, false.
Collection of k-dimensional tree nodes.
The class type for the nodes of the tree.
This class is used to store neighbor nodes when running one of the
search algorithms for k-dimensional trees.
Gets or sets the maximum number of elements on this
collection, if specified. A value of zero indicates
this instance has no upper limit of elements.
Gets the minimum distance between a node
in this collection and the query point.
Gets the maximum distance between a node
in this collection and the query point.
Gets the farthest node in the collection (with greatest distance).
Gets the nearest node in the collection (with smallest distance).
Creates a new with a maximum size.
The maximum number of elements allowed in this collection.
Attempts to add a value to the collection. If the list is full
and the value is more distant than the farthest node in the
collection, the value will not be added.
The node to be added.
The node distance.
Returns true if the node has been added; false otherwise.
Attempts to add a value to the collection. If the list is full
and the value is more distant than the farthest node in the
collection, the value will not be added.
The node to be added.
The node distance.
Returns true if the node has been added; false otherwise.
Adds the specified item to the collection.
The distance from the node to the query point.
The item to be added.
Removes all elements from this collection.
Gets the
at the specified index. Note: this method will iterate over the entire collection
until the given position is found.
Gets the number of elements in this collection.
Gets a value indicating whether this instance is read only.
For this collection, always returns false.
true if this instance is read only; otherwise, false.
Returns an enumerator that iterates through this collection.
An object
that can be used to iterate through the collection.
Returns an enumerator that iterates through this collection.
An object that
can be used to iterate through the collection.
Determines whether this instance contains the specified item.
The object to locate in the collection.
The value can be null for reference types.
true if the item is found in the collection; otherwise, false.
Copies the entire collection to a compatible one-dimensional , starting
at the specified index of the target
array.
The one-dimensional that is the destination of the
elements copied from tree. The must have zero-based indexing.
The zero-based index in at which copying begins.
Adds the specified item to this collection.
The item.
Not supported.
Removes the farthest tree node from this collection.
Removes the nearest tree node from this collection.
Space-Partitioning Tree.
Gets the dimension of the space covered by this tree.
Initializes a new instance of the class.
The dimensions of the space partitioned by the tree.
Creates a new space-partitioning tree from the given points.
The points to be added to the tree.
A populated with the given data points.
Inserts a point in the Space-Partitioning tree.
Computes non-edge forces using Barnes-Hut algorithm.
Computes edge forces.
Vantage-Point Tree.
Creates a new vantage-point tree from the given points.
The points to be added to the tree.
Whether to perform operations in place, altering the
original array of points instead of creating an extra copy.
A populated with the given data points.
Creates a new vantage-point tree from the given points.
The points to be added to the tree.
Whether to perform operations in place, altering the
original array of points instead of creating an extra copy.
A populated with the given data points.
Creates a new vantage-point tree from the given points.
The points to be added to the tree.
The distance function to use.
Whether to perform operations in place, altering the
original array of points instead of creating an extra copy.
A populated with the given data points.
Creates a new vantage-point tree from the given points.
The type for the position vectors.
The points to be added to the tree.
The distance function to use.
Whether to perform operations in place, altering the
original array of points instead of creating an extra copy.
A populated with the given data points.
Creates a new vantage-point tree from the given points.
The type of the value to be stored.
The points to be added to the tree.
The corresponding values at each data point.
Whether to perform operations in place, altering the
original array of points instead of creating an extra copy.
A populated with the given data points.
Creates a new vantage-point tree from the given points.
The type for the position vectors.
The type of the value to be stored.
The points to be added to the tree.
The corresponding values at each data point.
The distance function to use.
Whether to perform operations in place, altering the
original array of points instead of creating an extra copy.
A populated with the given data points.
Initializes a new instance of the class.
The distance to use when comparing points. Default is
.
Initializes a new instance of the class.
Base class for Vantage-Point Trees.
The type for the position vector of each node.
The class type for the nodes of the tree.
Gets or set the distance function used to
measure distances amongst points on this tree
Gets or sets the radius of the nodes in the tree.
Initializes a new instance of the class.
The distance to use when comparing points.
Retrieves a fixed point of nearest points to a given point.
The queried point.
The number of neighbors to retrieve.
A list of neighbor points, ordered by distance.
Retrieves a fixed point of nearest points to a given point.
The queried point.
The number of neighbors to retrieve.
The list where to store results.
A list of neighbor points, ordered by distance.
Node of a .
The type for the position vector (e.g. double[]).
The type for the value stored at the node.
Gets or sets a value associated with this node.
Returns a that represents this instance.
A that represents this instance.
Base class for nodes.
The type for the position vector (e.g. double[]).
The class type for the nodes of the tree.
Gets or sets the current position for this Vantage-Point Tree Node.
Gets or sets the threshold radius for this node.
Indicates whether the current object is equal to another object of the same type.
An object to compare with this object.
true if the current object is equal to the parameter; otherwise, false.
Returns a that represents this instance.
A that represents this instance.
Node of a .
The type for the position vector (e.g. double[]).
Solver types allowed in LibSVM/Liblinear model files.
Unknown solver type.
L2-regularized logistic regression in the primal (-s 0, L2R_LR).
L2-regularized L2-loss support vector classification
in the dual (-s 1, L2R_L2LOSS_SVC_DUAL, the default).
L2-regularized L2-loss support vector classification
in the primal (-s 2, L2R_L2LOSS_SVC).
L2-regularized L1-loss support vector classification
in the dual (-s 3, L2R_L1LOSS_SVC_DUAL).
Support vector classification by
Crammer and Singer (-s 4, MCSVM_CS).
L1-regularized L2-loss support vector
classification (-s 5, L1R_L2LOSS_SVC).
L1-regularized logistic regression (-s 6, L1R_LR).
L2-regularized logistic regression in the dual (-s 7, L2R_LR_DUAL).
L2-regularized L2-loss support vector regression
in the primal (-s 11, L2R_L2LOSS_SVR).
L2-regularized L2-loss support vector regression
in the dual (-s 12, L2R_L2LOSS_SVR_DUAL).
L2-regularized L1-loss support vector regression
in the dual (-s 13, L2R_L1LOSS_SVR_DUAL).
Reads support vector machines
created from LibSVM or Liblinear. Not all solver types are supported.
This class can be used to import LibSVM or LibLINEAR models into .NET
and use them to make predictions in .NET/C# applications.
If you are looking for ways to load and save SVM models in the Accord.NET
Framework without necessarily being compatible with LibSVM or LIBLINEAR,
please use the class instead.