SharePointTaskMaster

This example employs several unsupervised learning techniques, (Affinity Propagation clustering, Manifold learning techniques ..), to extract the ETFs market structure from variations of high and low prices in historical quotes.

The quantity that we use is the daily variation, 6053 days, in quote price: quotes that are linked tend to fluctuate in relation to each other during a day.

This example employs several unsupervised learning techniques, (Affinity Propagation clustering, Manifold learning techniques ..), to extract the ETFs market structure from variations of close and open prices in historical quotes.

The quantity that we use is the daily variation, 6053 days, in quote price: quotes that are linked tend to fluctuate in relation to each other during a day.

How is Principal Component Analysis (PCA) of a DDoS data? Here we use DDoS data from https://github.com/ThomasTrungVo/public/blob/main/APA-DDoS-Dataset.csv ( that I have downloaded from 

https://www.kaggle.com/datasets/yashwanthkumbam/apaddos-dataset )

List of real labels are not only 3 as the sample data, may be 30 or 40 labels.

1. PCA of APA-DDoS-Dataset index by ip.src and tcp.dstport

2. PCA of APA-DDoS-Dataset index by ip.src and ip.proto

3.

Incremental principal component analysis (IPCA) is typically used as a replacement for principal component analysis (PCA) when the dataset to be decomposed is too large to fit in memory. IPCA builds a low-rank approximation for the input data using an amount of memory which is independent of the number of input data samples. It is still dependent on the input data features, but changing the batch size allows for control of memory usage.

Principal Component Analysis (PCA) applied to this data identifies the combination of attributes (principal components, or directions in the feature space) that account for the most variance in the data. Here we plot the different samples on the 2 first principal components.

Linear Discriminant Analysis (LDA) tries to identify attributes that account for the most variance between classes. In particular, LDA, in contrast to PCA, is a supervised method, using known class labels.

Principal Component Analysis (PCA) applied to this data identifies the combination of attributes (principal components, or directions in the feature space) that account for the most variance in the data. Here we plot the different samples on the 2 first principal components.

Linear Discriminant Analysis (LDA) tries to identify attributes that account for the most variance between classes. In particular, LDA, in contrast to PCA, is a supervised method, using known class labels.

An example comparing nearest neighbors classification with and without Neighborhood Components Analysis.

It will plot the class decision boundaries given by a Nearest Neighbors classifier when using the Euclidean distance on the original features, versus using the Euclidean distance after the transformation learned by Neighborhood Components Analysis. The latter aims to find a linear transformation that maximises the (stochastic) nearest neighbor classification accuracy on the training set.

Demonstrate the resolution of a regression problem using a k-Nearest Neighbor and the interpolation of the target using both barycenter and constant weights.

Dimensional reduction using Principal component analysis (PCA) consists of finding the features that maximize the variance. If one feature varies more than the others only because of their respective scales, PCA would determine that such feature dominates the direction of the principal components.

We can visualize the distribution of the principal components in both cases:

1. Unscaled training dataset after PCA

2.

The dataset used is the Wine recognition dataset available at UCI. This dataset has continuous features that are heterogeneous in scale due to differing properties that they measure (e.g. alcohol content and malic acid).

For the sake of visualizing the decision boundary of a KNeighborsClassifier, in this section we select a subset of 2 features that have values with different orders of magnitude.