TransWikia.com

Agglomerative Hierarchial Clustering in python using DTW distance

Data Science Asked by user3806649 on January 5, 2021

I am new to both data science and python.
I have a dataset of the time-dependent samples, which I want to run agglomerative hierarchical clustering on them. I have found that Dynamic Time Warping (DTW) is a useful method to find alignments between two time series which may vary in time or speed.

I have found dtw_std in mlpy library and scipy.cluster.hierarchy in SciPy in order to cluster my data.

From the scipy docs, I find that I could use my custom distance function:

metric : str or function, optional The distance metric to use in the
case that y is a collection of observation vectors; ignored otherwise.
See the pdist function for a list of valid distance metrics. A custom
distance function can also be used.

But I am stuck matching this information to implement clustering.

My dataset is in the format of dataframe which each row corresponds to a sample.

Here is my questions:

1- How can I provide distance matrics for the linkage function?

2- How to set my custom distance function?

import pandas as pd
import scipy.cluster.hierarchy as hac
import mlpy

dataset = pd.read_csv ( "dataset.csv",encoding='utf-8' )
X # distance matrics
Z = hac.linkage(X, metrics=mlpy.dtw_std, method='average')
cluster = hac.fcluster(Z, t=100, criterion='maxclust')
leader = hac.leaders(Z, t=100, criterion='maxclust')


fig = plt.figure(figsize=(25, 10))
dn = dendrogram(Z)
plt.show()

edit:

Here is how I compute distance matrix, then I pass it to linkage:

# computing distance matrix
dm = pdist ( dataset ,lambda u,v: mlpy.dtw_std ( pd.Series(u).dropna().values.tolist(),pd.Series(v).dropna().values.tolist(),dist_only=True ))
z = hac.linkage(dm, method='average')
cluster = hac.fcluster(z, t=100, criterion='maxclust')
leader = scipy.cluster.hierarchy.fcluster(z, t=100, criterion='maxclust')

One Answer

Use a precomputed distance matrix, and distance="precomputed".

HAC will compute a distance matrix anyway.

Answered by Has QUIT--Anony-Mousse on January 5, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP