A: What is clustering in Data Science?
In a data-science context, clustering refers to organizing data into categories by using some sort of distance metric. "K-means clustering" is a common technique for doing so, but other clustering algorithms exist. An example of applying K-means clustering to a two-dimensional data set is shown below. The points have been grouped into three different clusters, and each cluster represented by a color:
Abstractly, clustering takes a large set of data points and a distance function between each pair of points. It assigns each data point a label, called a "cluster".
The goal in clustering is to find data points that "belong together" in some way. This can be used purely as a descriptive/explanatory model, but also as a predictive model. In the latter case, new data points are assigned to a category based on their distance from the clusters discovered during training. You would typically use a clustering technique when the data points you have aren't already labelled with a "correct" answer.
For example, suppose you have a large set of Steemit posts, represented by word vectors or tag vectors. A word vector is just a count of how often each word appears in the post. One analysis you might run is to find posts that are similar to each other, in terms of which words or tags they use. Similar posts can be labelled as a "cluster" which could correspond to a particular topic (or language, or even author.) I did an example of this sort of analysis here, using tags: https://steemit.com/steemstem/@markgritter/topic-extraction-from-steem-tags
The "distance" between data points can be very abstract. For word vectors, it is typical to use the standard Euclidean distance definition, treating the articles as points in a multi-dimensional space, with one dimension per word. "Distance" could also mean the number of edits required to change one string to another; this definition is often used in genomics. Or, the distance could be very concrete; it is possible to do clustering based on geographical location.
StemQ Notice: This post was originally submitted on StemQ.io, a Q&A application for STEM subjects powered by the Steem blockchain.