# Detecting exact nearest neighbors

Aaron Lun1

1Cancer Research UK Cambridge Institute, Cambridge, United Kingdom

#### Revised: 2 December 2018

#### Package

BiocNeighbors 1.4.2

# 1 Introduction

The BiocNeighbors package implements a few algorithms for exact nearest neighbor searching:

- The k-means for k-nearest neighbors (KMKNN) algorithm (Wang 2012) uses k-means clustering to create an index. Within each cluster, the distance of each of that cluster’s points to the cluster center are computed and used to sort all points. Given a query point, the distance to each cluster center is determined and the triangle inequality is applied to determine which points in each cluster warrant a full distance calculation.
- The vantage point (VP) tree algorithm (Yianilos 1993) involves constructing a tree where each node is located at a data point and is associated with a subset of neighboring points. Each node progressively partitions points into two subsets that are either closer or further to the node than a given threshold. Given a query point, the triangle inequality is applied at each node in the tree to determine if the child nodes warrant searching.

Both methods involve a component of randomness during index construction, though the k-nearest neighbors result is fully deterministic11 Except in the presence of ties, see `?findKNN` for details..

# 2 Identifying k-nearest neighbors

The most obvious application is to perform a k-nearest neighbors search. We’ll mock up an example here with a hypercube of points, for which we want to identify the 10 nearest neighbors for each point.

    nobs <- 10000
    ndim <- 20
    data <- matrix(runif(nobs*ndim), ncol=ndim)

The `findKNN()` method expects a numeric matrix as input with data points as the rows and variables/dimensions as the columns. We indicate that we want to use the KMKNN algorithm by setting `BNPARAM=KmknnParam()` (which is also the default, so this is not strictly necessary here). We could use a VP tree instead by setting `BNPARAM=VptreeParam()`.

    fout <- findKNN(data, k=10, BNPARAM=KmknnParam())
    head(fout$index)

    ##      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
    ## [1,] 1881 8278 3628 5229 6391 9639 6161 7671 5375  5037
    ## [2,] 3432 6646 4652  782 7068 5634 4408 5663 2795   751
    ## [3,] 9029 8006 4575    7 3232  774 1207 1112 8270  7643
    ## [4,]  135 2543  613 2054 5954  797 7852 1271 5963  5719
    ## [5,]  521  752 8056 4628 1134 5878 8624 7083 6162  9431
    ## [6,] 6003 4150  574 8355 5948 6947 4176 3989 5718  5707

    head(fout$distance)

    ##           [,1]      [,2]      [,3]      [,4]      [,5]      [,6]      [,7]
    ## [1,] 0.8053135 0.8344659 0.8597889 0.8675684 0.8903521 0.9510813 0.9541194
    ## [2,] 0.9231167 0.9317375 0.9677469 0.9691288 0.9693556 0.9707015 0.9762002
    ## [3,] 0.9674346 1.0275486 1.0556525 1.0923730 1.1129987 1.1219975 1.1402440
    ## [4,] 1.0107910 1.0346099 1.0550829 1.0576011 1.0749991 1.0769960 1.0805129
    ## [5,] 0.9196937 0.9447228 0.9824839 0.9871147 0.9942311 1.0174962 1.0189802
    ## [6,] 0.9658588 0.9816334 0.9958824 1.0291708 1.0416947 1.0525788 1.0570698
    ##           [,8]      [,9]     [,10]
    ## [1,] 0.9567863 0.9773559 0.9800960
    ## [2,] 0.9766540 0.9814470 0.9872668
    ## [3,] 1.1422397 1.1547846 1.1638458
    ## [4,] 1.0839071 1.0840470 1.0903321
    ## [5,] 1.0193400 1.0205089 1.0397422
    ## [6,] 1.0711706 1.0814136 1.0856588

Each row of the `index` matrix corresponds to a point in `data` and contains the row indices in `data` that are its nearest neighbors. For example, the 3rd point in `data` has the following nearest neighbors:

    fout$index[3,]

    ##  [1] 9029 8006 4575    7 3232  774 1207 1112 8270 7643

… with the following distances to those neighbors:

    fout$distance[3,]

    ##  [1] 0.9674346 1.0275486 1.0556525 1.0923730 1.1129987 1.1219975 1.1402440
    ##  [8] 1.1422397 1.1547846 1.1638458

Note that the reported neighbors are sorted by distance.

# 3 Querying k-nearest neighbors

Another application is to identify the k-nearest neighbors in one dataset based on query points in another dataset. Again, we mock up a small data set:

    nquery <- 1000
    ndim <- 20
    query <- matrix(runif(nquery*ndim), ncol=ndim)

We then use the `queryKNN()` function to identify the 5 nearest neighbors in `data` for each point in `query`.

    qout <- queryKNN(data, query, k=5, BNPARAM=KmknnParam())
    head(qout$index)

    ##      [,1] [,2] [,3] [,4] [,5]
    ## [1,] 3116 1922 7629  104   77
    ## [2,] 4330 8770 5493 8495 6613
    ## [3,]   96 2671 2717 5074 3563
    ## [4,] 6919 5293 7579 6739 8893
    ## [5,] 5898 1706  532 9060 1513
    ## [6,] 4944 3700  987 8835 9258

    head(qout$distance)

    ##           [,1]      [,2]      [,3]      [,4]      [,5]
    ## [1,] 0.8477125 0.9520432 0.9528356 0.9533169 0.9873199
    ## [2,] 1.0110129 1.0158369 1.0206459 1.0561397 1.0701870
    ## [3,] 0.9009902 0.9609223 0.9808124 0.9998865 1.0299645
    ## [4,] 0.9132506 0.9161617 0.9163441 0.9603837 0.9647815
    ## [5,] 0.9640249 0.9984658 1.0019975 1.0360102 1.0717339
    ## [6,] 0.8440024 0.8859069 0.8912034 0.9447737 0.9465425

Each row of the `index` matrix contains the row indices in `data` that are the nearest neighbors of a point in `query`. For example, the 3rd point in `query` has the following nearest neighbors in `data`:

    qout$index[3,]

    ## [1]   96 2671 2717 5074 3563

… with the following distances to those neighbors:

    qout$distance[3,]

    ## [1] 0.9009902 0.9609223 0.9808124 0.9998865 1.0299645

Again, the reported neighbors are sorted by distance.

# 4 Further options

Users can perform the search for a subset of query points using the `subset=` argument. This yields the same result as but is more efficient than performing the search for all points and subsetting the output.

    findKNN(data, k=5, subset=3:5)

    ## $index
    ##      [,1] [,2] [,3] [,4] [,5]
    ## [1,] 9029 8006 4575    7 3232
    ## [2,]  135 2543  613 2054 5954
    ## [3,]  521  752 8056 4628 1134
    ##
    ## $distance
    ##           [,1]      [,2]      [,3]      [,4]      [,5]
    ## [1,] 0.9674346 1.0275486 1.0556525 1.0923730 1.1129987
    ## [2,] 1.0107910 1.0346099 1.0550829 1.0576011 1.0749991
    ## [3,] 0.9196937 0.9447228 0.9824839 0.9871147 0.9942311

If only the indices are of interest, users can set `get.distance=FALSE` to avoid returning the matrix of distances. This will save some time and memory.

    names(findKNN(data, k=2, get.distance=FALSE))

    ## [1] "index"

It is also simple to speed up functions by parallelizing the calculations with the BiocParallel framework.

    library(BiocParallel)
    out <- findKNN(data, k=10, BPPARAM=MulticoreParam(3))

For multiple queries to a constant `data`, the pre-clustering can be performed in a separate step with `buildIndex()`. The result can then be passed to multiple calls, avoiding the overhead of repeated clustering22 The algorithm type is automatically determined when `BNINDEX` is specified, so there is no need to also specify `BNPARAM` in the later functions..

    pre <- buildIndex(data, BNPARAM=KmknnParam())
    out1 <- findKNN(BNINDEX=pre, k=5)
    out2 <- queryKNN(BNINDEX=pre, query=query, k=2)

The default setting is to search on the Euclidean distance. Alternatively, we can use the Manhattan distance by setting `distance="Manhattan"` in the `BiocNeighborParam` object.

    out.m <- findKNN(data, k=5, BNPARAM=KmknnParam(distance="Manhattan"))

Advanced users may also be interested in the `raw.index=` argument, which returns indices directly to the precomputed object rather than to `data`. This may be useful inside package functions where it may be more convenient to work on a common precomputed object.

# 5 Session information

    sessionInfo()

    ## R version 3.6.2 (2019-12-12)
    ## Platform: x86_64-apple-darwin15.6.0 (64-bit)
    ## Running under: OS X El Capitan 10.11.6
    ##
    ## Matrix products: default
    ## BLAS:   /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib
    ## LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib
    ##
    ## locale:
    ## [1] C/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    ##
    ## attached base packages:
    ## [1] stats     graphics  grDevices utils     datasets  methods   base
    ##
    ## other attached packages:
    ## [1] BiocParallel_1.20.1 BiocNeighbors_1.4.2 knitr_1.28
    ## [4] BiocStyle_2.14.4
    ##
    ## loaded via a namespace (and not attached):
    ##  [1] Rcpp_1.0.3          bookdown_0.17       lattice_0.20-40
    ##  [4] digest_0.6.25       grid_3.6.2          stats4_3.6.2
    ##  [7] magrittr_1.5        evaluate_0.14       rlang_0.4.4
    ## [10] stringi_1.4.6       S4Vectors_0.24.3    Matrix_1.2-18
    ## [13] rmarkdown_2.1       tools_3.6.2         stringr_1.4.0
    ## [16] parallel_3.6.2      xfun_0.12           yaml_2.2.1
    ## [19] compiler_3.6.2      BiocGenerics_0.32.0 BiocManager_1.30.10
    ## [22] htmltools_0.4.0

# References

Wang, X. 2012. “A Fast Exact k-Nearest Neighbors Algorithm for High Dimensional Search Using k-Means Clustering and Triangle Inequality.” Proc Int Jt Conf Neural Netw 43 (6):2351–8.

Yianilos, P. N. 1993. “Data Structures and Algorithms for Nearest Neighbor Search in General Metric Spaces.” In SODA, 93:311–21. 194.