https://pinboard.in/u:jm/public/ recent bookmarks from jm 2023-10-03T10:24:40+00:00 https://platform.openai.com/docs/guides/embeddings/what-are-embeddings jm Text [vector] embeddings measure the relatedness of text strings. Embeddings are commonly used for: Search (where results are ranked by relevance to a query string); Clustering (where text strings are grouped by similarity); Recommendations (where items with related text strings are recommended); Anomaly detection (where outliers with little relatedness are identified); Diversity measurement (where similarity distributions are analyzed); Classification (where text strings are classified by their most similar label); An embedding is a vector (list) of floating point numbers. The distance between two vectors measures their relatedness. Small distances suggest high relatedness and large distances suggest low relatedness. Commonly used as a storage format in vector databases (cf. https://vercel.com/guides/vector-databases). Search using text embeddings is therefore implemented using cosine similarity or k-nearest neighbour to find vector similarity. Looks like https://www.trychroma.com/ is the current open source vector DB of choice, at the moment. (via Simon Willison)]]> ai openai via:simonw vector-embeddings text-embeddings text storage databases search similarity clustering recommendations anomaly-detection classification vector-databases https://pinboard.in/ https://pinboard.in/u:jm/b:95c03aa49119/ 2020-03-04T11:48:22+00:00 https://aws.amazon.com/about-aws/whats-new/2020/03/build-k-nearest-neighbor-similarity-search-engine-with-amazon-elasticsearch/ jmAmazon Elasticsearch Service now offers k-Nearest Neighbor (k-NN) search which can enhance search by similarity use cases like product recommendations, fraud detection, and image, video and semantic document retrieval. Built using the lightweight and efficient Non-Metric Space Library (NMSLIB), k-NN enables high scale, low latency nearest neighbor search on billions of documents across thousands of dimensions with the same ease as running any regular Elasticsearch query.   ]]> elasticsearch aws knn algorithms similarity searching search nmslib https://pinboard.in/ https://pinboard.in/u:jm/b:1ac9f1d202a0/ 2020-01-08T20:58:09+00:00 https://nelsonslog.wordpress.com/2020/01/07/facial-recognition-for-the-public-yandex/ jmYou can use Yandex Image Search right now as a pretty good facial recognition system for anyone who has labelled photos on the Web. I believe this is the first generally accessible facial recognition system with a large database. Yandex isn’t designed for this purpose. The trick is to upload photos cropped to a face and it’ll work more or less to find similar faces. this is really odd. Definitely seems like they designed the image similarity engine to support faces as a special case.]]> privacy face-recognition yandex search similarity images web https://pinboard.in/ https://pinboard.in/u:jm/b:3a385d71b7b1/ 2017-04-13T20:53:10+00:00 https://www.quora.com/On-what-basis-does-FFFFOUND-recommend-related-images jm ffffound images similarity algorithms via:pheezy mltshp quora https://pinboard.in/ https://pinboard.in/u:jm/b:6b6c3508dac8/ 2017-04-12T21:22:06+00:00 http://stackoverflow.com/questions/843972/image-comparison-fast-algorithm/844113#844113 jm algorithms hashing comparison diff images similarity search ffffound mltshp https://pinboard.in/ https://pinboard.in/u:jm/b:7cb94c5de107/ 2015-03-05T21:50:41+00:00 https://vividcortex.com/blog/2015/03/05/analyzing-related-metrics-with-vividcortex/ jm metrics k-means-clustering clustering algorithms discovery similarity vividcortex analysis data https://pinboard.in/ https://pinboard.in/u:jm/b:71d69a91d3df/ 2014-01-20T15:43:05+00:00 http://www.mlsec.org/harry/ jma small tool for comparing strings and measuring their similarity. The tool supports several common distance and kernel functions for strings as well as some exotic similarity measures. The focus of Harry lies on implicit similarity measures, that is, comparison functions that do not give rise to an explicit vector space. Examples of such similarity measures are the Levenshtein distance and the Jaro-Winkler distance. For comparison Harry loads a set of strings from input, computes the specified similarity measure and writes a matrix of similarity values to output. The similarity measure can be computed based on the granularity of characters as well as words contained in the strings. The configuration of this process, such as the input format, the similarity measure and the output format, are specified in a configuration file and can be additionally refined using command-line options. Harry is implemented using OpenMP, such that the computation time for a set of strings scales linear with the number of available CPU cores. Moreover, efficient implementations of several similarity measures, effective caching of similarity values and low-overhead locking further speedup the computation. via kragen.]]> via:kragen strings similarity levenshtein-distance algorithms openmp jaro-winkler edit-distance cli commandline hamming-distance compression https://pinboard.in/ https://pinboard.in/u:jm/b:7f75587e4bd7/ 2013-04-19T12:23:54+00:00 http://www.fmwconcepts.com/imagemagick/similar/index.php jmSIMILAR computes the normalized cross correlation similarity metric between two equal dimensioned images. The normalized cross correlation metric measures how similar two images are, not how different they are. The range of ncc metric values is between 0 (dissimilar) and 1 (similar). If mode=g, then the two images will be converted to grayscale. If mode=rgb, then the two images first will be converted to colorspace=rgb. Next, the ncc similarity metric will be computed for each channel. Finally, they will be combined into an rms value. (via Dan O'Neill)]]> image photos pictures similar imagemagick via:dano metrics similarity https://pinboard.in/ https://pinboard.in/u:jm/b:b8099e62ccd6/ 2011-09-29T21:10:15+00:00 http://petermblair.com/fbl-n-gram-analyzer/ jm anti-spam spam fbl feedback filtering n-grams similarity hashing redis searching https://pinboard.in/ https://pinboard.in/u:jm/b:00bea3b79665/