WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Databases
0 likes48 views
Vector databases are transforming how we handle data, allowing us to search through text, images, and audio by converting them into vectors. Today, we'll dive into the basics of this exciting technology and discuss its potential to revolutionize our next-generation AI applications. We'll examine typical uses for these databases and the essential tools developers need. Plus, we'll zoom in on the advanced capabilities of vector search and semantic caching in Java, showcasing these through a live demo with Redis libraries. Get ready to see how these powerful tools can change the game!
![Visual representation
Vector Embeddings
https://jalammar.github.io/illustrated-word2vec
King
[ 0.50451 , 0.68607 , -0.59517 , -0.022801, 0.60046 , -0.13498 , -0.08813 , 0.47377 , -0.61798 , -0.31012
, -0.076666, 1.493 , -0.034189, -0.98173 , 0.68229 , 0.81722 , -0.51874 , -0.31503 , -0.55809 , 0.66421 ,
0.1961 , -0.13495 , -0.11476 , -0.30344 , 0.41177 , -2.223 , -1.0756 , -1.0783 , -0.34354 , 0.33505 , 1.9927 ,
-0.04234 , -0.64319 , 0.71125 , 0.49159 , 0.16754 , 0.34344 , -0.25663 , -0.8523 , 0.1661 , 0.40102 , 1.1685
, -1.0137 , -0.21585 , -0.15155 , 0.78321 , -0.91241 , -1.6106 , -0.64426 , -0.51042 ]](https://image.slidesharecdn.com/luigifugaro-unlockingthefutureofdatapoweringnext-genaiwithvectordatabases-wmf2024-240614101127-d59b66d2/85/WMF-2024-Unlocking-the-Future-of-Data-Powering-Next-Gen-AI-with-Vector-Databases-17-320.jpg)
![Demo - Plan B!
@Document
public class ImageData {
@Id
private String id;
@Indexed
private String name;
@Indexed
private int height;
@Indexed
private int width;
@Indexed(schemaFieldType = SchemaFieldType.VECTOR,
algorithm = VectorField.VectorAlgorithm.HNSW,
type = VectorType.FLOAT32,
dimension = 512,
distanceMetric = DistanceMetric.L2,
initialCapacity = 10)
private float[] imageEmbedding
;
@Vectorize(destination = "imageEmbedding", embeddingType = EmbeddingType.FACE)
private String imagePath;
@Indexed
private double score = 0;
...
}](https://image.slidesharecdn.com/luigifugaro-unlockingthefutureofdatapoweringnext-genaiwithvectordatabases-wmf2024-240614101127-d59b66d2/85/WMF-2024-Unlocking-the-Future-of-Data-Powering-Next-Gen-AI-with-Vector-Databases-34-320.jpg)
![Demo - Plan B!
@Service
public class BestOfMatchService {
@Autowired
private EntityStream entityStream;
@Autowired
public ZooModel <Image, float[]> faceEmbeddingModel ;
private List<ImageData > matchAll (byte[] image, int limit) {
List<ImageData > imageDataList = new ArrayList<>();
try (Predictor <Image, float[]> predictor = faceEmbeddingModel .newPredictor()) {
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream( image);
Image img = ImageFactory .getInstance().fromInputStream( byteArrayInputStream );
float[] embedding = predictor .predict( img);
byte[] embeddingAsByteArray = floatArrayToByteArray(embedding );
SearchStream<ImageData> stream = entityStream.of(ImageData.class);
List<Pair<ImageData,Double>> matchWithScore = stream
.filter(ImageData$.IMAGE_EMBEDDING.knn(K, embeddingAsByteArray))
.sorted(ImageData$._IMAGE_EMBEDDING_SCORE, SortedField.SortOrder.ASC)
.limit(limit)
.map(Fields.of(ImageData$._THIS, ImageData$._IMAGE_EMBEDDING_SCORE))
.collect(Collectors.toList());
for (Pair<ImageData ,Double> pair : matchWithScore ) {
ImageData imageData = pair.getFirst();
Double score = pair.getSecond();
imageData .setScore( score);
imageDataList .add(imageData );
}
return imageDataList ;
} catch (Exception e ) {
throw new RuntimeException( e);
}
}
}](https://image.slidesharecdn.com/luigifugaro-unlockingthefutureofdatapoweringnext-genaiwithvectordatabases-wmf2024-240614101127-d59b66d2/85/WMF-2024-Unlocking-the-Future-of-Data-Powering-Next-Gen-AI-with-Vector-Databases-35-320.jpg)
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Databases
- 1. Luigi Fugaro Senior Solution Architect @ Redis Unlocking the Future of Data: Powering Next-Gen AI with Vector Databases
- 2. Agenda 1. Data Review 2. Vector Embeddings 3. Vector Database 4. Demo - Lets see come code
- 4. Data Review Lets start with a metric Around 80% of the data generated by organizations is Unstructured Growth IDC Report 2023 - https://www.box.com/resources/unstructured-data-paper
- 5. Data Review Data Types Growth Unstructured Quasi-Structured Semi-Structured Structured No inherent structure ~ PDFs, images, audio, video Erratic patterns/formats ~ Clickstreams There's a discernible pattern ~ Spreadsheets / XML / JSON Schema/de鍖ned data model ~ Database IDC Report 2023 - https://www.box.com/resources/unstructured-data-paper
- 6. How to deal with Unstructured Data? Common approaches were: Labeling Tagging Data Review
- 7. Labeling and Tagging Feature Value Frame Color Green Tire Color Brown Has Rear Rack Yes Has Fenders Yes Has Safety Bell No Has Fat Tires Yes Feature Value Frame Color Matte Olive Tire Color Orange Has Rear Rack Yes Has Fenders Yes Has Safety Bell Yes Has Fat Tires Yes Data Review
- 8. Labeling and Tagging Feature Value Easy Assembly 皚皚皚皚皚 Chain Quality 皚皚皚 Seat Comfort 皚 Gear Smoothness 皚皚皚皚 Data Review
- 9. How to deal with Unstructured Data? Labeling and Tagging are labor intensive, subjective and error-prone Whats the new approach? Data Review
- 11. Vector Embeddings What is a Vector? Numeric representation of something in N-dimensional space using 鍖oating numbers Can represent anything entire documents, images, video, audio
- 12. Vector Embeddings How to turn Data into Vectors? Its quite a complex process, based primarily on Neural Networks
- 13. Vector Embeddings How to turn Data into Vectors? Dont be scared, Machine Learning and Deep Learning has leaped forward in the last decade and we all can bene鍖t from a huge ecosystem of Models, ready to use! Each Model has its own speci鍖c task!
- 14. Vector Embeddings Music Video Images Faces Poses Emotions Audio Model Video Model Vision Model Face Detection/Recognition Models Vision Model Trained on Poses Sentiment Model Embeddings
- 15. Models quanti鍖es features of the item Vector Embeddings Why vectors embeddings? They are comparable!
- 16. Visual representation Vector Embeddings Semantic Relationship Syntactic Relationship
- 17. Visual representation Vector Embeddings https://jalammar.github.io/illustrated-word2vec King [ 0.50451 , 0.68607 , -0.59517 , -0.022801, 0.60046 , -0.13498 , -0.08813 , 0.47377 , -0.61798 , -0.31012 , -0.076666, 1.493 , -0.034189, -0.98173 , 0.68229 , 0.81722 , -0.51874 , -0.31503 , -0.55809 , 0.66421 , 0.1961 , -0.13495 , -0.11476 , -0.30344 , 0.41177 , -2.223 , -1.0756 , -1.0783 , -0.34354 , 0.33505 , 1.9927 , -0.04234 , -0.64319 , 0.71125 , 0.49159 , 0.16754 , 0.34344 , -0.25663 , -0.8523 , 0.1661 , 0.40102 , 1.1685 , -1.0137 , -0.21585 , -0.15155 , 0.78321 , -0.91241 , -1.6106 , -0.64426 , -0.51042 ]
- 23. So, is it all about arithmetic operations? Vector Embeddings What else? There is one main operation that you can do, and its called Similarity Search!
- 24. Vector Similarity Search Algorithms Vector Embeddings
- 26. Now that we have Vector Embeddings? Vector Embeddings We need a database to store them! Nope, we need a Vector Database!
- 28. Vector Database Music Video Images Faces Poses Emotions Audio Model Video Model Vision Model Face Detection/Recognition Models Vision Model Trained on Poses Sentiment Model Embeddings REDIS
- 29. How does a Vector DB need to have? Store data Index data Query data Does Redis have all ofem? Avoja, and much more! Vector Database
- 30. Vector indexing algorithms Redis manages vectors in an index data structure to enable intelligent similarity search that balances search speed and search quality. Choose from two popular techniques, FLAT (a brute force approach) and HNSW (Hierarchical Navigable Small World - a faster, and approximate approach). Vector search distance metrics Redis uses a distance metric to measure the similarity between two vectors. Choose from three popular metrics Euclidean, Inner Product, and Cosine Similarity used to calculate how close or far apart two vectors are. Powerful hybrid 鍖ltering Take advantage of the full suite of search features available in Redis query and search. Enhance your work鍖ows by combining the power of vector similarity with more traditional geo, numeric, text, and tag 鍖lters. Incorporate more business logic into queries and simplify client application code. Redis as Vector DB Vector Database
- 31. Redis as Vector DB Real-time updates Real-time search and recommendation systems generate large volumes of changing data. New images, text, products, or metadata? Perform updates, insertions, and deletes to the search index seamlessly as your dataset changes overtime. Redis Enterprise reduces costly impacts of stagnant data. Vector range queries Traditional vector search is performed by 鍖nding the top K most similar vectors. Redis Enterprise also enables the discovery of relevant content within a prede鍖ned similarity range or threshold for an alternative, and offers a more 鍖exible search experience. Vector Database
- 32. Titolo Lets see some code 4 of 4
- 33. Demo - Plan B! spring.data.redis.host =35.187.74.111 spring.data.redis.port =12000 spring.data.redis.username =default spring.data.redis.password =redis server.port=8080 spring.mvc.hiddenmethod.filter.enabled =true com.redis.om.vss.useLocalImages =false com.redis.om.vss.maxLines =300 redis.om.spring.djl.enabled =true redis.om.spring.djl.image-embedding-model-engine =PyTorch redis.om.spring.djl.image-embedding-model-model-urls =djl://ai.djl.pytorch/resnet18_embedding redis.om.spring.djl.sentence-tokenizer-max-length =768 redis.om.spring.djl.sentence-tokenizer-model =sentence-transformers/all-mpnet-base-v2 redis.om.spring.djl.sentence-tokenizer-model-max-length =768 redis.om.spring.djl.face-detection-model-engine =PyTorch redis.om.spring.djl.face-detection-model-name =retinaface redis.om.spring.djl.face-detection-model-model-urls =https://resources.djl.ai/test-models/pytorch/retinaface.zip redis.om.spring.djl.face-embedding-model-engine =PyTorch redis.om.spring.djl.face-embedding-model-name =face_feature redis.om.spring.djl.face-embedding-model-model-urls =https://resources.djl.ai/test-models/pytorch/face_feature.zip
- 34. Demo - Plan B! @Document public class ImageData { @Id private String id; @Indexed private String name; @Indexed private int height; @Indexed private int width; @Indexed(schemaFieldType = SchemaFieldType.VECTOR, algorithm = VectorField.VectorAlgorithm.HNSW, type = VectorType.FLOAT32, dimension = 512, distanceMetric = DistanceMetric.L2, initialCapacity = 10) private float[] imageEmbedding ; @Vectorize(destination = "imageEmbedding", embeddingType = EmbeddingType.FACE) private String imagePath; @Indexed private double score = 0; ... }
- 35. Demo - Plan B! @Service public class BestOfMatchService { @Autowired private EntityStream entityStream; @Autowired public ZooModel <Image, float[]> faceEmbeddingModel ; private List<ImageData > matchAll (byte[] image, int limit) { List<ImageData > imageDataList = new ArrayList<>(); try (Predictor <Image, float[]> predictor = faceEmbeddingModel .newPredictor()) { ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream( image); Image img = ImageFactory .getInstance().fromInputStream( byteArrayInputStream ); float[] embedding = predictor .predict( img); byte[] embeddingAsByteArray = floatArrayToByteArray(embedding ); SearchStream<ImageData> stream = entityStream.of(ImageData.class); List<Pair<ImageData,Double>> matchWithScore = stream .filter(ImageData$.IMAGE_EMBEDDING.knn(K, embeddingAsByteArray)) .sorted(ImageData$._IMAGE_EMBEDDING_SCORE, SortedField.SortOrder.ASC) .limit(limit) .map(Fields.of(ImageData$._THIS, ImageData$._IMAGE_EMBEDDING_SCORE)) .collect(Collectors.toList()); for (Pair<ImageData ,Double> pair : matchWithScore ) { ImageData imageData = pair.getFirst(); Double score = pair.getSecond(); imageData .setScore( score); imageDataList .add(imageData ); } return imageDataList ; } catch (Exception e ) { throw new RuntimeException( e); } } }
- 36. Demo - Plan B!
- 37. Demo - Plan B!
- 38. Demo - Plan B!
- 39. Demo - Plan B!
- 41. 4 Wrap up Unlocking the Future of Data: Powering Next-Gen AI with Vector Databases #WMF2024 3 2 1 Data Vector Embeddings Vector Database Redis
- 42. VOTA LINTERVENTO SU IBRIDA Luigi Fugaro Senior Solution Architect @ Redis
- 43. TITOLO PASSAGGIO UNO Per ulteriori informazioni puoi scriverci a speaker@wemakefuture.it www.wemakefuture.it