AI is driving a remarkable transformation throughout the industry, delivering unprecedented productivity gains and enabling rapid insights from vast amounts of data. In the second of a two-episode season premiere, Tirthankar Lahiri, SVP of Mission-Critical Data and AI Engines, discusses how Oracle AI Vector and embedded machine learning search are harnessing the power of AI to unlock value from enterprise data. AI is triggering an incredible transformation across the industry, enabling breakthrough improvements in productivity and rapid insights across massive volumes of data. However, in order to fully harness the power of AI, it is necessary to enable AI processing where the data resides. Oracle AI vector and built-in-machine learning search bring the power of AI to enterprise data, and allow developers to build sophisticated RAG and Agentic frameworks that leverage the full power of the converged database architecture of Oracle Database — including its class-leading scalability, fault-tolerance, and enterprise-grade security. Furthermore, Oracle database provides several mechanisms to make data "AI-ready" by enabling declarative data intent for AI. In this session, we will describe these techniques, and more, to explain how to truly build an AI for data solution in this rapidly changing AI landscape!

------------------------------------------------

Episode Transcript:

00:00:00:00 - 00:00:36:04 Unknown Welcome to the Oracle Academy Tech Chat. This podcast provides educators and students in-depth discussions with thought leaders around computer science, cloud technologies, and software design to help students on their journey to becoming industry ready technology leaders of the future. Let's get started. Welcome to Oracle Academy Tech Chat, where we discuss how Oracle Academy prepares next generation's workforce. I'm your host, Tyra Peirce.

00:00:36:06 - 00:01:02:02 Unknown This is the second episode in our two part presentation. AI for data when data meets intelligence. In this episode, our guest speaker to thank Tirthankar Lahiri, senior vice President for mission critical data and AI engines at Oracle, concludes his presentation on AI and enterprise data. If you've not heard episode one yet, please go to oracle.com/podcast where you will find all episodes of Oracle Academy Tech Chat.

00:01:02:04 - 00:01:32:05 Unknown Now let me hand over two to thank for the conclusion of this presentation by. Hey guys, thank you very much for joining. It's a great pleasure to be presenting AI for data. Now let's talk about vector sequel. I've already mentioned this briefly earlier, so I won't spend too much time on this, but basically you can do vector distances to find similar, objects using the vector distance function and very simple query finding the customers that are similar to this photo.

00:01:32:07 - 00:01:55:00 Unknown And I wonder what the top five customers and we could add other complex conditions. Like I said earlier, maybe only one customers in San Francisco that match this photo. Again, that filter will be added to the query to find only the customers matching the photo in San Francisco. We could join, various with other tables as well.

00:01:55:01 - 00:02:25:20 Unknown For instance, we could join it with a status table that finds the customers only with a certain spending tier. So this sort of, you know, extension gives me, again, as I said earlier, the power of composing SQL with complex conditions to run very sophisticated searches. Okay. Once again, in the query, you can also specify how accurate you want the search to be, regardless of the index accuracy.

00:02:25:20 - 00:02:48:01 Unknown You can make a search more accurate by spending more time on the search by going deeper in the index. Same deal. The index. The default accuracy. Maybe the default was that 85%, but I want 95% for a certain search. I want to do a deeper search. This directive with the SQL lets me do exactly that. I can specify again how much I want in the SQL itself.

00:02:48:01 - 00:03:13:16 Unknown Maybe it's a different search. Maybe my recommendations initially were great. I want like a really deep recommendation based on my last search. My last browsing or buying patterns. So this lets me crank up the knob, crank up the dial rather for the accuracy based on my use case. Again, to allow the search to proceed deeper into the vector index.

00:03:13:18 - 00:03:34:07 Unknown Okay. All right. I'm going to, you know, vector index support transactions. There's not a whole lot to say about that. As you update the data, the vector indexes get maintained. The core indexes, you know, I'm not going to spend a whole lot of time on this because some of this is very oracle specific. But we scale our Oracle database in a number of different ways.

00:03:34:09 - 00:03:58:12 Unknown And each of those scale out mechanism, the Oracle database runs on multiple physical computers. All of those mechanisms support vector indexes. So vector vector search can scale out on nodes of a compute cluster easily. You could also scale out using something called sharding, which is a way to scale the vector, the result to a global scale.

00:03:58:12 - 00:04:18:02 Unknown So imagine I have, you know, a database that's logically sharding across multiple continents. I essentially can build a vector index on each shard and do a global search that runs very fast, because if I'm searching for products that are, let's say Italian coaches, for instance, I would only search the Italian shard and not the other, the other shards.

00:04:18:02 - 00:04:38:19 Unknown So you get performance improvements as well as localization of your search results. Okay. So I'm going to skip some of this and go to the next step. So I want to talk about. So some some just some simple use case vector search before I get to my other topic I wanted to cover quickly today I think I have ten minutes left.

00:04:38:23 - 00:05:03:02 Unknown So yeah. So there's lots of use cases. Like I said, for just plain vanilla I like to search lots of use case. I just find me similar support tickets. Find me someone matching this biometric pattern. Similar products find me manufacturing defects. You know, text search using some similarity. If I miss similar documents or, you know, browsing made, etc. those are very standard vector search case.

00:05:03:02 - 00:05:25:09 Unknown But let's talk about generative AI. That's, you know, the big new thing nowadays. And generative AI is really what made vector search very, very important as a primary use case for vector search because it lets you interact with business data using natural language. So why do I do a vector search for this? Because it lends to all of the work.

00:05:25:09 - 00:05:54:03 Unknown Anyway, you know, everyone here must have used GPT or Grok or Gemini. Why do I need vector search? Well, here's the reason why. Generative AI is, basically answers natural language questions. You ask a question, for instance, in the support example, I want to know why my laptop keeps rebooting as an example. I also love them. They basically uses the question and it's general knowledge to answer the question.

00:05:54:03 - 00:06:13:21 Unknown And maybe it doesn't know. It might get a bad response if you just toss them ungodly without any other other information. You may not get a good answer that you like. It might. It might tell you that just by buying your laptop with a laptop is rebooting. Buy a new one. That's not a good answer, but I spent $20 buying a laptop and I wouldn't want to hear that.

00:06:13:23 - 00:06:37:00 Unknown Especially if it was a MacBook that was expensive. That's a bad answer for me. This is not a great answer. And why is that? Because it doesn't know anything about your support incident or other incidents. So it alarms are very, very smart, but they're generalist. They're trained on broad data from the internet, not on specific private data.

00:06:37:02 - 00:06:58:07 Unknown And they can also hallucinate as we know. Right. So what we need to do is to add vector search to improve results. Search quality. What we do is using vector search, we could augment prompts with private database content to provide better answers, which avoids having to create an alarm on sensitive data and this is called drag.

00:06:58:07 - 00:07:21:01 Unknown As everyone knows, this is a very popular technique today. Retrieval augmented generation. So you could again include the question in a vector sent to a vector database to find relevant support incidents, that dig into the reports and augment them with the reports and the question, and then send the whole thing there alarm and then get a better answer.

00:07:21:03 - 00:07:54:05 Unknown It said, you know, it might say, hey, you might need to apply the OS update if that's what you're seeing, because there could be other reports that look similar. Now, you could do even better with Oracle database or the enterprise database to handle that search, because now I can also include customer data and product data, not just the support vectors, but other information that I can also use to augment the response with, to provide even further clarity on what was going on with that customer.

00:07:54:07 - 00:08:27:07 Unknown So again, same deal. I send the question to the customer, to the to the vector database to generate the vector, to look up relevant support request. So I have the question why does my laptop keep reporting a convert that to a vector? I use that to search the vector database, and then I use the vector database to not only generate the incident reports, but I also return the matching product and customer data to find the reports, the support, incident description and resolutions augment them with product information.

00:08:27:07 - 00:08:52:13 Unknown So this is the laptop with a certain middle manufacturing profile. And the customer was in Las Vegas for instance. All of that is set now back to their lab, and they will now be able to get a much better answer because now it has both the vector information as well as the business information. And this might give you a much better, more elaborate response that tells you what's really going on.

00:08:52:15 - 00:09:22:08 Unknown You need a certain update number with what's with the firmware. You got to do this. The cool place. All of that could be summarized because it has the information now for both the incident reports and the customer product, profile data. This is how I run with an enterprise database gives you a better answer. And there's many use cases for generative AI.

00:09:22:10 - 00:09:45:11 Unknown You know, customers want to know what is a document all about? The, very common use case, the one I just showed you, by the way, the support chatbot that I tried to get something done on a website, I can't, and I asked the chat bot, what's going on here? Or maybe I can't log into my, you know, internal universe to database for whatever reason, before I call tech support.

00:09:45:16 - 00:10:10:01 Unknown Can I find out of this? The known problem or what the issue is support. Chat bots are very commonplace using generative AI. You know, travel chatbot. There's a lot of different use cases where you basically have to converse with your business data. And that's what vector search lets you do. The power of vector search lets you do, combined with the power of large language models.

00:10:10:03 - 00:10:43:14 Unknown I'm going to show you a very quick demo here, built by one of our customers, a partner. We have a, an application developed platform, apex, that lets you develop Oracle apps extremely quickly. And how does apex do this? Using retrieval, augmented generation to make such a lot more intelligent and sort of chat oriented. So the first thing we want to do here is to upload a bunch of PDF documents to the database to facilitate retrieval, augmented generation submit as quickly walk you through this hopefully as the work.

00:10:43:15 - 00:11:05:05 Unknown All right. So if you can see here there's a couple of PDF documents that I want to basically augment my database with. So one is on sovereign cloud Oracle sovereign cloud. The other is how to do spatial analysis with Oracle Spatial Studio. These are big PDF documents. These are not available directly. So I want to get this to the realm by adding it to my database.

00:11:05:05 - 00:11:28:12 Unknown And I add them to this apex application. Now that they're in the in the system, I basically convert them into vectors for the loaded in the database in the schema. Now the convert into vectors and you can see these are big vectors that these PDF documents perform. So now I have my vectors in the database, I have a PDF documents and I have my vectors.

00:11:28:14 - 00:11:50:17 Unknown All inside of the database. Right. If the question you know, you have basically what we've done is we've taken the document, we've taken the document and chunk in a chunks. Each chunk has a separate vector. And all these vectors are now inside the database. You typically don't create a single vector per document. You typically divide the document, the text chunks just because the token limits for albums.

00:11:50:19 - 00:12:17:23 Unknown And then each chunk is separately vectorized. So that's basically step one though. You can do basically a have a chat session. That essentially does this. You can into this application just type your question. You know, the question is what is sovereign cloud. And you can see if you ask JNI, what happens is you're going to get a response that this is going to take a while because this is talking to a public.

00:12:18:01 - 00:12:42:22 Unknown This is not Oracle database respondent. This is a public LRM. You're going to get this, information from the from the LM. But it's augmented with Oracle information. So let's stop here for a second. It's open or I don't know how to stop this. Then how did it stop anyway? Sorry, but that,

00:12:43:00 - 00:13:01:21 Unknown Basically, the idea here is that you use the LM to generate the response augmented with the information I just uploaded. Now, if I just add vector such a message. Okay. So I don't know how to replay the stupid thing. Sorry about that. This video is something I'm not very comfortable pausing and restarting, but let's see. This will cooperate.

00:13:01:21 - 00:13:27:19 Unknown This then. So I typed in the question here for step one. You got that? The question was sent to get a lab and, hopefully it'll cooperate to give me a response in a short time. There we are, almost there. When Queequeg, it's basically taking its time talking to the end. But you can see that the information returned includes documentation from the Oracle documents.

00:13:27:20 - 00:13:50:21 Unknown You can also look about vectors such itself that. So what this question did was match itself to these text chunks in the document. And that's the document chunk. It matched in the vector search. And to generate this context, there was that said that the loan was sent the user's question plus the text chunks that matched that user question.

00:13:50:23 - 00:14:17:14 Unknown This was the augmentation payload that the lab used to generate its correct answer. Essentially, it shows you rag in action that if you give it business data and you combine it with, vector data, then you get the best possible outcome, because now it has all the information needed to answer the question. And you can keep going, can ask about things like Oracle Spatial that were the second document we uploaded.

00:14:17:14 - 00:14:36:22 Unknown So again, because it has that document and its vector chunks, it can answer questions about Oracle Spatial as well. And you know again you could iteratively go on and interrogate them and do cool things with it. So it gives you an idea of that, you know, given the documents and the vector embeddings for each chunk, you can use them to.

00:14:37:03 - 00:15:17:14 Unknown It's a run, essentially interactive chat sessions that give you a lot of information that combined the knowledge with the additional knowledge you just added to the them. Using those documents. Okay. Oracle database, essentially vector search is designed to power the entire pipeline for generative AI that you can take the data sources you have, load data from those from the sources, convert those, documents into, you know, essentially summaries or, or segments rather than through the vector embedding models, load them with the vector database and then run similarity searches on them.

00:15:17:16 - 00:15:49:23 Unknown And this vectorized data can be used as part of the interactive flow with the large language model. So a complete gen AI pipeline is really what brings AI to your data. Remember, the data has not moved to Oracle database. All the data is here. And all of this additional intelligence is being added to that data through the sophisticated gen AI pipeline.

00:15:50:01 - 00:16:19:09 Unknown You know, using you know, so we have native calls for generative AI. We have tight integration with third party frameworks. So a data science developer can very easily build a complete every data solution using, using the Oracle Gen AI pipeline. Okay, I think I managed just the outline. So let me quickly wrap up here. The key takeaways here is that vector search is really what brings AI to your business data.

00:16:19:11 - 00:16:46:11 Unknown And the Avoca search includes a new vector data type for a new set of SQL extensions, very simple extensions that allow you to search data by semantics or similarity. A new set of vector indexes for performance. It's very fast because as I showed you, this index lets you search data within milliseconds. The very, very fast indexes for very rapid retrievals and highly accurate retrievals.

00:16:46:13 - 00:17:10:13 Unknown And the AI vector search is not just for similarity search. It's also to enable conversations with your data using generative AI. Lots of places you can go get more information. So there's live labs that you could go try this out. You're welcome to take a screenshot of this if you'd like to see, to get more information in the future.

00:17:10:15 - 00:17:38:18 Unknown We do have a free version Oracle database. I would encourage students and faculty to explore getting this. It's a great way to get started. Oracle database to understand everything. All the concepts we have in the database are free to use. And it's a it's a, it's a basically a container image. You could download that makes it easy to develop applications with that to understand some of the, you know, concepts around things like I like vector search very quickly and that's it for my site.

00:17:38:18 - 00:18:04:02 Unknown I think I stayed on time. You did really great to thank her. And I actually have some questions for you. So does using SQL compromise on performance on vector databases compared to mechanisms by other vendors? Actually does not. SQL is a very thin wrapper. SQL basically is a declarative specification of access methods that the other vector providers use.

00:18:04:02 - 00:18:23:17 Unknown So whatever the vector access method is, is directly exposed to SQL, there's no additional overhead from the SQL itself. If you think of SQL as a very simple, like a work plan for what operation I want, and it directly goes to vector database. If you look at what happened in vector search, the actual time is spent in the search.

00:18:23:17 - 00:18:44:07 Unknown The search has to look at lots of vectors. Go to an index. The SQL engine adds like negligible overhead, to that process. So yeah, there is no, no one since then. I have two more for you. How does the system handle updates inserts, deletes, modifies in the layers? And in parentheses add the multilayer in-memory graph.

00:18:44:12 - 00:19:11:23 Unknown Does it rebalance layers or delay maintenance. Very good question. So there's a lot of literature on this. And I would encourage you to look at the people in W basically index maintenance is done just like index search. So if I delete a vector from an index, the way the delete works, essentially I have to find again the the vectors neighbors that I'm deleting from and essentially effectively rebalance the index links so that the number of links to each neighbor has remained the same.

00:19:12:01 - 00:19:37:01 Unknown This is standard literature. Other techniques for graph maintenance. We do try to optimize this by batching the operations together. So instead of doing one at a time, we wait for several to accumulate and then do them in a batch. So the way we do it is we accumulate changes in the journal to make sure that the user query can cancel both the index and the journal, and when the journal reached a certain size, we applied the journal to the index.

00:19:37:06 - 00:20:00:15 Unknown But that's a great question. It is true that vector indexes are not as easy to maintain as traditional relational indexes, but again, they're a standard techniques often, to advertise the cost. Okay, one final question when combining tabular structure with unstructured columns makes it flexible. Will this be logistically more expensive to create maintain? No, not really at all.

00:20:00:15 - 00:20:23:20 Unknown I mean, if you look at the way the storage manager works, right, it gives you full flexibility. The vectors are vec vector, the base basically just another column. And you know, creation and maintenance is done just like a regular index. So we know how to maintain indexes. In fact we support it. I like to think of this as being a superset of Mongo Atlas, because we can store Json documents just fine as well.

00:20:24:01 - 00:20:45:11 Unknown If you want to use Oracle in the document store, you can also do that. You can imagine that the call it the table is basically a Json collection and have one of the fields be a vector field for instance. So it's actually a superset. There is no additional cost. What you do get though is flexibility, which you don't get with the document stores, because now I can combine all kinds of data.

00:20:45:11 - 00:21:05:04 Unknown I can have spatial columns for a very common requirement, as I want to find similar restaurants first by description, but only within a certain search. Treatise on my current location on my phone. So that's a spatial search. It's not just a vector search. There's no you know, I've seen this. I've asked for restaurant recommendations, and I had location tracking off my phone.

00:21:05:05 - 00:21:23:10 Unknown I gave you a recommendation for a restaurant in Tokyo. I thought, that's helpful. I'm not in Tokyo right now. And asked the top sushi restaurant this. Oh, go to poke you. This is a fantastic place. I said. I would love to go to Tokyo, but I happen to not be in Oregon. I can't go to Tokyo. So that's a case where you want a spatial search combine with vector search.

00:21:23:12 - 00:21:43:10 Unknown And I did find a very nice sushi restaurant in Oregon, Portland, but it requires spatial filters, so you get flexibility. You don't really get any additional, overheads from, from the tabular representations. I love that example because that's a question that I would ask is give me a sushi restaurant in in California in Palo Alto. So I really learned so much there.

00:21:43:10 - 00:22:12:09 Unknown I think that's all of our questions. One final announcement Oracle Academy is excited. Announce new free artificial intelligence resources, including hands on workshops that introduce generative AI technologies in education bites, featuring practical labs and projects on digital assistants, natural language processing and vector search to help you and your students build essential world AI skills, faculty members log in to the member hub and explore the course catalog in our air resources.

00:22:12:11 - 00:22:34:02 Unknown Thank you to Tirthankar for a fascinating presentation and for being our guest on the Oracle Academy Tech Chop podcast. For more information about Oracle AI, visit Oracle Akamai and if you would like to learn more about Oracle Academy, our website is academy.oracle.com. If you enjoyed this podcast, please like and subscribe on your podcast platform. Thanks for listening.

00:22:34:03 - 00:22:46:06 Unknown That wraps up this episode. Thanks for listening and stay tuned for the next Oracle Academy Tech Chat podcast.