#1066 - Invoking OCI GenAI from OIC

Introduction 

The OCI GenAI service offers access to pre-trained models, as well as allowing you to host your own custom models.

The following OOTB models are available to you - 

OIC GenAI service is currently available in 5 regions - 

The service offers I will be using the Chicago endpoint in this post. 

The objective of the post is to detail how easy it is to invoke the chat interface from an integration. I will detail this both for Cohere and llama.

Pre-Requisites 

First step - generate private and public keys - 

openssl genrsa -out oci_api_key.pem 2048

and

openssl rsa -pubout -in oci_api_key.pem -out oci_api_key_public.pem

User Settings - API Keys - 

I also created the following Policy - 

allow group myAIGroup to manage generative-ai-family in tenancy

Invoking GenAI chat interface from OIC

Now to OIC where I create a new project - adding 2 connections - 

https://inference.generativeai.us-chicago-1.oci.oraclecloud.com

As already mentioned, OCI GenAI is currently available in 5 regions. Check out the documentation for the endpoint for other regions.

The Response payload - 

I test the integration -

The invoke is configured with the following request / response -

Request - 

 Response - 

Maybe it's just me, but the llama response is better.

{

  "text": "Here is some additional context Expenses Rules no alcohol can be expensed. bill limit per meal is €100. you cannot expense extreme left wing literature.  Please answer my question: can I expense the book the Anarchist's cookbook?"}

Here is the test result - 

Ok, so let's try with a steak and a beer for €130.

Request -

{

  "text": "Here is some additional context Expenses Rules no alcohol can be expensed. bill limit per meal is €100. you cannot expense extreme left wing literature.  Please answer my question: can I expense a meal of a steak and a beer for €130?"}

Response -

{

  "summary" : "No, you cannot expense a meal of steak and a beer for €130 as it violates two of the Expenses Rules: the bill limit per meal is €100, and no alcohol can be expensed."

}