AI Agents: Confidential Company Data

Sharing of confidential company data such as customer names, prices, or quantities of products with chatbot services (ChatGPT / Claude / Gemini) might be perceived as a security risk by the company management. For this reason, in the OFFISEEK production planning platform, chatbots are deployed inside the infrastructure and data is not sent outside.

Why?

• LLMs (Large Language Models) are generative models that are predicting the next word in the response, when given a question and some context.
• The LLMs are trained on big loads of conversations that were processed in its history
• If your company agreed to use its data for LLMs training, the LLMs might learn the connection between the company name, products and prices. (some companies are having data retention of up to 5 years and the setup is ON by default)
• And such a response might be generated for any user outside of the company (if learned during training)

There are arguments that you’re exposing your company's sensitive data similarly as you’re using cloud email such as Gmail. However, there is one big difference. Gmail is used mainly for STORING data. However, LLMs are AI models that are GENERATING content based on previously seen data.

Impact on AI adoption

Due to security risks, the following could occur:

• Companies will be hesitating with adopting external AI agents
• Companies will deploy AI agents in their own infrastructure in order to prevent sharing of the data.

Deploying AI models in own infrastructure is expensive

If companies would want to deploy agents locally, they should keep in mind that such models require GPUs in order to run quickly. Running the same prompts on CPUs might take several minutes for each prompt. In other words, deploying custom models might become expensive, as GPUs are required.

Example for a small company

Imagine you want to ask the AI daily 10 questions about your internal company data. F.e. you’re having a manufacturing company and you would want to consult the production plan and energy consumption with the AI.

For external APIs (ChatGPT, Claude or Gemini), the calculation could be following:

• Context: 100 product cards on the production dashboard. Each card consists of product name, customer name, required dates, description, prices, etc. On average - the context has 200 tokens.
• Questions: 10 questions each day.
• AI Monthly Input: 10x100x200x30= 6 x 1 000 000 = 6M
• AI Monthly Output: 10x500x30=150 000
• Input price: 5$ / MTok = 30$
• Output price: 25$ / Mtok …… 3.75$

Totaling 34$ monthly. 

Approach at OFFISEEK

At OFFISEEK, we are offering a SaaS platform for production planning. All customers' data is living inside our cloud infrastructure. And based on our philosophy, we do not want the data to be shared with 3rd party services.

That is why we are deploying AI agents directly to our infrastructure. At the same time, we are trying to optimize AI costs by generating answers only once a day and being able to use cheaper resources.

Choosing datasets to be shared with AI agents

In Offiseek, customers can choose what data will be shared with the AI agents. The customer might want to query only some data or some date period. The following datasets can be selected:

• Products
• Energy
• Stock

Based on the separated context, the AI agent knows what data to use for answering the prompts. 

Questions answering

In OFFISEEK you can set up questions about the selected data and the platform will overnight generate responses with daily, weekly or monthly periodicity.