[RFC] Support for air gapped mode

Signed-off-by: Lianhao Lu <[email protected]>

Author: lianhao

community/rfcs/25-23-04-GenAIComps-001-Air-Gap-Support.md

@@ -0,0 +1,81 @@
+# Support Air-gapped environment
+
+This RFC talks about how to support running OPEA microservices in an air-gapped environment.
+
+## Author(s)
+
+[Lianhao Lu](https://github.com/lianhao)
+
+## Status
+
+`Under Review`
+
+## Objective
+
+An air-gapped computer or network is one that has no network interfaces, either wired or wireless, connected to outside networks(e.g. Internet, etc.). Many enterprises have network security policies that prohibit the computers in the enterprise internal network to have any kind of ability to send/receive data to/from the outside networks. 
+
+This RFC discusses about how to support running OPEA microservices in such an air-gapped environment, including the methodology to find out the what kind of data need to be pre-downloaded, where to store the pre-downloaded data, and how this will affect the deployment, etc.
+
+### Online data types
+There are some OPEA microservices requires downloading data from Internet during the runtime, these kind of data includes:
+
+#### Type1: User configurable AI model data
+Some OPEA microservices allow the user to configure the AI model it will run against with. We've already supported pre-download these kind of data and run the OPEA microservices with them in air-gapped environment. 
+
+#### Type2: User non-configurable AI model data
+Some OPEA microservice will silently download some AI model data from Internet during runtime. The kind of data to be downloaded is either hardcoded in the OPEA microservice, or is not exposed to the end user to be configurable. For example, the `dataprep` microservice requires to download the AI model `unstructuredio/yolo_x_layout` during runtime to process unstructured input data, and currently there are no exposed configuration options for the `dataprep` end users to specify which AI model to be downloaded for that purpose. 
+
+#### Type3: Other data
+Some OPEA microservices needs download additional data other than AI models during runtime, e.g. `dataprep`, `retriever` and `gpt-sovits` need to download a subset of `nltk` data, `speecht5` needs to download data from [intel-extension-for-transformers](https://github.com/intel/intel-extension-for-transformers/tree/main/intel_extension_for_transformers/neural_chat/assets/speaker_embeddings), etc.
+
+In this RFC, we'll mainly cover the `Type2` and `Type3` online data type, since most of the microservices have already supported using the offline data of `Type1`. 
+
+## Motivation
+
+When trying to deploy the OPEA microservices in some customers air-gapped environment, we've found that there are quite some OPEA microservices needs to download data from internet during runtime, which requires special tweaks for each of them. We want to make sure that all OPEA microservices can be run in the air-gapped environment in a uniform way.
+
+## Design Proposal
+
+### Ways to verify
+To quickly verify if a OPEA microservice support air-gapped mode or not, and what kind of online data it's downloading, we can use the following steps:
+
+ 1. Deploy the microservice in one of the 3 following environment:
+    - Deploy it in a real air gapped environment(docker or K8s)
+    - Deploy it in a tweak K8s environment by disable K8s DNS forwarding(i.e. remove the `forward` part in `kubectl -n kube-system edit cm coredns`, and restart coredns related pods)
+    - Deploy it with the fake proxy settings by setting both `http_proxy` and `https_proxy` to a non existent proxy servers, e.g. `http://localhost:54321`.
+ 2. Send requests to the microservice under verification
+     We need to make sure that all the sent out requests should have a decent coverage of the internal data flow of that microservices, because sometimes it's not the microservice itself is downloading online data, but the dependent modules.
+ 3. Check the requests return status and microservice logs to find out whether it supports air-gapped mode and what kind of online data it's downloading if it's not.
+
+### Type 2 online data
+
+Since we're not allowed the distribute AI model data in the microserivce's container image, we need to make sure that there is a way to `mount` the user pre downloaded AI model data into the microservice's runtime so that the microservice itself can run in the air gapped mode. We also need to document what to be download and how to `mount` in the deployment document of that microservice.
+
+### Type 3 online data
+
+To minimize the deployment complexity, for small size online data which is NOT shared by multiple microservices, we should have them downloaded in the container image, so that the microservice itself doesn't need to download it during runtime.
+
+For online data which are used by multiple microservices, e.g. `nltk` data,  depends on its size, we can have them pre downloaded in the container image if it will not increase the image size more than 5%. For large size data, we should follow the `Type 2` online data method to support running in air gapped mode.
+
+
+## Alternatives Considered
+
+Using the same `Type 2` method for `Type 3` online data
+
+## Compatibility
+
+n/a.
+
+## Miscellaneous
+
+List other information user and developer may care about, such as:
+
+- Engineering Impact: 
+  - increase container image build time
+  - decrease the container image startup time.
+
+- Staging plan:
+  - Using the methods listed in the above section `Ways to verify` to find all the microservices which doesn't support air-gapped mode, and create corresponding github issues
+  - For each found microservice, figure out the online data type and enhance it to support air gapped mode
+
+- CI env to test this functionality: Since this is a common requirements to all OPEA microservices, we probably need to setup an CI/CD test task to test it?