Deploying custom Watson NLP Models with Terraform

IBM Watson NLP (Natural Language Understanding) and Watson Speech containers can be run locally, on-premises or Kubernetes and OpenShift clusters. Via REST and gRCP APIs AI can easily be embedded in applications. This post describes how custom Watson NLP models can be deployed with TechZone Deployer, an opinionated deployment and operations toolkit based on Terraform and ArgoCD.

Watson NLP

To set some context, check out the landing page IBM Watson NLP Library for Embed. The Watson NLP containers can be run on different container platforms, they provide REST and gRCP interfaces, they can be extended with custom models and they can easily be embedded in solutions. While this offering is new, the underlaying functionality has been used and optimized for a long time in IBM offerings like the IBM Watson Assistant and NLU (Natural Language Understanding) SaaS services and IBM Cloud Pak for Data.

TechZone Deployer

With TechZone Deployer (also known as TechZone Accelerator Toolkit, TechZone Automation, Software Everywhere, Cloud Native Toolkit) IBM software, open source projects and custom applications can easily be deployed to various clouds. Check out my earlier blog that introduces the toolkit: Introducing IBM’s Toolkit to handle Everything as Code. The toolkit leverages Terraform and GitOps and is based on best practices from IBM projects with partners and clients. With the toolkit both infrastructure like Kubernetes clusters as well as Kubernetes resources within clusters can be deployed. Infrastructure resources are deployed via Terraform, resources within clusters via Argo CD.

Automatic Deployments of the Watson NLP Runtime and Models

Based on TechZone Deployer my team has created an asset to deploy 1. OpenShift clusters, 2. Watson NLP and 3. custom applications in these clusters in one hour. Watch the short video Automation for IBM Watson Deployments for an introduction.

The usage of TechZone Deployer is very easy:

Install CLI
Define which modules to deploy from a module catalog
Configure modules in variables.yaml and credentials.properties files
Use CLI to create Terraform modules
Launch local tools container and apply Terraform modules

This sample Watson NLP configuration uses one predefined model hosted in the IBM Cloud Pak registry.

- name: terraform_gitops_watson_nlp_runtime_image
  value: watson-nlp-runtime:1.0.18
- name: terraform_gitops_watson_nlp_runtime_registry
  value: watson
- name: terraform_gitops_watson_nlp_accept_license
  value: false
- name: terraform_gitops_watson_nlp_imagePullSecrets
  value:
    - ibm-entitlement-key
- name: terraform_gitops_watson_nlp_models
  value:
    - registry: watson
      image: watson-nlp_syntax_izumo_lang_en_stock:1.0.7
- name: terraform_gitops_watson_nlp_registries
  value:
    - name: watson
      url: cp.icr.io/cp/ai
- name: terraform_gitops_watson_nlp_registryUserNames
  value:
    - registry: watson
      userName: cp

Deployments of multiple Models

It’s also possible to deploy in addition to the Watson NLP runtime multiple models, both predefined models as well as custom models.

At a minimum you need the Watson NLP runtime image. The NLP runtime container runs in the Watson NLP pod at runtime.

Additionally you can have 1 to N ‘model images’ which run as Kubernetes init containers. They are triggered when pods are created. Their purpose is to put the model artifacts on disk so that the Watson NLP runtime container can access them. Once they have done this, these containers terminate.

Images reside in registries which are typically protected. Pull secrets need to be provided to access them. Sealed Secrets for Kubernetes are used to protect the secrets.

There can be multiple registries (N >= 1) and multiple secrets (M >= 0). Registries can use secrets, but don’t have to (N > M). There needs to be one registry to access the NLP runtime image which is stored in a protected registry.

The configuration is done in two files:

variables.yaml
credentials.yaml

Pull secrets have to contain the following information:

Secret name: Defined in the “imagePullSecrets” array in variables.yaml.
Registry URL: Defined in the “registries” array in variables.yaml.
Registry user name: Defined in the “registryUserNames” array in variables.yaml. the “registry” name needs to map to the same name under registries.
Registry password: Defined in “TF_VAR_terraform-gitops-watson-nlp_registry_credentials” in credentials.properties. This variable can include a comma delimited list of registry passwords/tokens. For multiple secrets the order needs to be the same one as in variables.yaml for “registryUserNames”.

The screenshot shows the deployed containers.

To find out more about Watson NLP and TechZone Deployer, check out these resources: