= Orchestration Example = == ONAP == The [http://onap.org Open Network Automation Platform (ONAP)] is an open-source software platform that enables the design, creation, and orchestration of networking services. ONAP was formed as a result of a [https://www.lightreading.com/nfv/nfv-mano/open-o-ecomp-combine-to-create-onap/d/d-id/730522 merger of the Linux Foundation’s OPEN-Orchestrator (OPEN-O), and the AT&T ECOMP (Enhanced Control, Orchestration, Management and Policy) projects]. [https://www.onap.org/architecture ONAP architecture] consists of two major architectural frameworks: * design-time environment * run-time environment Both of these environments consist of a numerous separate subsystems. This tutorial simplified deployment no OpenStack only Kubernties; 4 machines: controller and 3 workers Basic set of subsystems: === Prerequisites === In order to access the test bed, create a reservation and have it approved by the reservation service. Access to the resources are granted after the reservation is confirmed. Please follow the process shown on [wiki:cosmos_workflow the COSMOS work flow page] to get started. === Resources required === At least 2 nodes are needed to run this example - one node for the Kuberneties controller and one node for the Kuberneties worker. === Execution === 1. Determine the set of nodes that you're using. If you have a topology assigned, it will be entered in the form "system:topo:group-black" for example 1. run {{{ onaptutorial-min.rb -g group-black }}} 1. Test if the nodes are up, with {{{omf stat -t system:topo:group-black}}} 1. If not in state POWERON, run {{{ omf tell -a on -t system:topo:group-black}}} 1. Once they're up, they will respond to ping, e.g. {{{ping node13-8 }}} 1. Download the cluster setup script {{{ wget -O create-cluster.sh }}} 1. Run the cluster setup script on the prepared nodes {{{ ./create-cluster.sh -c "node14-7" -w "node13-8" }}} ==== Imaging the nodes ==== {{{ onaptutorial-min.rb -g group-black }}} will load controller and worker images on the two nodes in the group and turn them on. Please note which node is the controller and which one is the worker since you will need it for the setup script. ==== Setting up physical server cluster ==== In order to create the Kubernetes cluster that will run ONAP, download and execute the cluster creation script on the console of the domain where you have node (please note that you will need to use corresponding node names in the script arguments) {{{ wget https://wiki.cosmos-lab.org/raw-attachment/wiki/tutorials/orchestration-example/create-kube-cluster.sh chmod 755 create-kube-cluster.sh ./create-kube-cluster.sh -c node13-8 -w node14-7 }}} [[CollapsibleStart(create-kube-cluster.sh)]] create-kube-cluster.sh {{{ #!/bin/bash rm -f cluster.yml rm -f kube_config_cluster.yml rm -f rke rm -f config omf tell -t all -a offh sleep 60 wget https://github.com/rancher/rke/releases/download/v0.2.1/rke_linux-amd64 mv rke_linux-amd64 rke chmod 754 rke usage () { echo "Usage:" echo " ./$(basename $0) -c \"cNode1 cNode2 ... cnodeN\" -w \"wNode1 wnode2 ... wnodeN\"" echo "Note: controllers hostnames and workers hostnames are to be enclosed in \"\"" exit 0 } if [[ ( $# == "--help") || $# == "-h" ]] then usage exit 0 fi if [ "$#" -lt 2 ]; then echo "Missing Kubernetes control and worker nodes" usage fi echo "# An example of an HA Kubernetes cluster for ONAP" >> cluster.yml echo "nodes:" >> cluster.yml while getopts c:w: option do case "${option}" in c) CONTROLLERS=${OPTARG};; w) WORKERS=${OPTARG};; esac done IFS=' ' read -ra C <<< "$CONTROLLERS" IFS=' ' read -ra W <<< "$WORKERS" echo "Testing node availability. This might take some time" for i in "${C[@]}"; do while ! ping -c 1 -n -w 1 $i &> /dev/null do printf "%c" "." done echo "127.0.0.1 localhost" > hosts echo "`ping $i -c 1 | grep "PING" | grep '('|awk '{gsub(/[()]/,""); print $3}'` ${i}" >> hosts scp hosts root@$i:/etc/hosts done for i in "${W[@]}"; do while ! ping -c 1 -n -w 1 $i &> /dev/null do printf "%c" "." done echo "127.0.0.1 localhost" > hosts echo "`ping $i -c 1 | grep "PING" | grep '('|awk '{gsub(/[()]/,""); print $3}'` ${i}" >> hosts scp hosts root@$i:/etc/hosts done echo "Availability check successful" for i in "${C[@]}"; do echo "- address: `ping $i -c 1 | grep "PING" | grep '('|awk '{gsub(/[()]/,""); print $3}'`" >> cluster.yml echo ' port: "22"' >> cluster.yml echo " role:" >> cluster.yml echo " - controlplane" >> cluster.yml echo " - etcd" >> cluster.yml echo " hostname_override: `ping $i -c 1 | grep 'PING' | awk '{print $2}' | awk -F . '{print $1}'`" >> cluster.yml echo " user: root" >> cluster.yml echo " ssh_key_path: '~/.ssh/id_rsa'" >> cluster.yml done echo "# worker nodes start " >> cluster.yml for i in "${W[@]}"; do echo "- address: `ping $i -c 1 | grep "PING" | grep '('|awk '{gsub(/[()]/,""); print $3}'`" >> cluster.yml echo ' port: "22"' >> cluster.yml echo " role:" >> cluster.yml echo " - worker" >> cluster.yml echo " hostname_override: `ping $i -c 1 | grep 'PING' | awk '{print $2}' | awk -F . '{print $1}'`" >> cluster.yml echo " user: root" >> cluster.yml echo " ssh_key_path: '~/.ssh/id_rsa'" >> cluster.yml done echo 'services: kube-api: service_cluster_ip_range: 10.43.0.0/16 pod_security_policy: false always_pull_images: false kube-controller: cluster_cidr: 10.42.0.0/16 service_cluster_ip_range: 10.43.0.0/16 kubelet: cluster_domain: cluster.local cluster_dns_server: 10.43.0.10 fail_swap_on: false network: plugin: canal authentication: strategy: x509 ssh_key_path: "~/.ssh/id_rsa" ssh_agent_auth: false authorization: mode: rbac ignore_docker_version: false kubernetes_version: "v1.13.5-rancher1-2" private_registries: - url: nexus3.onap.org:10001 user: docker password: docker is_default: true cluster_name: "onap" restore: restore: false snapshot_name: ""' >> cluster.yml ./rke up for i in "${C[@]}"; do scp kube_config_cluster.yml root@$i:~/.kube/config done exit 0 }}} [[CollapsibleEnd]] Perform this on kubernetes control node: 1. Set the context of kubernetes to onap by default: {{{ kubectl config set-context --current --namespace=onap }}} 2. Verify the Kubernetes cluster: \\ {{{ kubectl get nodes -o=wide }}} 3. Initialize Kubernetes Cluster for use by Helm {{{ kubectl -n kube-system create serviceaccount tiller kubectl create clusterrolebinding tiller --clusterrole=cluster-admin --serviceaccount=kube-system:tiller helm init --service-account tiller kubectl -n kube-system rollout status deploy/tiller-deploy }}} 4. Set only the required ONAP component to true {{{ cd overrides/ //edit the onap-all.yaml file and set the ONAP components to false nano onap-all.yaml }}} in our example mariadb and portal are turned on: {{{ # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. ################################################################### # This override file enables helm charts for all ONAP applications. ################################################################### cassandra: enabled: false mariadb-galera: enabled: true aaf: enabled: false aai: enabled: false appc: enabled: false cds: enabled: false clamp: enabled: false cli: enabled: false consul: enabled: false contrib: enabled: false dcaegen2: enabled: false dmaap: enabled: false esr: enabled: false log: enabled: false sniro-emulator: enabled: false oof: enabled: false msb: enabled: false multicloud: enabled: false nbi: enabled: false policy: enabled: false pomba: enabled: false portal: enabled: true robot: enabled: false sdc: enabled: false sdnc: enabled: false so: enabled: false uui: enabled: false vfc: enabled: false vid: enabled: false vnfsdk: enabled: false modeling: enabled: false }}} 5. Start helm server and add local helm repository {{{ // start server helm serve & //hit enter if the process is running to let the process run in background // add repository helm repo add local http://127.0.0.1:8879 }}} 6. Make onap helm charts available in local helm repository {{{ cd ~/oom/kubernetes make all; make onap }}} The output should look like this: {{{ root@node:make all; make onap [common] make[1]: Entering directory '/root/oom/kubernetes' make[2]: Entering directory '/root/oom/kubernetes/common' [common] make[3]: Entering directory '/root/oom/kubernetes/common' ==> Linting common [INFO] Chart.yaml: icon is recommended ... Update Complete. ⎈Happy Helming!⎈ Saving 34 charts Downloading aaf from repo http://127.0.0.1:8879 Downloading aai from repo http://127.0.0.1:8879 Downloading appc from repo http://127.0.0.1:8879 Downloading cassandra from repo http://127.0.0.1:8879 Downloading cds from repo http://127.0.0.1:8879 Downloading clamp from repo http://127.0.0.1:8879 Downloading cli from repo http://127.0.0.1:8879 Downloading common from repo http://127.0.0.1:8879 Downloading consul from repo http://127.0.0.1:8879 Downloading contrib from repo http://127.0.0.1:8879 Downloading dcaegen2 from repo http://127.0.0.1:8879 Downloading dmaap from repo http://127.0.0.1:8879 Downloading esr from repo http://127.0.0.1:8879 Downloading log from repo http://127.0.0.1:8879 Downloading sniro-emulator from repo http://127.0.0.1:8879 Downloading mariadb-galera from repo http://127.0.0.1:8879 Downloading msb from repo http://127.0.0.1:8879 Downloading multicloud from repo http://127.0.0.1:8879 Downloading nbi from repo http://127.0.0.1:8879 Downloading nfs-provisioner from repo http://127.0.0.1:8879 Downloading pnda from repo http://127.0.0.1:8879 Downloading policy from repo http://127.0.0.1:8879 Downloading pomba from repo http://127.0.0.1:8879 Downloading portal from repo http://127.0.0.1:8879 Downloading oof from repo http://127.0.0.1:8879 Downloading robot from repo http://127.0.0.1:8879 Downloading sdc from repo http://127.0.0.1:8879 Downloading sdnc from repo http://127.0.0.1:8879 Downloading so from repo http://127.0.0.1:8879 Downloading uui from repo http://127.0.0.1:8879 Downloading vfc from repo http://127.0.0.1:8879 Downloading vid from repo http://127.0.0.1:8879 Downloading vnfsdk from repo http://127.0.0.1:8879 Downloading modeling from repo http://127.0.0.1:8879 Deleting outdated charts ==> Linting onap Lint OK 1 chart(s) linted, no failures Successfully packaged chart and saved it to: /root/oom/kubernetes/dist/packages/onap-5.0.0.tgz make[1]: Leaving directory '/root/oom/kubernetes' root@node:~/oom/kubernetes# }}} This '''make''' command takes quite some time to finish (i.e. 12+ minutes) so please be patient. 7. Deploy ONAP {{{ helm deploy demo local/onap --namespace onap -f ~/overrides/onap-all.yaml }}} {{{ root@node:~/oom/kubernetes# helm deploy demo local/onap --namespace onap -f ~/overrides/onap-all.yaml fetching local/onap release "demo" deployed release "demo-cassandra" deployed release "demo-mariadb-galera" deployed root@node:~/oom/kubernetes# }}} 8. Verify the deploy {{{ helm ls }}} resulting in: {{{ NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE demo 1 Sun Oct 20 02:14:55 2019 DEPLOYED onap-5.0.0 El Alto onap demo-cassandra 1 Sun Oct 20 02:14:55 2019 DEPLOYED cassandra-5.0.0 onap demo-mariadb-galera 1 Sun Oct 20 02:14:56 2019 DEPLOYED mariadb-galera-5.0.0 onap }}} -- Openstack installed -- VM cloud image ubuntu 18.04-server-image