Apache Ignite is a high-performance, integrated and distributed in-memory platform for computing and transacting on large-scale data sets in real-time, orders of magnitude faster than possible with traditional disk-based or flash technologies.
Apache Ignite In-Memory Data Fabric comprises the following set of components shared across the Ignite platform.
| Transactions & Analytics | Hadoop & Spark | Runs Everywhere | |---------------------------|------------------------|--------------------| | Data Grid | Spark Shared RDD | Java | | Sql Grid | In-Memory File System | .NET | | Compute Grid | In-Memory MapReduce | C++ | | Service Grid | | Client Protocols | | Streaming & CEP | | Deployment Options | | RDBMS Integration | | | | Data Structures | | | | Messaging & Events | | |
Apache Ignite discovery for Kubernetes.
Kubernetes IP Finder:
To enable Apache Ignite nodes auto-discovery in Kubernetes, you need to enable
IgniteConfiguration. Let’s create an example configuration file called example-kube.xml
Amazon AWS IP finder, Google Compute Engine IP finder or JClouds IP finder will suit your needs but Kubernetes has to be deployed in one of these cloud environments. Alternatively, a shared file system or a relational database can be used for the sake of Apache Ignite nodes auto-discovery but you will have to maintain the database or the shared file system separately.
Here we provides information about the IP finder developed specifically for cases when Apache Ignite nodes are containerized by Kubernetes. The IP finder automatically lookups IP addresses of all the alive Ignite pods by talking to a distinct Kubernetes service that keeps all the endpoints up to date.
KubernetesIPFinderrequires users to configure and deploy a special Kubernetes service that maintains a list of the IP addresses of all the alive Ignite pods (nodes).
Every time a new Ignite pod is started, the IP finder will connect to the service via the Kubernetes API to obtain the list of the existing Ignite pods’ addresses. Using these addresses, the new node will be able to discover the rest of the cluster nodes and finally join the Apache Ignite cluster.
First, to create the service we need to prepare
ignite-svc.yaml file with the content below:
apiVersion: v1 kind: Service metadata: # Name of Ignite Service used by Kubernetes IP finder. # The name must be equal to TcpDiscoveryKubernetesIpFinder.serviceName. name: ignite spec: clusterIP: None # custom value. ports: - port: 9042 # custom value. selector: # Must be equal to one of the labels set in Ignite pods' # deployement configuration. app: ignite
- Create Ignite Pods Deployment file
apiVersion: extensions/v1beta1 kind: Deployment metadata: name: apache-ignite-cluster spec: replicas: 3 template: metadata: labels: app: ignite spec: containers: - name: apache-ignite-node image: vishnunair/docker-ignite:latest env: - name: OPTION_LIBS value: ignite-kubernetes - name: CONFIG_URI value: file:///data/example-kube.xml ports: - containerPort: 11211 # REST port number. - containerPort: 47100 # communication SPI port number. - containerPort: 47500 # discovery SPI port number. - containerPort: 49112 # JMX port number.
Create the Kubernetes service for Apache Ignite
$ kubectl create -f ignite-svc.yaml
Create the Kubernetes deployments for Apache Ignite
$ kubectl create -f ignite-deployment.yaml
To see the pods
$ kubectl get pods OUTPUT WILL BE SOMETHING LIKE BELOW: NAME READY STATUS RESTARTS AGE apache-ignite-cluster-2708371603-7l3x0 1/1 Running 0 4m apache-ignite-cluster-2708371603-mbv8j 1/1 Running 0 4m apache-ignite-cluster-2708371603-q82q6 1/1 Running 0 4m