IDG Contributor Network: What IT should understand before implementing an in-memory computing solution | Computing

As I explained in ”Scale out and conquer architectural decisions behind distributed in memory systems,” choosing the right open source solution (such as a combination of Apache Ignite, Apache Kafka, Apache Spark, and Kubernetes) for an in-memory computing (IMC) infrastructure ensures a simpler and more cost-effective architecture for applications to support digital transformation and omnichannel customer engagement initiatives.

It is also critical when deciding on a solution to ensure the flexibility to deploy and manage the architecture to meet your organization’s requirements related to simplicity, security, availability, and cost. As you evaluate various IMC architecture alternatives, here are some key considerations to keep in mind.

Architecture deployment flexibility

The solution should offer the flexibility to be deployed on-premises; on an internal cloud, private virtual cloud, or public cloud; or on a hybrid environment. This flexibility is essential for balancing infrastructure costs and management requirements.

Server configuration

Be sure the in-memory computing cluster can be built using commodity servers. This is essential for controlling costs and increasing redundancy as the IMC infrastructure scales to dozens or even hundreds or thousands of servers.

Using containers

With containers, the IMC solution can be abstracted from the physical environment in which it runs. This makes it easier to deploy the solution consistently no matter which of the above locations is used. It frees IT operations to focus on deployment management without worrying about the specific applications and lets developers focus on their applications without worrying about the infrastructure. Kubernetes, for example, can automate the deployment, scaling and management of containerized applications, such as Apache Ignite and Apache Kafka, across a server cluster that has been deployed in any location.

Monitoring and visualization

Being able to monitor and visualize the entire environment is essential to ensuring the optimal use of the IMC resources as well as the optimal operation of each component in the stack. Look for an IMC solution that includes ODBC and JDBC APIs, which enable customers to connect to the solution from any platform using standard SQL queries, without custom coding. This makes it easy to integrate many of the common data monitoring and visualization tools such as Tableau or Microstrategy.

Native integrations

Ensure there are native integrations for your other open source solutions of choice to reduce the effort required to integrate and establish communications between the various components in your architecture.

Rolling upgrades to eliminate downtime

Many organizations using their IMC infrastructure in production applications will want to update individual nodes in the cluster without bringing down the entire cluster (and application) as a way to deploy a software update.

Data center replication

Another key capability to consider for production environments is data center replication. This enables the full data set to be continuously kept in synch on two clusters located in multiple, geographically separated datacenters. If one data center goes down, the other data center can immediately take over. As with rolling upgrades, data center replication is available in a commercial version of Apache Ignite.

With the pressure to achieve application speed and scale in a highly manageable solution at a reasonable cost, many organizations will recognize that their path to success starts with in-memory computing. In addition to choosing the right architecture to support the demands of the business applications, it is essential that the system be easy and cost-effective to deploy and manage. Most organizations will find that open source solutions offer the ultimate combination of capabilities, flexibility and cost effectiveness to enable them to more easily progress toward their digital transformation and omnichannel customer engagement goals.

This article is published as part of the IDG Contributor Network. Want to Join?