CLOUD
Architecture Summit

Enterprise-Grade Integration Across Cloud and On-Premise

REGISTER

Online Conference

The latest update from SnapLogic is using AI/ML to automate a rich array of intelligent integration features – making them easier and more efficient for less technical users, as well as long-time integration professionals.  

The latest release of SnapLogic Intelligent Integration Platform leverages a number of AI/ML-powered updates aimed at boosting productivity and performance – as well as yield greater insights, Diby Malakar, SnapLogic’s vice president of product management.

“In our view, enterprise automation begins with AI and integration,” Malakar told IDN.

SnapLogic’s growing focus on bringing AI to complex integration tasks is directly linked to the growing demand from its customers. 

“We’ve heard from customers. They say, ‘Our world is shifting to the citizen integrator,’” he told IDN. “Years ago, when SnapLogic started, we helped customers support their hard-core [integration] developers. But now, companies are looking to shift the bulk of that work to less technical users, such as ad hoc or citizen integrators. AI and automation are powerful ways for us to meet this exploding need we’re seeing,” Malakar said.

SnapLogic’s ability to deliver on these and other types of intelligent automation arises from its approach to combine AI models, on-going machine learning and access to tons of operational data. SnapLogic is combining all these ingredients to provide non-technical users a drag-and-drop way to build accurate and efficient end-to-end integrations that focus on delivering business benefits.

“Less technical users don’t know the exact semantics of a question [and how that affects data integration]. So we’re using AI/ML to assist them.  It all starts with our engineers asking, ‘With minimal input for users, how can we speed up, and automate, the output.  With AI and machine learning, we’re automating highly-repetitive tasks,” Diby said.

To understand how SnapLogic delivers this AI assistance, Malakar spelled out for IDN the connections SnapLogic sees between automation and AI when it comes to integration: 

Before end-to-end processes can be automated, a complex web of apps, APIs, data sources, and other systems must be connected.  

 

To effectively navigate and deliver these integrations, SnapLogic’s view is to make it possible for AI, machine learning and intelligent workflows to all work together.

 

With such smart technologies in place, it’s possible to automate the builds -- and even the on-going management functions.

Since May 2017, “we’ve used advanced algorithms to automate repetitive tasks for integration specialists,” Malakar told IDN.

“We started on this journey using decision tree algorithm and we now have neural networks.  So when a user is dragging something to the canvas, the AI/ML can make at least 80% accurate suggestions,” Malakar said.  To deliver faster – even automated – responses, AI/ML is aided by SnapLogic’s inclusion of data repositories, where users can access and reuse integration patterns and expression logic – both of which speed pipeline assembly and data mappings.

While SnapLogic continues to push the envelope for helping skilled integrators get their jobs done faster, the new focus is to bring AI and automation to rank-and-file workers.  “We are taking what we’ve learned [about automation and AI] to empower the less skilled citizen integrator and ad hoc integrator,” he told IDN.

SnapLogic’s AI for Integration in Action

Malakar shared an example of how AI and automation are opening up integration tasks for less technical workers.  In the current May 2019 version, “a user can go to our canvas and just start with a target, and we will tell you the next step,” he said

“Thanks to AI, we can be a pipeline suggestor, and we can do this based on minimal information,” Malakar told IDN.  SnapLogic has a visual designer interface, that lets users assemble integrations without coding – simply drag-and-drop components together. “If you go to our [designer] canvas, you can drag a Snap [SnapLogic’s term for components] and we can tell you what the next Snap should be,” he said.

Malakar said the idea is to synchronize all these technologies in a simple-to-navigate visual designer. “This is very easy to use. It’s like using a macro in Excel. The whole focus is we want to leverage AI to traverse the history of all expressions for any [integration]  target,” he added.  

SnapLogic Also Adds Updates To Accelerate Integration Lifecycle

AI/ML has the added advantage of also being able to help seasoned app and data integrators. “We’re also able to automate tasks for highly-skilled integrators to drive greater productivity and accelerate innovation,” he added.

Notably, with SnapLogic’s Data Prep/Preview feature, users can visualize output at every step of the pipeline in preview mode (displaying in bar charts, line charts, pie charts or scatter charts).

Malakar described this capability is a recent blog post. 

Data engineers and data scientists spend 70% of their time collecting and cleaning data. With the May 2019 release of the SnapLogic Intelligent Integration Platform, we are empowering these users to spend less time normalizing data sets. The ‘Match Snap’ performs record linkages across multiple data sources that may represent the same entity but have different values – .g., comparing two data sets where Spain is represented in two different ways: España vs. Espana.

SnapLogic’s other innovations permeate the integration lifecycle – from design, testing and operations. Among some notable examples:

• Temp Disable Snap lets users disable a snap during testing and troubleshooting without removing or modifying Snaps from the pipeline
• Improved Performance via newly-added local caching for triggered tasks running during operations.
• Operational Integrity is supported by smart alerts that deliver telemetry when CPU and memory usage on nodes exceeds the configured threshold so proactive action can be taken to maintain enterprise SLAs

For even more complex data science tasks, SnapLogic’s AI/ML updates let users capture and prepare data to build machine learning models for many real-life business use cases.

Malakar described it this way.

Data engineers and citizen data scientists still need a fair bit of knowledge and background in data science to produce high-performing machine learning models. With 4.17 release, non-experts can perform exploratory data analysis with clustering, the most common unsupervised learning technique. A ‘Clustering Snap’ has been added as part of the ML Core Snap Pack and is used to find hidden patterns or groupings in data.  It supports machine learning scenarios such as gene sequence analysis, market segmentation, and object recognition.

Further, SnapLogic provides deep fuzzy matching s via its new ‘Match Snap,’ which can perform record linkage, which will identify documents from different data sources that may represent the same entity but have a different column name.

Another new SnapLogic ‘snap’ is also bringing automation to data science with what it calls “automated data feature engineering.’  The ‘Feature Synthesis Snap’ provides the ability to transform raw data into features that better represent the underlying problem to the ML predictive models, resulting in improved model accuracy on as yet unseen data.

SnapLogic’s latest update follows an AI study commissioned by the company.  The research found that while a vast majority of organizations (93%) are committed to adopting AI, but more than half (51%) admit don’t have the required in-house skilled talent to execute their strategy.

Behind lack of access to skilled talent, IT decision makers mentioned several other barriers to AI adoption. Among them: Lack of budget (32%), Lack of access to the right technologies and tools (28%), and lack of access to useful data (26%).

“Taking an enterprise approach to automation [ensures] strategies and practices can be standardized and repeated,” Malakar said. Further, by including automation capabilities natively in the SnapLogic platform, AI/ML can better solve complex IT and business challenges, he added.