SnapLogic iPaaS Adds Machine Learning; AI Seen as Key To ‘Self-Driving’ Integration for Apps, Data and Analytics

SnapLogic is bringing machine learning and AI to the complex tasks of integration and analytics. The capability, named Iris, is one of the key upgrades in SnapLogic Spring 2017 release of its cloud-based integration platform as a service (iPaaS)  IDN talks with SnapLgic CEO Gaurav Dhillon.

by Vance McCarthy

Tags: AI, cloud, integration, iPaaS, machine learning, SnapLogic,

Gaurav Dillon, SnapLogic
Gaurav Dhillon
CEO and co-founder
SnapLogic


"Machine learning. . . .will power a self-service model for integration that’s important to the success of SaaS and analytics."

Enterprise Integration Summit
Powering Digital Transformation with APIs, Cloud, IoT, Mobile & SOA
An Online Conference

SnapLogic is bringing machine learning and AI to the complex tasks of integration and analytics. The capability is one of the key upgrades in SnapLogic Spring 2017 release of its cloud-based integration platform as a service (iPaaS).

 

“Today’s explosion of apps and datasets has created a need for these assets to constantly talk and interact with each other. This is driving the need for a new approach to integration, one that is more agile, responsive and even intelligent integration,” Gaurav Dhillon, SnapLogic CEO and co-founder, told IDN.

 

SnapLogic’s new AI/machine learning technology, dubbed Iris, uses advanced algorithms to automate highly repetitive development tasks, eliminating integration backlogs that stifle most technology initiatives. Iris learns from millions of metadata elements and billions of data flows across SnapLogic’s cloud-based platform to determine what’s popular, what works, and what doesn’t work.

 

The idea is to shorten (even eliminate) the learning curve for line-of-business users to manage their own data flows, while freeing up technology teams for higher-value design and deployment needs.

Iris does its ‘learning’ as the algorithms sift through telltale data trails left behind from millions of customer integration tasks now successfully running in the SnapLogic Enterprise Integration Cloud iPaaS. Especially valuable to SnapLogic’s machine learning capabilities will be metadata elements and billions of data flows. Iris will also learn from failures, analyzing unsuccessful integration projects to diagnose what went wrong and help future users avoid those mistakes.

 

The result for the user: Iris machine learnings is the engine powering SnapLogic’s first recommendation engine, called the SnapLogic Integration Assistant. It can deliver expert step-by-step guidance with up to 90 percent accuracy according to tests, Dhillon added.  

 

Dhillon has broad ambitions for Iris, far beyond recommendations. Iris is the first step of our technology vision to deliver fully ‘self-service’ or ‘self-driving’ integration,” Dhillon told IDN. “Digital transformation shouldn’t depend on manual labor. To net it out, Iris is an attempt to take our power of and ease of use [of SnapLogic] to the next logical conclusion to promote more self-service integration,” he added

This vision reflects a new level of ‘self-service’ that aims to go beyond the use of models or guided wizards, Dhillon added.   Powered by AI and complex algorithms, SnapLogic’s approach to ‘self-service’ integration will be able to rapidly understand, learn, suggest, and even execute integration tasks, he explained.

 

Inside SnapLogic’s Vision for AI-Powered Self-Service Integration

SnapLogic’s ability to bring Iris’ machine learning to complex integration tasks combines several key elements – specially-designed algorithms, SnapLogic’s cloud-based integration platform architecture, and (perhaps most importantly) access to multi-millions of bits of integration-focused metadata and its drag-and-drop approach to integration called ‘pipelines’.

 

By putting all these ingredients together Dhillon believes his team has a recipe for ‘self-driving’ integration – one that can abstract a lot of the gnarly complexity once faces with such tasks.  

 

“Today, the demand for apps and data to talk to each other is so huge, we need to put this power in the hands of many more users than simply trained integration staff. We need to promote the citizen integrator and that is why machine learning is crucial. It will power a self-service model for integration that’s important to the success of SaaS and analytics,” he said.

 

To be specific, a SnapLogic ‘pipeline’ is a visually-built flow that can connect APIs, apps, data and business logic together without coding. Pipelines can be assembled (snapped together) using a library of more than 400 pre-built intelligent connectors (SnapLogic appropriately calls these Snaps). These Snaps receive input from a website, application or data source – and return data to a requested endpoint. Snaps are designed to be smarter than traditional end-point connectors, as they can be inter-locked together, one at a time, to create customizable and configurable end-to-end pipelines.

 

SnapLogic Pipeline

 

The ‘pipeline’ approach also provides another key benefit, It allows for step-to-step processes (or special handling of a payload along the flow), such as callbacks, transformations and so forth that many end-to-end integrations require to be successful.  These pipelines are executed on a Snaplex, an application that runs on a multitude of platforms: on a customer’s infrastructure, on the SnapLogic cloud and on Hadoop.

 

Thanks to this granular attention to what happens with these pipelines at runtime (to APIs, data, calls for data, or business logic along a flow), the metadata within the SnapLogic iPaaS cloud provides the specially built algorithms multiple benefits, Dhillon noted.

 

“We spent a lot of time building this algorithm so it can drive all types of self-service and improve speed and quality of integrations across data, applications, and business processes,” Gaurav said. “Consumer technologies like [Amazon] Alexa and [Apple] Siri have done wonders to bring self-service to consumers. We’re looking at Iris to do the same for businesses coping with integration.”

 

SnapLogic’s CTO James Markarian also echoed the time has come to apply smarter tools and simpler non-tech friendly interfaces to integration.

 

“For the longest time, the common wisdom was that the only way to solve an integration problem was to throw armies of developers at it. We are embarking on a new direction to bring machine learning to data and application integration. Customers now have a choice that uses intelligent software, not manual labor, to make integration fast and easy with no coding,” he said in a statement.

 

One early SnapLogic customer agreed. “SnapLogic’s vision for AI-assisted data integration will greatly assist the citizen developer and data integration developer alike. AI will have a positive impact to an already exceptionally powerful platform, and this is only the beginning,” Jim Teal, senior cloud solution architect at iRobot, said in a statement.

 

The capability, called SnapLogic Integration Assistant is available now as part of SnapLogic’s Spring 2017 Release, and is available to all customers at no charge.

 

Other Updates to SnapLogic Spring 2017
SnapLogic Spring 2017 also sports other updates and new features to speed and simplify iCRM, HR and cloud data warehousing projects:

Microsoft Dynamics 365 CRM integration: To help users create, read, and update records in-cloud and on-premises versions of Microsoft Dynamics CRM. Users can also delete a record based on account ID and also search with various filter options.

 

Workday integration: An updated Workday Read Snap to improve performance. The Workday Write Snap now supports bulk operations for many objects.

 

Confluent integration: An updated Confluent Acknowledge Snap to support the Kafka Connect API. This allows seamless integration with the Confluent Connector ecosystem. The new implementation reduces potential data losses, and eliminates duplicates by externally acknowledging each message.

 

Apache Hive integration: The Hive Snap is now certified on Cloudera version 5.8 and also works on SnapLogic’s Groundplex with Kerberos-enabled clusters.

Enhanced Snaps for Amazon Redshift, Anaplan, and Tableau.

 

Field encryption and decryption: The Transform Snap Pack adds new Snaps to encrypt and decrypt field values for sensitive data, or entire documents, providing a greater level of data security.

 

SnapLogic platform enhancements: Updates to save time and enhance productivity, including Snaplex restart; enhanced Asset Search; network statistics display to aid troubleshooting and performance optimization; and parameterizable accounts that make it easier to automate or dynamically assign environments in a development lifecycle.

Study Suggests The Need To Better, More Quickly Connect ‘Disconnected Data’

SnapLogic’s latest release comes as the company released a survey report that estimates the lack of ‘connected data’ is costing businesses nearly $140 billion a year.

 

The study, conducted by research firm Vanson Bourne, surveyed some 500 business and IT leaders from the U.S. and U.K. and found that nearly half of respondents (47%) believe disconnected data is negatively impacting their organization’s ability to innovate, develop new products and services and get them to market quickly.

 

Such ‘disconnected data’ was also costing them with customers, as some 4 46% admitted it was hampering their ability to engage, support, and meet customer needs. 

 

More than three-quarters of respondents (76%) admitted having data trapped in this legacy systems that cannot be accessed or linked to cloud services. More than four in ten (41%) said critical company data is trapped.

 

“We continue to see the same issues around legacy technology, disconnected systems, and trapped data,” Dhillon said of the survey. “This is hampering collaboration, decision-making and timely business outcomes, and with the cost reaching into the billions, we need to act.”

Here are some other hazards and negative effects of ‘disconnected’ or ‘trapped’ legacy data:

Leaves revenue (and opportunity) on the table. A shocking 72% felt that siloed data was causing their business to miss out on opportunities, and a third stated that it was holding back innovation in product and services.

 

It hinders teamwork. Some 44% of respondents admitted their company suffers a lack collaboration between business departments. In fact, 21% of respondents admitted they don’t know what data other departments hold.

 

Workers spend more time in finding data than on analysing and making decisions. Workers need to spend more time searching for, acquiring, entering, or moving data (8 hours per week) than they do making decisions on that data (7 hours per week), according to the study.

 

It reinforces silos (and silo-thinking). More than half (57%) admitted that their organization is struggling with data silos. Nearly the same percentage said data silos are a barrier to meeting their organization’s business objectives. More than a fifth were unaware of what data other departments actually held and one in six didn’t even know how many data sources actually existed.

The survey results also seemed to echo Dhillon’s call for more intelligent integration approaches.

 

Even while IT tried to keep pace for business needs for app and data integration, 48% of respondents want to compromise coding shortcuts within their organization. What’s more, even though many ad hoc projects are successful, there’s a high price as nearly a third (31%) admitted they have experienced errors, data loss, or privacy violations in their organization as a result of manual moves or ad hoc coding to integrate data.

 

 “Organizations need to be rethinking how they approach the access to and use of data within, and across, teams. Self-service for thousands and thousands of non-technical workers will prove a key to delivering it,” Dhillon noted.

 

 “If a business cannot be innovative, bring products to market in a timely manner, or deliver the level of service their customers expect, then they simply aren’t going to be around for much longer,” he added.

 

Readers can obtain the full survey / report The High Cost of Disconnected Data here.




back