Category: XOps

“Impetus has the opportunity to make StreamAnalytix (now known as Gathr) the de facto tooling standard for Spark and future streaming engines”- States the report.

Forrester has positioned Gathr as a strong performer among 13 most significant streaming analytics providers in The Forrester Wave™: Streaming Analytics, Q3 2017; one of the few, and perhaps the most comprehensively evaluated market insight reports focussed on streaming analytics today.

Gathr is a platform that makes creating real-time stream processing and machine learning applications on Apache Spark extremely easy. It is aimed at enterprises looking for a single visual platform that leverages popular open source, big data platforms for streaming ETL and advanced analytics, which is easy to use for both business and technical users.

Forrester’s evaluation of Gathr resonates with our key focus areas; the report’s vendor profile section highlights the following aspects of Gathr:

• Gathr offers use of both Apache Storm and Apache Spark and is architecturally positioned to support future use of other open-source streaming engines such as Apache Flink
• Gathr embeds EsperTech to provide advanced streaming analytics capabilities such as complex event processing
• Gathr tooling also unifies streaming and batch by supporting arbitrary Spark jobs such as machine learning

For more information on Gathr coverage in the Forrester Wave, read the full press release.

Every time an independent research firm identifies Gathr (earlier known as StreamAnalytix) as a leading platform in the increasingly competitive space of streaming analytics, it is an exciting moment for us. Inclusion in a recently published report by Aragon Research, a technology focused research and advisory firm, as one of the ‘Hot Vendors in Streaming analytics 2017’ is one such proud moment.

The fact that we are one of only four players covered in the report, makes it even more exciting. With this report, Aragon Research provides insight on new and noteworthy data management and streaming analytics providers. Each year, Aragon Research recognizes Hot Vendors across multiple markets that are doing something new or differently. They may have new technology that expands capabilities, a new strategy that opens up markets, or just a new way of doing business that makes them worth assessing.

The report validates our focus on the use of open source big data technologies such as Spark Streaming and Apache Storm for real- time data insight, and recommends evaluating Gathr to enterprises that need a single visual platform that leverages popular open source, big data platforms for streaming ETL and advanced analytics, and that is easy-to-use for business and technical users.

Streaming data represents new avenues for creating value, and enterprises are beginning to pay attention to this new source of competitive advantage.  The value is driven by new business insights from sensor data, web clickstreams, geolocation data, weather reports, market data, social media and other event streams. Often it is the combination of multiple streaming and static sources of data that reveals new powerful insights. However, the successful use of stream processing engines such as Apache Spark ™ to build such advanced analytical applications can be a challenge as it typically requires deep technical and data science skills.

Solution? Gathr ! A platform that makes creating real-time stream processing and machine learning applications on Apache Spark extremely easy. It now offers a Visual Spark Studio for development and life-cycle management of Apache Spark applications in both streaming and batch mode. Earlier this year, within a short span of six weeks, engineers who were even new to Apache Spark were able to build complex machine learning applications for anomaly detection leveraging the Gathr platform – as part of a contest that we had organised.

Back to the topic, and in closing…we feel very thankful and immensely encouraged by the Aragon Research recognition as a ‘Hot Vendor’ and the validation of the benefit we strive to bring to enterprises i.e. “powerful tooling and ease of use – over open source big data and fast data technologies”. For more information, read the full press release.

Last week the Gathr team hosted a webinar on Structured Streaming, “The Structured Streaming Upgrade to Apache Spark and How Enterprises Can Benefit” and received overwhelming participation from the industry, including many of you reading this. Amit Assudani (Sr. Technical Architect – Spark, Gathr) and I took a deep dive into Structured Streaming and shared our views on how it enables the real-time enterprise and is simplifying building stream processing applications on Spark. Here is a summary of our current view on Structured Streaming-

Streaming Big Data ETL with Impetus Gathr and Syncsort

Today we are announcing a partnership between Syncsort and Impetus Technologies, and our entry into an integration of batch processing and real-time stream processing that we call “Streaming ETL”. The mix of batch and real-time processing has also been referred to as the Lambda Architecture. Streaming ETL allows a mixing of the best batch and streaming technologies under the umbrella of tools which abstract the complexity of the underlying platforms.

The huge increase in types and sources of data has placed pressure on companies to blend and summarize that data quickly to create actionable information. A combination of real time and batch processing is needed to meet the new demands.

There’s a grab bag of technologies that excel in specific aspects: Hadoop Mapreduce, Storm and Spark for massively parallel processing; Kafka and Spark Streaming along with traditional messaging and queuing software for real time data movement; Mesos and YARN for cluster management. These components can be mixed and matched, but there are many APIs to learn and different skill sets needed to leverage them well.

Gathr definition with Syncsort DMX bolt

DMX task that performs lookup for IP