Case Study

An analytics firm moved about 17,000 listening tool queries onto raw provider data

Queries of up to 18,000 characters, running in full and delivered to its own cloud storage.

About 17,000
jobs
Queries up to 18,000
characters
Fix shipped within
1 week

At a glance

Customer
Analytics firm
Sources
Raw provider data, replacing a listening tool
Volume
About 17,000 jobs
Delivery
Its own cloud storage
In production
On its own stack
About the customer

An analytics firm that delivers insight work to its own clients.

Industry: Analytics and Insights

The Story

Challenge, solution, results.

The Challenge

Paying tool prices for data

The firm was using a social listening tool as its data provider, and paying listening tool contract prices to do it. It also wanted to control its own tools and add to them, which it couldn't do inside a listening tool. So it moved to raw data from a provider.

The hard part was the queries. There were about 17,000 of them, written as long Boolean expressions in the listening tool's own syntax. Some ran past 18,000 characters, far longer than the new provider would accept in a standard request.

The Solution

Queries that run in full

Datastreamer worked through the syntax differences between the two. When the longest queries started failing, the team traced the cause to a request-size limit on the provider's side, at about 8 KB.

Within 1 week, Datastreamer changed how long queries are sent so they run in full, and rolled the fix out to every customer using that provider. Results land in the firm's own cloud storage, ready for its analysts.

The Results

Control over its own stack

About 17,000 jobs now run on the firm's own stack, at provider data prices, with the freedom to build what its clients need.

How It Works

How the pipeline works

1
Queries
About 17,000, carried over from the listening tool.
2
Translation
Syntax differences worked through for the new provider.
3
Collection
Long queries sent in full, past the provider's standard size limit.
4
Delivery
Into the firm's own cloud storage.
Get Started

Using a listening tool as a data provider?

See what your queries look like running on raw provider data.

Used by market-leading intelligence platforms. Supported by a dedicated success team.