2025: A Year Of Progress
By Tyler Logtenberg
2025 in Review!
2025 has been a huge year for Datastreamer’s customers. Here are all the major releases that are helping our customers’ pipelines bring new capabilities to market, faster.
Many new data sources fed into customer pipelines.
For customers, the ability to rapidly integrate new data sources has enabled them to free up major engineering time.
Some of the highlights were:
- Reddit (multiple providers)
- Apify marketplace (with over 5k+ options)
- YouTube (multiple providers)
- Snapchat
- Google Search
- Google Maps
- TikTok
- Twitter+ Firehoses
- Auto Sources (more on that next!)
Auto Sources freed customers from selection stress
Many customers gave us the feedback and insights that they just needed stable data. The choosing of providers, merging them, comparing them, was too much.
Auto Sources is the answer to that.
Use Auto Sources within your pipelines, and it handles the right provider selection for that query, and conversion to a common format and schema. Allowing you to just ingest, without all the stress.
Datastreamer rolled out an in-pipeline storage option.
Pipelines are great, but pulling/pushing data from pipelines can be difficult.
Searchable Storage is Datastreamer’s own offering to use high-performance searchable storage within your pipelines.
Some customers use for primary storage, others as a cache, and others as a manner to augment data.
It’s not just storage though. It added APIs to do trend analysis, counting, histogram aggregation and more!
Storage integrations with partners too!
Searchable Storage wasn’t the only solution to ease storage integrations. We expanded our ecosystem with integrations to:
- Kafka
- Cloudflare R2
- Fivetran ETL
- Databricks
And options of delivery beyond specific providers:
- Webhook
- Firehose delivery
- SFTP
Storage wasn’t just pipeline posts, files got a big addition.
Files, profiles, videos, and enrichments. So much possible within.
Getting content into your platforms and products using pipelines running on Datastreamer is easy. Getting that data to its maximum value is the role of enrichments.
Enrichments included:
- Video analysis, transcription, logo detection, etc (from partners like SocialVoice)
- Deep content analysis of abusive content or dangerous content (from partners like Tisane)
- File and profile downloading and enrichments (from partners like WebSightLine)
- and even our own enrichments from the team at Datastreamer. Including hits like the influence/reach prediction models.
Query creation got agentic too.
Queries are often tricky, understanding exactly how to get useful data with as little noise as possible.
Throughout the year, the team released:
- Query builder wizards to help with more complex sources.
- Query validation logic to help identify when a query may fail.
- and Agentic query creation! Using the power of LLMs to understand the objectives and design a query to get the data. (More on that)
"Datastreamer is the interface layer between autonomous agents and the dynamic, high-signal world of social data."
Your code and your pipelines got even closer.
Saving one of the best for last, Custom Functions brought the ability to add your own python functionality right into your pipelines.
Beyond using for integration of your own capabilities and scripts, these functions come with a boatload of templates.
These “Recipes” include functionality like:
- Bot detection
- High-level sentiment analysis
- Emoji detection
- Urgency detection
- Subjectivity scoring
- and more!
Better cost insights and reports to support all the new functionality.
With new features, came the need for new releases on cost control and cost prediction. To help with that, the Datastreamer team rolled out:
- Brand new billing dashboards with greater source-specific details.
- Add-On estimator to forecast and estimate future spend.
- Budget and usage alerting to measure and alert on cost milestones.
- In-platform support elements, like tagging of content, deduplication components, and other filtering capabilities. Removing noise inside the flow of data, and reducing downstream costs.
We aren’t done yet! With the biggest update (customized detailing cost measurements) coming right at the start of 2026!
It wasn’t just cost insights that rolled out. Detailed insights into the pipelines itself got multiple releases:
- Detailed logging was adding to every component, and detailed metrics.
- Many tools (and APIs) were rolled out to help engineers monitor, assess, and action parts of the pipelines.
- Improvements to how components recovered, handled schema changes, rate limits, updates, and more brought even greater stability to customer’s pipelines.
A great 2025, a greater 2026!
With close to 100+ feature releases during the year, our customers helped us create the best pipeline platform for working with social and web data.
2026’s releases are already in progress! The team is working to roll out updates in Auto sources, detailed billing insights, cost forecasting, and more sources. 2026’s roadmap brings even more value.
From our team at Datastreamer, we wish you a happy holidays!
