Skip to main content

Changelog

Subscribe to Updates
August2023

The team has been heads down this month bringing improvements and squashing bugs to all things Meltano. Let's take a look at some of the bigger items we shipped recently!

☁️ Meltano Cloud​

🚀 Automated deployments​

Now when you set your deployment to track a git branch, any new commits that appear on that branch will automatically be deployed in Meltano Cloud. Previously users had to complete an additional manual step to get any new committed changes deployed.

Static deployments are still possible by referencing a particular git hash.

🚨 Custom Notifications​

Cloud customers now have greater flexibility over their notifications. With the updated config CLI command, you can specify exactly how you want to receive notifications. Email and webhooks are the initially supported destinations and the payload of the notification can be filtered to exactly how you want it - failures only, successes only, or anything!

meltano cloud config notification set

➕ Add new projects from the CLI​

New projects can now easily be added from the CLI to your Meltano Cloud account. This simplifies onboarding for new users as well as our power users that have 10s or 100s of projects they need to onboard.

Meltano Core​

v2.20.0 of Meltano Core was released. This was mostly a bug, performance, and doc update release.

MeltanoHub​

We added a "Most Popular" listing to the Extractors page. Check it out!

The Meltano Team and Community have also been busy adding new connectors and utilities to the Hub. Many of the new connectors are related to our recent blog post about how LLMs are mostly data pipelines.

Extractors​

  • Added the MeltanoLabs variant of tap-jira - Link
  • Added the MeltanoLabs variant of tap-mysql. This tap is still under active development. - Link
  • Added the MeltanoLabs variant of tap-beautifulsoup - Link
  • Added the sehnem variant of tap-shopify which supports accessing data via graphql - Link

Mappers​

  • Added the map-gpt-embeddings mapper. This can be used to connect to OpenAI to generate embeddings for any data from an extractor - Link

Loaders​

  • Added the MeltanoLabs variant of target-pinecone which is a Vector Database for storing embeddings. - Link

Utilities​

  • Added Tableau utility which can be used to trigger a refresh of Tableu data source - Link
  • Added the dbt Artifacts utility which can be used to process dbt-generated artifacts for other use cases such as model lineage - Link

SDK​

We shipped 2 releases of the SDK:

Highlights include:

  • Improved connection handling in SQL targets by sharing a connector instance among stream sinks
  • Expose builtin add_record_metadata and batch_config target settings
July2023

Welcome to the new Meltano Changelog!

This is going to be our new place to share and keep a record of all of the product updates happening around the Meltano ecosystem. Previously we'd shared these in our newsletter and in Slack. We'll likely keep putting updates there, but this will be the primary destination for updates going forward.

Let's dive in to what's improved over the past month or so!

☁️ Meltano Cloud entered Public Beta​

Right at the end of May this year we launched Meltano Cloud into Public Beta. This was a key milestone for us at it represented the threshold where early Alpha users were seeing enough value to pay for Cloud use. And it means we can start working through our large waitlist to onboard more users.

Request Access to the Meltano Cloud Beta

Since then we've:

  • Added the ability to cancel a running job from the CLI
  • Added syntax highlighting to the logs in the Cloud UI
  • Added support for manual, once, and none aliases for schedules. Useful for one-off runs!
  • Improved the error code response when creating or updating a deployment
  • Squashed quite a few bugs. Thanks to our early users for all of their feedback!

Core​

We released v2.20.0 which was primarily a bugfix and documentation release. We're also incrementally starting to improve the settings definitions for plugins. We now recommend using hidden: true instead of kind: hidden on plugins. This is part of an effort to make kind indicate just the data type and not any sort of UX choices that may be desired.

MeltanoHub​

The Meltano Community has been busy adding new connectors to MeltanoHub. In June and July so far we've had the following additions and updates:

Extractors​

  • Updated the Google Analytics connectors and docs due to the GA4 change - Link
  • Added the MeltanoLabs variant of tap-hubspot - Link
  • Added the datarts-tech variant of tap-cassanda - Link
  • Added the acarter24 variant of tap-mailchimp - Link
  • Updated the default variant fo SaaSOptics to the datarts-tech variant - Link
  • Updated the default variant of tap-postgres and tap-snowflake to the MeltanoLabs variant - Link
  • Added tap-theme-parks - Link
  • Added the sehnem variant of tap-netsuite - Link
  • Updated the default variant of tap-recruitee to the rawwar variant - Link

Loaders​

  • Added a loader for Clickhouse - Link
  • Added target-sqs and target-parquet variants from Hotglue - Link
  • Updated the default variant of target-postgres and target-snowflake to the MeltanoLabs variant - Link

We've also got an introductory video on how to go from Zero to Snowflake with the help of target-snowflake.

Connectors​

We're continuing to invest in building first-party connectors based on the Meltano SDK.

We also have new extractors for Jira and Salesforce landing soon.

SDK​

We shipped 3 SDK releases

Some highlights from these releases are:

  • Supporting optional headers in OAuth requests
  • Publishing the supported python version in --about
  • Supporting union schemas which is useful for connectors that accept mutually exclusive settings (e.g. OAuth vs API key auth)
  • Validation of a parsed/transformed record against the SCHEMA message
  • Add Apache 2.0 as an optional license when developing a connector

And of course lots of fixes, under the hood changes, and improved documentation.