Making a custom plugin discoverable
If you've added a custom plugin (or variant) to your project that could be discoverable and supported out of the box for new users, please contribute its description to Meltano Hub to save the next user the hassle of setting up the custom plugin. GitHub makes it easy to contribute changes without requiring you to leave your browser.
The format and further requirements are laid out in more detail in the Meltano Hub plugin definition syntax document.
Adopting a plugin
When the maintainer of the default variant of a discoverable plugin becomes unresponsive to issues and contributions filed by the community, that plugin is considered up for adoption, which means that we are looking for a different variant of the plugin with a more engaged maintainer to become the new default.
This new variant can either be a fork of the original default variant, or an alternative implementation for the same source or destination, as long as it is actively maintained.
As a plugin's primary maintainer, you do not have to spend a lot of time improving the plugin yourself. In fact, attracting more users and getting the community involved is likely to recude your personal maintenance burden, since you'll receive contributions with bug fixes and new features that you will only be expected to review, not build yourself.
Taps & Targets Development
For existing taps/targets
We should be good citizen about these, and use the default workflow to contribute. Most of these are on GitHub so:
- Fork (using Meltano organization)
- Add a webhook to trigger the
- Modify and submit PRs
- If there is resistance, fork as our tap
How to test a tap?
We qualify taps with the capabilities it supports:
- properties: the tap uses the old
--propertiesformat for the catalog
- catalog: the tap uses the new
--catalogformat for the catalog
- discover: the tap supports catalog extraction
- state: the tap supports incremental extraction
You should look at the tap's documentation to see which one is supported.
Try to run the tap with the
--discover switch, which should output a catalog on STDOUT.
- Try to run the tap connect and extract data first, watching for
- Do two EL runs with
target-postgres, then validate that:
- All the tables in the schema created have a PRIMARY KEY constraint. (this is important for incremental updates)
- There is no duplicates after multiple extractions
Tables are lacking primary keys
This might be a configuration issue with the catalog file that is sent to the tap. Take a look at the tap's documentation and look for custom metadata on the catalog.