Data platform provider AirByte has released an open source Python library to help speed the process for Python programmers to connect data sources with other destinations that would use that data.
PyAirbyte provides over 250 data connectors for bridging different applications via API. It also works with open source orchestration tools such as Dagster. On the LLM side, a connector can work with LangChain for the creation of applications using large language models (LLMs).
According to the documentation, PyAirbyte “is meant to be used in situations where setting up an Airbyte server or cloud account is not possible or desirable, for example in a Jupyter notebook or when iterating on early prototypes on a developer’s workstation,”
“A majority of existing data pipelines are written in Python today,” said Michel Tricot, co-founder and CEO, Airbyte, in a statement. “We’ve seen growing demand for this capability, which provides a natural fit into the coding workflow and only requires running one command.”
PyAirByte works with Python 3.9 and later versions. An installation can be done from the Python command line:
pip install airbyte
Among the 290 connectors are those for:
- Amazon Ads
- Amazon Seller Partner
- Amazon SQS
- Auth0
- AWS CloudTrail
- Azure Blob Storage
- Azure Table Storage
- GitHub
- GitLab
Thanks to its simplicity, Python is widely used as a programming language in the data science community and is integrated across many data analysis tools, such as SQL databases, Hadoop, and Spark.
Airbyte itself specializes in synchronizing data across applications, APIs, and databases to data warehouses, lakes, and other destinations, through offerings such as Airbyte Open Source, Airbyte Self-Managed, Airbyte Cloud, and Powered by Airbyte. The open source community supporting the platform through connectors has garnered over 800 contributors.
Other data pipeline enhancements from the company include the Airbyte API and Terraform Provider,
To learn more, register for the webinar to be held on March 14.
The post PyAirbyte: Airbyte’s New Python Library for Moving Data appeared first on The New Stack.
AirByte has released an open source Python library to help speed the process for Python programmers to connect data sources with other destinations that would use that data.