Sources & Sinks¶

QuickETL reads data from sources and writes data to sinks. This section covers all supported source and sink types.

Overview¶

File Sources

Read CSV, Parquet, and JSON files.

File Sources
Database Sources

Read from PostgreSQL, MySQL, and other databases.

Database Sources
Cloud Storage

Read from S3, GCS, and Azure.

Cloud Storage
File Sinks

Write Parquet and CSV files.

File Sinks
Database Sinks

Write to databases.

Database Sinks

Type	Description	Formats
`file`	Local or cloud files	CSV, Parquet, JSON
`database`	Relational databases	SQL query or table

Type	Description	Formats
`file`	Local or cloud files	Parquet, CSV
`database`	Relational databases	Table writes

source:
  type: file
  path: data/sales.parquet
  format: parquet

source:
  type: database
  connection: postgresql://localhost/db
  table: sales

sink:
  type: file
  path: output/results.parquet
  format: parquet

sink:
  type: database
  connection: ${DATABASE_URL}
  table: results
  mode: truncate

All file sources and sinks support cloud URIs:

Provider	URI Format
AWS S3	`s3://bucket/path/file.parquet`
Google Cloud	`gs://bucket/path/file.parquet`
Azure ADLS	`abfs://container@account.dfs.core.windows.net/path/file.parquet`

See Cloud Storage for authentication setup.