class sources.Parquet

Source reading data from Parquet files.


Add data to the source.

sources.Parquet.create(file=None, *, time_column, key_column, schema=None, subsort_column=None, grouping_name=None, time_unit=None)

Create a Parquet source.

  • file (Optional[str], default: None)

    The url or path of the Parquet file to add. Paths should be relative to the

    current working directory or absolute. URLs may describe local file paths or

    object-store locations.

  • time_column (str)

    The name of the column containing the time.

  • key_column (str)

    The name of the column containing the key.

  • schema (Optional[Schema], default: None)

    The schema to use. If not provided, it will be inferred from the input.

  • subsort_column (Optional[str], default: None)

    The name of the column containing the subsort.

    If not provided, the subsort will be assigned by the system.

  • grouping_name (Optional[str], default: None)

    The name of the group associated with each key.

    This is used to ensure implicit joins are only performed between data grouped

    by the same entity.

  • time_unit (Optional[TimeUnit], default: None)

    The unit of the time column. One of ns, us, ms, or s.

    If not specified (and not specified in the data), nanosecond will be assumed.