Package org.apache.spark.sql

Class TableValuedFunction

Object
org.apache.spark.sql.TableValuedFunction
public abstract class TableValuedFunction extends Object
Interface for invoking table-valued functions in Spark SQL.

Since:
4.0.0

  • Constructor Summary

    Constructors
    Constructor
    Description
    TableValuedFunction()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    abstract Dataset<Row>
    collations()
    Gets all of the Spark SQL string collations.
    abstract Dataset<Row>
    explode(Column collection)
    Creates a DataFrame containing a new row for each element in the given array or map column.
    abstract Dataset<Row>
    explode_outer(Column collection)
    Creates a DataFrame containing a new row for each element in the given array or map column.
    abstract Dataset<Row>
    inline(Column input)
    Creates a DataFrame containing a new row for each element in the given array of structs.
    abstract Dataset<Row>
    inline_outer(Column input)
    Creates a DataFrame containing a new row for each element in the given array of structs.
    Dataset<Row>
    json_tuple(Column input, Column... fields)
    Creates a DataFrame containing a new row for a json column according to the given field names.
    abstract Dataset<Row>
    json_tuple(Column input, scala.collection.immutable.Seq<Column> fields)
    Creates a DataFrame containing a new row for a json column according to the given field names.
    abstract Dataset<Row>
    posexplode(Column collection)
    Creates a DataFrame containing a new row for each element with position in the given array or map column.
    abstract Dataset<Row>
    posexplode_outer(Column collection)
    Creates a DataFrame containing a new row for each element with position in the given array or map column.
    abstract Dataset<Long>
    range(long end)
    Creates a Dataset with a single LongType column named id, containing elements in a range from 0 to end (exclusive) with step value 1.
    abstract Dataset<Long>
    range(long start, long end)
    Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with step value 1.
    abstract Dataset<Long>
    range(long start, long end, long step)
    Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with a step value.
    abstract Dataset<Long>
    range(long start, long end, long step, int numPartitions)
    Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with a step value, with partition number specified.
    abstract Dataset<Row>
    sql_keywords()
    Gets Spark SQL keywords.
    Dataset<Row>
    stack(Column n, Column... fields)
    Separates col1, ..., colk into n rows.
    abstract Dataset<Row>
    stack(Column n, scala.collection.immutable.Seq<Column> fields)
    Separates col1, ..., colk into n rows.
    abstract Dataset<Row>
    variant_explode(Column input)
    Separates a variant object/array into multiple rows containing its fields/elements.
    abstract Dataset<Row>
    variant_explode_outer(Column input)
    Separates a variant object/array into multiple rows containing its fields/elements.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • TableValuedFunction

      public TableValuedFunction()

  • Method Details

    • collations

      public abstract Dataset<Row> collations()
      Gets all of the Spark SQL string collations.

      Returns:
      (undocumented)
      Since:
      4.0.0

    • explode

      public abstract Dataset<Row> explode(Column collection)
      Creates a DataFrame containing a new row for each element in the given array or map column. Uses the default column name col for elements in the array and key and value for elements in the map unless specified otherwise.

      Parameters:
      collection - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • explode_outer

      public abstract Dataset<Row> explode_outer(Column collection)
      Creates a DataFrame containing a new row for each element in the given array or map column. Uses the default column name col for elements in the array and key and value for elements in the map unless specified otherwise. Unlike explode, if the array/map is null or empty then null is produced.

      Parameters:
      collection - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • inline

      public abstract Dataset<Row> inline(Column input)
      Creates a DataFrame containing a new row for each element in the given array of structs.

      Parameters:
      input - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • inline_outer

      public abstract Dataset<Row> inline_outer(Column input)
      Creates a DataFrame containing a new row for each element in the given array of structs. Unlike inline, if the array is null or empty then null is produced for each nested column.

      Parameters:
      input - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • json_tuple

      public Dataset<Row> json_tuple(Column input, Column... fields)
      Creates a DataFrame containing a new row for a json column according to the given field names.

      Parameters:
      input - (undocumented)
      fields - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • json_tuple

      public abstract Dataset<Row> json_tuple(Column input, scala.collection.immutable.Seq<Column> fields)
      Creates a DataFrame containing a new row for a json column according to the given field names.

      Parameters:
      input - (undocumented)
      fields - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • posexplode

      public abstract Dataset<Row> posexplode(Column collection)
      Creates a DataFrame containing a new row for each element with position in the given array or map column. Uses the default column name pos for position, and col for elements in the array and key and value for elements in the map unless specified otherwise.

      Parameters:
      collection - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • posexplode_outer

      public abstract Dataset<Row> posexplode_outer(Column collection)
      Creates a DataFrame containing a new row for each element with position in the given array or map column. Uses the default column name pos for position, and col for elements in the array and key and value for elements in the map unless specified otherwise. Unlike posexplode, if the array/map is null or empty then the row (null, null) is produced.

      Parameters:
      collection - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • range

      public abstract Dataset<Long> range(long end)
      Creates a Dataset with a single LongType column named id, containing elements in a range from 0 to end (exclusive) with step value 1.

      Parameters:
      end - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • range

      public abstract Dataset<Long> range(long start, long end)
      Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with step value 1.

      Parameters:
      start - (undocumented)
      end - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • range

      public abstract Dataset<Long> range(long start, long end, long step)
      Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with a step value.

      Parameters:
      start - (undocumented)
      end - (undocumented)
      step - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • range

      public abstract Dataset<Long> range(long start, long end, long step, int numPartitions)
      Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with a step value, with partition number specified.

      Parameters:
      start - (undocumented)
      end - (undocumented)
      step - (undocumented)
      numPartitions - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • sql_keywords

      public abstract Dataset<Row> sql_keywords()
      Gets Spark SQL keywords.

      Returns:
      (undocumented)
      Since:
      4.0.0

    • stack

      public Dataset<Row> stack(Column n, Column... fields)
      Separates col1, ..., colk into n rows. Uses column names col0, col1, etc. by default unless specified otherwise.

      Parameters:
      n - (undocumented)
      fields - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • stack

      public abstract Dataset<Row> stack(Column n, scala.collection.immutable.Seq<Column> fields)
      Separates col1, ..., colk into n rows. Uses column names col0, col1, etc. by default unless specified otherwise.

      Parameters:
      n - (undocumented)
      fields - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • variant_explode

      public abstract Dataset<Row> variant_explode(Column input)
      Separates a variant object/array into multiple rows containing its fields/elements. Its result schema is struct&lt;pos int, key string, value variant&gt;. pos is the position of the field/element in its parent object/array, and value is the field/element value. key is the field name when exploding a variant object, or is NULL when exploding a variant array. It ignores any input that is not a variant array/object, including SQL NULL, variant null, and any other variant values.

      Parameters:
      input - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0

    • variant_explode_outer

      public abstract Dataset<Row> variant_explode_outer(Column input)
      Separates a variant object/array into multiple rows containing its fields/elements. Its result schema is struct&lt;pos int, key string, value variant&gt;. pos is the position of the field/element in its parent object/array, and value is the field/element value. key is the field name when exploding a variant object, or is NULL when exploding a variant array. Unlike variant_explode, if the given variant is not a variant array/object, including SQL NULL, variant null, and any other variant values, then NULL is produced.

      Parameters:
      input - (undocumented)
      Returns:
      (undocumented)
      Since:
      4.0.0