Skip to content

API, Core: Add UDF leaf types — Representation and Parameter #15994

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
huaxingao wants to merge 6 commits into
base: main
Choose a base branch
from
Open

API, Core: Add UDF leaf types — Representation and Parameter #15994

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8a21601
API, Core: Add UDF leaf types — Representation and Parameter
huaxingao Apr 16, 2026
5afb02f
Add UdfType
huaxingao Apr 20, 2026
b96a2fe
address comments
huaxingao May 2, 2026
eed9f23
address comments
huaxingao May 4, 2026
0db77f5
Address comments
huaxingao May 11, 2026
690408d
fix checkstyle
huaxingao May 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions api/src/main/java/org/apache/iceberg/udf/SQLUdfRepresentation.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.iceberg.udf;

/** A SQL representation of a UDF, containing the SQL expression and dialect. */
public interface SQLUdfRepresentation extends UdfRepresentation {

@Override
default String type() {
return Type.SQL;
}

/** The SQL expression text that defines the function body. */
String sql();

/** The SQL dialect identifier (e.g., "spark", "trino"). */
String dialect();
}
35 changes: 35 additions & 0 deletions api/src/main/java/org/apache/iceberg/udf/UdfParameter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.iceberg.udf;

import javax.annotation.Nullable;

/** A parameter declared in a UDF definition. */
public interface UdfParameter {

/** The parameter name. */
String name();

/** The parameter data type. */
UdfType type();

/** A documentation string for the parameter. */
@Nullable
String doc();
}
38 changes: 38 additions & 0 deletions api/src/main/java/org/apache/iceberg/udf/UdfRepresentation.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.iceberg.udf;

/**
* Describes how a UDF's logic is expressed, for example as a SQL body with a specific dialect. A
* UDF definition version may carry one or more representations so that engines can pick a form they
* understand.
*/
public interface UdfRepresentation {

/** Standard representation type names used in UDF metadata. */
class Type {
private Type() {}
Copy link
Copy Markdown
Member

@RussellSpitzer RussellSpitzer May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: missing javadoc


/** A SQL body representation, see {@link SQLUdfRepresentation}. */
public static final String SQL = "sql";
}

/** Returns the representation type, e.g., {@link Type#SQL}. */
String type();
}
70 changes: 70 additions & 0 deletions api/src/main/java/org/apache/iceberg/udf/UdfType.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.iceberg.udf;

/**
* Represents a UDF data type as defined in the UDF spec. UDF types are based on Iceberg types but
* intentionally omit field IDs and element nullability. Concrete implementations live as static
* nested classes on {@link UdfTypes}: {@link UdfTypes.PrimitiveType} for primitive and
* semi-structured types, and the nested types {@link UdfTypes.ListType}, {@link UdfTypes.MapType},
* and {@link UdfTypes.StructType}.
*/
public interface UdfType {
Copy link
Copy Markdown
Member

@RussellSpitzer RussellSpitzer May 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Organization suggestion -

I feel like we have a lot of small class files here and there are a few ways to make it cleaner

Option 1, make a type package put all the files in there
Option 2 (or option 1 extension) - Follow the Iceberg Types model with UdfType Interfaces and UdfTypes Concerete Implemetnations

I would probably suggest doing both, but just option 2 is probably fine.

How Types is set up

// Type.java — the interface
public interface Type extends Serializable {
  enum TypeID { BOOLEAN, INTEGER, ... STRUCT, LIST, MAP, VARIANT }

  TypeID typeId();
  default boolean isPrimitiveType() { ... }
  default boolean isStructType() { ... }
  // ...

  abstract class PrimitiveType implements Type { ... }
  abstract class NestedType implements Type { ... }
}
// Types.java — ALL concrete implementations as static nested classes in ONE file
public class Types {
  private Types() {}

  // Primitives
  public static class BooleanType extends PrimitiveType { ... }
  public static class IntegerType extends PrimitiveType { ... }
  public static class StringType extends PrimitiveType { ... }
  public static class DecimalType extends PrimitiveType { ... }
  // ... ~15 more primitives ...

  // Field (equivalent to UdfFieldType)
  public static class NestedField implements Serializable {
    public static NestedField optional(int id, String name, Type type) { ... }
    public static NestedField required(int id, String name, Type type) { ... }
    // ...
  }

  // Nested types
  public static class StructType extends NestedType {
    public static StructType of(NestedField... fields) { ... }
    // ...
  }

  public static class ListType extends NestedType {
    public static ListType ofOptional(int elementId, Type elementType) { ... }
    // ...
  }

  public static class MapType extends NestedType {
    public static MapType ofOptional(int keyId, int valueId, Type keyType, Type valueType) { ... }
    // ...
  }
}

Copy link
Copy Markdown
Contributor Author

@huaxingao huaxingao May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Went with Option 2: collapsed into UdfTypes.java with PrimitiveType / ListType / MapType / StructType / NestedField as static nested classes, mirroring org.apache.iceberg.types.Types.


enum TypeID {
PRIMITIVE,
LIST,
MAP,
STRUCT
}

TypeID typeId();

default boolean isPrimitiveType() {
return typeId() == TypeID.PRIMITIVE;
}

default boolean isListType() {
return typeId() == TypeID.LIST;
}

default boolean isMapType() {
return typeId() == TypeID.MAP;
}

default boolean isStructType() {
return typeId() == TypeID.STRUCT;
}

default UdfTypes.PrimitiveType asPrimitiveType() {
throw new IllegalArgumentException("Not a primitive type: " + this);
}

default UdfTypes.ListType asListType() {
throw new IllegalArgumentException("Not a list type: " + this);
}

default UdfTypes.MapType asMapType() {
throw new IllegalArgumentException("Not a map type: " + this);
}

default UdfTypes.StructType asStructType() {
throw new IllegalArgumentException("Not a struct type: " + this);
}
}
Loading