Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support UDF in planner #2995

Open
wants to merge 79 commits into
base: main
Choose a base branch
from
Open

Support UDF in planner #2995

wants to merge 79 commits into from

Conversation

pengpeng-lu
Copy link
Contributor

@pengpeng-lu pengpeng-lu commented Dec 5, 2024

No description provided.

This reverts commit 3a5a44b.
@foundationdb-ci
Copy link
Contributor

Result of fdb-record-layer-pr on Linux CentOS 7

  • Commit ID: 6ff48c0
  • Duration 0:48:01
  • Result: ❌ FAILED
  • Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of fdb-record-layer-pr on Linux CentOS 7

  • Commit ID: bd1cd47
  • Duration 0:55:36
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@@ -20,9 +20,9 @@

package com.apple.foundationdb.relational.util;

import com.apple.foundationdb.annotation.API;
import static com.apple.foundationdb.record.expressions.RecordKeyExpressionProto.*;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use static import to make file more readable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checkstyle doesn't allow "*" in imports, reverted the change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can static import them separately. Sometimes, IntelliJ is configured to group them into a * import.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, but unfortunately build still fails because checkStyle doesn't allow * in import lines.

return MacroFunction.fromProto(serializationContext, proto.getMacroFunction());
} else {
return null;
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can add more if blocks when we have more items in the oneof

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will do dynamic dispatch in a follow-up PR

}

@Nonnull
public static MacroFunction fromProto(@Nonnull final PlanSerializationContext serializationContext, @Nonnull final PMacroFunctionValue functionValue) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that I can inject serializationContext with @autoservice, but this method is called from RecordMetaDataBuilder, which doesn't have serializationContext injected. So it seems that I'll need to create a serializationContext in RecordMetaDataBuilder and pass it to MacroFunction.fromProto anyways.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The context is normally used to do dynamic dispatch among the implementing classes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. will do it in a follow-up PR.

*/
public class MacroFunction implements SerializableFunction {
@Nonnull
private final String functionName;
Copy link
Contributor

@normen662 normen662 Feb 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All functions will have this in common:

  1. function name
  2. declared parameters (types at least)
  3. return type

Can you move that up into SerializableFunction? You can give SerializableFunction the appropriate getters forcing all implementors to provide methods for them. As a second step, have an abstract base class implementing SerializableFunction. Let's call that AbstractCatalogedFunction. All your methods extends from that. It's the same approach Value, and AbstractValue (for instance) have taken.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, restructured the code.

import javax.annotation.Nonnull;
import javax.annotation.Nullable;

public interface SerializableFunction {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be called something that encompasses more than just serialization. I had suggested CatalogedFunction.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discussed offline, renamed.

* limitations under the License.
*/

package com.apple.foundationdb.record.metadata;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach you have taken is not what I had in mind:

  • BuiltInFunction should give up most of its attributes/functionality, encapsulation to this class. Only the encapsulation lambda should stay that is used to implement the encapsulate() method.
  • BuiltInFunction should be THE base for all internal functions that are not domain specific (like length(string) or similar).
  • MacroFunction should be extending this class and adding the functionality that you coded up to expand an arbitrary body

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discussed offline, restructured the code.

@pengpeng-lu pengpeng-lu added the enhancement New feature or request label Feb 21, 2025
@pengpeng-lu pengpeng-lu changed the title Resolves #3013: Support UDF Support UDF in planner Feb 21, 2025
}

@Nonnull
public static MacroFunction fromProto(@Nonnull final PlanSerializationContext serializationContext, @Nonnull final PMacroFunctionValue functionValue) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The context is normally used to do dynamic dispatch among the implementing classes.

}

@Nullable
public static UserDefinedFunction fromProto(@Nonnull PlanSerializationContext serializationContext, @Nonnull RecordMetaDataProto.UserDefinedFunction proto) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can autoservice the extends for UserDefinedFunction and then call the appropriate dispatch routine using the serialization context. We currently do this for Value, RecordQueryPlan, and for Type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will do the dispatch in a follow-up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants