Skip to content

Conversation

@shangyian
Copy link
Contributor

@shangyian shangyian commented May 10, 2024

Summary

This PR adds support for writing SELECT * in node queries.

During the Wildcard compile stage, it expands out SELECT * to all available columns on the table references. This may include DJ node columns, CTE columns, or subquery columns. These expanded columns are saved to a node revision's associated columns, but in the saved query, we keep the wildcard syntax (i.e., we always preserve what the user wrote).

During node updates, we also update these columns based on the parent nodes' columns, if any has changed.

Test Plan

Deployment Plan

@netlify
Copy link

netlify bot commented May 10, 2024

Deploy Preview for thriving-cassata-78ae72 canceled.

Name Link
🔨 Latest commit 0734396
🔍 Latest deploy log https://app.netlify.com/sites/thriving-cassata-78ae72/deploys/665e8a9992c45a0008796a85

@shangyian shangyian force-pushed the support-wildcards branch from 5cf0317 to 3a4253e Compare May 10, 2024 15:48
@shangyian shangyian requested a review from agorajek May 10, 2024 15:52
Copy link
Member

@agorajek agorajek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one! Thank you.

# Check if any columns have been updated
existing_columns = {col.name: col for col in node.current.columns} # type: ignore
updated_columns = False
updated_columns = len(current_node_revision.columns) != len(node_validator.columns)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch!

),
)
wildcard_parent.projection.remove(self)
else:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it may be easier to invert this conditional:

if not isinstance(self.parent, Select):
    return super().compile(ctx)

# and then the main code w/o the need to be indented

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point!

@shangyian shangyian force-pushed the support-wildcards branch 10 times, most recently from 299d00d to ce182ec Compare May 15, 2024 02:04
for node_name in node_names:
impacts[node_name] = await hard_delete_node(
node_name,
for node in reversed(topological_sort(nodes)):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we don't have to do a topological sort here, doing so will mean that we process the deletes for nodes in the namespace in an order that minimizes unnecessary invalidation followed by deletion.

display_name=labelize(column_name),
type=column_type,
attributes=existing_column.attributes if existing_column else [],
attributes=[
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was an annoying issue: the attributes field on the Column database object is actually a ColumnAttribute object and not an Attribute object. Since ColumnAttribute is the relational link between an Attribute and a Column, we actually need to recreate this for any copied columns. Otherwise, it'll cause problems downstream since it may reference a column that doesn't exist.

@shangyian shangyian force-pushed the support-wildcards branch 2 times, most recently from ba853a0 to d51de7c Compare June 3, 2024 17:47
@shangyian shangyian force-pushed the support-wildcards branch from d51de7c to 0734396 Compare June 4, 2024 03:31
@shangyian shangyian marked this pull request as ready for review June 4, 2024 03:32
@agorajek
Copy link
Member

agorajek commented Dec 2, 2024

@shangyian what happened here? Was there a major issue or just a missed PR? I see that we can create nodes with SELECT * but we get one Wildcard column for them.

@shangyian
Copy link
Contributor Author

@agorajek yeah, unfortunately I couldn't get one of the unit tests to pass (there was some non-deterministic issue that was hard to track down). I'll take another stab at it soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants