[rust] Procedural macros for Tup generation #5323

gz · 2025-12-21T23:22:20Z

Also add IsNone trait for future null optimization.
Remove some unused/old example benchmark code.

Copilot

Pull request overview

This PR introduces procedural macros for tuple generation and adds an IsNone trait for future null optimization. It replaces the macro-based tuple declaration system with a procedural macro approach and removes unused example/benchmark code.

Adds feldera-macros crate with declare_tuple! macro and IsNone derive macro
Migrates tuple generation from declarative macros to procedural macros
Removes unused GDELT benchmarks and JSON example code

Reviewed changes

Copilot reviewed 55 out of 59 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
crates/feldera-macros/*	New crate providing procedural macros for tuple generation and IsNone trait
crates/dbsp/src/utils/tuple/gen.rs	Removed old declarative macro implementation
crates/dbsp/src/utils/tuple.rs	Updated to use new procedural macros
crates/dbsp/src/utils/is_none.rs	New trait and implementations for null optimization
crates/dbsp/src/trace.rs	Added IsNone bound to DBData trait
sql-to-dbsp-compiler/SQL-compiler/src/main/java/org/dbsp/sqlCompiler/compiler/backend/rust/*.java	Updated Java code generators to use new macro syntax
crates/dbsp/examples/json.rs	Removed unused JSON example
crates/dbsp/benches/gdelt/*	Removed unused GDELT benchmarks
Various test and example files	Added IsNone derive to struct definitions

crates/dbsp/examples/tutorial/tutorial9.rs

crates/dbsp/examples/tutorial/tutorial8.rs

crates/dbsp/examples/tutorial/tutorial7.rs

crates/dbsp/examples/tutorial/tutorial6.rs

crates/dbsp/examples/tutorial/tutorial5.rs

crates/dbsp/examples/tutorial/tutorial4.rs

crates/dbsp/examples/tutorial/tutorial3.rs

crates/dbsp/examples/tutorial/tutorial2.rs

crates/dbsp/examples/orgchart.rs

crates/dbsp/Cargo.toml

mihaibudiu · 2025-12-22T18:37:51Z

Let me know when this is ready for review, I still see a lot of churn

gz · 2025-12-22T18:41:17Z

it is ready

Cargo.toml

mihaibudiu · 2025-12-22T19:07:27Z

crates/dbsp/Cargo.toml

 serde_json = { workspace = true }
 ptr_meta = { workspace = true }
 feldera-types = { workspace = true }
+feldera-macros = { workspace = true }


why remove examples and benchmarks?

removed gdelt because no one has touched it in years
removed ijson because we never ended up using it

The fact no one has touched it may be a sign that it needs to be kept; check with Leonid

@ryzhyk can comment

It's ok for it to go

crates/dbsp/src/time/product.rs

crates/dbsp/src/utils/is_none.rs

mihaibudiu · 2025-12-22T19:12:41Z

crates/feldera-macros/src/lib.rs

+    if std::env::var_os("FELDERA_DEV_MACROS_DUMP").is_some() {
+        let parsed_file: syn::File = syn::parse2(expanded.clone()).expect("Failed to parse output");
+        let formatted = prettyplease::unparse(&parsed_file);
+        eprintln!("{}", formatted);


is this needed?

crates/feldera-macros/src/lib.rs

mihaibudiu · 2025-12-22T19:44:05Z

crates/feldera-macros/src/tuples.rs

+    expanded
+}
+
+pub(super) struct TupleDef {


putting this first would make the code easier to review

mihaibudiu · 2025-12-22T19:49:52Z

crates/feldera-macros/src/tuples.rs

+            .into_iter()
+            .filter_map(|param| {
+                if let syn::GenericParam::Type(ty) = param {
+                    Some(ty.ident)


I wonder why Some(...) has a type of Punctuated<>.

crates/feldera-macros/src/tuples.rs

Also add IsNone trait for future null optimization. Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

ryzhyk

I don't yet know how to read procedural macros, so didn't review that part. I wonder if IsNone is sufficient for the upcoming rkyv optimization. Don't we also need to IsOption to determine if the type is an option and needs a bit in the bitmap?

ryzhyk · 2025-12-24T01:14:25Z

crates/dbsp/Cargo.toml

 serde_json = { workspace = true }
 ptr_meta = { workspace = true }
 feldera-types = { workspace = true }
+feldera-macros = { workspace = true }


It's ok for it to go

gz requested review from Copilot, mihaibudiu and ryzhyk December 21, 2025 23:22

Copilot AI reviewed Dec 21, 2025

View reviewed changes

gz force-pushed the none-optimziation-part1 branch 3 times, most recently from d3a4243 to d2301f9 Compare December 22, 2025 00:48

gz had a problem deploying to ci December 22, 2025 01:07 — with GitHub Actions Failure

gz force-pushed the none-optimziation-part1 branch from d2301f9 to e017654 Compare December 22, 2025 01:24

gz temporarily deployed to ci December 22, 2025 01:39 — with GitHub Actions Inactive

gz force-pushed the none-optimziation-part1 branch from e017654 to 11e82fa Compare December 22, 2025 02:41

gz temporarily deployed to ci December 22, 2025 02:56 — with GitHub Actions Inactive

gz force-pushed the none-optimziation-part1 branch from 11e82fa to b4751f6 Compare December 22, 2025 04:28

gz temporarily deployed to ci December 22, 2025 04:44 — with GitHub Actions Inactive

gz force-pushed the none-optimziation-part1 branch from b4751f6 to 7d1bc0a Compare December 22, 2025 07:28

gz temporarily deployed to ci December 22, 2025 07:43 — with GitHub Actions Inactive

gz force-pushed the none-optimziation-part1 branch from 7d1bc0a to 5744281 Compare December 22, 2025 17:11

gz temporarily deployed to ci December 22, 2025 17:27 — with GitHub Actions Inactive

gz force-pushed the none-optimziation-part1 branch 2 times, most recently from ca7a6b7 to 3e3c091 Compare December 22, 2025 18:12

gz temporarily deployed to ci December 22, 2025 18:40 — with GitHub Actions Inactive

gz force-pushed the none-optimziation-part1 branch 2 times, most recently from 618b224 to aad3b1f Compare December 22, 2025 19:18

mihaibudiu reviewed Dec 22, 2025

View reviewed changes

[rust] Procedural macros for Tup generation

37130ab

Also add IsNone trait for future null optimization. Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

gz force-pushed the none-optimziation-part1 branch from aad3b1f to 37130ab Compare December 22, 2025 20:07

ryzhyk approved these changes Dec 24, 2025

View reviewed changes

[rust] Procedural macros for Tup generation #5323

Are you sure you want to change the base?

[rust] Procedural macros for Tup generation #5323

Uh oh!

Conversation

gz commented Dec 21, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mihaibudiu commented Dec 22, 2025

Uh oh!

gz commented Dec 22, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ryzhyk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants