8000 GH-46410: [C++] Add parquet options to Meson configuration by WillAyd · Pull Request #46647 · apache/arrow · GitHub
[go: up one dir, main page]

Skip to content

GH-46410: [C++] Add parquet options to Meson configuration #46647

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

WillAyd
Copy link
Contributor
@WillAyd WillAyd commented May 30, 2025

Rationale for this change

This continues adding functionality to the Meson configuration

What changes are included in this PR?

This adds the parquet directory to the Meson configuration

Are these changes tested?

Yes

Are there any user-facing changes?

No

@WillAyd WillAyd requested a review from wgtmac as a code owner May 30, 2025 13:38
@WillAyd WillAyd added the CI: Extra Run extra CI label May 30, 2025
Copy link

⚠️ GitHub issue #46410 has been automatically assigned in GitHub to PR creator.

10000
@@ -66,6 +66,22 @@ option(
type: 'string',
description: 'Arbitrary string that identifies the kind of package (for informational purposes)',
)
option('parquet', type: 'feature', description: 'Build the Parquet libraries')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few differences with the CMake configuration worth calling out.

First, CMake has an extra option for PARQUET_MINIMAL_DEPENDENCY. This seems to maybe not be used all of that often, so I did not bother implementing here. Let me know if that is a mistake.

CMake also requires that when either the executables or examples are built that you use static libraries. My assumption is that was done to make the CMake configuration easier, so I did not place that same restriction in the Meson configuration.

Finally, CMake will fail if a pre-installed version of OpenSSL is not found and encryption is required. This configuration is not as strict, and will use Meson's wrap system to resolve the OpenSSL dependency if it is not provided by the system

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels May 30, 2025
@WillAyd WillAyd force-pushed the meson-add-parquet branch 2 times, most recently from 840cc24 to ee277eb Compare June 12, 2025 16:45
@WillAyd WillAyd force-pushed the meson-add-parquet branch 3 times, most recently from 2a8b860 to 764f5a2 Compare June 27, 2025 03:24
@WillAyd
Copy link
Contributor Author
WillAyd commented Jun 27, 2025

@kou this one might have gotten missed - mind taking a look when you have time?

Copy link
Member
@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for missing this...

},
}

if get_option('parquet_require_encryption').enabled()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use needs_parquet_encryption here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

cpp/meson.build Outdated
Comment on lines 80 to 82
' Meson >= 1.8.0 is required for fuzzing support, found @0@'.format(
meson.version(),
),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use f'... @meson_version@ here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also done!

option('parquet', type: 'feature', description: 'Build the Parquet libraries')
option(
'parquet_build_executables',
type: 'feature',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, should we use boolean not feature for option that doesn't depend on any dependency?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm not sure. I think feature is nice because it lets you easily toggle everything on/off (particularly useful for our current CI setup) but don't have a strong point of view here

)
option(
'parquet_build_examples',
type: 'feature',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

Comment on lines +63 to +64
if needs_parquet_encryption
openssl_dep = dependency('openssl')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, we may need to use the feature object not feature.enabled() to support auto:

Suggested change
if needs_parquet_encryption
openssl_dep = dependency('openssl')
openssl_dep = dependency('openssl', required: get_option('parquet_require_encryption'))
if openssl_dep

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated this logic slightly, although I'm not sure if I accomplished what you are after. Let me know!

@github-actions github-actions bot added awaiting changes Awaiting changes awaiting change review Awaiting change review and removed awaiting committer review Awaiting committer review awaiting changes Awaiting changes labels Jun 27, 2025
@WillAyd WillAyd force-pushed the meson-add-parquet branch from ff43c0f to 19aa192 Compare June 27, 2025 21:18
@WillAyd
Copy link
Contributor Author
WillAyd commented Jul 24, 2025

Hey @kou just wanted to toss a friendly ping on this. If you have time, please let me know what you think of the latest updates. Thanks!

Copy link
Member
@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for not re-reviewing this...

Comment on lines 63 to 64
openssl_dep = dependency('openssl', required: needs_parquet_encryption)
if openssl_dep.found()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that Parquet encryption is enabled even when parquet_require_encryption=disabled and system OpenSSL exists.

I think that the following is expected:

  • parquet_require_encryption=enabled:
    • dependency('openssl') must be found
    • Parquet encryption is built
  • parquet_require_encryption=auto:
    • Parquet encryption is built only when dependency('opnessl') is found
  • parquet_require_encryption=disabled:
    • Don't execute dependency('openssl')
    • Parquet encryption is not built

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I think the latest iteration simplifies this by branching only when the parquet_require_encryption option is enabled.

Note that the auto feature in Meson is still either in an enabled or disabled state; it just depends if auto_features=enabled or auto_features=disabled when the project is configured

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, really?
Do you know the auto feature documentation?

I saw https://mesonbuild.com/Reference-manual_returned_feature.html . It includes feature.allowed() and feature.auto(). It seems that get_option('parquet_require_encryption') may be auto.

@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Jul 25, 2025
@WillAyd WillAyd force-pushed the meson-add-parquet branch from 19aa192 to 2484174 Compare July 25, 2025 14:02
@github-actions github-actions bot removed the awaiting changes Awaiting changes label Jul 25, 2025
@github-actions github-actions bot added the awaiting change review Awaiting change review label Jul 25, 2025
@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Jul 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0