HADOOP-19466. Fix Dockerfile_windows_10 #7417

mgmgwi · 2025-02-20T14:20:57Z

Description of PR

Fix a few outdated dependencies.

How was this patch tested?

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2025-02-20T14:23:46Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/1/console in case of problems.

hadoop-yetus · 2025-02-20T15:40:43Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s< 8000 /td>		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+0 🆗	mvndep	6m 1s		Maven dependency ordering for branch
+1 💚	shadedclient	40m 52s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	34m 8s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		78m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/1/artifact/out/Dockerfile
GITHUB PR	#7417
Optional Tests	dupname asflicense
uname	Linux 3963ac263972 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `9a47c0d`
Max. process+thread count	545 (vs. ulimit of 5500)
modules	C: U:
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/1/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2025-02-20T22:46:45Z

@GauthamBanasandra Could you please review this PR? Thank you very much!

GauthamBanasandra · 2025-02-21T00:39:06Z

@mgmgwi Thank you so much for the PR. I've started the CI run for Windows - https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-win10-x86_64/806/console.
We can merge this once the CI run is completed. FYI, it'll take about a day to finish.

mgmgwi · 2025-02-23T00:00:58Z

Hmm, that build runs now for 2 days already. @GauthamBanasandra can you check what's going on?

mgmgwi · 2025-02-24T08:53:09Z

Can someone give me a hint, what went wrong?

GauthamBanasandra · 2025-02-24T08:58:29Z

@mgmgwi I guess there was something wrong with the CI node thus it was stuck at the mvn install step.

I had thus started the CI run on a couple of other nodes and it has passed the required steps - https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-win10-x86_64/813/

GauthamBanasandra · 2025-02-24T08:59:27Z

Please see - https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-win10-x86_64/813/artifact/out/report.html

GauthamBanasandra · 2025-02-24T09:05:43Z

The ones in green are expected to pass and it's looking fine.

@mgmgwi could you please address the minor comment that I've added? We can then proceed with the merge.

GauthamBanasandra · 2025-02-21T00:36:12Z

dev-support/docker/Dockerfile_windows_10

@@ -51,6 +51,11 @@ RUN powershell git clone https://github.com/microsoft/vcpkg.git \
    && cd vcpkg \
    && git checkout 7ffa425e1db8b0c3edf9c50f2f3a0f25a324541d \
    && .\bootstrap-vcpkg.bat
+
+# fix wrong download link


@mgmgwi could you please elaborate on why we're installing 7-zip here? Please add the same as a comment.

We are not installing 7-zip here, we are replacing the download link that is configured in .\vcpkg\scripts\vcpkgTools.xml using the Linux tool sed.

https://www.nuget.org/api/v2/package/7-Zip.CommandLine/18.1.0 (not working) is replaced with

https://globalcdn.nuget.org/packages/7-zip.commandline.18.1.0.nupkg?packageVersion=18.1.0 (working)

hadoop-yetus · 2025-02-24T09:40:37Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/2/console in case of problems.

hadoop-yetus · 2025-02-24T09:40:42Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/3/console in case of problems.

hadoop-yetus · 2025-02-24T09:46:13Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/4/console in case of problems.

mgmgwi · 2025-02-24T09:46:46Z

@GauthamBanasandra thanks for the feedback. I updated the comment explaining what we do there.

hadoop-yetus · 2025-02-24T10:26:18Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 19s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+0 🆗	mvndep	5m 28s		Maven dependency ordering for branch
+1 💚	shadedclient	25m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	18m 55s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 23s		The patch does not generate ASF License warnings.
		46m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/3/artifact/out/Dockerfile
GITHUB PR	#7417
Optional Tests	dupname asflicense
uname	Linux d1a346334361 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b6b9539`
Max. process+thread count	553 (vs. ulimit of 5500)
modules	C: U:
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/3/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-02-24T10:38:36Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	6m 57s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+0 🆗	mvndep	5m 26s		Maven dependency ordering for branch
+1 💚	shadedclient	25m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	19m 17s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 25s		The patch does not generate ASF License warnings.
		53m 16s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/4/artifact/out/Dockerfile
GITHUB PR	#7417
Optional Tests	dupname asflicense
uname	Linux f13f89fac95b 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0d3e08e`
Max. process+thread count	677 (vs. ulimit of 5500)
modules	C: U:
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/4/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-02-24T11:01:03Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+0 🆗	mvndep	5m 49s		Maven dependency ordering for branch
+1 💚	shadedclient	41m 26s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	37m 6s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		81m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/2/artifact/out/Dockerfile
GITHUB PR	#7417
Optional Tests	dupname asflicense
uname	Linux 19b7ec36eb35 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b6b9539`
Max. process+thread count	553 (vs. ulimit of 5500)
modules	C: U:
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/2/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

GauthamBanasandra

Thanks for the fix @mgmgwi. LGTM.

* This PR fixes a few outdated dependencies needed for the Hadoop build environment on Windows.

github-actions bot added trunk Infra labels Feb 20, 2025

slfan1989 requested a review from GauthamBanasandra February 20, 2025 22:45

GauthamBanasandra requested changes Feb 24, 2025

View reviewed changes

mgmgwi force-pushed the HADOOP-19466-fix-windows-dockerfile branch 2 times, most recently from 9707cb0 to b6b9539 Compare February 24, 2025 09:38

HADOOP-19466. fix Dockerfile_windows_10

0d3e08e

mgmgwi force-pushed the HADOOP-19466-fix-windows-dockerfile branch from b6b9539 to 0d3e08e Compare February 24, 2025 09:44

mgmgwi requested a review from GauthamBanasandra February 24, 2025 11:57

GauthamBanasandra approved these changes Feb 24, 2025

View reviewed changes

GauthamBanasandra merged commit acc3e50 into apache:trunk Feb 24, 2025
4 checks passed

GauthamBanasandra assigned mgmgwi Feb 24, 2025

mgmgwi deleted the HADOOP-19466-fix-windows-dockerfile branch February 24, 2025 14:47

adideshpande pushed a commit to adideshpande/hadoop that referenced this pull request Feb 27, 2025

HADOOP-19466. fix Dockerfile_windows_10 (apache#7417)

d2e964c

* This PR fixes a few outdated dependencies needed for the Hadoop build environment on Windows.

YanivKunda pushed a commit to YanivKunda/hadoop that referenced this pull request Mar 23, 2025

HADOOP-19466. fix Dockerfile_windows_10 (apache#7417)

95b1f91

* This PR fixes a few outdated dependencies needed for the Hadoop build environment on Windows.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-19466. Fix Dockerfile_windows_10 #7417

HADOOP-19466. Fix Dockerfile_windows_10 #7417

HADOOP-19466. Fix Dockerfile_windows_10 #7417

Description of PR

How was this patch tested?

For code changes:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment