8000 async-profiler 4.0 (new formula) by bric3 · Pull Request #222217 · Homebrew/homebrew-core · GitHub
[go: up one dir, main page]

Skip to content

async-profiler 4.0 (new formula) #222217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

bric3
Copy link
Contributor
@bric3 bric3 commented May 2, 2025
  • Have you followed the guidelines for contributing?
  • Have you ensured that your commits follow the commit style guide?
  • Have you checked that there aren't other open pull requests for the same formula update/change?
  • Have you built your formula locally with HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
  • Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
  • Does your build pass brew audit --strict <formula> (after doing HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

Also : brew style --formula async-profiler


Contributing my formula for async-profiler/async-profiler (8.1 K stars).

Note this formula is only for version 4.0, however my original formulas started with 2.8 (which was mostly a script and and lib until 3.0).

@github-actions github-actions bot added new formula PR adds a new formula to Homebrew/homebrew-core missing license Formula has a missing license which should be added labels May 2, 2025
@github-actions github-actions bot added the autosquash Automatically squash pull request commits according to Homebrew style. label May 2, 2025
@botantony
Copy link
Contributor

Here's how I'd create this formula:

class AsyncProfiler < Formula
  desc "Sampling CPU and HEAP profiler for Java"
  homepage "https://github.com/async-profiler/async-profiler/releases"
  url "https://github.com/async-profiler/async-profiler/archive/refs/tags/v4.0.tar.gz"
  sha256 "7beb736868af485d6b0b624e42141f78df0ca8403188adc17965b7153261aa55"
  license "Apache-2.0"

  depends_on "openjdk"

  def install
    system "make", "all"
    bin.install Dir["build/bin/*"]
    lib.install Dir["build/lib/*"]
    libexec.install Dir["build/jar/*"]
  end

  test do
    assert_match "Async-profiler #{version}", shell_output("#{bin}/asprof --version")
    # TODO: add better test
    # https://docs.brew.sh/Formula-Cookbook#add-a-test-to-the-formula
  end
end

@bric3 bric3 marked this pull request as draft May 3, 2025 07:49
@github-actions github-actions bot added the java Java use is a significant feature of the PR or issue label May 3, 2025
@github-actions github-actions bot removed the missing license Formula has a missing license which should be added label May 3, 2025
@bric3 bric3 force-pushed the contribute-async-profiler-formula branch 2 times, most recently from b5e56e0 to b64e5e9 Compare May 3, 2025 20:26
@bric3 bric3 force-pushed the contribute-async-profiler-formula branch from b64e5e9 to 37ca88f Compare May 3, 2025 20:31
Comment on lines +20 to +22
bin.install Dir["build/bin/*"]
lib.install Dir["build/lib/*"]
libexec.install Dir["build/jar/*"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No make install?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with make, but I didn't see any install target there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be better to also report the install target missing issue

head "https://github.com/async-profiler/async-profiler.git", branch: "master"

depends_on "cmake" => :build
depends_on "openjdk" => [:build, :test]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it really not needed at runtime? It seems like this should just be

Suggested change
depends_on "openjdk" => [:build, :test]
depends_on "openjdk"

Copy link
Contributor Author
@bric3 bric3 May 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the jars are supplementary, but Async-profiler itself is native. Of course it connects to / interact with some running JVM to perform it's main duty, but it's not dependent on a JDK to be able to run.

async-profiler can then be "deployed" anywhere, e.g. in a container.

@bric3
Copy link
Contributor Author
bric3 commented May 5, 2025

Hi, also what would be the strategy (if it's even possible) to make my formula redirect to this one?

@bric3 bric3 force-pushed the contribute-async-profiler-formula branch from 82539c5 to 673197d Compare May 5, 2025 09:49
@SMillerDev
Copy link
Member

You can add a tap_migrations.json, it marks a formula as migrated.

@bric3
Copy link
Contributor Author
bric3 commented May 5, 2025

So the tests are failing https://github.com/Homebrew/homebrew-core/actions/runs/14833693346/job/41640450001?pr=222217#step:3:57

The error : Could not start attach mechanism: No such file or directory

Is there something specific to know about macOs tests, how is the JVM run ?

Error log
  /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.4.3/bin/bundle clean
  ==> Testing async-profiler
  ==> /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof --version
  ==> /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof -d 2 -f /private/tmp/async-profiler-test-20250505-14530-kx2x51/test-profile-via-attach.html jps
  Could not start attach mechanism: No such file or directory
  Error: async-profiler: failed
  An exception occurred within a child process:
    BuildError: Failed executing: /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof -d 2 -f /private/tmp/async-profiler-test-20250505-14530-kx2x51/test-profile-via-attach.html jps
  /opt/homebrew/Library/Homebrew/formula.rb:3119:in 'block in Formula#system'
  /opt/homebrew/Library/Homebrew/formula.rb:3055:in 'IO.open'

Locally on macOs 15.4

brew test --verbose async-profiler
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.4.3/bin/bundle clean
==> Testing bric3/tap/async-profiler
==> /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof --version
Picked up _JAVA_OPTIONS: -Duser.home=/Users/brice.dutheil/Library/Caches/Homebrew/java_cache -Djava.io.tmpdir=/private/tmp
==> /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof -d 2 -f /private/tmp/async-profiler-test-20250505-66666-ak7y1g/test-profile-via-attach.html jps
Profiling for 2 seconds
Done
==> /opt/homebrew/opt/openjdk/bin/java -agentpath:/opt/homebrew/Cellar/async-profiler/4.0/lib/libasyncProfiler.dylib=start,event=cpu,lock=10ms,file=test-profile-via-lib.jfr /private/tmp/async-profiler-test-20250505-66666-ak7y1g/Main.java 2
Picked up _JAVA_OPTIONS: -Duser.home=/Users/brice.dutheil/Library/Caches/Homebrew/java_cache -Djava.io.tmpdir=/private/tmp
Profiling started
==> /opt/homebrew/Cellar/async-profiler/4.0/bin/jfrconv -o pprof /private/tmp/async-profiler-test-20250505-66666-ak7y1g/test-profile-via-lib.jfr /private/tmp/async-profiler-test-20250505-66666-ak7y1g/test-profile-via-lib.pprof
Picked up _JAVA_OPTIONS: -Duser.home=/Users/brice.dutheil/Library/Caches/Homebrew/java_cache -Djava.io.tmpdir=/private/tmp
Converting test-profile-via-lib.jfr -> test-profile-via-lib.pprof # 0.017 s

FYI here's what async-profiler says on this error.

https://github.com/async-profiler/async-profiler/blob/73f04869464df9dce09fe4755e7a1c05712ae480/docs/Troubleshooting.md#could-not-start-attach-mechanism-no-such-file-or-directory

Could not start attach mechanism: No such file or directory

The profiler cannot establish communication with the target JVM through UNIX domain socket.
Usually this happens in one of the following cases:

  1. Attach socket /tmp/.java_pidNNN has been deleted. It is a common
    practice to clean /tmp automatically with some scheduled script.
    Configure the cleanup software to exclude .java_pid* files from deletion.

    • How to check: run lsof -p PID | grep java_pid. If it lists a socket file, but the file does not exist, then this is exactly
      the described problem.
  2. JVM is started with -XX:+DisableAttachMechanism option.

  3. /tmp directory of Java process is not physically the same directory
    as /tmp of your shell, because Java is running in a container or in
    chroot environment. asprof attempts to solve this automatically,
    but it might lack the required permissions to do so.

    • Check strace asprof PID jcmd
  4. JVM is busy and cannot reach a safepoint. For instance,
    JVM is in the middle of long-running garbage collection.

    • How to check: run kill -3 PID. Healthy JVM process should print
      a thread dump and heap info in its console.

@SMillerDev
Copy link
Member

Attach socket /tmp/.java_pidNNN has been deleted.

You can't just write anywhere in the tests, brew usage, especially for tests, is heavily sandboxed. It might be best to set the tmp directory to testpath/"tmp" in the test.

@bric3 bric3 force-pushed the contribute-async-profiler-formula branch from ee72c06 to ebc08d8 Compare May 5, 2025 16:45
@bric3 bric3 force-pushed the contribute-async-profiler-formula branch from ebc08d8 to 329d787 Compare May 5, 2025 16:48
@bric3
Copy link
Contributor Author
bric3 commented May 5, 2025

Unfortunately the JVM doesn't work that way, the JVM temp directory is hardcoded to /tmp on linux, and to the the user temp folder on OSX

In order for the attach mechanism to work a socket FD is needed to be present.

Locally I have the following file descriptor opened.

==> bash -c 'lsof -p 84540'
Picked up _JAVA_OPTIONS: -Duser.home=/Users/brice.dutheil/Library/Caches/Homebrew/java_cache -Djava.io.tmpdir=/private/tmp
==> COMMAND   PID          USER   FD   TYPE             DEVICE  SIZE/OFF                NODE NAME
java    84540 brice.dutheil  cwd    DIR               1,15       128           251759721 /private/tmp/async-profiler-test-20250505-84530-cz8bev
java    84540 brice.dutheil  txt    REG               1,15     69856           225322662 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/bin/java
java    84540 brice.dutheil  txt    REG               1,15    130416           225323381 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjli.dylib
java    84540 brice.dutheil  txt    REG               1,15     32768           251759730 /private/var/folders/5_/gltgq4390wx07yvypgl1mc400000gq/T/hsperfdata_brice.dutheil/84540
java    84540 brice.dutheil  txt    REG               1,15     76144           225323377 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjimage.dylib
java    84540 brice.dutheil  txt    REG               1,15     66964           250200055 /Library/Preferences/Logging/.plist-cache.NXoAOdnc
java    84540 brice.dutheil  txt    REG               1,15       110 1152921500312087311 /System/Library/CoreServices/SystemVersion.bundle/English.lproj/SystemVersion.strings
java    84540 brice.dutheil  txt    REG               1,15    187024           225323364 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjava.dylib
java    84540 brice.dutheil  txt    REG               1,15   2288832 1152921500312522782 /usr/lib/dyld
java    84540 brice.dutheil  txt    REG               1,15  15633904           225323448 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/server/libjvm.dylib
java    84540 brice.dutheil  txt    REG               1,15 143206280           225323256 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/modules
java    84540 brice.dutheil  txt    REG               1,15  33979312 1152921500312533288 /usr/share/icu/icudt76l.dat
java    84540 brice.dutheil  txt    REG               1,15  15630336           225323267 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/server/classes.jsa
java    84540 brice.dutheil    0u   CHR               16,8     0t471                1043 /dev/ttys008
java    84540 brice.dutheil    1u   CHR               16,8     0t471                1043 /dev/ttys008
java    84540 brice.dutheil    2u   CHR               16,8     0t471                1043 /dev/ttys008
java    84540 brice.dutheil    3r   REG               1,15 143206280           225323256 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/modules
java    84540 brice.dutheil    4u  unix 0x71c935f907907507       0t0                     /var/folders/5_/gltgq4390wx07yvypgl1mc400000gq/T//.java_pid84540.tmp

Notice the .java_pid84540 file, while on the CI it is not present:

  ==> bash -c 'lsof -p 14483'
  Picked up _JAVA_OPTIONS: -Duser.home=/Users/brew/Library/Caches/Homebrew/java_cache -Djava.io.tmpdir=/private/tmp
  ==> COMMAND   PID USER   FD    TYPE             DEVICE  SIZE/OFF    NODE NAME
  java    14483 brew  cwd     DIR               1,15       128 4021049 /private/tmp/async-profiler-test-20250505-14459-udwwfr
  java    14483 brew  txt     REG               1,15     69856 4020034 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/bin/java
  java    14483 brew  txt     REG               1,15    131136 4020734 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjli.dylib
  java    14483 brew  txt     REG               1,15     52392 3957305 /Library/Preferences/Logging/.plist-cache.0qxpMgpi
  java    14483 brew  txt     REG               1,15     32768 4021072 /private/var/folders/wp/mgkns75d6dd_kq5800l78p080000gq/T/hsperfdata_brew/14483
  java    14483 brew  txt     REG               1,15     76864 4020733 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjimage.dylib
  java    14483 brew  txt     REG               1,15    187744 4020729 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjava.dylib
  java    14483 brew  txt     REG               1,15  15987024 4020757 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/server/libjvm.dylib
  java    14483 brew  txt     REG               1,15 143205520 4020628 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/modules
  java    14483 brew  txt     REG               1,15  15630336 4020639 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/server/classes.jsa
  java    14483 brew    0u    CHR               16,0     0t453     703 /dev/ttys000
  java    14483 brew    1u    CHR               16,0     0t453     703 /dev/ttys000
  java    14483 brew    2u    CHR               16,0     0t453     703 /dev/ttys000
  java    14483 brew    3r    REG               1,15 143205520 4020628 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/modules
  java    14483 brew  136u  systm 0x47efe4af3f450a85       0t0         [ctl com.apple.netsrc id 6 unit 34]
  java    14483 brew  140u  systm 0x47efe4af3f450005       0t0         [ctl com.apple.netsrc id 6 unit 33]
  java    14483 brew  149    PIPE 0x4d5c62a40ea9cfc0     65536         ->0x235b49b07fa21557
  java    14483 brew  152    PIPE 0x669a58636e94ab49     16384         ->0x2778f7377120f743

What is strange is that the hsperfdata also generated by the JVM is correctly created in what seems like to be an usual temp folder /private/var/folders/wp/mgkns75d6dd_kq5800l78p080000gq/T. Not sure what's going on there.

Note I used the -XX:+StartAttachListener which creates this file upon JVM start.

I wonder if there might be another thing at play (maybe due to this sandboxing), because this test works locally (brew test --verbose async-profiler).

@SMillerDev
Copy link
Member

an usual temp folder /private/var/folders/wp/mgkns75d6dd_kq5800l78p080000gq/T

That looks like the default tmp location for macOS.

@SMillerDev
Copy link
Member

The error is pretty clear, it's not finding openjdk.

  ==> /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof -d 2 -f /private/tmp/async-profiler-test-20250505-14459-udwwfr/test-profile-via-attach.html jps
  The operation couldn’t be completed. Unable to locate a Java Runtime.

@bric3
Copy link
Contributor Author
bric3 commented May 5, 2025

That's the last run when I introduced the lsof on java to debug. But in previous runs, Async profiler had the error described in my earlier comment.

See : https://github.com/Homebrew/homebrew-core/actions/runs/14833693346/job/41640450001#step:3:64

@bric3
Copy link
Contributor Author
bric3 commented May 7, 2025

By any chance is there a way to reproduce in some kind of macos container what happens in the this workflow?

Copy link
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale No recent activity label May 28, 2025
@chenrui333
Copy link
Member

By any chance is there a way to reproduce in some kind of macos container what happens in the this workflow?

I usually just leverage the macos github action workflow to do such debugging https://github.com/chenrui333/github-action-test/blob/main/.github/workflows/brew-debug.yml

@github-actions github-actions bot removed the stale No recent activity label May 29, 2025
@bric3
Copy link
Contributor Author
bric3 commented May 30, 2025

Thanks @chenrui333 I'll take a look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
autosquash Automatically squash pull request commits according to Homebrew style. java Java use is a significant feature of the PR or issue new formula PR adds a new formula to Homebrew/homebrew-core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0