8000 Speed up executable command completions by mqudsi · Pull Request #7153 · fish-shell/fish-shell · GitHub
[go: up one dir, main page]

Skip to content

Speed up executable command completions#7153

Merged
mqudsi merged 2 commits intofish-shell:masterfrom
mqudsi:fast_waccess
Jun 29, 2020
Merged

Speed up executable command completions#7153
mqudsi merged 2 commits intofish-shell:masterfrom
mqudsi:fast_waccess

Conversation

@mqudsi
Copy link
Contributor
@mqudsi mqudsi commented Jun 24, 2020

This brings down the number of syscalls per potential completion result from three or four to just one.

It passes the tests and I'm comfortable with how it works on Linux and BSD, but putting this up here before merging in case anyone spots something non-portable or can think of any edge cases that aren't addressed.

@mqudsi mqudsi added the performance Purely performance-related enhancement without any changes in black box output label Jun 24, 2020
@mqudsi mqudsi requested a review from faho June 24, 2020 20:45
src/wildcard.cpp Outdated
std::vector<gid_t> groups;
while (true) {
int ngroups = getgroups(0, nullptr);
assert(ngroups > 0 && "getgroups(0, nullptr) returned 0 group memberships!");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think technically you can have no supplementary groups? The spec is not very clear.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I forgot that the effective group/primary group is not guaranteed to be included and was presuming that would be the at least one response. You're right, we should probably remove that hard assert.

@zanchey
Copy link
Member
zanchey commented Jun 27, 2020

How much faster do you think this makes things?

@mqudsi
Copy link
Contributor Author
mqudsi commented Jun 28, 2020

Completions are around 25% faster (on Linux 5.3 with pti enabled), taking 34% less system cpu time. I've been leery of syscall-heavy code because it just seems to get slower and slower these days. (Benchmark was after rebasing against master.)

Note that this is on an NVMe and after priming the cache. The syscalls involve inode metadata access, so on spinning rust you can expect much bigger improvements.

hyperfine 'fish exec_completion_benchmark.fish' 'build/fish exec_completion_benchmark.fish'
Benchmark #1: fish exec_completion_benchmark.fish
  Time (mean ± σ):      2.142 s ±  0.076 s    [User: 1.421 s, System: 0.722 s]
  Range (min … max):    2.058 s …  2.262 s    10 runs
 
Benchmark #2: build/fish exec_completion_benchmark.fish
  Time (mean ± σ):      1.699 s ±  0.178 s    [User: 1.225 s, System: 0.475 s]
  Range (min … max):    1.554 s …  2.137 s    10 runs

return COMPLETE_SOCKET_DESC;
} else if (S_ISDIR(buf.st_mode)) {
return COMPLETE_DIRECTORY_DESC;
} else if (buf.st_mode & (S_IXUSR | S_IXGRP | S_IXGRP) && waccess(filename, X_OK) == 0) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure this is a bug and was supposed to be S_IXUSR | S_IXGRP | S_IXOTH

@mqudsi
Copy link
Contributor Author
mqudsi commented Jun 28, 2020

On Linux and BSD it is not possible to effectively* apply separate permissions to a symlink than the destination it points to. Does anyone know if there are any platforms that do not follow this rule (@krobelus @zanchey)?

e.g.

> touch ./executable
> chmod 777 ./executable
> ln -s ./executable ./symlink
> chmod 077 ./symlink
> ls -alhtr ./executable ./symlink
----rwxrwx 1 mqudsi mqudsi  0 Jun 28 16:04 ./executable
lrwxrwxrwx 1 mqudsi mqudsi 12 Jun 28 16:04 ./symlink -> ./executable

from chmod(1) on Linux:

   chmod never changes the permissions of symbolic links; the chmod system call cannot  change  their  permis‐
   sions.   This  is  not a problem since the permissions of symbolic links are never used.  However, for each
   symbolic link listed on the command line, chmod changes the permissions of the pointed-to  file.   In  con‐
   trast, chmod ignores symbolic links encountered during recursive directory traversals.

* the symlink always has full permissions so the target is always the determinate

It's not surprising that this never came up, as I cannot imagine a more
useless chmod value. Perhaps in the context of nologin or something?
*shrug*
@zanchey
Copy link
Member
zanchey commented Jun 29, 2020

At least on FreeBSD, the symbolic link can have a different mode than the target file - the chmod -h option will do so.

@zanchey
Copy link
Member
zanchey commented Jun 29, 2020

Oh, sorry - effectively - you are right

This brings down the number of syscalls per potential completion result
from three or four to just one.
@mqudsi mqudsi merged commit 1c74335 into fish-shell:master Jun 29, 2020
@zanchey zanchey added this to the fish 3.2.0 milestone Jul 1, 2020
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 29, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

performance Purely performance-related enhancement without any changes in black box output

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0