feat: Compress and extract slim binaries with zstd #2533
Conversation
scripts/build_go_slim.sh
Outdated
| if [[ -n $compress ]]; then | ||
| dependencies tar zstd | ||
|
|
||
| if [[ ! $compress == [0-9]* ]] || ((compress > 22)) || ((compress < 1)); then |
There was a problem hiding this comment.
| if [[ ! $compress == [0-9]* ]] || ((compress > 22)) || ((compress < 1)); then | |
| if [[ "$compress" != [0-9]* ]] || [ "$compress" -gt 22 ] || [ "$compress" -lt 1 ]; then |
There was a problem hiding this comment.
Just curious why we'd want to avoid (( ))? Neither handles non-numbers gracefully (but we've verified that with the glob match).
There was a problem hiding this comment.
Consistency, we use lt and gt in other scripts
There was a problem hiding this comment.
Ok, fair enough. I'd like for us to drop the use of [ though. [[ is superior and also supports -gt and -lt. For instance, with [[ there is no need to quote the right-hand variable to avoid errors when it's empty.
There was a problem hiding this comment.
Feel free to change it across all scripts 👍
There was a problem hiding this comment.
I won't do it in this PR to keep the noise down, but I can do a follow-up tomorrow.
| @@ -254,6 +254,7 @@ func server() *cobra.Command { | |||
| Logger: logger.Named("coderd"), | |||
| Database: databasefake.New(), | |||
| Pubsub: database.NewPubsubInMemory(), | |||
| CacheDir: cacheDir, | |||
There was a problem hiding this comment.
cacheDir defaults to /tmp/ on my system. We should probably change that to not use /tmp at all and instead use ~/.cache so we're not dumping 300MB into RAM on startup
There was a problem hiding this comment.
Indeed, I've hinted at this in #2200 but we should create a separate issue for it.
Co-authored-by: Dean Sheather <dean@deansheather.com>
scripts/build_go_slim.sh
Outdated
| if [[ -n $compress ]]; then | ||
| dependencies tar zstd | ||
|
|
||
| if [[ ! $compress == [0-9]* ]] || ((compress > 22)) || ((compress < 1)); then |
There was a problem hiding this comment.
Just curious why we'd want to avoid (( ))? Neither handles non-numbers gracefully (but we've verifie
F7FC
d that with the glob match).
| @@ -254,6 +254,7 @@ func server() *cobra.Command { | |||
| Logger: logger.Named("coderd"), | |||
| Database: databasefake.New(), | |||
| Pubsub: database.NewPubsubInMemory(), | |||
| CacheDir: cacheDir, | |||
There was a problem hiding this comment.
Indeed, I've hinted at this in #2200 but we should create a separate issue for it.
| @@ -254,6 +254,7 @@ func server() *cobra.Command { | |||
| Logger: logger.Named("coderd"), | |||
| Database: databasefake.New(), | |||
| Pubsub: database.NewPubsubInMemory(), | |||
| CacheDir: cacheDir, | |||
This PR adds support for compressing coder slim binaries. The binaries will be embedded as
site/bin/coder.tar.zstand unpacked on coder startup.Cases:
bin/coder.tar.zstarchive is present -> extracted to$CODER_CACHE_DIR/site/binand served viahttp.DirGITKEEPin embedded fs underbin/-> serve from embedd fsbin/or onlybin/GITKEEPin embedd fs -> serve from empty$CODER_CACHE_DIR/site/bin(allows manual placement)Fixes #2202
Changes here may also affect #1547 (i.e. 404 for /bin won't return index.html).
TODO:
Only extract archive once (currently extracted on each startup)(This will be a separate PR)Notes
Decompression is sufficiently fast at around 0.6 - 1 second and will not delay startup significantly.
The following results show that compression after level 6 or 7 show diminishing returns until "ultra" levels of 20+.