8000 chore: implement oom/ood processing component by DanielleMaywood · Pull Request #16436 · coder/coder · GitHub
[go: up one dir, main page]

Skip to content

chore: implement oom/ood processing component #16436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 44 commits into from
Feb 17, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
6c6240b
chore: add workspace reached resource threshold notification
DanielleMaywood Jan 24, 2025
b3081de
chore: split out into two notifications
DanielleMaywood Jan 30, 2025
a9c8676
chore: update golden files
DanielleMaywood Jan 30, 2025
1a84f96
chore: begin impl of processing logic for oom/ood
DanielleMaywood Jan 29, 2025
78ede46
chore: appease the linter for now
DanielleMaywood Jan 29, 2025
0d2b970
chore: use latest changes to #247, start debounce logic
DanielleMaywood Jan 30, 2025
0df2fd5
chore: add more tests
DanielleMaywood Jan 30, 2025
854d81a
chore: remove mock db for workspace monitor agentapi test
DanielleMaywood Jan 30, 2025
9d9d7b4
chore: remove todo comment
DanielleMaywood Jan 30, 2025
4c21ce7
Merge branch 'main' into dm-internal-247
DanielleMaywood Feb 4, 2025
944fdb5
chore: rewrite ood notification
DanielleMaywood Feb 4, 2025
6444176
chore: updaten golden file
DanielleMaywood Feb 4, 2025
bc87268
chore: silly me
DanielleMaywood Feb 4, 2025
d2265f6
chore: rename test
DanielleMaywood Feb 4, 2025
82a9852
Merge branch 'dm-internal-247' into dm-internal-248
DanielleMaywood Feb 4, 2025
62621d4
chore: add more tests, fix broken sql query
DanielleMaywood Feb 4, 2025
81f43d3
Merge branch 'main' into dm-internal-248
DanielleMaywood Feb 4, 2025
7522b37
chore: update to match main
DanielleMaywood Feb 4, 2025
69c4f42
chore: run 'make gen'
DanielleMaywood Feb 4, 2025
44ebf65
chore: run 'make fmt'
DanielleMaywood Feb 5, 2025
714e743
chore: remove cruft
DanielleMaywood Feb 5, 2025
7cf5212
chore: align interface
DanielleMaywood Feb 5, 2025
d08e713
chore: add another test
DanielleMaywood Feb 6, 2025
ed42eae
chore: improve volume monitor test
DanielleMaywood Feb 6, 2025
1b0d0d2
chore: rename fields
DanielleMaywood Feb 11, 2025
51b16c6
Merge branch 'main' into dm-internal-248
DanielleMaywood Feb 11, 2025
4e43bab
chore: align with other branch
DanielleMaywood Feb 11, 2025
4e144ae
Merge branch 'main' into dm-internal-248
DanielleMaywood Feb 12, 2025
da25ecc
chore: bump migration number
DanielleMaywood Feb 12, 2025
fe1e805
chore: add test and align better
DanielleMaywood Feb 12, 2025
abbd522
chore: appease linter
DanielleMaywood Feb 12, 2025
456989e
Merge branch 'main' into dm-internal-248
DanielleMaywood Feb 14, 2025
1550cc6
chore: update rbac
DanielleMaywood Feb 14, 2025
7998f89
chore: handle missing datapoints
DanielleMaywood Feb 14, 2025
bda8f29
chore: add tests for unknown state on memory monitor
DanielleMaywood Feb 14, 2025
9d662a3
chore: add tests for missing datapoints in volume monitors
DanielleMaywood Feb 14, 2025
bff48dc
chore: add default debounce of 5 minutes
DanielleMaywood Feb 14, 2025
c343a70
chore: implement feedback
DanielleMaywood Feb 14, 2025
babc48f
chore: feedback
DanielleMaywood Feb 17, 2025
01ca549
chore: feedback
DanielleMaywood Feb 17, 2025
a975810
chore: forgot to run the linter
DanielleMaywood Feb 17, 2025
ee35d85
chore: use percentages for alert config
DanielleMaywood Feb 17, 2025
d2fa8df
Merge branch 'main' into dm-internal-248
DanielleMaywood Feb 17, 2025
27d78d1
chore: fmt and bump migration number
DanielleMaywood Feb 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
chore: rewrite ood notification
  • Loading branch information
DanielleMaywood committed Feb 4, 2025
commit 944fdb51e5864e147451e41006e08118876a46e1
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,16 @@ INSERT INTO notification_templates
VALUES (
'f047f6a3-5713-40f7-85aa-0394cce9fa3a',
'Workspace Out Of Disk',
E'Your workspace "{{.Labels.workspace}}" is low on disk',
E'Your workspace "{{.Labels.workspace}}" is low on volume space',
E'Hi {{.UserName}},\n\n'||
E'Your workspace **{{.Labels.workspace}}** has reached the usage threshold set at **{{.Labels.threshold}}** for volume `{{.Labels.volume}}`.',
E'{{ if eq (len .Data.volumes) 1 }}{{ $volume := index .Data.volumes 0 }}'||
E'Volume **`{{$volume.path}}`** is over {{$volume.threshold}} full in workspace **{{.Labels.workspace}}**.'||
E'{{ else }}'||
E'The following volumes are nearly full in workspace **{{.Labels.workspace}}**\n\n'||
E'{{ range $volume := .Data.volumes }}'||
E'- **`{{$volume.path}}`** is over {{$volume.threshold}} full\n'||
E'{{ end }}'||
E'{{ end }}',
'Workspace Events',
'[
{
Expand Down
43 changes: 39 additions & 4 deletions coderd/notifications/notifications_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1078,16 +1078,50 @@ func TestNotificationTemplates_Golden(t *testing.T) {
},
},
{
name: "TemplateWorkspaceOutOfDisk",
name: "TemplateWorkspaceOutOfDisk/SingleVolume",
id: notifications.TemplateWorkspaceOutOfDisk,
payload: types.MessagePayload{
UserName: "Bobby",
UserEmail: "bobby@coder.com",
UserUsername: "bobby",
Labels: map[string]string{
"workspace": "bobby-workspace",
"threshold": "90%",
"volume": "/home/coder",
},
Data: map[string]any{
"volumes": []map[string]any{
{
"path": "/home/coder",
"threshold": "90%",
},
},
},
},
},
{
name: "TemplateWorkspaceOutOfDisk/MultipleVolumes",
id: notifications.TemplateWorkspaceOutOfDisk,
payload: types.MessagePayload{
UserName: "Bobby",
UserEmail: "bobby@coder.com",
UserUsername: "bobby",
Labels: map[string]string{
"workspace": "bobby-workspace",
},
Data: map[string]any{
"volumes": []map[string]any{
{
"path": "/home/coder",
"threshold": "90%",
},
{
"path": "/dev/coder",
"threshold": "80%",
},
{
"path": "/etc/coder",
"threshold": "95%",
},
},
},
},
},
Expand All @@ -1099,7 +1133,8 @@ func TestNotificationTemplates_Golden(t *testing.T) {
for _, name := range allTemplates {
var found bool
for _, tc := range tests {
if tc.name == name {
tcName, _, _ := strings.Cut(tc.name, "/")
if tcName == name {
found = true
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
From: system@coder.com
To: bobby@coder.com
Subject: Your workspace "bobby-workspace" is low on disk
Subject: Your workspace "bobby-workspace" is low on volume space 8000
Message-Id: 02ee4935-73be-4fa1-a290-ff9999026b13@blush-whale-48
Date: Fri, 11 Oct 2024 09:03:06 +0000
Content-Type: multipart/alternative; boundary=bbe61b741255b6098bb6b3c1f41b885773df633cb18d2a3002b68e4bc9c4
Expand All @@ -12,8 +12,7 @@ Content-Type: text/plain; charset=UTF-8

Hi Bobby,

Your workspace bobby-workspace has reached the usage threshold set at 90% f=
or volume /home/coder.
Volume /home/coder is over 90% full in workspace bobby-workspace.


View workspace: http://test.com/@bobby/bobby-workspace
Expand All @@ -28,7 +27,7 @@ Content-Type: text/html; charset=UTF-8
<meta charset=3D"UTF-8" />
<meta name=3D"viewport" content=3D"width=3Ddevice-width, initial-scale=
=3D1.0" />
<title>Your workspace "bobby-workspace" is low on disk</title>
<title>Your workspace "bobby-workspace" is low on volume space</title>
</head>
<body style=3D"margin: 0; padding: 0; font-family: -apple-system, system-=
ui, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 'Ubuntu', 'Cantarel=
Expand All @@ -43,14 +42,13 @@ er Logo" style=3D"height: 40px;" />
</div>
<h1 style=3D"text-align: center; font-size: 24px; font-weight: 400; m=
argin: 8px 0 32px; line-height: 1.5;">
Your workspace "bobby-workspace" is low on disk
Your workspace "bobby-workspace" is low on volume space
</h1>
<div style=3D"line-height: 1.5;">
<p>Hi Bobby,</p>

<p>Your workspace <strong>bobby-workspace</strong> has reached the usage th=
reshold set at <strong>90%</strong> for volume <code>/home/coder</code>.</p=
>
<p>Volume <strong><code>/home/coder</code></strong> is over 90% full in wor=
kspace <strong>bobby-workspace</strong>.</p>
</div>
<div style=3D"text-align: center; margin-top: 32px;">
=20
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,19 @@
}
],
"labels": {
"threshold": "90%",
"volume": "/home/coder",
"workspace": "bobby-workspace"
},
"data": null
"data": {
"volumes": [
{
"path": "/home/coder",
"threshold": "90%"
}
]
}
},
"title": "Your workspace \"bobby-workspace\" is low on disk",
"title_markdown": "Your workspace \"bobby-workspace\" is low on disk",
"body": "Hi Bobby,\n\nYour workspace bobby-workspace has reached the usage threshold set at 90% for volume /home/coder.",
"body_markdown": "Hi Bobby,\n\nYour workspace **bobby-workspace** has reached the usage threshold set at **90%** for volume `/home/coder`."
"title": "Your workspace \"bobby-workspace\" is low on volume space",
"title_markdown": "Your workspace \"bobby-workspace\" is low on volume space",
"body": "Hi Bobby,\n\nVolume /home/coder is over 90% full in workspace bobby-workspace.",
"body_markdown": "Hi Bobby,\n\nVolume **`/home/coder`** is over 90% full in workspace **bobby-workspace**."
}
Loading
0