8000 feat: implement thin vertical slice of system-generated notifications by dannykopping · Pull Request #13537 · coder/coder · GitHub
[go: up one dir, main page]

Skip to content

feat: implement thin vertical slice of system-generated notifications #13537

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
Jul 8, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
53c9cbb
feat: system-generated notifications
dannykopping Jun 11, 2024
4856aed
Fixing lint errors & minor tests
dannykopping Jun 11, 2024
cda6efb
Fixing dbauthz test
dannykopping Jun 11, 2024
86f937a
TestBufferedUpdates does not need a real db, altering test details sl…
dannykopping Jun 11, 2024
e8f1af2
Correct TestBufferedUpdates to count updated entries, use real db again
dannykopping Jun 12, 2024
a056f54
Use UUID for notifier IDs
dannykopping Jun 27, 2024
8c64d30
Small improvements from review suggestions
dannykopping Jun 27, 2024
ac149ec
Protect notifiers from modification during Stop()
dannykopping Jun 27, 2024
884fadf
Split out enqueuer as separate responsibility, get rid of singleton
dannykopping Jun 28, 2024
4e362e7
Remove unnecessary handler registry
dannykopping Jun 28, 2024
8097290
Remove unused context
dannykopping Jun 28, 2024
1b841ad
Centralise markdown rendering
dannykopping Jun 28, 2024
61f5bd6
Appease the linter
dannykopping Jun 28, 2024
3c8e33b
Only enqueue notification when not initiated by self
dannykopping Jul 1, 2024
757327c
Hide config flags which are unlikely to be modified by operators
dannykopping Jul 1, 2024
6f909ae
Remove unnecessary Labels struct
dannykopping Jul 1, 2024
36698c5
Enable experiment as safe
dannykopping Jul 1, 2024
c5701a6
Correcting bad refactor
dannykopping Jul 1, 2024
9d4c312
Initialize Enqueuer on API startup
dannykopping Jul 1, 2024
9380d8e
Only start one notifier since all dispatches are concurrent anyway
dannykopping Jul 1, 2024
4b7214d
Fix docs
dannykopping Jul 1, 2024
6679ef1
Fix lint error
dannykopping Jul 1, 2024
337997d
Merge branch 'main' of github.com:/coder/coder into dk/system-notific…
dannykopping Jul 2, 2024
ba5f7c6
Merge branch 'main' of github.com:/coder/coder into dk/system-notific…
dannykopping Jul 3, 2024
0f29293
Review feedback
dannykopping Jul 3, 2024
7c6c486
Merge branch 'main' of github.com:/coder/coder into dk/system-notific…
dannykopping Jul 4, 2024
c6e75c2
Fix lint failures
dannykopping Jul 4, 2024
aff9e6c
Review comments
dannykopping Jul 4, 2024
613e074
Avoid race by exposing number of pending updates
dannykopping Jul 4, 2024
faea7fc
Merge branch 'main' of github.com:/coder/coder into dk/system-notific…
dannykopping Jul 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fixing lint errors & minor tests
Signed-off-by: Danny Kopping <danny@coder.com>
  • Loading branch information
dannykopping committed Jun 28, 2024
commit 4856aed2c5ccd8fc91daa21ed8763022b1742e31
62 changes: 62 additions & 0 deletions cli/testdata/coder_server_--help.golden
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,68 @@ can safely ignore these settings.
Minimum supported version of TLS. Accepted values are "tls10",
"tls11", "tls12" or "tls13".

NOTIFICATIONS OPTIONS:
--notifications-dispatch-timeout duration, $CODER_NOTIFICATIONS_DISPATCH_TIMEOUT (default: 1m0s)
How long to wait while a notification is being sent before giving up.

--notifications-fetch-interval duration, $CODER_NOTIFICATIONS_FETCH_INTERVAL (default: 15s)
How often to query the database for queued notifications.

--notifications-lease-count int, $CODER_NOTIFICATIONS_LEASE_COUNT (default: 10)
How many notifications a notifier should lease per fetch interval.

--notifications-lease-period duration, $CODER_NOTIFICATIONS_LEASE_PERIOD (default: 2m0s)
How long a notifier should lease a message. This is effectively how
long a notification is 'owned' by a notifier, and once this period
expires it will be available for lease by another notifier. Leasing is
important in order for multiple running notifiers to not pick the same
messages to deliver concurrently. This lease period will only expire
if a notifier shuts down ungracefully; a dispatch of the notification
releases the lease.

--notifications-max-send-attempts int, $CODER_NOTIFICATIONS_MAX_SEND_ATTEMPTS (default: 5)
The upper limit of attempts to send a notification.

--notifications-method string, $CODER_NOTIFICATIONS_METHOD (default: smtp)
Which delivery method to use (available options: 'smtp', 'webhook').

--notifications-retry-interval duration, $CODER_NOTIFICATIONS_RETRY_INTERVAL (default: 5m0s)
The minimum time between retries.

--notifications-store-sync-buffer-size int, $CODER_NOTIFICATIONS_STORE_SYNC_BUFFER_SIZE (default: 50)
The notifications system buffers message updates in memory to ease
pressure on the database. This option controls how many updates are
kept in memory. The lower this value the lower the change of state
inconsistency in a non-graceful shutdown - but it also increases load
on the database. It is recommended to keep this option at its default
value.

--notifications-store-sync-interval duration, $CODER_NOTIFICATIONS_STORE_SYNC_INTERVAL (default: 2s)
The notifications system buffers message updates in memory to ease
pressure on the database. This option controls how often it
synchronizes its state with the database. The shorter this value the
lower the change of state inconsistency in a non-graceful shutdown -
but it also increases load on the database. It is recommended to keep
this option at its default value.

--notifications-worker-count int, $CODER_NOTIFICATIONS_WORKER_COUNT (default: 2)
How many workers should be processing messages in the queue; increase
this count if notifications are not being processed fast enough.

NOTIFICATIONS / EMAIL OPTIONS:
--notifications-email-from string, $CODER_NOTIFICATIONS_EMAIL_FROM
The sender's address to use.

--notifications-email-hello string, $CODER_NOTIFICATIONS_EMAIL_HELLO (default: localhost)
The hostname identifying the SMTP server.

--notifications-email-smarthost host:port, $CODER_NOTIFICATIONS_EMAIL_SMARTHOST (default: localhost:587)
The intermediary SMTP host through which emails are sent.

NOTIFICATIONS / WEBHOOK OPTIONS:
--notifications-webhook-endpoint url, $CODER_NOTIFICATIONS_WEBHOOK_ENDPOINT
The endpoint to which to send webhooks.

OAUTH2 / GITHUB OPTIONS:
--oauth2-github-allow-everyone bool, $CODER_OAUTH2_GITHUB_ALLOW_EVERYONE
Allow all logins, setting this option means allowed orgs and teams
Expand Down
59 changes: 59 additions & 0 deletions cli/testdata/server-config.yaml.golden
Original file line number Diff line number Diff line change
Expand Up @@ -493,3 +493,62 @@ userQuietHoursSchedule:
# compatibility reasons, this will be removed in a future release.
# (default: false, type: bool)
allowWorkspaceRenames: false
notifications:
# The upper limit of attempts to send a notification.
# (default: 5, type: int)
max-send-attempts: 5
# The minimum time between retries.
# (default: 5m0s, type: duration)
retry-interval: 5m0s
# The notifications system buffers message updates in memory to ease pressure on
# the database. This option controls how often it synchronizes its state with the
# database. The shorter this value the lower the change of state inconsistency in
# a non-graceful shutdown - but it also increases load on the database. It is
# recommended to keep this option at its default value.
# (default: 2s, type: duration)
store-sync-interval: 2s
# The notifications system buffers message updates in memory to ease pressure on
# the database. This option controls how many updates are kept in memory. The
# lower this value the lower the change of state inconsistency in a non-graceful
# shutdown - but it also increases load on the database. It is recommended to keep
# this option at its default value.
# (default: 50, type: int)
store-sync-buffer-size: 50
# How many workers should be processing messages in the queue; increase this count
# if notifications are not being processed fast enough.
# (default: 2, type: int)
worker-count: 2
# How long a notifier should lease a message. This is effectively how long a
# notification is 'owned' by a notifier, and once this period expires it will be
# available for lease by another notifier. Leasing is important in order for
# multiple running notifiers to not pick the same messages to deliver
# concurrently. This lease period will only expire if a notifier shuts down
# ungracefully; a dispatch of the notification releases the lease.
# (default: 2m0s, type: duration)
lease-period: 2m0s
# How many notifications a notifier should lease per fetch interval.
# (default: 10, type: int)
lease-count: 10
# How often to query the database for queued notifications.
# (default: 15s, type: duration)
fetch-interval: 15s
# Which delivery method to use (available options: 'smtp', 'webhook').
# (default: smtp, type: string)
method: smtp
# How long to wait while a notification is being sent before giving up.
# (default: 1m0s, type: duration)
dispatch-timeout: 1m0s
email:
# The sender's address to use.
# (default: <unset>, type: string)
from: ""
# The intermediary SMTP host through which emails are sent.
# (default: localhost:587, type: host:port)
smarthost: localhost:587
# The hostname identifying the SMTP server.
# (default: localhost, type: string)
hello: localhost
webhook:
# The endpoint to which to send webhooks.
# (default: <unset>, type: url)
hello:
3 changes: 2 additions & 1 deletion coderd/notifications/dispatch/webhook.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"net/http"

Expand Down Expand Up @@ -95,7 +96,7 @@ func (w *WebhookHandler) dispatch(msgPayload types.MessagePayload, title, body,
// Body could be quite long here, let's grab the first 500B and hope it contains useful debug info.
var respBody []byte
respBody, err = abbreviatedRead(resp.Body, 500)
if err != nil && err != io.EOF {
if err != nil && !errors.Is(err, io.EOF) {
return true, xerrors.Errorf("non-200 response (%d), read body: %w", resp.StatusCode, err)
}
w.log.Warn(ctx, "unsuccessful delivery", slog.F("status_code", resp.StatusCode),
Expand Down
7 changes: 2 additions & 5 deletions coderd/notifications/events.go
3419
Original file line number Diff line numberDiff line change
Expand Up @@ -2,8 +2,5 @@ package notifications

import "github.com/google/uuid"

var (
// Workspaces.
TemplateWorkspaceDeleted = uuid.MustParse("'f517da0b-cdc9-410f-ab89-a86107c420ed'")
// ...
)
// Workspaces.
var TemplateWorkspaceDeleted = uuid.MustParse("'f517da0b-cdc9-410f-ab89-a86107c420ed'") // ...
4 changes: 3 additions & 1 deletion coderd/notifications/manager_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,8 @@ func TestBufferedUpdates(t *testing.T) {
}

func TestBuildPayload(t *testing.T) {
t.Parallel()

// given
const label = "Click here!"
const url = "http://xyz.com/"
Expand Down Expand Up @@ -198,6 +200,6 @@ func (e *enqueueInterceptor) EnqueueNotificationMessage(_ context.Context, arg d
return database.NotificationMessage{}, err
}

func (e *enqueueInterceptor) FetchNewMessageMetadata(_ context.Context, arg database.FetchNewMessageMetadataParams) (database.FetchNewMessageMetadataRow, error) {
func (e *enqueueInterceptor) FetchNewMessageMetadata(_ context.Context, _ database.FetchNewMessageMetadataParams) (database.FetchNewMessageMetadataRow, error) {
return e.metadataFn(), nil
}
4 changes: 1 addition & 3 deletions coderd/notifications/notifier.go
Original file line number Diff line number Diff line change
Expand Up @@ -170,9 +170,7 @@ func (n *notifier) prepare(ctx context.Context, msg database.AcquireNotification
return nil, xerrors.Errorf("resolve handler: %w", err)
}

var (
title, body string
)
var title, body string
if title, err = render.GoTemplate(msg.TitleTemplate, payload, nil); err != nil {
return nil, xerrors.Errorf("render title: %w", err)
}
Expand Down
8 changes: 6 additions & 2 deletions coderd/notifications/system/system_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,20 @@ import (
"testing"
"time"

"github.com/google/uuid"
"github.com/stretchr/testify/require"

"github.com/coder/coder/v2/coderd/notifications"
"github.com/coder/coder/v2/coderd/notifications/system"
"github.com/coder/coder/v2/coderd/notifications/types"
"github.com/coder/coder/v2/testutil"
"github.com/google/uuid"
"github.com/stretchr/testify/require"
)

// TestNotifyWorkspaceDeleted tests the "public" interface for enqueueing notifications.
// Calling system.NotifyWorkspaceDeleted uses the Enqueuer singleton to enqueue the notification.
func TestNotifyWorkspaceDeleted(t *testing.T) {
t.Parallel()

// given
manager := newFakeEnqueuer()
notifications.RegisterInstance(manager)
Expand All @@ -42,5 +45,6 @@ func newFakeEnqueuer() *fakeEnqueuer {

func (f *fakeEnqueuer) Enqueue(context.Context, uuid.UUID, uuid.UUID, types.Labels, string, ...uuid.UUID) (*uuid.UUID, error) {
f.enqueued <- true
// nolint:nilnil // Irrelevant.
return nil, nil
}
30 changes: 18 additions & 12 deletions codersdk/deployment.go
Original file line number Diff line number Diff line change
Expand Up @@ -2104,19 +2104,21 @@ Write out the current server config as YAML to stdout.`,
Default: (time.Minute * 5).String(),
Group: &deploymentGroupNotifications,
YAML: "retry-interval",
Annotations: serpent.Annotations{}.Mark(annotationFormatDuration, "true"),
},
{
Name: "Notifications: Store Sync Interval",
Description: "The notifications system buffers message updates in memory to ease pressure on the database. " +
"This option controls how often it synchronizes its state with the database. The shorter this value the " +
"lower the change of state inconsistency in a non-graceful shutdown - but it also increases load on the " +
"database. It is recommended to keep this option at its default value.",
Flag: "notifications-store-sync-interval",
Env: "CODER_NOTIFICATIONS_STORE_SYNC_INTERVAL",
Value: &c.Notifications.StoreSyncInterval,
Default: (time.Second * 2).String(),
Group: &deploymentGroupNotifications,
YAML: "store-sync-interval",
Flag: "notifications-store-sync-interval",
Env: "CODER_NOTIFICATIONS_STORE_SYNC_INTERVAL",
Value: &c.Notifications.StoreSyncInterval,
Default: (time.Second * 2).String(),
Group: &deploymentGroupNotifications,
YAML: "store-sync-interval",
Annotations: serpent.Annotations{}.Mark(annotationFormatDuration, "true"),
},
{
Name: "Notifications: Store Sync Buffer Size",
Expand Down Expand Up @@ -2149,12 +2151,13 @@ Write out the current server config as YAML to stdout.`,
"is important in order for multiple running notifiers to not pick the same messages to deliver concurrently. " +
"This lease period will only expire if a notifier shuts down ungracefully; a dispatch of the notification " +
"releases the lease.",
Flag: "notifications-lease-period",
Env: "CODER_NOTIFICATIONS_LEASE_PERIOD",
Value: &c.Notifications.LeasePeriod,
Default: (time.Minute * 2).String(),
Group: &deploymentGroupNotifications,
YAML: "lease-period",
Flag: "notifications-lease-period",
Env: "CODER_NOTIFICATIONS_LEASE_PERIOD",
Value: &c.Notifications.LeasePeriod,
Default: (time.Minute * 2).String(),
Group: &deploymentGroupNotifications,
YAML: "lease-period",
Annotations: serpent.Annotations{}.Mark(annotationFormatDuration, "true"),
},
{
Name: "Notifications: Lease Count",
Expand All @@ -2175,6 +2178,7 @@ Write out the current server config as YAML to stdout.`,
Default: (time.Second * 15).String(),
Group: &deploymentGroupNotifications,
YAML: "fetch-interval",
Annotations: serpent.Annotations{}.Mark(annotationFormatDuration, "true"),
},
{
Name: "Notifications: Method",
Expand All @@ -2195,6 +2199,7 @@ Write out the current server config as YAML to stdout.`,
Default: time.Minute.String(),
Group: &deploymentGroupNotifications,
YAML: "dispatch-timeout",
Annotations: serpent.Annotations{}.Mark(annotationFormatDuration, "true"),
},
{
Name: "Notifications: Email: From Address",
Expand All @@ -2210,6 +2215,7 @@ Write out the current server config as YAML to stdout.`,
Description: "The intermediary SMTP host through which emails are sent.",
Flag: "notifications-email-smarthost",
Env: "CODER_NOTIFICATIONS_EMAIL_SMARTHOST",
Default: "localhost:587", // To pass validation.
Value: &c.Notifications.SMTP.Smarthost,
Group: &deploymentGroupNotificationsEmail,
YAML: "smarthost",
Expand Down
1 change: 1 addition & 0 deletions docs/cli/server.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

62 changes: 62 additions & 0 deletions enterprise/cli/testdata/coder_server_--help.golden
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,68 @@ can safely ignore these settings.
Minimum supported version of TLS. Accepted values are "tls10",
"tls11", "tls12" or "tls13".

NOTIFICATIONS OPTIONS:
--notifications-dispatch-timeout duration, $CODER_NOTIFICATIONS_DISPATCH_TIMEOUT (default: 1m0s)
How long to wait while a notification is being sent before giving up.

--notifications-fetch-interval duration, $CODER_NOTIFICATIONS_FETCH_INTERVAL (default: 15s)
How often to query the database for queued notifications.

--notifications-lease-count int, $CODER_NOTIFICATIONS_LEASE_COUNT (default: 10)
How many notifications a notifier should lease per fetch interval.

--notifications-lease-period duration, $CODER_NOTIFICATIONS_LEASE_PERIOD (default: 2m0s)
How long a notifier should lease a message. This is effectively how
long a notification is 'owned' by a notifier, and once this period
expires it will be available for lease by another notifier. Leasing is
important in order for multiple running notifiers to not pick the same
messages to deliver concurrently. This lease period will only expire
if a notifier shuts down ungracefully; a dispatch of the notification
releases the lease.

--notifications-max-send-attempts int, $CODER_NOTIFICATIONS_MAX_SEND_ATTEMPTS (default: 5)
The upper limit of attempts to send a notification.

--notifications-method string, $CODER_NOTIFICATIONS_METHOD (default: smtp)
Which delivery method to use (available options: 'smtp', 'webhook').

--notifications-retry-interval duration, $CODER_NOTIFICATIONS_RETRY_INTERVAL (default: 5m0s)
The minimum time between retries.

--notifications-store-sync-buffer-size int, $CODER_NOTIFICATIONS_STORE_SYNC_BUFFER_SIZE (default: 50)
The notifications system buffers message updates in memory to ease
pressure on the database. This option controls how many updates are
kept in memory. The lower this value the lower the change of state
inconsistency in a non-graceful shutdown - but it also increases load
on the database. It is recommended to keep this option at its default
value.

--notifications-store-sync-interval duration, $CODER_NOTIFICATIONS_STORE_SYNC_INTERVAL (default: 2s)
The notifications system buffers message updates in memory to ease
pressure on the database. This option controls how often it
synchronizes its state with the database. The shorter this value the
lower the change of state inconsistency in a non-graceful shutdown -
but it also increases load on the database. It is recommended to keep
this option at its default value.

--notifications-worker-count int, $CODER_NOTIFICATIONS_WORKER_COUNT (default: 2)
How many workers should be processing messages in the queue; increase
this count if notifications are not being processed fast enough.

NOTIFICATIONS / EMAIL OPTIONS:
--notifications-email-from string, $CODER_NOTIFICATIONS_EMAIL_FROM
The sender's address to use.

--notifications-email-hello string, $CODER_NOTIFICATIONS_EMAIL_HELLO (default: localhost)
The hostname identifying the SMTP server.

--notifications-email-smarthost host:port, $CODER_NOTIFICATIONS_EMAIL_SMARTHOST (default: localhost:587)
The intermediary SMTP host through which emails are sent.

NOTIFICATIONS / WEBHOOK OPTIONS:
--notifications-webhook-endpoint url, $CODER_NOTIFICATIONS_WEBHOOK_ENDPOINT
The endpoint to which to send webhooks.

OAUTH2 / GITHUB OPTIONS:
--oauth2-github-allow-everyone bool, $CODER_OAUTH2_GITHUB_ALLOW_EVERYONE
Allow all logins, setting this option means allowed orgs and teams
Expand Down
0