We have already outlined the new query parameters and fields we introduced along with some example scenarios. While that’s more than enough to get started, we thought it would be useful to demonstrate how, using our Go SDK, you would keep your Realtime API connections up to date and fault-tolerant.

package main

import (
	"context"
	"log"
	"os"

	"github.com/wikimedia-enterprise/wme-sdk-go/pkg/api"
	"github.com/wikimedia-enterprise/wme-sdk-go/pkg/auth"
)

func main() {
	// initialize the context and get the access token
	context := context.Background()
	authClient := auth.NewClient()

	// login to the API using the username and password
	loginResponse, err := authClient.Login(context, &auth.LoginRequest{
		Username: os.Getenv("WME_USERNAME"),
		Password: os.Getenv("WME_PASSWORD"),
	})

	if err != nil {
		log.Fatalln(err)
	}

	// revoke the token after we are done
	defer authClient.RevokeToken(context, &auth.RevokeTokenRequest{
		RefreshToken: loginResponse.RefreshToken,
	})

	// initialize the API client and set the access token
	apiClient := api.NewClient()
	apiClient.SetAccessToken(loginResponse.AccessToken)

	// create a request to get the Realtime API endpoint
	// we limit the request to only three fields and filter by language
	// also, we connect only to the first two parts of the stream
	// if you have another consumer you can just use this code to connect to the next set of parts ({5, 6, 7, 8, 9})
	// important part is to include the `event` field in the request
	apiRequest := &api.Request{
		Parts:  []int{0, 1, 2, 3, 4},
		Fields: []string{"name", "abstract", "event.*"},
		Filters: []*api.Filter{
			{
				Field: "in_language.identifier",
				Value: "en",
			},
		},
	}

	// list of offsets for each partition
	// this needs to stored in persistent storage and used to reconnect
	// there's a paramter called Offsets in the api.Request object
	offsets := map[int]int64{}

	// create a callback to read the articles
	// will be called every time you get a new article from the stream
	apiReadCallback := func(art *api.Article) error {
		println(art.Name)
		offsets[*art.Event.Partition] = *art.Event.Offset
		log.Println(offsets)
		log.Println("----------------------------------")
		return nil
	}

	// connect to the Realtime API and stream the articles
	// this is a blocking call
	err = apiClient.StreamArticles(context, apiRequest, apiReadCallback)

	if err != nil {
		log.Fatalln(err)
	}
}

The code above demonstrates how to extract the data needed to make your connection fault-tolerant using offsets of the messages you have consumed. To run it, you’ll need to have Go installed on your machine and have set WME_USERNAME and WME_PASSWORD from your Wikimedia Enterprise credentials.

As mentioned in the code comments, you would store those offsets somewhere in persistent storage and use them each time you reconnect to the stream to resume consumption exactly where you left off.

For more extensive examples and technical details, please see our SDKs (Go and Python) for more information.