The Beam Playground is a web application to run Beam code snippets in a modern browser. This directory holds code to build, test, and deploy the frontend and backend services.

Setup development prerequisites
Common tasks
Run Beam Playground locally
How to add your own example
Deployment guide
- Manual deployment
- Manual CloudBuild setup
Project structure
Contribution guide

Setup development prerequisites

Google Cloud Shell note: Google Cloud Shell already has most of the prerequisites installed. Only the following has to be installed manually:

Flutter - run flutter precache

Go protobuf dependencies

Dart protobuf dependencies

buf

sbt

Install Go 1.23+

Ubuntu 22.04 and newer:
```
sudo apt install golang
```
Other Linux variants: Follow manual at https://go.dev/doc/install
Install flutter

Ubuntu 22.04 or newer:
```
sudo apt install flutter
```
Other Linux variants: Follow manual at https://flutter.dev/docs/get-started/install/linux
Install protoc

Ubuntu 22.04 or newer/Debian 11 or newer:
```
sudo apt install protobuf-compiler
```
Other Linux variants: Follow manual at https://grpc.io/docs/protoc-installation/
Install Go protobuf dependencies: Go gRPC Quickstart

Do not forget to update your PATH environment variable
Install Dart protobuf dependencies: Dart gRPC Quickstart

Do not forget to update your PATH environment variable
Install npm Ubuntu 22.04 and newer/Debian 11 and newer:
```
sudo apt install npm
```
Other Linux variants: Follow manual at https://docs.npmjs.com/downloading-and-installing-node-js-and-npm
Install buf
```
npm install -g @bufbuild/buf
```
Install Docker Ubuntu 22.04 and newer/Debian 11 and newer:
```
sudo apt install docker.io
```
Other Linux variants: Follow manual at https://docs.docker.com/engine/install/

To verify your docker installation, run:
```
docker ps
```
It should finish without any errors. If you get a permission denied error, you need to add your user to the docker group:
```
sudo usermod -aG docker $USER
```
Then, log out and log back in to apply the changes.
Install Docker Compose Ubuntu 22.04 and newer/Debian 11 and newer:
```
sudo apt install docker-compose
```
Other Linux variants: Follow manual at https://docs.docker.com/compose/install/
Install gcloud CLI by following the manual for your system

Install Cloud Datastore Emulator

gcloud components install cloud-datastore-emulator

or, if you have install gcloud CLI using APT:

sudo apt install google-cloud-cli-datastore-emulator

Install sbt following instruction at https://www.scala-sbt.org/1.x/docs/Installing-sbt-on-Linux.html

Optional: Run sbt comamnd once after installation to let it cache its dependencies
Make sure that you have the full installation of Python 3.8 or newer. On Debian or Ubuntu it can be installed using sudo apt install python3-full.

Common tasks

To get an overview of common tasks, see TASKS.md

Run Beam Playground locally

Configure frontend to use local backend

Note: Follow this step only if you want to have a local deployment of Playground. Skip this step entirely if you want to deploy Playground to Google Cloud.

Uncommend lines after // Uncomment the following lines to use local backend. in frontend/playground_components/lib/src/constants/backend_urls.dart

Run local deployment using Gradle task

For more information read the corresponding section in TASKS.md

Run the deployment script:

./gradlew :playground:dockerComposeLocalUp

To shut down the playground, run:

./gradlew :playground:dockerComposeLocalDown

Deploy examples

Go to /playground/infrastucture directory
```
cd playground/infrastucture
```
Setup python venv
```
python3 -m venv venv
```
Activate venv
```
source venv/bin/activate
```
Install dependencies
```
pip install -r requirements.txt
```

Setup the environment variables

export BEAM_ROOT_DIR=$(realpath ../../)
export SDK_CONFIG="../../playground/sdks.yaml"
export BEAM_EXAMPLE_CATEGORIES="../categories.yaml"
export BEAM_USE_WEBGRPC=yes
export BEAM_CONCURRENCY=4
export DATASTORE_EMULATOR_HOST=localhost:8081

Run the ci_cd.py script

export SERVER_ADDRESS=<runner_address> # see the note below
python ci_cd.py --step CD \
                --sdk <SDK> \ # SDK_GO, SDK_JAVA, SDK_PYTHON, SDK_SCIO
                --namespace Playground \
                --datastore-project test \
                --origin PG_EXAMPLES \
                --subdirs $BEAM_ROOT_DIR/sdks $BEAM_ROOT_DIR/examples $BEAM_ROOT_DIR/learning/katas # For SCIO examples see note below

Note: The SERVER_ADDRESS variable should be set to the address of the runner server for the particular SDK. For the local deployment the default values are:

SDK Address

Go localhost:8084

Java localhost:8086

Python localhost:8088

SCIO localhost:8090

Note: SCIO examples are not committed to the Beam tree. You will need to use fetch_scala_examples.py script to download them to some directory and then use --subdirs option to point to that directory. For example:
python fetch_scala_examples.py --output-dir /tmp/scio-examples
python ci_cd.py --step CD \
                --sdk SDK_SCIO \
                --namespace Playground \
                --datastore-project test \
                --origin PG_EXAMPLES \
                --subdirs /tmp/scio-examples
See this section for more information.

How to add your own example

Please refer to this document.

Deployment guide

Manual deployment

See deployment guide at terraform/README.md

Manual CloudBuild setup

To set up CloudBuild triggers manually please refer to this guide

Project structure

Several directories in this repository are used for the Beam Playground project. The list of the directories used can be seen below.

Directory	Purpose
`/examples`	Contains code of the examples for Java SDK. The examples are getting loaded into the main catalog.
`/learning/beamdoc`	Contains code of the examples which should not be available in the main Playground catalog.
`/learning/katas`	Containes small code examples, loaded into main catalog.
`/playground`	Root of Playground sources.
`/playground/api`	Protobuf definitions for the Playground gRPC API.
`/playground/backend`	Root of Playground backend sources. See this document for detailed description.
`/playground/frontend`	Root of Playground frontend sources. See this document for detailed description.
`/playground/infrastructure`	Scripts used in Playground CI/CD pipelines for verifying and uploading examples.
`/playground/kafka-emulator`	Sources of a Kafka emulator used for demonstrating KafkaIO examples.
`/playground/terraform`	Terraform configuration files for Playground deployment to Google Cloud Platform.
`/sdks`	Source of the BEAM SDKs. Used by Playground as a source of examples for Go and Python SDKs.

Contribution guide

Backend: see backend/README.md and backend/CONTRIBUTE.md
Frontend: see frontend/README.md and frontend/CONTRIBUTE.md