8000 add meshage to monorepo · github1/meshage@1afd996 · GitHub
[go: up one dir, main page]
Skip to content

Commit 1afd996

Browse files
github1github1
github1
authored and
github1
committed
add meshage to monorepo
0 parents  commit 1afd996

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

46 files changed

+9736
-0
lines changed

‎README.md

Lines changed: 257 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
# @github1/meshage
2+
3+
A simple service mesh. Messages sent within the service mesh can be consistently partitioned across members of the cluster.
4+
5+
[![build status](https://img.shields.io/travis/github1/meshage/master.svg?style=flat-square)](https://travis-ci.org/github1/meshage)
6+
[![npm version](https://img.shields.io/npm/v/meshage.svg?style=flat-square)](https://www.npmjs.com/package/meshage)
7+
[![npm downloads](https://img.shields.io/npm/dm/meshage.svg?style=flat-square)](https://www.npmjs.com/package/meshage)
8+
9+
## Install
10+
11+
```shell
12+
npm install meshage --save
13+
```
14+
15+
## Usage
16+
17+
Initialize a node:
18+
19+
```javascript
20+
const meshage = require('meshage');
21+
meshage
22+
.init(
23+
// Initialize the cluster to join (Grapevine or Consul)
24+
new meshage.GrapevineCluster(
25+
process.env.CLUSTER_PORT,
26+
(process.env.SEEDS || '').split(',')),
27+
process.env.HTTP_PORT
28+
)
29+
.register('echo', message => {
30+
// Register a message handler on the 'echo' stream
31+
return { echoed: message };
32+
})
33+
.start();
34+
```
35+
36+
Start one or more instances:
37+
38+
```shell
39+
CLUSTER_PORT=9742 SERVICE_PORT=8080 node index.js
40+
CLUSTER_PORT=9743 SEEDS=127.0.0.1:9742 HTTP_PORT=8081 node index.js
41+
```
42+
43+
Each node exposes an HTTP endpoint which accepts messages for registered streams. When a request is received by any instance registered to the cluster, a consistent hashing algorithm is used to determine which node should handle the request. If the node which receives the initial HTTP request is the designated handler it will respond directly, otherwise the receiving node will route the request to the designated node within the cluster.
44+
45+
*Request:*
46+
47+
```shell
48+
curl -sX POST http://localhost:8080/api/echo/$RANDOM \
49+
-H 'Content-Type: application/json' \
50+
-d '{"hello":"world"}'
51+
```
52+
53+
*Response:*
54+
55+
```json
56+
{
57+
"echoed": {
58+
"hello": "world"
59+
}
60+
}
61+
```
62+
63+
## HTTP API
64+
65+
### Send
66+
67+
Sends a message to be handled consistently by a registered handler for the specified stream.
68+
69+
**URL** : `/api/:stream/:partitionKey`
70+
71+
**URL Parameters** :
72+
- `stream` - the logical name for the message handler.
73+
- `partitionKey` - the identifier for the `entity` receiving the message.
74+
75+
**Example** :
76+
77+
*Request:*
78+
79+
```shell
80+
curl -sX POST http://localhost:8080/api/echo/$RANDOM \
81+
-H 'Content-Type: application/json' \
82+
-d '{"hello":"world"}'
83+
```
84+
85+
*Response:*
86+
87+
```json
88+
{
89+
"echoed": {
90+
"hello": "world"
91+
}
92+
}
93+
```
94+
95+
### Broadcast
96+
97+
Sends a message to all registered handlers for the specified stream.
98+
99+
**URL** : `/api/broadcast/:stream/:partitionKey`
100+
101+
**URL Parameters** :
102+
- `stream` - the logical name for the message handler.
103+
- `partitionKey` - the identifier for the `entity` receiving the message.
104+
105+
**Example** :
106+
107+
*Request:*
108+
109+
```shell
110+
curl -sX POST http://localhost:8080/api/broadcast/echo/$RANDOM \
111+
-H 'Content-Type: application/json' \
112+
-d '{"hello":"world"}'
113+
```
114+
115+
*Response:*
116+
117+
```json
118+
[
119+
{
120+
"echoed": {
121+
"hello": "world"
122+
}
123+
},
124+
{
125+
"echoed": {
126+
"hello": "world"
127+
}
128+
}
129+
]
130+
```
131+
132+
## JS API
133+
134+
### Init
135+
Configure the cluster to join.
136+
137+
**init(cluster : Cluster) : MessageRouter**
138+
- `cluster` - an instance of `Cluster` which is responsible for advertising and discovering message handlers.
139+
- `address` - (optional) an *host:port* pair (string) or simply a numeric port (number) to listen for HTTP requests on
140+
141+
```javascript
142+
const node = meshage.init(cluster, 8080);
143+
```
144+
145+
#### *Cluster Implementations:*
146+
147+
#### GrapevineCluster
148+
149+
Leverages an implementation of the Gossip protocol to discover nodes and services.
150+
151+
**GrapevineCluster(address : (string | number), seeds : (string | number)[])**
152+
- `address` - accepts a *host:port* pair (string) or simply a numeric port (number). If only a port is provided, the host defaults to `127.0.0.1`.
153+
- `seeds` - accepts an array of `address` values (following the same behavior as the *address* argument).
154+
155+
```javascript
156+
// The initial node in the cluster will not have seeds
157+
new meshage.GrapevineCluster(9473);
158+
// Subsequent nodes in the cluster need to specify at least one existing node as a seed
159+
new meshage.GrapevineCluster(9474, [9473]);
160+
```
161+
162+
#### ConsulCluster
163+
164+
Connects to a consul agent/cluster for service registration.
165+
166+
**ConsulCluster(address : (string | number), seeds : (string | number)[])**
167+
- `address` - an *host:port* pair (string) or simply a numeric port (number). *The cluster address should point to the associated consul agents HTTP API (typically port 8500)*.
168+
- `seeds` - an array of `address` values (following the same behavior as the *address* argument). *The seed address should be point to a consul agents serf_lan port (typically port 8301)*.
169+
170+
```javascript
171+
new meshage.ConsulCluster('127.0.0.1:8500');
172+
```
173+
174+
#### Custom Implementations
175+
176+
Custom cluster types may be provided by implementing the `core/cluster/Cluster` interface.
177+
178+
## Register
179+
180+
Registers a message handler on the node.
181+
182+
**register(stream : string, handler : (message : Message) => any) : MessageRouter**
183+
- `stream` - the stream name to accept messages for.
184+
- `handler` - the message handler function.
185+
186+
```javascript
187+
node.register('someStream', message => {
188+
return {};
189+
});
190+
```
191+
192+
## Start
193+
194+
Joins the cluster and begins advertising the nodes message handlers.
195+
196+
**start(callback : (router : ConnectedMessageRouter) => void)**
197+
- `callback` - (optional) accepts a callback function which is provided a router instance. The router instance can be used to send or broadcast messages to nodes in the cluster.
198+
199+
```javascript
200+
node.start(router => {
201+
router
202+
.send({ stream: 'echo', partitionKey: 'c6c5e7f3-6228-41ce-a7ea-23ac24a08a32', data: 'hello' })
203+
.then(res => {
204+
console.log(res);
205+
});
206+
});
207+
```
208+
209+
The `router` instance passed to the `start` callback exposes two methods:
210+
211+
### Send
212+
213+
Sends a message to be handled consistently by a registered handler for the specified stream. Depending on how the message is routed, it could be handled by the node itself.
214+
215+
**send(message : Message) : Promise<{}>**
216+
- `message` - the message to send
217+
218+
### Broadcast
219+
220+
Sends a message to all registered handlers for the specified stream.
221+
222+
**broadcast(message : Message) : Promise<{}>**
223+
- `message` - the message to send
224+
225+
## Address Formats
226+
227+
Address may be supplied in the following formats:
228+
229+
### Host and port string
230+
231+
The host and port separated by a colon.
232+
233+
_Example_
234+
235+
`localhost:8080`
236+
237+
### Port number
238+
239+
Just the port (as a number or string). If no explicit hostname is provided, `os.hostname()` is used to determine the host.
240+
241+
_Example_
242+
243+
`8080`
244+
245+
### Finding open ports
246+
247+
By suffixing the address with the keyword `find`, the library will attempt to find an open port to listen on.
248+
249+
_Example_
250+
251+
- `localhost:find` - use localhost, but find an open port
252+
- `localhost:8080/find` - use localhost and port 8080 if available, otherwise find an open port
253+
- `8080/find` - use port 8080 if available, otherwise find an open port
254+
- `find` - find any open port
255+
256+
## License
257+
[MIT](LICENSE.md)

‎example/.dockerignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
node_modules
2+
dist

‎example/Dockerfile

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
FROM node:latest
2+
3+
WORKDIR /opt/app
4+
5+
ADD . /opt/app
6+
7+
RUN npm install --unsafe-perm --production
8+
9+
CMD ["bash","-c","if [ -d /meshage ]; then rm -rf node_modules/meshage/dist; cp -R /meshage/dist node_modules/meshage/dist; fi; node index.js"]

‎example/docker-compose.consul.yml

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
version: '3'
2+
services:
3+
consul1:
4+
image: consul:latest
5+
hostname: 'consul1'
6+
command: 'agent -server -bootstrap-expect 1 -client 0.0.0.0'
7+
consul2:
8+
image: consul:latest
9+
hostname: 'consul2'
10+
command: 'agent -join consul1 -client 0.0.0.0'
11+
depends_on:
12+
- consul1
13+
consul3:
14+
image: consul:latest
15+
hostname: 'consul3'
16+
command: 'agent -join consul1 -client 0.0.0.0'
17+
depends_on:
18+
- consul1
19+
node-a:
20+
build: ./
21+
ports:
22+
- '8080:8080'
23+
depends_on:
24+
- consul2
25+
environment:
26+
- 'SERVICE_PORT=8080'
27+
- 'CLUSTER_TYPE=consul'
28+
- 'CLUSTER_HOST=consul2'
29+
- 'CLUSTER_PORT=8500'
30+
- 'LIB=meshage'
31+
- 'SEED=consul1:8301'
32+
- 'DEBUG=meshage,meshage:*'
33+
- 'DELAY_STARTUP_MS=2000'
34+
volumes:
35+
- '../:/meshage:ro'
36+
node-b:
37+
build: ./
38+
depends_on:
39+
- consul3
40+
environment:
41+
- 'CLUSTER_TYPE=consul'
42+
- 'CLUSTER_HOST=consul3'
43+
- 'CLUSTER_PORT=8500'
44+
- 'LIB=meshage'
45+
- 'SEED=consul1:8301'
46+
- 'DEBUG=meshage,meshage:*'
47+
- 'DELAY_STARTUP_MS=2000'
48+
volumes:
49+
- '../:/meshage:ro'

‎example/docker-compose.grapevine.yml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
version: '3'
2+
services:
3+
node-seed:
4+
build: ./
5+
ports:
6+
- '8080:8080'
7+
environment:
8+
- 'SERVICE_PORT=8080'
9+
- 'CLUSTER_PORT=9374'
10+
- 'LIB=meshage'
11+
- 'DEBUG=meshage,meshage:*'
12+
volumes:
13+
- '../:/meshage:ro'
14+
node-member:
15+
build: ./
16+
environment:
17+
- 'SEED=node-seed:9374'
18+
- 'LIB=meshage'
19+
- 'DEBUG=meshage,meshage:*'
20+
volumes:
21+
- '../:/meshage:ro'

‎example/example.sh

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/usr/bin/env bash
2+
3+
docker exec \
4+
-it $(docker ps | grep node-a | awk '{print $1}') \
5+
curl http://localhost:8080/api/echo/$RANDOM | jq .

‎example/index.js

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
const meshage = require(process.env.LIB || '../dist/src');
2+
const os = require('os');
3+
4+
const serviceHost = process.env.SERVICE_HOST || os.hostname();
5+
const servicePort = process.env.SERVICE_PORT || '8080/find';
6+
const serviceAddress = `${serviceHost}:${servicePort}`;
7+
8+
const clusterType = process.env.CLUSTER_TYPE;
9+
const clusterHost = process.env.CLUSTER_HOST || serviceHost;
10+
const initClusterPort = `${parseInt(servicePort, 10) - 10}`;
11+
const clusterPort = process.env.CLUSTER_PORT || `${initClusterPort}/find`;
12+
const clusterAddress = `${clusterHost}:${clusterPort}`;
13+
14+
const delayStartupMs = process.env.DELAY_STARTUP_MS || 0;
15+
16+
const seeds = (process.env.SEED || `${clusterHost}:${initClusterPort}`).split(/,/);
17+
18+
console.log(`starting on ${clusterAddress} in ${delayStartupMs} ms with ${seeds.length} seed(s) ${seeds}`.trim());
19+
20+
setTimeout(() => {
21+
22+
let cluster;
23+
if (clusterType === 'consul') {
24+
cluster = new meshage.ConsulCluster(clusterAddress, seeds);
25+
} else {
26+
cluster = new meshage.GrapevineCluster(clusterAddress, seeds);
27+
}
28+
29+
meshage
30+
.init(cluster, serviceAddress)
31+
.register('echo', (message, header) => ({
32+
header,
33+
echo: message
34+
}))
35+
.start();
36+
37+
}, delayStartupMs);
38+
39+
process.on('unhandledRejection', error => {
40+
console.log('unhandledRejection', error);
41+
});

0 commit comments

Comments
 (0)
0