8000 Avro Compact Format by alexec · Pull Request #578 · cloudevents/sdk-java · GitHub
[go: up one dir, main page]

Skip to content

Avro Compact Format #578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Jul 20, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
8000
Diff view
Next Next commit
feat: Add Avro EvenFormat
Signed-off-by: Alex Collins <alex_collins@intuit.com>
  • Loading branch information
alexec committed Jun 6, 2023
commit 989c0fa519e52dfbf39e15724034c1a32ef6355b
96 changes: 96 additions & 0 deletions docs/avro.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
title: CloudEvents Avro
nav_order: 4
---

# CloudEvents Avro

[![Javadocs](http://www.javadoc.io/badge/io.cloudevents/cloudevents-avro.svg?color=green)](http://www.javadoc.io/doc/io.cloudevents/cloudevents-avro)

This module provides the Avro Buffer (avro) `EventFormat` implementation using the Java
avro runtime and classes generated from the CloudEvents
[avro spec](https://github.com/cloudevents/spec/blob/v1.0.1/spec.avro).

# Setup
For Maven based projects, use the following dependency:

```xml
<dependency>
<groupId>io.cloudevents</groupId>
<artifactId>cloudevents-avro</artifactId>
<version>x.y.z</version>
</dependency>
```

No further configuration is required is use the module.

## Using the avro Event Format

### Event serialization

```java
import io.cloudevents.CloudEvent;
import io.cloudevents.core.format.EventFormatProvider;
import io.cloudevents.core.builder.CloudEventBuilder;
import io.cloudevents.avro.avroFormat;

CloudEvent event = CloudEventBuilder.v1()
.withId("hello")
.withType("example.vertx")
.withSource(URI.create("http://localhost"))
.build();

byte[]serialized = EventFormatProvider
.getInstance()
.resolveFormat(avroFormat.CONTENT_TYPE)
.serialize(event);
```

The `EventFormatProvider` will automatically resolve the `avroFormat` using the
`ServiceLoader` APIs.

## Passing avro messages as CloudEvent data.

The `AvroCloudEventData` capability provides a convenience mechanism to handle avro message object data.

### Building

```java
// Build my business event message.
com.google.avro.Message myMessage = ..... ;

// Wrap the avro message as CloudEventData.
CloudEventData ceData = AvroCloudEventData.wrap(myMessage);

// Build the CloudEvent
CloudEvent event = CloudEventBuilder.v1()
.withId("hello")
.withType("example.avrodata")
.withSource(URI.create("http://localhost"))
.withData(ceData)
.build();
```

### Reading

If the `AvroFormat` is used to deserialize a CloudEvent that contains a avro message object as data you can use
the `AvroCloudEventData` to access it as an 'Any' directly.

```java

// Deserialize the event.
CloudEvent myEvent = eventFormat.deserialize(raw);

// Get the Data
CloudEventData eventData = myEvent.getData();

if (ceData instanceOf AvroCloudEventData) {

// Obtain the avro 'any'
Any anAny = ((AvroCloudEventData) eventData).getAny();

...
}

```

9 changes: 9 additions & 0 deletions formats/avro/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# CloudEvents Avro Format

This project provides functionality for the Java SDK to handle the
[avro format](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/formats/avro-format.md).

The Avro definition file is located in src/main/avro/spec.proto. The file was directly
copied from [https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/formats/cloudevents.avsc]().

The namespace has been changed so it does not clash with the core namespace.
97 changes: 97 additions & 0 deletions formats/avro/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Copyright 2021-Present The CloudEvents Authors
~ <p>
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~ <p>
~ http://www.apache.org/licenses/LICENSE-2.0
~ <p>
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
~
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>io.cloudevents</groupId>
<artifactId>cloudevents-parent</artifactId>
<version>2.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>cloudevents-avro</artifactId>
<name>CloudEvents - Avro</name>

<build>
<plugins>
<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>1.11.1</version>
<configuration>
<stringType>String</stringType>
</configuration>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>schema</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

<dependencies>
<dependency>
<groupId>io.cloudevents</groupId>
<artifactId>cloudevents-core</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
<version>1.11.1</version>
</dependency>

<!-- Test deps -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.36</version>
<scope>test
</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>${junit-jupiter.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>${assertj-core.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.cloudevents</groupId>
<artifactId>cloudevents-core</artifactId>
<classifier>tests</classifier>
<type>test-jar</type>
<version>${project.version}</version>
<scope>test</scope>
</dependency>

</dependencies>

</project>
73 changes: 73 additions & 0 deletions formats/avro/src/main/avro/cloudevents.avsc
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
{
"namespace": "io.cloudevents.v1.avro",
"type": "record",
"name": "CloudEvent",
"version": "1.0",
"doc": "Avro Event Format for CloudEvents",
"fields": [
{
"name": "attribute",
"type": {
"type": "map",
"values": [
"null",
"boolean",
"int",
"string",
"bytes"
]
}
},
{
"name": "data",
"type": [
"bytes",
"null",
"boolean",
{
"type": "map",
"values": [
"null",
"boolean",
{
"type": "record",
"name": "CloudEventData",
"doc": "Representation of a JSON Value",
"fields": [
{
"name": "value",
"type": {
"type": "map",
"values": [
"null",
"boolean",
{
"type": "map",
"values": "CloudEventData"
},
{
"type": "array",
"items": "CloudEventData"
},
"double",
"string"
]
}
}
]
},
"double",
"string"
]
},
{
"type": "array",
"items": "CloudEventData"
},
"double",
"string"
],
"default": "null"
}
]
}
Loading
0