8000 feat: prepare the 0.9.1 release by SemyonSinchenko · Pull Request #650 · graphframes/graphframes · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 13 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
<img src="docs/img/GraphFrames-Logo-Large.png" alt="GraphFrames Logo" width="400"/>
<img src="docs/img/GraphFrames-Logo-Large.png" alt="GraphFrames Logo" width="500"/>

[![Scala CI](https://github.com/graphframes/graphframes/actions/workflows/scala-ci.yml/badge.svg)](https://github.com/graphframes/graphframes/actions/workflows/scala-ci.yml)
[![Python CI](https://github.com/graphframes/graphframes/actions/workflows/python-ci.yml/badge.svg)](https://github.com/graphframes/graphframes/actions/workflows/python-ci.yml)
[![pages-build-deployment](https://github.com/graphframes/graphframes/actions/workflows/pages/pages-build-deployment/badge.svg)](https://github.com/graphframes/graphframes/actions/workflows/pages/pages-build-deployment)
[![scala-central-publish](https://github.com/graphframes/graphframes/actions/workflows/scala-publish.yml/badge.svg)](https://github.com/graphframes/graphframes/actions/workflows/scala-publish.yml)
[![python-pypi-publish](https://github.com/graphframes/graphframes/actions/workflows/python-publish.yml/badge.svg)](https://github.com/graphframes/graphframes/actions/workflows/python-publish.yml)
![GitHub Release](https://img.shields.io/github/v/release/graphframes/graphframes)
![GitHub License](https://img.shields.io/github/license/graphframes/graphframes)



# GraphFrames: DataFrame-based Graphs

Expand All @@ -26,19 +30,19 @@ GraphFrames scala core and Spark-Connect plugin are published in the Sonatype Ce
# Interactive Scala/Java

# For Spark 3.5.x, scala 2.12
$ spark-shell --packages io.graphframes:graphframes-spark3_2.12:0.9.0
$ spark-shell --packages io.graphframes:graphframes-spark3_2.12:0.9.1

# For Spark 3.5.x, scala 2.13
$ spark-shell --packages io.graphframes:graphframes-spark3_2.13:0.9.0
$ spark-shell --packages io.graphframes:graphframes-spark3_2.13:0.9.1

# For Spark 4.0.x
$ spark-shell --packages io.graphframes:graphframes-spark4_2.13:0.9.0
$ spark-shell --packages io.graphframes:graphframes-spark4_2.13:0.9.1

# Interactive Python, Spark 3.5.x
$ pyspark --packages io.graphframes:graphframes-spark3_2.12:0.9.0
$ pyspark --packages io.graphframes:graphframes-spark3_2.12:0.9.1

# Interactive Python, Spark 4.0.x
$ pyspark --packages io.graphframes:graphframes-spark4_2.13:0.9.0
$ pyspark --packages io.graphframes:graphframes-spark4_2.13:0.9.1
```

### GraphFrames Python API
Expand All @@ -58,7 +62,7 @@ To add GraphFrames to your spark connect server, you need to specify the plugin
```bash
./sbin/start-connect-server.sh \
--conf spark.connect.extensions.relation.classes=org.apache.spark.sql.graphframes.GraphFramesConnect \
--packages io.graphframes.graphframes-connect-spark4_2.13:0.9.0
--packages io.graphframes.graphframes-connect-spark4_2.13:0.9.1
--conf spark.checkpoint.dir=${CHECKPOINT_DIR}
```

Expand Down Expand Up @@ -208,14 +212,14 @@ Maven:
<dependency>
<groupId>io.graphframes</groupId>
<artifactId>graphframes-spark4_2.13</artifactId>
<version>0.9.0</version>
<version>0.9.1</version>
</dependency>
</dependencies>
```

SBT:
```sbt
libraryDependencies += "io.graphframes" %% "graphframes-spark4" % "0.9.0"
libraryDependencies += "io.graphframes" %% "graphframes-spark4" % "0.9.1"
```

**WARNING!**
Expand Down
2 changes: 1 addition & 1 deletion docs/_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ include:

# These allow the documentation to be updated with newer releases
# of Spark, Scala, and Mesos.
GRAPHFRAMES_VERSION: 0.8.4
GRAPHFRAMES_VERSION: 0.9.1
#SCALA_BINARY_VERSION: "2.10"
#SCALA_VERSION: "2.10.4"
#MESOS_VERSION: 0.21.0
Expand Down
2 changes: 1 addition & 1 deletion docs/_plugins/copy_api_dirs.rb
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@

# Copy over the unified ScalaDoc for all projects to api/scala.
# This directory will be copied over to _site when `jekyll` command is run.
source = "../target/scala-2.12/api"
source = "../core/target/scala-2.13/api"
dest = "api/scala"

puts "Making directory " + dest
Expand Down
40 changes: 33 additions & 7 deletions python/README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,43 @@
# GraphFrames `graphframes-py` Python Package
# GraphFrames Python Package

The is the officila [graphframes-py PyPI package](https://pypi.org/project/graphframes-py/), which is a Python wrapper for the Scala GraphFrames library. This package is maintained by the GraphFrames project and is available on PyPI.
![PyPI - Downloads](https://img.shields.io/pypi/dm/graphframes-py) ![PyPI - License](https://img.shields.io/pypi/l/graphframes-py) ![PyPI - Version](https://img.shields.io/pypi/v/graphframes-py)

For instructions on GraphFrames, check the project [../README.md](../README.md). See [Installation and Quick-Start](#installation-and-quick-start) for the best way to install and use GraphFrames.
<img src=https://raw.githubusercontent.com/graphframes/graphframes/refs/heads/master/docs/img/GraphFrames-Logo-Large.png width=500>

https://graphframes.io/

The is the officila [graphframes-py PyPI package](https://pypi.org/project/graphframes-py/), which is a Python wrapper for the Scala GraphFrames library.
This package is maintained by the GraphFrames project and is available on PyPI.

For instructions on GraphFrames, check the project [README.md](https://github.com/graphframes/graphframes?tab=readme-ov-file#graphframes-dataframe-based-graphs).

See [Installation and Quick-Start](https://github.com/graphframes/graphframes?tab=readme-ov-file#installation-and-quick-start) for the best way to install and use GraphFrames.

## Installation

```bash
pip install graphframes-py
```

**NOTE!** *Python distribution does not include JVM-core. You need to add it to your cluster or Spark-Connect server!*

## Running `graphframes-py`

You should use GraphFrames via the `--packages` argument to `pyspark` or `spark-submit`, but this package is helpful in development environments.

```bash
# Interactive Python
$ pyspark --packages graphframes:graphframes:0.8.4-spark3.5-s_2.12
# Interactive Python, Spark 3.5.x
$ pyspark --packages io.graphframes:graphframes-spark3_2.12:0.9.1

# Submit a script in Scala/Java/Python
$ spark-submit --packages graphframes:graphframes:0.8.4-spark3.5-s_2.12 script.py
# Interactive Python, Spark 4.0.x
$ pyspark --packages io.graphframes:graphframes-spark4_2.13:0.9.1
```

## Documentation

- [API Reference](https://graphframes.io/api/python/index.html)

## Spark-Connect Note

GraphFrames PySpark is choosing connect or classic implementation implicitly based on the result of `is_remote()`.
To enforce usage of connect-based implementation, you may export this variable `SPARK_CONNECT_MODE_ENABLED=1`
0