Closed
Description
As we gear up towards our 1.0 release, documentation needs to be added about the following:
New docs:
How to's
- Connecting to Remote Storage from your Local Machine
- Connect to ADLS gen2 account locally (How to connect to ADLS Gen 2 in Spark.Net? #337) - PR link
- Connect to Azure Blobs (WASB) account locally (How to connect to ADLS Gen 2 in Spark.Net? #337) - PR link
- Connecting to Data Sources
- Use .NET for Apache Spark locally using Jupyter notebooks (Using with .NET Interactive notebooks locally #529) - PR link
- Use .NET for Apache Spark locally using VS Code - PR link
- User-defined Functions in .NET for Apache Spark
- Using .NET for Spark on Azure Synapse Analytics - Update links after how to guides published
Dotnet-interactive guide/good to know's
- Dotnet interactive serialization issue ([BUG]: UDF Serialization bug in .Net interactive #619) - PR link
- Running UDFs in notebooks - PR link
Migration guide - #721
- Document moving
Microsoft.Spark.ExperimentaltoMicrosoft.Spark. - Move Microsoft.Spark.Experimental to Microsoft.Spark #691 - Known issues: e.g.) grouped map UDF for Spark 3.0
- .NET Delta/Hyperspace version | OSS Delta/Hyperspace version | Spark versions matrix
- Dropping support for .NET Framework
- Show Scala version in microsoft-spark*.jar filename
Existing docs
- Update this documentation to reflect how to zip binaries for .NET for Spark batch submission. - PR link
- Update FAQs to include how
ArrayListandHashTablecannot be supported as UDF return type [FEATURE REQUEST]: Support ArrayList / Hashtable udf chaining #684 - PR link - Update links after how to guides published - Verify (and simplify) this - PR link
- Update this for Submitting batch jobs (with additional dependencies) - PR link
- Verify all public facing documentation
- GitHub documentation
- .NET Website (https://dot.net/spark) - PR link
- Azure Synapse Analytics - PR link - Update links after how to guides published